Character Set Conversion

Converter Classes

Converter objects are created like this:

ByteToCharConverter toUnicode
= ByteToCharConverter.getConverter("ISO-8859-1");
CharToByteConverter fromUnicode
= CharToByteConverter.getConverter("CP850");

The static getConverter() methods return a subclass that will perform the appropriate conversion. The name strings used to select the converter must be valid IANA character set names¹. These are the same names as are used by the MIME charset parameter.

Both ByteToCharConverter and CharToByteConverter contain methods that are used to perform the actual conversion. The following simple example shows how a Unicode string could be converted into the Japanese Shift JIS character encoding using the convertAll() method:

String str = "The quick brown fox";

byte[] output;

CharToByteConverter fromUnicode

    = CharToByteConverter.getConverter("SJIS");

output = fromUnicode.convertAll( str.toCharArray() );

Unknown Characters

Stateful Conversions

As a result of the stateful nature of the converters, the flush() method must be called after a series of calls to convert(). This signals that a particular conversion is done and allows the converter to emit any final bytes or characters. A call to flush() also resets the converter to its initial state. An explicit reset() call is also available if the caller does not care about finalizing the previous conversion.

Exception Handling

To recover from an UnsupportedCharacterException or MalformedInputException, getBadInputLength() can be called to determine the length of the bad input and throw it out or fix it. In all cases, the nextCharIndex() and nextByteIndex() methods can be used to determine where conversion should resume in the input and output buffers.

The following example shows how to use the convert() method to translate between files:

public static void convertFile( DataInputStream in,

										DataOutputStream out,

										ByteToCharConverter conv )

    throws java.io.IOException

{

	// Set up output buffer.

	int outBufLen = 1200;

	char[] outBuf = new char [outBufLen];



	// Set up input buffer and index.

	int inBufLen = 1200;

	byte[] inBuf = new byte [inBufLen];

	int inStart = 0;



	// Use the ByteToCharConverter's substitution mode.

	// Substitute '?' for unknown characters.

	char[] subChars = { '?' };

	conv.setSubstitutionMode(true);

	conv.setSubstitutionChars(subChars);



	// If we don't know where this converter is coming from,

	// it's a good idea to reset it.

	conv.reset();



	// Read an input buffer's worth.

	while((inBufLen = in.read(inBuf, 0, inBuf.length)) != -1 ) {



		// Loop while there's input to convert. This loop only

		// needs to run more than once if MalformedInputException

		// or ConversionBufferFullException occurs.

		while( inStart < inBufLen ) {

			try {

				// convert it.

				conv.convert(inBuf, inStart, inBufLen,

							outBuf, 0, outBufLen);

				inStart = conv.nextByteIndex();

			} catch (MalformedInputException e) {

				// Throw away bad input.

				inStart += conv.getBadInputLength();

			} catch(ConversionBufferFullException e) {

				inStart = conv.nextByteIndex();

			} finally {

				// write it out!

				out.writeChars(

					new String(outBuf,0, conv.nextCharIndex()));

			}

		}

	}



	int flushedOutputLength = 0;

	try {

		flushedOutputLength = conv.flush(outBuf, 0, outBufLen);

	} catch (MalformedInputException e) {

		// The bad input will just be ignored.

	} finally {

		// write it out!

		out.writeChars(

			new String(outBuf,0, flushedOutputLength));

	}

}

Supported Encodings

OS Support For Character Conversions

• How character sets are designated.
  
  
• Which character sets are supported.
  
  
• How failure modes are handled.
  
  
• How easy it is to install additional conversion tables.