Issue: java.nio.charset.CharsetDecoder can corrupt data during Load on IBM 1.6/1.7 VMs

Olivier Thomann
Last Updated: October 22nd, 2012
Build Basis: Rational Team Concert 3.0.1 and 4.0

Summary

When file content encoding is changed during load, some characters that cannot be mapped to the targeted encoding may be replaced with the ‘?’ character causing a loss of data.

More Information

There is a bug in the java.nio.charset.CharsetDecoder implementation on the IBM 1.6/1.7 VMs that prevents the conversion from terminating when a character cannot be mapped to the targeted encoding. The java.nio.charset.CharsetDecoder instantiation sets the action to be java.nio.charset.CodingErrorAction.REPORT when an unmappable character is encountered. The problem is that this action is ignored and the corresponding character is replaced with a ‘?’ character.

There is no workaround at this time for users that do not have access to an Oracle VM on their platforms. A fix for the IBM VM is expected to be ready in Java 7 SR4 release.

Related Information

The following links point to related information:

• 230440: Change in behavior in Java 6 may cause issue in Filesystem Core