eclipse/transformer

Converting a text file which is not encoded in UTF-8 causes garbled characters

11rx4f opened this issue · 2 comments

11rx4f commented

Environment Details

  • Transformer Version: 0.5.0
  • JDK version: openjdk 11.0.12
  • OS: Red Hat Enterprise Linux 7.4

Problem Description

Converting a text file which is not encoded in UTF-8 causes garbled characters.
In some cases, comments in the source code will not be recognized correctly and will fail to compile.

Steps to reproduce

Specify a text file such as java or xml which isn't encoded in UTF-8 (for example, Shift_JIS) as the conversion target.

Cause

This is because Java and Text files are always read and written in UTF-8.

transformer/org.eclipse.transformer/src/main/java/org/eclipse/transformer/util/FileUtils.java

Line 164 in 2411075

OutputStreamWriter outputWriter = new OutputStreamWriter(outputStream, UTF_8);

bjhargrave commented

@11rx4f, If we used the encoding specified by the file.encoding system property as the input and output encoding, would that work for your use case?

bjhargrave commented

If we used the encoding specified by the file.encoding system property as the input and output encoding, would that work for your use case?

This would be useful in the command line usage but not as useful in the maven plugin or bnd plugin (and eventually gradle plugin) scenarios. So we may need an encoding option which can be set.