CSV command line export encoding -u flag no effect?

Hello I am trying to convert dlt files to csv via the command line interface.

Should the flag -u have an effect on the encoding of the csv file?

At first I tried this conversion command and it works perfectly for some dlt files:

dlt-viewer.exe -s -csv -c <input.dlt> <output.csv>

Most of the time it works and I get a csv file that I can work with.

But for some csv files I get encoding errors while reading file. My guess is that there is some strange logging output in some payload messages.

To fix the encoding problems i tried with the -u flag:

dlt-viewer.exe -s -csv -c <input.dlt> <output.csv>

But the problem stayed the same.

For the moment my solution is to replace encoding errors while reading the file but of course it would be better to have a "perfect" csv file in the first place. :)

Perhaps someone had a similar problem and found a solution. Thanks in advance.

Hi @suiluj. Could it be that you have corrupted messages in your DLT file or some Linefeeds/Cariage Returns in your AppIds/CtxIds. There exists a fix already, that fixed the issue:
Remove Linefeeds and Cariage Returns in CSV Export of ECUId, AppId and CtxId.
Please check if this helps.

Hi @alexmucde yes I noticed that there was a pull request for a fix.

But in my case it seems that it is part of the payload column

When I tried to load the exported csv with python pandas or duckdb I got an error that said there is an error in the payload column.

Ich checked the row but did not notice any strange characters in dlt viewers.

@suiluj So could be another issue in your payload. Please check the source code. Perhaps some characters must be removed or replaced.

dlt-viewer/src/dltexporter.cpp

Line 53 in 5ddd642

void DltExporter::writeCSVLine(int index, QFile *to, QDltMsg msg)

Please provide more details which characters could cause the issues.
Best would be to provide a patch file.

hi @alexmucde

I just kept my solution with the file read and "replace" on error.

(I use python for processing the generated csv file)
https://docs.python.org/3/library/functions.html#open

Since then I never noticed any problems in my workflow so we can close this issue.