Non-ascii characters prepended with UTF (IDFGH-11966)
nebkat opened this issue · 5 comments
When using various non-ascii characters in serial output over esp-idf-monitor powershell prepends replacement character 0xEF 0xBF 0xBD in Windows PowerShell.
Have tried changing the PowerShell encoding and source file encoding to no avail.
Seems to work as expected on esptool-js.
Example
ESP_LOGI(TAG, "╔════════════════════════════════════════════════╗");
ESP_LOGI(TAG, "║ Version │ %-32s " "║", app_desc->version);
ESP_LOGI(TAG, "║ ESP-IDF │ %-32s " "║", app_desc->idf_ver);
ESP_LOGI(TAG, "║ ELF SHA256 │ %-32s " "║", app_desc.elf_sha256_hex(8).c_str());
ESP_LOGI(TAG, "╟─────────────┼──────────────────────────────────╢");
ESP_LOGI(TAG, "║ Chip │ %-32s " "║", chip_info.model_name());
ESP_LOGI(TAG, "║ Revision │ %-32d " "║", chip_info->revision);
ESP_LOGI(TAG, "║ Cores │ %-32d " "║", chip_info->cores);
ESP_LOGI(TAG, "╟─────────────┼──────────────────────────────────╢");
ESP_LOGI(TAG, "║ Partition │ %-32s " "║", running_partition->label);
ESP_LOGI(TAG, "║ Boot reason │ %-32s " "║", sys::reset_reason_string());
ESP_LOGI(TAG, "╚═════════════╧══════════════════════════════════╝");I (848) APP : ���╔���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���╗
I (888) APP : ���║ Version ���│ 0d46b8c-dirty ���║
I (898) APP : ���║ ESP-IDF ���│ v5.2-beta2-dirty ���║
I (908) APP : ���║ ELF SHA256 ���│ 4ebdab159d1674d4 ���║
I (918) APP : ���╟���─���─���─���─���─���─���─���─���─���─���─���─���─���┼���─���─���─���─���─���─���─���─���─���─���─���─���─���─���─���─���─���─���─���─���─���─���─���─���─���─���─���─���─���─���─���─���─���─���╢
I (928) APP : ���║ Chip ���│ ESP32 ���║
I (938) APP : ���║ Revision ���│ 300 ���║
I (948) APP : ���║ Cores ���│ 2 ���║
I (958) APP : ���╟���─���─���─���─���─���─���─���─���─���─���─���─���─���┼���─���─���─���─���─���─���─���─���─���─���─���─���─���─���─���─���─���─���─���─���─���─���─���─���─���─���─���─���─���─���─���─���─���─���╢
I (968) APP : ���║ Partition ���│ ota_0 ���║
I (978) APP : ���║ Boot reason ���│ POWERON ���║
I (988) APP : ���╚���═���═���═���═���═���═���═���═���═���═���═���═���═���╧���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���═���╝
Same result with emojis.
Hi @nebkat, can you please provide more details about your environment? Like the version of idf-monitor, PowerShell, and Windows you are using? I have tried the example that you provided and everything is working for me (including emojis). I am using esp-idf-monior 1.3.4, PowerShell - 5.1.22621.1 on Windows 11. Please also check if you are not using some non-standard font in your PowerShell.
idf-monitor: v1.3.4
powershell: v5.1 and v7.4.1 both tested
windows: 11 22H2 22621.3007
Same result in CLion PowerShell-based terminal. My colleagues have the same problem on PCs which are not specifically kept in sync on any versions.
I believe the problem started about a year ago, but I never properly investigated. I know originally when I wrote the box drawing characters code they displayed correctly.
@nebkat One more thing: How are you executing the monitor? Are you using idf.py wrapper or are you running the monitor directly? If you are using the idf.py monitor command, can you please disable hints by running idf.py --no-hints monitor or run the package directly by running python -m esp_idf_monitor?
I was using idf.py, and --no-hints fixes it! I will close this and open an issue in esp-idf. Thank you for your help.
On closer inspection if I disable get_ansi_converter from this package in idf/tools.py the problem goes away, so might still need to be addressed here?
# use ANSI color converter for Monitor on Windows
output_converter = output_stream # get_ansi_converter(output_stream) if self.convert_output else output_stream