Wrong text encoding

Question

Wrong text encoding

lukasini opened this issue 9 months ago · 3 comments

I´m facing issue with encoding symbols š,č etc. when scrapping slovak pages. I guess this is limitation of SQLite that supports only ISO-8859-1. Also exporting to UTF8 change nothing. Do you have some workaround or better way to fix it?

Within application it is looking fine but inside database its encoded wrongly

Answer 1 · 2024-01-21T07:58:53.000Z

Hi lukasini, I guess the encoding is correct but you need to handle it in your subsequent workflows. How do you process the data? When you export a csv file, does it come out with the right encoding?

For R, we have an experimental package that loads the database, maybe it helps you? See https://github.com/strohne/datavana/tree/main/facepager

Answer 2 · 2024-01-28T09:27:52.000Z

I suggest you use the export button of Facepager, that handles the encoding :)

Answer 3 · 2024-01-28T12:29:23.000Z

Yes correct, direct export to CSV is working. Thank you for support.