amphi-ai/amphi-etl

Read and write compressed files

Opened this issue · 1 comments

Hello,

More than once, we received files that are zipped. We can of course dezipped it before reading in amphi but that's an extra step I would like to spare.

On the other hand, writing zipped files is also great, since it means we can save disk space.

There are of course a lot of format but .zip and .gz are the more common I think and it will be enough for a first iteration.

Best regards,

Simon

Thanks Simon, just checked and it's pretty easy with pandas as it's natively supported (for both read and write). Will add the option shortly.