Warnings shown when quiet flag
SDKongstad opened this issue · 4 comments
I am mostly running odbc2parquet from Powershell.
I noticed that I always get warnings when running - which powershell interprets as errors. To fix this I tried using the -q flag.
However warnings are still shown
Is it possible to introduce a flag so only errors are displayed?
Thanks for reporting this. Seems odbc2parquet had a bug which caused verbosity always be one level too high. odbc2parquet 0.13.1 is released just now. Update to it, and --quite should work as intended.
Does it work for you now? I use powershell myself if on windows and did not experience trouble before, so I am curious if a different issue emerges from this one.
Sorry about the late reply - Yes this fixed the issue!
Our operations team will be happy when the warnings disappear :)
We are using odbc2parquet to move data between a SQL Server on a an Azure vm and into a Delta lake.
I've set up a PS script that exports all tables in parallel to parquets, and then uses AZCopy to move it to the blob storage.
This is leagues faster - and indeed simpler - than using Data Factory and a gateway, and so far I've had no luck setting up a shared vnet between the databricks environment and the VM.
It's a part of a process of moving the SQL server from the VM and into a Delta lake, so it's not gonna be forever, but we needed to monitor each step - which is why the warnings were a nuisance to us!
Thanks for your reply!
Cheers, Markus