more complete async support and async bug
Opened this issue · 2 comments
So I got arrow-odbc (almost) working with async-polling with these two PRs: pacman82/arrow-odbc#102 and #577
But I'm running into an issue with the snowflake ODBC driver. We're using the snowflake driver, which complains that we free the statement before we've completed executing (I've concluded that based on the error message and the MS docs here (https://learn.microsoft.com/en-us/sql/odbc/reference/syntax/sqlfreestmt-function?view=sql-server-ver16#diagnostics).
The query seems to stream fine, but the driver fails when the inner BlockCursorPolling is dropped in the AsyncBatchedOdbcReader struct, panicking in the drop impl for BlockCursorPolling with the following error:
Unexpected error unbinding columns: Diagnostics { record: State: HY010, Native error: 0, Message: [unixODBC][Driver Manager]Function sequence error, function: "SQLFreeStmt" }
I'm unsure how this is possible since all methods that use wait_for() do not return until StillExecuting is not returned. And if a method didn't handle StillExecuting, then it would panic here https://github.com/nobie-org/odbc-api/blob/a20309759e83ebdfbeac5c115b24b97917d6b5fd/odbc-api/src/error.rs#L244
Hello @matthewgapp ,
thanks for your effort. I took the liberty and already merged a fix for the missing enablement of async in execute polling you mentioned in #577. Please take a look at the tests and note that most drivers actually do not support polling.
It was mostly this poor support for polling (not even starting to think about event based async on windows) within ODBC drivers, which made me write ConcurrentBlockCursor which uses system threads and works reliably with many drivers.
For your use-case it seems to me worth verifying that the snowflake driver does indeed support polling.
Currently I am heavily occupied by my day job, so my ability to review large pull requests is inhibited. Splitting things out into smaller features, with their own test cases would help me. I have some further thoughts, but (again) I think the first thing to check if this is indeed actually helping you at all and snowflake supports this behavior.
Best, Markus
FYI: I just verified this is still working: If you want to execute all the tests locally you can just open it remotely in vscode in the devcontainer. It will boot up all databases and install all drivers in a devcontainer, provided you have docker compose installed.