Throw an error with mismatched Apache Arrow version
willium opened this issue · 8 comments
What happens?
Right now, if you have a mismatched apache arrow version, DuckDB WASM fails silently or throws very confusing errors. I was trying to create a table, and nothing happened.
To Reproduce
Target Arrow 15 or 16, and try to use DuckDB.
Browser/Environment:
Chrome
Device:
Mac
DuckDB-Wasm Version:
Latest
DuckDB-Wasm Deployment:
App
Full Name:
William Wolf
Affiliation:
Gestalt
This keeps coming up. I wonder what the best way to resolve this is. Maybe only include arrow as a peer dependency or a more robust check for isArrowTable?
Related vega/vega-loader-arrow#350
Is the arrow esm vs csj problem I reported here #1708 ?
That's related. The problem can also happen when you have e.g. arrow 14 but duckdb wasm uses arrow 15.
Problem seems quite hard, it's the problem of being successful, but instanceof feel like the wrong solution here. Would it not be possible to duck-type this and assume that if something looks like an arrow table has to be an arrow table?
Or possibly using stuff like obj.constructor.someStaticFieldThatEncodesTheArrowVersion ?
One note is that DuckDB-Wasm has possibly a simpler job that regular arrow, since we can implement detection our side, once a semi-reliable detection function is approved.
One option might be to re-export apache arrow? At least that's what some Rust libraries do for their dependencies.
That doesn't help if another library has arrow as a dependency. This really needs to be solved at the npm/yarn/etc level but we can try to warn people when they may have multiple versions.