astronomer/astro-sdk

DuckDB integration does not use published wheel on Astro runtime

jeremybeard opened this issue · 0 comments

jeremybeard commented

Describe the bug
When building an Astro runtime image with the Astro SDK DuckDB submodule the published DuckDB wheel is not used and instead it is compiled locally, which takes many minutes and also requires installing the gcc compiler on the image.

Version

  • Astro SDK: 1.6.1
  • Astro runtime: 8.4.0
  • OS: macOS

To Reproduce
Steps to reproduce the behavior:

  1. Create a new Astro CLI project with astro dev init
  2. Modify requirements.txt to have the line astro-sdk-python[duckdb]==1.6.1
  3. Modify packages.txt to have the line g++ (gcc is required to build the wheel)
  4. Start the Airflow instance with astro dev start
  5. See that it takes ~5 minutes to build the DuckDB wheel

Expected behavior
The published DuckDB wheels to be used instead of building it locally.