snowflakedb/snowflake-cli

SNOW-1246906: `snow build` snowflake/core dependency bundling broken

Closed this issue · 5 comments

fletchertyler914 commented

SnowCLI version

2.1.0

Python version

3.11.7

Platform

macOS-10.16-x86_64-i386-64bit

What happened

I am deploying a snowpark project that uses snowflake and snowflake.core packages.
If I add snowflake and snowflake.core to the requirements.txt, and run snow build, the snowflake and snowflake.core dependencies are added to a requirements.other.txt instead of the requirements.snowflake.txt, so running snow deploy will not include the snowflake packages in the packages list of the procedure or function being deployed, so the deployment fails.

If I manually copy the snowflake and snowflake.core into the requirements.snowflake.txt file, after building and before deploying, it works as expected. This seems to just be a matter of adding snowflake (and related) dependencies to the correct requirements.txt file.

Console output

build succeeds, deploy fails do to missing `snowflake.core` package.

How to reproduce

  1. create a new directory
  2. navigate into new directory
  3. initialize a new snowpark project
  4. update procedure/function to use snowflake.core, like retrieving a TaskResource from a db/schema.
  5. add snowflake and/or snowflake.core to the requirements.txt
  6. build the project using snow build

you will see the packages are add to requirements.other.txt instead of requirements.snowflake.txt

sfc-gh-pjob commented

It looks like the issue is caused by the logic of finding packages available in Snowflake. We get the list of available packages from snowflake anaconda channel info instead of using the query presented in snowpark docs (select * from information_schema.packages where language = 'python';).

We'll work on a fix to make it working correctly.

See:

sfc-gh-pjob commented

To extend my previous answer:
https://repo.anaconda.com/pkgs/snowflake/channeldata.json doesn't contain snowflake and snowflake.core packages while select * from information_schema.packages where language = 'python'; returns them.

  • 👍2
fletchertyler914 commented

Ah, got it! Will be glad to see this improvement.

sfc-gh-pjob commented

The fix can be tested using the release candidate (pre-release) for 2.2.0 which can be installed the following way:

pip install snowflake-cli-labs==2.2.0rc0

However, as it is only the release candidate, it should be used only for tests, not in the production.

sfc-gh-pjob commented

We released CLI v2.2.0. The fix is there.