Dependency Consolidation

[sfc-gh-twhite]

There is likely some overlap in our environment.yml file, where we can consolidate some dependencies. I think pip install jupyter snowflake-ml-python gets us everything. After this, running a pip freeze gives me quite a lot. (There still might be an issue with XGBoost that we need to research)

ValueError: Mismatched version between the Python package and the native shared object.  Python package version: 1.7.6. Shared object version: 1.7.3. Shared object is loaded from: [/envs/Intro_SnowML/lib/libxgboost.dylib.](/envs/Intro_SnowML/lib/libxgboost.dylib.)
Likely cause:
  * XGBoost is first installed with anaconda then upgraded with pip. To fix it please remove one of the installations.

absl-py==1.4.0
aiobotocore==2.11.2
aiohttp==3.9.3
aioitertools==0.11.0
aiosignal==1.3.1
anyio==3.7.1
appnope==0.1.4
argon2-cffi==23.1.0
argon2-cffi-bindings==21.2.0
arrow==1.3.0
asn1crypto==1.5.1
asttokens==2.4.1
async-lru==2.0.4
async-timeout==4.0.3
attrs==23.2.0
Babel==2.14.0
beautifulsoup4==4.12.3
bleach==6.1.0
botocore==1.34.34
cachetools==4.2.4
certifi==2024.2.2
cffi==1.16.0
charset-normalizer==3.3.2
cloudpickle==2.2.1
comm==0.2.1
cryptography==41.0.7
debugpy==1.8.0
decorator==5.1.1
defusedxml==0.7.1
exceptiongroup==1.2.0
executing==2.0.1
fastjsonschema==2.19.1
filelock==3.13.1
fqdn==1.5.1
frozenlist==1.4.1
fsspec==2023.12.2
h11==0.14.0
httpcore==1.0.2
httpx==0.26.0
idna==3.6
importlib-resources==5.13.0
ipykernel==6.29.1
ipython==8.21.0
ipywidgets==8.1.1
isoduration==20.11.0
jedi==0.19.1
Jinja2==3.1.3
jmespath==1.0.1
joblib==1.3.2
json5==0.9.14
jsonpointer==2.4
jsonschema==4.21.1
jsonschema-specifications==2023.12.1
jupyter==1.0.0
jupyter-console==6.6.3
jupyter-events==0.9.0
jupyter-lsp==2.2.2
jupyter_client==8.6.0
jupyter_core==5.7.1
jupyter_server==2.12.5
jupyter_server_terminals==0.5.2
jupyterlab==4.1.0
jupyterlab-widgets==3.0.9
jupyterlab_pygments==0.3.0
jupyterlab_server==2.25.2
MarkupSafe==2.1.5
matplotlib-inline==0.1.6
mistune==3.0.2
multidict==6.0.5
nbclient==0.9.0
nbconvert==7.15.0
nbformat==5.9.2
nest-asyncio==1.6.0
notebook==7.0.7
notebook_shim==0.2.3
numpy==1.26.4
overrides==7.7.0
packaging==23.2
pandas==1.5.3
pandocfilters==1.5.1
parso==0.8.3
pexpect==4.9.0
platformdirs==3.11.0
prometheus-client==0.19.0
prompt-toolkit==3.0.43
psutil==5.9.8
ptyprocess==0.7.0
pure-eval==0.2.2
pyarrow==15.0.0
pycparser==2.21
Pygments==2.17.2
PyJWT==2.8.0
pyOpenSSL==23.3.0
python-dateutil==2.8.2
python-json-logger==2.0.7
pytimeparse==1.1.8
pytz==2024.1
PyYAML==6.0.1
pyzmq==25.1.2
qtconsole==5.5.1
QtPy==2.4.1
referencing==0.33.0
requests==2.31.0
retrying==1.3.4
rfc3339-validator==0.1.4
rfc3986-validator==0.1.1
rpds-py==0.17.1
s3fs==2023.12.2
scikit-learn==1.3.2
scipy==1.12.0
Send2Trash==1.8.2
six==1.16.0
sniffio==1.3.0
snowflake-connector-python==3.7.0
snowflake-ml-python==1.2.1
snowflake-snowpark-python==1.12.0
sortedcontainers==2.4.0
soupsieve==2.5
sqlparse==0.4.4
stack-data==0.6.3
terminado==0.18.0
threadpoolctl==3.2.0
tinycss2==1.2.1
tomli==2.0.1
tomlkit==0.12.3
tornado==6.4
traitlets==5.14.1
types-python-dateutil==2.8.19.20240106
typing_extensions==4.9.0
uri-template==1.3.0
urllib3==2.0.7
wcwidth==0.2.13
webcolors==1.13
webencodings==0.5.1
websocket-client==1.7.0
widgetsnbextension==4.0.9
wrapt==1.16.0
xgboost==1.7.6
yarl==1.9.4

This is something else we might want to add though, just to prevent the warning:

Package 'fastparquet' is not installed in the local environment. Your UDF might not work when the package is installed on the server but not on your local environment.