[Bug] psycopg2 dependency is not installed correctly
Auric-Manteo opened this issue · 4 comments
Is this a new bug in dbt-redshift?
- I believe this is a new bug in dbt-redshift
- I have searched the existing issues, and I could not find an existing issue for this bug
Current Behavior
When I install the following Pipfile and run dbt seed
afterwards I get an import error.
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"
[packages]
dbt-redshift = "==1.8.0"
sqlfluff = "~=2.2.0"
requests = "2.32.0"
jsonpath-ng="1.5.3"
[dev-packages]
[requires]
python_version = "3.9.9"
Expected Behavior
dbt seed
should run successfully.
Steps To Reproduce
- Run pipenv install in a directory with the above Pipfile.
- Run dbt seed
Relevant log output
06:53:16 Running with dbt=1.8.1
06:53:16 Encountered an error:
No module named 'psycopg2'
06:53:16 Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/dbt/cli/requires.py", line 138, in wrapper
result, success = func(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/dbt/cli/requires.py", line 101, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/dbt/cli/requires.py", line 215, in wrapper
profile = load_profile(flags.PROJECT_DIR, flags.VARS, flags.PROFILE, flags.TARGET, threads)
File "/usr/local/lib/python3.9/site-packages/dbt/config/runtime.py", line 71, in load_profile
profile = Profile.render(
File "/usr/local/lib/python3.9/site-packages/dbt/config/profile.py", line 403, in render
return cls.from_raw_profiles(
File "/usr/local/lib/python3.9/site-packages/dbt/config/profile.py", line 369, in from_raw_profiles
return cls.from_raw_profile_info(
File "/usr/local/lib/python3.9/site-packages/dbt/config/profile.py", line 325, in from_raw_profile_info
credentials: Credentials = cls._credentials_from_profile(
File "/usr/local/lib/python3.9/site-packages/dbt/config/profile.py", line 149, in _credentials_from_profile
cls = load_plugin(typename)
File "/usr/local/lib/python3.9/site-packages/dbt/adapters/factory.py", line 239, in load_plugin
return FACTORY.load_plugin(name)
File "/usr/local/lib/python3.9/site-packages/dbt/adapters/factory.py", line 96, in load_plugin
self.load_plugin(dep)
File "/usr/local/lib/python3.9/site-packages/dbt/adapters/factory.py", line 68, in load_plugin
mod: Any = import_module("." + name, "dbt.adapters")
File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 850, in exec_module
File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
File "/usr/local/lib/python3.9/site-packages/dbt/adapters/postgres/__init__.py", line 4, in <module>
from dbt.adapters.postgres.connections import PostgresConnectionManager, PostgresCredentials
File "/usr/local/lib/python3.9/site-packages/dbt/adapters/postgres/connections.py", line 13, in <module>
import psycopg2
ModuleNotFoundError: No module named 'psycopg2'
Environment
- OS: MacOS and also tried on Ubuntu
- Python: 3.9.9
- dbt-core: 1.7.15
- dbt-redshift: 1.8.0
Additional Context
No response
@Auric-Manteo On Ubuntu, I expect this would be a result of:
Which we're going to resolve by:
- installing
psycopg2-binary
by default, on all systems - removing the dependency of
dbt-redshift
ondbt-postgres
(#811)
However, on MacOS, I would expect the install to work successfully (with psycopg2-binary
). Could you try:
python3 -m pip install --upgrade --force-reinstall dbt-core dbt-postgres dbt-redshift
Oh, nice. It's inside a docker container which I guess is why it doesn't work on my Mac either.
I wasn't sure if that is relevant.
Ah yes, that would explain it! The easiest way to fix this is by adding the psycopg2
build prerequisites to your docker image:
I'm going to close this issue as a duplicate in the meantime
Adding psycopg2 didn't work in my case, but reverting to dbt-redshift 1.7.7 does, which is fine for now.
Thanks for your help jtcohen6!