Problem with SQLAlchemy 1.4.0
kromash opened this issue · 17 comments
Hello,
New version of SQLAlchemy (1.4.0) was released 15.03.2020 and since then I got following error:
File "/layers/google.python.pip/pip/lib/python3.8/site-packages/promise/promise.py", line 489, in _resolve_from_executor
executor(resolve, reject)
File "/layers/google.python.pip/pip/lib/python3.8/site-packages/promise/promise.py", line 756, in executor
return resolve(f(*args, **kwargs))
File "/layers/google.python.pip/pip/lib/python3.8/site-packages/graphql/execution/middleware.py", line 75, in make_it_promise
return next(*args, **kwargs)
File "/layers/google.python.pip/pip/lib/python3.8/site-packages/graphene_sqlalchemy/fields.py", line 78, in connection_resolver
return on_resolve(resolved)
File "src/adtrender_api/connections.py", line 52, in resolve_connection
return super().resolve_connection(connection_type, model, info, args, resolved)
File "/layers/google.python.pip/pip/lib/python3.8/site-packages/graphene_sqlalchemy/fields.py", line 53, in resolve_connection
_len = resolved.count()
File "/layers/google.python.pip/pip/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3009, in count
return self._from_self(col).scalar()
File "/layers/google.python.pip/pip/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 2753, in scalar
ret = self.one()
File "/layers/google.python.pip/pip/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 2730, in one
return self._iter().one()
File "/layers/google.python.pip/pip/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 2768, in _iter
result = self.session.execute(
File "/layers/google.python.pip/pip/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1653, in execute
result = conn._execute_20(statement, params or {}, execution_options)
File "/layers/google.python.pip/pip/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1520, in _execute_20
return meth(self, args_10style, kwargs_10style, execution_options)
File "/layers/google.python.pip/pip/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 313, in _execute_on_connection
return connection._execute_clauseelement(
File "/layers/google.python.pip/pip/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1381, in _execute_clauseelement
compiled_sql, extracted_params, cache_hit = elem._compile_w_cache(
File "/layers/google.python.pip/pip/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 533, in _compile_w_cache
compiled_sql = self._compiler(
File "/layers/google.python.pip/pip/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 566, in _compiler
return dialect.statement_compiler(dialect, self, **kw)
File "/layers/google.python.pip/pip/lib/python3.8/site-packages/pybigquery/sqlalchemy_bigquery.py", line 137, in __init__
super(BigQueryCompiler, self).__init__(dialect, statement, column_keys, inline, **kwargs)
TypeError: __init__() got multiple values for argument 'cache_key'"
I rolled back to SQLAlchemy==1.3.20 and it works fine.
Facing the same problem.
Can you please share a code snippet which generates this error?
And also as much details as possible regarding your environment (python version, package versions, etc...)
Here is snippet. Just use table that you have access to
from sqlalchemy import create_engine, Column, Integer, String
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker
Base = declarative_base()
class ExampleTable(Base):
__tablename__ = 'dataset.example_table'
id = Column(Integer, primary_key=True)
name = Column(String)
def get_data():
engine = create_engine(f'bigquery://project/dataset')
session_maker = sessionmaker(bind=engine)
session = session_maker()
return session.query(ExampleTable).count()
if __name__ == '__main__':
print(get_data())
here is requirements.txt for this example
pybigquery==0.5.0
pip freeze:
cachetools==4.2.1
certifi==2020.12.5
cffi==1.14.5
chardet==4.0.0
future==0.18.2
google-api-core==1.26.1
google-auth==1.28.0
google-cloud-bigquery==2.12.0
google-cloud-core==1.6.0
google-crc32c==1.1.2
google-resumable-media==1.2.0
googleapis-common-protos==1.53.0
greenlet==1.0.0
grpcio==1.36.1
idna==2.10
packaging==20.9
proto-plus==1.18.0
protobuf==3.15.6
pyasn1==0.4.8
pyasn1-modules==0.2.8
pybigquery==0.5.0
pycparser==2.20
pyparsing==2.4.7
pytz==2021.1
requests==2.25.1
rsa==4.7.2
six==1.15.0
SQLAlchemy==1.4.0
urllib3==1.26.4
python --version:
Python 3.8.5
when you change requirements.txt to:
pybigquery==0.5.0
SQLAlchemy==1.3.20
example works just fine
it seems like SQLAlchemy 1.4.0, also breaks a lot of other features (23 tests fail)
still not sure how big are the changes to fix those
I updated the maximum sqlalchemy version in 37e584e#diff-60f61ab7a8d1910d86d9fda2261620314edcae5894d5aaa236b821c7256badd7R67 to limit to 1.3.x and below.
As such, I'm converting this issue to a feature request (to support 1.4.x).
From https://docs.sqlalchemy.org/en/14/changelog/migration_14.html is sounds like there have been conceptual changes, so this may require significant work to get right.
Any update on the timeline for support for sqlalchemy 1.4?
I'm working on it now. :)
Hopefully, this week.
@jimfulton Started a PR, couldn't find the time to finish it until now,
Did you see it?
Willing to help finish if needed
@jimfulton Started a PR, couldn't find the time to finish it until now,
Did you see it?
I didn't until you pointed it out.
Willing to help finish if needed
I'm nearly finished with my work, which builds on:
- Adding use of the sqlalchemy dialect-compliance test suite and
- 100% unit test coverage.
My work is here: https://github.com/googleapis/python-bigquery-sqlalchemy/compare/riversnake-sqla-14
ATM, I'm trying to figure out why some code paths I added while debugging the compliance tests initially are no-longer being hit. :/
Lots of code there!
i did some work to rebase my branch, but i think that you implemented all the fixes i did already.
so i have a branch with those applied and where all current tests work.
does your branch also work on older versions of SQLA?
Lots of code there!
There was lots of code in the tests work. With that in place, the 1.4 port isn't too bad. :)
i did some work to rebase my branch, but i think that you implemented all the fixes i did already.
Cool.
does your branch also work on older versions of SQLA?
It should work with the latest 1.2, 1.3, and 1.4 versions.
I dropped support for 1.1 (on master) when I added comment support.
The PR: #177
More eyes would be welcome. Feel free to ask questions or to request more comments.
Release 0.8.0 supports SQLAlchemy 1.4.