PyAthena 3.0.3 + AthenaPandasDialect/AthenaRestDialect has no attribute dbapi
mdeshmu opened this issue · 5 comments
With SQLAlchemy 1.4.x, getting below errors when using Athena Rest or Pandas dialect with PyAthena 3.0.3
type object 'AthenaPandasDialect' has no attribute 'dbapi'
Below is the stack trace.
2023-05-29 09:48:52,990:ERROR:flask_appbuilder.api: type object 'AthenaPandasDialect' has no attribute 'dbapi'
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/flask_appbuilder/api/init.py", line 110, in wraps
return f(self, *args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/flask_appbuilder/api/init.py", line 182, in wraps
return f(self, *args, **kwargs)
File "/app/superset/views/base_api.py", line 122, in wraps
raise ex
File "/app/superset/views/base_api.py", line 113, in wraps
duration, response = time_function(f, self, *args, **kwargs)
File "/app/superset/utils/core.py", line 1586, in time_function
response = func(*args, **kwargs)
File "/app/superset/utils/log.py", line 266, in wrapper
value = f(*args, **kwargs)
File "/app/superset/databases/api.py", line 551, in schemas
schemas = database.get_all_schema_names(
File "/app/superset/utils/cache.py", line 131, in wrapped_f
return f(*args, **kwargs)
File "/app/superset/models/core.py", line 720, in get_all_schema_names
raise self.db_engine_spec.get_dbapi_mapped_exception(ex) from ex
File "/app/superset/models/core.py", line 717, in get_all_schema_names
with self.get_inspector_with_context(ssh_tunnel=ssh_tunnel) as inspector:
File "/usr/local/lib/python3.8/contextlib.py", line 113, in enter
return next(self.gen)
File "/app/superset/models/core.py", line 690, in get_inspector_with_context
with self.get_sqla_engine_with_context(
File "/usr/local/lib/python3.8/contextlib.py", line 113, in enter
return next(self.gen)
File "/app/superset/models/core.py", line 409, in get_sqla_engine_with_context
yield self._get_sqla_engine(
File "/app/superset/models/core.py", line 471, in _get_sqla_engine
raise self.db_engine_spec.get_dbapi_mapped_exception(ex)
File "/app/superset/models/core.py", line 469, in _get_sqla_engine
return create_engine(sqlalchemy_url, **params)
File "", line 2, in create_engine
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/deprecations.py", line 309, in warned
return fn(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/create.py", line 557, in create_engine
for k in util.get_func_kwargs(dialect_cls.dbapi):
AttributeError: type object 'AthenaPandasDialect' has no attribute 'dbapi'
2023-05-30 01:00:09,934:ERROR:flask_appbuilder.api:type object 'AthenaRestDialect' has no attribute 'dbapi'
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/flask_appbuilder/api/init.py", line 110, in wraps
return f(self, *args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/flask_appbuilder/api/init.py", line 182, in wraps
return f(self, *args, **kwargs)
File "/app/superset/views/base_api.py", line 122, in wraps
raise ex
File "/app/superset/views/base_api.py", line 113, in wraps
duration, response = time_function(f, self, *args, **kwargs)
File "/app/superset/utils/core.py", line 1586, in time_function
response = func(*args, **kwargs)
File "/app/superset/utils/log.py", line 266, in wrapper
value = f(*args, **kwargs)
File "/app/superset/databases/api.py", line 551, in schemas
schemas = database.get_all_schema_names(
File "/app/superset/utils/cache.py", line 131, in wrapped_f
return f(*args, **kwargs)
File "/app/superset/models/core.py", line 720, in get_all_schema_names
raise self.db_engine_spec.get_dbapi_mapped_exception(ex) from ex
File "/app/superset/models/core.py", line 717, in get_all_schema_names
with self.get_inspector_with_context(ssh_tunnel=ssh_tunnel) as inspector:
File "/usr/local/lib/python3.8/contextlib.py", line 113, in enter
return next(self.gen)
File "/app/superset/models/core.py", line 690, in get_inspector_with_context
with self.get_sqla_engine_with_context(
File "/usr/local/lib/python3.8/contextlib.py", line 113, in enter
return next(self.gen)
File "/app/superset/models/core.py", line 409, in get_sqla_engine_with_context
yield self._get_sqla_engine(
File "/app/superset/models/core.py", line 471, in _get_sqla_engine
raise self.db_engine_spec.get_dbapi_mapped_exception(ex)
File "/app/superset/models/core.py", line 469, in _get_sqla_engine
return create_engine(sqlalchemy_url, **params)
File "", line 2, in create_engine
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/deprecations.py", line 309, in warned
return fn(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/create.py", line 557, in create_engine
for k in util.get_func_kwargs(dialect_cls.dbapi):
AttributeError: type object 'AthenaRestDialect' has no attribute 'dbapi'
https://github.com/sqlalchemy/sqlalchemy/blob/rel_2_0_15/lib/sqlalchemy/engine/default.py#L322
In SQLAlchemy 2.0, the dbapi field is already defined in the parent class of Dialect, DefaultDialect; I think Superset should support SQLAlchemy 2.0.
It will eventually support but it may take months since a lot of its dependencies are yet to support SQLAlchemy 2.0.
It would be possible to port changes made in 3.0 to 2.x. The only major change would be support for ICEBERG tables, which may not be a much needed feature in Superset (I don't use Superset, so I don't know if it is needed or not).
1f6bca1
if version 3 won't support sqlalchemy 1.4 this should be put in the dependencies. I have a use case where I use sqlalchemy to move query data from Google Cloud and then insert it into a ICEBERG table in AWS Athena. sqlalchemy-bigquery does not support sqlalchemy 2.X so far so I have to use 1.4.
@dominik-reiner
This issue seems to be an error with Superset; if you are not using Superset and the error still occurs, please register another issue. 🙏