Airflow 2.4.1 ab_permission_view_role database error
ybantes-esure opened this issue · 3 comments
Apache Airflow version
2.4.1
What happened
I am trying to update airflow to version 2.4.1. The current version is 2.2.1 . I tried running airflow db upgrade, I tried to drop the database and run db init, I also tried to run it without running airflow db init in a fresh database. I am always getting the same error:
Traceback (most recent call last): File "/usr/local/bin/airflow", line 8, in <module> sys.exit(main()) File "/usr/local/lib/python3.10/site-packages/airflow/__main__.py", line 39, in main args.func(args) File "/usr/local/lib/python3.10/site-packages/airflow/cli/cli_parser.py", line 52, in command return func(*args, **kwargs) File "/usr/local/lib/python3.10/site-packages/airflow/utils/cli.py", line 99, in wrapper return f(*args, **kwargs) File "/usr/local/lib/python3.10/site-packages/airflow/cli/commands/sync_perm_command.py", line 28, in sync_perm appbuilder = cached_app().appbuilder File "/usr/local/lib/python3.10/site-packages/airflow/www/app.py", line 161, in cached_app app = create_app(config=config, testing=testing) File "/usr/local/lib/python3.10/site-packages/airflow/www/app.py", line 124, in create_app init_api_experimental_auth(flask_app) File "/usr/local/lib/python3.10/site-packages/airflow/www/extensions/init_security.py", line 57, in init_api_experimental_auth auth = import_module(backend.strip()) File "/usr/local/lib/python3.10/importlib/__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1050, in _gcd_import File "<frozen importlib._bootstrap>", line 1027, in _find_and_load File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 688, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 883, in exec_module File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed File "/etc/airflow/config/token_auth.py", line 8, in <module> from flask_appbuilder.security.sqla.models import User File "/usr/local/lib/python3.10/site-packages/flask_appbuilder/security/sqla/models.py", line 48, in <module> assoc_permissionview_role = Table( File "<string>", line 2, in __new__ File "/usr/local/lib/python3.10/site-packages/sqlalchemy/util/deprecations.py", line 309, in warned return fn(*args, **kwargs) File "/usr/local/lib/python3.10/site-packages/sqlalchemy/sql/schema.py", line 594, in __new__ raise exc.InvalidRequestError( sqlalchemy.exc.InvalidRequestError: Table 'ab_permission_view_role' is already defined for this MetaData instance. Specify 'extend_existing=True' to redefine options and columns on an existing Table object.
Any ideas?
What you think should happen instead
It should run
How to reproduce
No response
Operating System
python:3.10-slim docker image
Versions of Apache Airflow Providers
apache-airflow==2.4.1
apache-airflow-providers-amazon==6.0.0
apache-airflow-providers-apache-spark==3.0.0
apache-airflow-providers-celery==3.0.0
apache-airflow-providers-cncf-kubernetes==4.4.0
apache-airflow-providers-common-sql==1.2.0
apache-airflow-providers-ftp==3.1.0
apache-airflow-providers-http==4.0.0
apache-airflow-providers-imap==3.0.0
apache-airflow-providers-mysql==3.2.1
apache-airflow-providers-redis==3.0.0
apache-airflow-providers-sqlite==3.2.1
Deployment
Other Docker-based deployment
Deployment details
No response
Anything else
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct
Thanks for opening your first issue here! Be sure to follow the issue template!
In future it will be helpful to have properly formatted traceback posted. As per traceback you have custom auth_backend class at /etc/airflow/config/token_auth.py where you are importing User from flask_appbuilder.security.sqla.models that causes Role from flask_appbuilder to get registered meanwhile there is already Role defined at airflow/www/fab_security/sqla/models.py . Please import from airflow related model files to avoid this issue. This can be closed as there is no issue with Airflow.
Traceback (most recent call last):
File "/usr/local/bin/airflow", line 8, in <module>
sys.exit(main())
File "/usr/local/lib/python3.10/site-packages/airflow/__main__.py", line 39, in main
args.func(args)
File "/usr/local/lib/python3.10/site-packages/airflow/cli/cli_parser.py", line 52, in command
return func(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/airflow/utils/cli.py", line 99,
in wrapper return f(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/airflow/cli/commands/sync_perm_command.py", line 28,
in sync_perm appbuilder = cached_app().appbuilder
File "/usr/local/lib/python3.10/site-packages/airflow/www/app.py", line 161, in cached_app
app = create_app(config=config, testing=testing)
File "/usr/local/lib/python3.10/site-packages/airflow/www/app.py", line 124, in create_app
init_api_experimental_auth(flask_app)
File "/usr/local/lib/python3.10/site-packages/airflow/www/extensions/init_security.py", line 57, in init_api_experimental_auth
auth = import_module(backend.strip())
File "/usr/local/lib/python3.10/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/etc/airflow/config/token_auth.py", line 8, in <module>
from flask_appbuilder.security.sqla.models import User # <---- Causes error
File "/usr/local/lib/python3.10/site-packages/flask_appbuilder/security/sqla/models.py", line 48, in <module>
assoc_permissionview_role = Table( File "<string>", line 2, in __new__
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/util/deprecations.py", line 309, in warned
return fn(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/sql/schema.py", line 594, in __new__
raise exc.InvalidRequestError( sqlalchemy.exc.InvalidRequestError: Table 'ab_permission_view_role' is already defined for this MetaData instance. Specify 'extend_existing=True' to redefine options and columns on an existing Table object.Thanks @tirkarthi. Closing.