ara_api role should setup a cron for "ara expire" and "ara playbook prune"
Closed this issue · 7 comments
See: https://ara.readthedocs.io/en/latest/cli.html#ara-playbook-prune
The role should be able to set up a cron that runs the ara playbook prune command in order to keep the database tidy.
A new similar command was added to expire old running playbooks, ara expire: https://review.opendev.org/#/c/752478/
So I'm taking a stab at this, I have configured the following cronjobs on my system, which has been installed with the ara_api role. But I get some weird errors, also when running them outside of cron:
@daily /opt/ara/api/virtualenv/bin/ara expire --confirm --hours 24
@daily /opt/ara/api/virtualenv/bin/ara playbook prune --confirm --days 30
However, even running the command from my commandline (with source activate for the venv or without), I get the following errors:
root@ara:/opt/ara/api/server# /opt/ara/api/virtualenv/bin/ara playbook prune --confirm --days 30 --server http://localhost:8000
2021-01-05 21:38:38,032 ERROR django.request: Internal Server Error: /api/v1/playbooks
Traceback (most recent call last):
File "/opt/ara/api/virtualenv/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
File "/opt/ara/api/virtualenv/lib/python3.7/site-packages/django/db/backends/sqlite3/base.py", line 383, in execute
return Database.Cursor.execute(self, query, params)
sqlite3.OperationalError: no such table: playbooks
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/opt/ara/api/virtualenv/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
response = get_response(request)
File "/opt/ara/api/virtualenv/lib/python3.7/site-packages/django/core/handlers/base.py", line 115, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/opt/ara/api/virtualenv/lib/python3.7/site-packages/django/core/handlers/base.py", line 113, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/opt/ara/api/virtualenv/lib/python3.7/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
return view_func(*args, **kwargs)
File "/opt/ara/api/virtualenv/lib/python3.7/site-packages/rest_framework/viewsets.py", line 125, in view
return self.dispatch(request, *args, **kwargs)
File "/opt/ara/api/virtualenv/lib/python3.7/site-packages/rest_framework/views.py", line 509, in dispatch
response = self.handle_exception(exc)
File "/opt/ara/api/virtualenv/lib/python3.7/site-packages/rest_framework/views.py", line 469, in handle_exception
self.raise_uncaught_exception(exc)
File "/opt/ara/api/virtualenv/lib/python3.7/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
raise exc
File "/opt/ara/api/virtualenv/lib/python3.7/site-packages/rest_framework/views.py", line 506, in dispatch
response = handler(request, *args, **kwargs)
File "/opt/ara/api/virtualenv/lib/python3.7/site-packages/rest_framework/mixins.py", line 40, in list
page = self.paginate_queryset(queryset)
File "/opt/ara/api/virtualenv/lib/python3.7/site-packages/rest_framework/generics.py", line 171, in paginate_queryset
return self.paginator.paginate_queryset(queryset, self.request, view=self)
File "/opt/ara/api/virtualenv/lib/python3.7/site-packages/rest_framework/pagination.py", line 379, in paginate_queryset
self.count = self.get_count(queryset)
File "/opt/ara/api/virtualenv/lib/python3.7/site-packages/rest_framework/pagination.py", line 522, in get_count
return queryset.count()
File "/opt/ara/api/virtualenv/lib/python3.7/site-packages/django/db/models/query.py", line 392, in count
return self.query.get_count(using=self.db)
File "/opt/ara/api/virtualenv/lib/python3.7/site-packages/django/db/models/sql/query.py", line 504, in get_count
number = obj.get_aggregation(using, ['__count'])['__count']
File "/opt/ara/api/virtualenv/lib/python3.7/site-packages/django/db/models/sql/query.py", line 489, in get_aggregation
result = compiler.execute_sql(SINGLE)
File "/opt/ara/api/virtualenv/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 1142, in execute_sql
cursor.execute(sql, params)
File "/opt/ara/api/virtualenv/lib/python3.7/site-packages/django/db/backends/utils.py", line 67, in execute
return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
File "/opt/ara/api/virtualenv/lib/python3.7/site-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/opt/ara/api/virtualenv/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
File "/opt/ara/api/virtualenv/lib/python3.7/site-packages/django/db/utils.py", line 89, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/opt/ara/api/virtualenv/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
File "/opt/ara/api/virtualenv/lib/python3.7/site-packages/django/db/backends/sqlite3/base.py", line 383, in execute
return Database.Cursor.execute(self, query, params)
django.db.utils.OperationalError: no such table: playbooks
2021-01-05 21:38:38,045 ERROR ara.clients.http: Failed to get on /api/v1/playbooks: {'params': {'started_before': '2020-12-06T21:38:37.980613', 'order': 'started', 'limit': 200}}
2021-01-05 21:38:38,045 ERROR ara.clients.http: Failed to get on /api/v1/playbooks: {'params': {'started_before': '2020-12-06T21:38:37.980613', 'order': 'started', 'limit': 200}}
2021-01-05 21:38:38,045 ERROR ara: Expecting value: line 1 column 1 (char 0)
Probing the endpoint api/v1/playbooks via my browser or curl works just fine, I get the results back.
I think it's some configuration thing, but I can't find it :-)
Oh and using ara-manage also doesn't work
Hey and thanks for looking into this !
root@ara:/opt/ara/api/server# /opt/ara/api/virtualenv/bin/ara playbook prune --confirm --days 30 --server http://localhost:8000
Strangely, I see the logs mention ara.clients.http but there's no --client http in that command so I wonder if there is a bug there. The default client is offline -- are you able to reproduce the issue with --client http ?
Oh and using ara-manage also doesn't work
The ara-manage implementation is deprecated but it should still work -- same thing, requires the --client http argument.
Bingo! That fixed it :-)
I'll update my code and make a PR tomorrow-ish ;-)
root@ara:~# /opt/ara/api/virtualenv/bin/ara playbook prune --confirm --days 30 --server http://localhost:8000 --client http
Found 196 playbooks matching query
Deleting playbook 403 (/opt/ansible/projects/element-networks/playbooks/main.yml), start date: 2020-09-01T00:02:43.576702Z
etc...
Fixed by: https://review.opendev.org/c/recordsansible/ara-collection/+/773407
(not sure if Github picks up on this, we'll see)
@dmsimard You can close this one now ;-)
@Thulium-Drake thanks!