Issue: Runtime error database disk image is malformed
MichaelGMoore opened this issue · 4 comments
We have run into this issue twice now.
We are running ~/Pose-app/app.py to run lightning pose.
It seems like something gets messed up in sqlite. The problem is we haven't been able to clear the error. We only solved this the first time with a complete re-install.
If you understand how to reset this database to somehow get lightning-pose going again, that would be helpful.
We tried removing and re-installing ~/venv-label-studio, but that did not fix the problem.
Here is the graphical output from app.py:
Runtime error
database disk image is malformed
Traceback (most recent call last):
File "/home/plafave/venv-label-studio/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
File "/home/plafave/venv-label-studio/lib/python3.8/site-packages/django/db/backends/sqlite3/base.py", line 423, in execute
return Database.Cursor.execute(self, query, params)
sqlite3.DatabaseError: database disk image is malformed
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/plafave/venv-label-studio/lib/python3.8/site-packages/rest_framework/views.py", line 506, in dispatch
response = handler(request, *args, **kwargs)
File "/home/plafave/venv-label-studio/lib/python3.8/site-packages/django/utils/decorators.py", line 43, in _wrapper
return bound_method(*args, **kwargs)
File "/home/plafave/venv-label-studio/lib/python3.8/site-packages/label_studio/projects/api.py", line 165, in get
return super(ProjectListAPI, self).get(request, *args, **kwargs)
File "/home/plafave/venv-label-studio/lib/python3.8/site-packages/rest_framework/generics.py", line 239, in get
return self.list(request, *args, **kwargs)
File "/home/plafave/venv-label-studio/lib/python3.8/site-packages/rest_framework/mixins.py", line 40, in list
page = self.paginate_queryset(queryset)
File "/home/plafave/venv-label-studio/lib/python3.8/site-packages/rest_framework/generics.py", line 171, in paginate_queryset
return self.paginator.paginate_queryset(queryset, self.request, view=self)
File "/home/plafave/venv-label-studio/lib/python3.8/site-packages/rest_framework/pagination.py", line 204, in paginate_queryset
self.page = paginator.page(page_number)
File "/home/plafave/venv-label-studio/lib/python3.8/site-packages/django/core/paginator.py", line 76, in page
number = self.validate_number(number)
File "/home/plafave/venv-label-studio/lib/python3.8/site-packages/django/core/paginator.py", line 54, in validate_number
if number > self.num_pages:
File "/home/plafave/venv-label-studio/lib/python3.8/site-packages/django/utils/functional.py", line 48, in get
res = instance.dict[self.name] = self.func(instance)
File "/home/plafave/venv-label-studio/lib/python3.8/site-packages/django/core/paginator.py", line 103, in num_pages
if self.count == 0 and not self.allow_empty_first_page:
File "/home/plafave/venv-label-studio/lib/python3.8/site-packages/django/utils/functional.py", line 48, in get
res = instance.dict[self.name] = self.func(instance)
File "/home/plafave/venv-label-studio/lib/python3.8/site-packages/django/core/paginator.py", line 97, in count
return c()
File "/home/plafave/venv-label-studio/lib/python3.8/site-packages/django/db/models/query.py", line 412, in count
return self.query.get_count(using=self.db)
File "/home/plafave/venv-label-studio/lib/python3.8/site-packages/django/db/models/sql/query.py", line 528, in get_count
number = obj.get_aggregation(using, ['__count'])['__count']
File "/home/plafave/venv-label-studio/lib/python3.8/site-packages/django/db/models/sql/query.py", line 513, in get_aggregation
result = compiler.execute_sql(SINGLE)
File "/home/plafave/venv-label-studio/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1175, in execute_sql
cursor.execute(sql, params)
File "/home/plafave/venv-label-studio/lib/python3.8/site-packages/sentry_sdk/integrations/django/init.py", line 596, in execute
return real_execute(self, sql, params)
File "/home/plafave/venv-label-studio/lib/python3.8/site-packages/django/db/backends/utils.py", line 66, in execute
return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
File "/home/plafave/venv-label-studio/lib/python3.8/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/home/plafave/venv-label-studio/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
File "/home/plafave/venv-label-studio/lib/python3.8/site-packages/django/db/utils.py", line 90, in exit
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/home/plafave/venv-label-studio/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
File "/home/plafave/venv-label-studio/lib/python3.8/site-packages/django/db/backends/sqlite3/base.py", line 423, in execute
return Database.Cursor.execute(self, query, params)
django.db.utils.DatabaseError: database disk image is malformed
following up on your comment from danbider/lightning-pose#101:
- changing list of body parts: I am not surprised this led to some issues, I have not explicitly tried this in the past. were some of the frames already labeled when the list was updated? I can try to make it so that this is possible, or so that it is at least very clear this is not possible
- leaving overnight: I have not tried this before, but I am working on some pointers/FAQ for the app and I'll explicitly mention in there that the app should not be left running for long periods of time. afterwards I can actually try to dig into this error a bit more and see if there is anything we can do on our side to make the app more stable.
Yes. I understand that these were unusual circumstances. We tasked an undergraduate student to process some videos and these are the issues that arose. My main concern is just how to recover from the database error without complete re-install, if possible.
We tried removing the two data-bases
~/Pose-app/data/labelstudio_db
~/Pose-app/.shared/data/labelstudio_db
and this allowed us to start a new project and proceed as far as the 'label frames' tab without seeing the error. Looks like it worked. We might want to experiment with removing only some files in those directories if something becomes corrupted in the future.
glad to hear! closing this issue for now, but if you run into the issue again please reopen and let me know, we can work on recovering already-labeled frames.