Database is locked (again)
Opened this issue · 1 comments
An instance of rover retrieve request.txt
, attached to this issue, run on Python 3.6 has lead to a critical error sqlite3.OperationalError: data base is locked
. The rover retrieve.log shows two stack traces that are leading to the data base is locked
error.
CRITICAL 2019-06-25 18:00:08,691: Traceback (most recent call last):
File "/.conda/envs/rover/lib/python3.6/site-packages/rover/__init__.py", line 116, in main
with ProcessManager(config):
File "/.conda/envs/rover/lib/python3.6/site-packages/rover/process.py", line 48, in __enter__
self._check_command(self._command)
File "/.conda/envs/rover/lib/python3.6/site-packages/rover/process.py", line 87, in _check_command
pid, command = self._current_command_inside_transaction()
File "/.conda/envs/rover/lib/python3.6/site-packages/rover/process.py", line 69, in _current_command_inside_transaction
self._db.execute('DELETE FROM rover_processes WHERE pid = ?', (pid,))
sqlite3.OperationalError: database is locked
CRITICAL 2019-06-25 19:47:33,962: Traceback (most recent call last):
File "/.conda/envs/rover/lib/python3.6/site-packages/rover/__init__.py", line 117, in main
execute(config.command, config)
File "/.conda/envs/rover/lib/python3.6/site-packages/rover/__init__.py", line 96, in execute
commands[command][0](config).run(config.args)
File "/.conda/envs/rover/lib/python3.6/site-packages/rover/retrieve.py", line 213, in run
return self.do_run(args, True, RETRIEVE)
File "/.conda/envs/rover/lib/python3.6/site-packages/rover/retrieve.py", line 152, in do_run
return self._fetch()
File "/.conda/envs/rover/lib/python3.6/site-packages/rover/retrieve.py", line 180, in _fetch
n_downloads = self._download_manager.download()
File "/.conda/envs/rover/lib/python3.6/site-packages/rover/manager.py", line 882, in download
self.step(quiet=False)
File "/.conda/envs/rover/lib/python3.6/site-packages/rover/manager.py", line 854, in step
self._update_stats()
File "/.conda/envs/rover/lib/python3.6/site-packages/rover/manager.py", line 906, in _update_stats
self._db.execute('DELETE FROM rover_download_stats', tuple())
sqlite3.OperationalError: database is locked
Please note that both database is locked operations originate from a DELETE FROM
sql command.
request.txt
It seems like this database locking is associated with multiple open transactions that are all trying to write or read from the database at the same time. The busy timeout has been increased to 120 seconds. The busy timeout allows incoming SQL transactions to sleep while another transaction is interfacing with the database, the maximum amount of time that the transaction can sleep is set in seconds by the integer value above Busy Timeout.
A connection.commit() command also been added after the DELETE FROM
statements. Committing the Delete FROM
statements to the database should allow the database to unlock and be ready for another transaction commit .
The request.txt, above, has been run through this modified code with no locking database exceptions.