grantjenks/python-diskcache

"sqlite3.DatabaseError: database disk image is malformed" caused by Network File System mounts?

Opened this issue · 0 comments

Hello,

I have encountered an sqlite3.DatabaseError: database disk image is malformed error during the execution of my scripts. After reading the documentation, I found that diskcache performs poorly with Network File System (NFS) mounts , which are used extensively in my setup. Can this be the origin of my issue?

In the case it is true, I believe this information is important enough to appear directly in the README of this Github in a Limitations section for example. If you agree with this, can it be done please?

Here is my environment:

  • Python 3.12
  • Diskcache version: 5.6.3
  • Multiple processes access the same FanoutCache with concurrent read access. Write access can happen but is rare.
  • These processes can be on other machines than the one hosting the cache, and by other machine, I mean the cache is accessed through the network.
  • I cannot consistently reproduce this error

And here is the traceback:

  ...
  File "/opt/venv/lib/python3.12/site-packages/diskcache/core.py", line 1872, in wrapper
    result = self.get(key, default=ENOVAL, retry=True)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/diskcache/fanout.py", line 285, in get
    return shard.get(key, default, read, expire_time, tag, retry)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/diskcache/core.py", line 1207, in get
    sql(cache_hit)
sqlite3.DatabaseError: database disk image is malformed

Best regards.