piskvorky/sqlitedict

Maximum recursion depth exceeded while trying to store nested dictionaries

Opened this issue · 1 comments

It seems that sqlitedict fails to pickle relatively nested dictionaries successfully. Sometimes, I get this error when trying to store a dictionary (around 5 MB) with a nesting level of around 3-4 in Python3.6.

File "/home/user/.local/lib/python3.6/site-packages/sqlitedict.py", line 253, in __setitem__
    self.conn.execute(ADD_ITEM, (key, self.encode(value)))
  File "/home/user/.local/lib/python3.6/site-packages/sqlitedict.py", line 100, in encode
    return sqlite3.Binary(dumps(obj, protocol=PICKLE_PROTOCOL))
RecursionError: maximum recursion depth exceeded while calling a Python object

Is it just me, or can anyone else verify that this happens during pickling? Any workarounds for this issue until then?

Thanks

There might be some recursion limitation in pickle. It's suspicious that would happen at a nesting level of 3-4 levels though; I'd expect any such limits to be orders of magnitude deeper.

If you post a minimal reproducible example, and a full stack trace, we'll have a look.