Maximum recursion depth exceeded while trying to store nested dictionaries
Opened this issue · 1 comments
kuriringohankamehameha commented
It seems that sqlitedict fails to pickle relatively nested dictionaries successfully. Sometimes, I get this error when trying to store a dictionary (around 5 MB) with a nesting level of around 3-4 in Python3.6.
File "/home/user/.local/lib/python3.6/site-packages/sqlitedict.py", line 253, in __setitem__
self.conn.execute(ADD_ITEM, (key, self.encode(value)))
File "/home/user/.local/lib/python3.6/site-packages/sqlitedict.py", line 100, in encode
return sqlite3.Binary(dumps(obj, protocol=PICKLE_PROTOCOL))
RecursionError: maximum recursion depth exceeded while calling a Python object
Is it just me, or can anyone else verify that this happens during pickling? Any workarounds for this issue until then?
Thanks
piskvorky commented
There might be some recursion limitation in pickle. It's suspicious that would happen at a nesting level of 3-4 levels though; I'd expect any such limits to be orders of magnitude deeper.
If you post a minimal reproducible example, and a full stack trace, we'll have a look.