Unsupported Operation: readline + Debug Feature

[SoundsSerious]

I have a bug that I cannot resolve what is causing is it.

Somewhere deep in my object I see that I have a file reference that I cannot resolve. Ignore encoding errors doesn't work as well.

deepdiff.DeepHash(vs,ignore_encoding_errors=True)

Results in a trace:

File ~/miniconda3/envs/nept/lib/python3.10/site-packages/deepdiff/deephash.py:514, in DeepHash._hash(self, obj, parent, parents_ids)
    511     result, counts = self._prep_tuple(obj=obj, parent=parent, parents_ids=parents_ids)
    513 elif isinstance(obj, Iterable):
--> 514     result, counts = self._prep_iterable(obj=obj, parent=parent, parents_ids=parents_ids)
    516 elif obj == BoolObj.TRUE or obj == BoolObj.FALSE:
    517     result = 'bool:true' if obj is BoolObj.TRUE else 'bool:false'

File ~/miniconda3/envs/nept/lib/python3.10/site-packages/deepdiff/deephash.py:405, in DeepHash._prep_iterable(self, obj, parent, parents_ids)
    402 counts = 1
    403 result = defaultdict(int)
--> 405 for i, item in enumerate(obj):
    406     new_parent = "{}[{}]".format(parent, i)
    407     if self._skip_this(item, parent=new_parent):

UnsupportedOperation: readline

In this case I would hope that I could add a debug flag so as to get to the bottom of the object, so i could apply rules to filter it using the existing functionality.

• OS: Ubuntu
• Version 20 LTS
• Python Version 3.10
• DeepDiff Version 6.3.1

Thanks!

[seperman]

Hi @SoundsSerious
Cool username. Sorry it took me a while to see your report.
Without having reproducible code, it is difficult to pin point the problem. It may be a file object you have that does not have a readline method.

[seperman]

Hi @SoundsSerious
Please post reproducible code. Otherwise, I will close this ticket due to inactivity. I can't help you if I can't reproduce it!

[SoundsSerious]

@seperman Hi sorry for the long delay

So it took me a while to find out where in my framework object this item was hiding, but it turns out to be a class logging value.

A python logging object has a logging.StreamHandler and logging.Manager that acts as a file references. They both are not able to be deep hashed.

ssb.keel_mass_material._log.__dict__
Out[30]: 
{'filters': [Rock(name='wet soil', elastic_modulus=67000000000.0, poissons_ratio=0.26, yield_strength=13000000.0, density=2600.0, color=(0.575563, 0.844566, 0.256415, 1.0))],
 'name': 'engforgelog_rock-wet soil_4141a959-710f-49c8-9a9a-22e685c3c3ad',
 'level': 20,
 'parent': <RootLogger root (WARNING)>,
 'propagate': True,
 'handlers': [<StreamHandler stdout (NOTSET)>],
 'disabled': False,
 '_cache': {},
 'manager': <logging.Manager at 0x7fdb8fa531c0>}

Inside the stream handler I found the watch_fd_thread item to be the culpret but other objects in this item also had issues.

{'session': <jupyter_client.session.Session at 0x7fdb8d3ba770>,
 'pub_thread': <ipykernel.iostream.IOPubThread at 0x7fdb8d4290c0>,
 'name': 'stdout',
 'topic': b'stream.stdout',
 'parent_header': {'msg_id': '1aa0e177-f5acbf1a51db28525e85bc3b_20941_133',
  'msg_type': 'execute_request',
  'username': 'olly',
  'session': '1aa0e177-f5acbf1a51db28525e85bc3b',
  'date': datetime.datetime(2024, 8, 18, 18, 55, 13, 755747, tzinfo=tzutc()),
  'version': '5.3'},
 '_master_pid': 21325,
 '_flush_pending': False,
 '_subprocess_flush_pending': False,
 '_io_loop': <tornado.platform.asyncio.AsyncIOLoop at 0x7fdb8d429150>,
 '_buffer_lock': <unlocked _thread.RLock object owner=0 count=0 at 0x7fdb8d3fc840>,
 '_buffer': <_io.StringIO at 0x7fd8e850e320>,
 'echo': None,
 '_isatty': False,
 '_should_watch': True,
 '_local': <_thread._local at 0x7fdb8d3faf20>,
 '_original_stdstream_fd': 1,
 '_original_stdstream_copy': 39,
 '_fid': 37,
 '_exc': None,
 'watch_fd_thread': <Thread(Thread-3 (_watch_pipe_fd), started daemon 140580822693440)>,
 'softspace': 0}

[SoundsSerious]

I'm not sure what your philosophy is on error proofing possible in-hashable items is but I would imagine there is little utility to hashing a thread, or even a python logging instance.

Perhaps there should be a global block list that gets activated with ignore_encoding_errors. Or simply ensure these errors get picked up and handled by that flag.