hashberg-io/multiformats

Various objects in library incompatible with pickle deserialization

emilyhunt opened this issue · 1 comments

multiformats is an upstream module in a package I'm using (MarshalX/atproto) and I've been having some issues with pickling objects from multiformats. (I need to pickle them to serialize them between threads.)

It seems that Pickle requires that all objects with a custom implementation of __new__() also include a function __getnewargs__() (see StackOverflow post). For instance, attempting to unpickle a pickled CID object with pickle.loads() gives an error

TypeError: CID.__new__() missing 4 required positional arguments: 'base', 'version', 'codec', and 'digest'

which can be fixed by adding the following function to multiformats/cid/__init__.py:CID:

def __getnewargs__(self):
    return self.base, self.version, self.codec, self.digest

allowing unpickling as expected.

Would be happy to submit a PR to fix this later this week (would do it right now but I should really go to bed...)

Thanks for your help!

sg495 commented

Hi @emilyhunt, this has now been fixed, with apologies for the delay :)