y-crdt/ypy

YBytes?

davidbrochart opened this issue · 4 comments

What would be the best way to handle a binary document?
A YText would not work, and a YArray seems overkill, as it can handle elements of arbitrary type.
Would it make sense to create a YBytes type?

cc @Horusiath @Waidhoferj @dmonad

I'd say just make it an embedded entry stored in a YMap, Array or YText. Binary data modifications are often scattered over the entire payload, so that tracking the individual changes has no sense (usually). In that case just treat them as a single object that's fully replaced on update.

Makes sense, thanks @Horusiath.

Except that it fails:

import y_py as Y

ydoc = Y.YDoc()
b = ydoc.get_map("map")
with ydoc.begin_transaction() as t:
    b.set(t, "bytes", b"012")

# TypeError: Cannot integrate this type into a YDoc: b'012'
#
# The above exception was the direct cause of the following exception:
#
# Traceback (most recent call last):
#   File "<stdin>", line 2, in <module>
# SystemError: <method 'set' of 'builtins.YMap' objects> returned a result with an exception set

Looks like y-py is missing conversion from byte arrays to lib0 Any::Buffer type (see: https://github.com/y-crdt/ypy/blob/main/src/shared_types.rs#L55 and https://github.com/y-crdt/ypy/blob/main/src/type_conversions.rs#L60). cc @Waidhoferj