weaviate/weaviate-python-client

[DX] Save a Collection's objects into a local JSON file

CShorten opened this issue · 1 comments

What?

Get the objects from a Collection in Weaviate and to a local JSON file.

Why?

This helps with quick experimentation, maybe other use cases.

How?

Thinking this API:

collection = client.collections.get("WineReview")

collection.to_json(savepath="wine_reviews.json", num_samples=100) # returns bool success/fail

That pretty much just wraps the cursor API,

collection = client.collections.get("WineReview")

for item in collection.iterator():
    print(item.uuid, item.properties)

And then dumps the JSON file with the savepath.

Maybe also makes the json Python library another dependency of Weaviate's Python Client (if not already).

Hey team, this notebook contains some more investigations into this idea of Weaviate Collections <> HuggingFace Dataset Hub or JSON files - https://github.com/weaviate/recipes/blob/main/weaviate-features/crud-apis/research/weaviate-collection-interoperability.ipynb