[DX] Save a Collection's objects into a local JSON file
CShorten opened this issue · 1 comments
CShorten commented
What?
Get the objects from a Collection in Weaviate and to a local JSON file.
Why?
This helps with quick experimentation, maybe other use cases.
How?
Thinking this API:
collection = client.collections.get("WineReview")
collection.to_json(savepath="wine_reviews.json", num_samples=100) # returns bool success/fail
That pretty much just wraps the cursor API,
collection = client.collections.get("WineReview")
for item in collection.iterator():
print(item.uuid, item.properties)
And then dumps the JSON file with the savepath
.
Maybe also makes the json
Python library another dependency of Weaviate's Python Client (if not already).
CShorten commented
Hey team, this notebook contains some more investigations into this idea of Weaviate Collections <> HuggingFace Dataset Hub or JSON files - https://github.com/weaviate/recipes/blob/main/weaviate-features/crud-apis/research/weaviate-collection-interoperability.ipynb