Add properties to a collection
Opened this issue · 13 comments
Is your feature request related to a problem? Please describe.
If theres any information that applies table-wide, theres currently no way to add it in unless we add it as a metadata to every row. This causes space wastage
Describe the solution you'd like
Ability to add any properties to the table. eg,
collection = client.get_collection("example_tablename")
print(collection.properties) # --> gives dictionary of properties. default {}
collection.set_property("main_column", "column a") --> sets it in property
print(collection.properties) # --> {"main_column":"column a"}Describe alternatives you've considered
Currently adding as a column metadata instead (so every row has a static value)
Additional context
Hey @Waffleboy, could you please describe your use-case a bit more - why would you need those values collection-wide?
Also, would it be possible to use special collection which would hold per-collection info instead?
Hi @generall ,
There are some metadata we want to add to the table, for example which is the main date column to filter by.
Ok, have you considered making another collection with collection-level-metadata? It seems that you would need to make 2 requests to the service to make this filtering requests anyway
How would that work? The collection has one row with the payload being the collection-level-metadata?
Rows need a vector, so some dummy vector?
Seems very inelegant, and there wouldnt be a duplicate call as Im already calling collection=client.get_collection(table_name) in code, I expect the properties to be inside that collection object?
How would that work? The collection has one row with the payload being the collection-level-metadata?
Yes.
Rows need a vector, so some dummy vector?
You can configure any vector size, and upload a point with a null vector.
Seems very inelegant, and there wouldnt be a duplicate call as Im already calling collection=client.get_collection(table_name) in code, I expect the properties to be inside that collection object?
It is, but its what is already supported in our current releases.
Since we've seen requests for this multiple times, it's something we might add later down the line. But currently we have higher priority tasks to focus on, so it's not on our road map in the short term.
You can configure any vector size, and upload a point with a null vector.
you can actually configure no vectors at all.
tbh, I don't see any advantages of the per-collection meta. Maybe you could provide a sample usage with "pseudocode" of how you would expect it to work
I have a similar situation of @Waffleboy, we as a team work with multiple embedding models and store vectors into collection, it would be helpful if we could store collection metadata like embedding model, dimensions, and other details like description, data title etc., right now we are dealing the situation as recommended by @generall, it would be helpful if we store the metadata along with its vectors.
Ok, have you considered making another collection with collection-level-metadata? It seems that you would need to make 2 requests to the service to make this filtering requests anyway
Whatever, we can make it as a separate text field and alter via collection update API.
/bounty $100
💎 $100 bounty • Qdrant
Steps to solve:
- Start working: Comment
/attempt #3957with your implementation plan - Submit work: Create a pull request including
/claim #3957in the PR body to claim the bounty - Receive payment: 100% of the bounty is received 2-5 days post-reward. Make sure you are eligible for payouts
Thank you for contributing to qdrant/qdrant!
Add a bounty • Share on socials
| Attempt | Started (GMT+0) | Solution |
|---|---|---|
| 🟢 @abhishek818 | Apr 24, 2024, 11:00:08 AM | #4107 |
/attempt #3957
| Algora profile | Completed bounties | Tech | Active attempts | Options |
|---|---|---|---|---|
| @abhishek818 | 1 Qdrant bounty + 2 bounties from 2 projects |
JavaScript, TypeScript |
Cancel attempt |
💡 @abhishek818 submitted a pull request that claims the bounty. You can visit your bounty board to reward.
Could I try my hand at this issue? @abhishek818 @generall