nomic-ai/nomic

Raise more informative better errors, change some warnings to errors.

bmschmidt opened this issue · 3 comments

User feedback.

I feel you should raise an error instead of only logging something if I’ve run out of storage. In prod I need to abort if I’ve hit the limit or I might end up serving a db with only half the points in. This also happens if I try to upload values which have conflicting ids. At the very least don’t log ‘Upload succeeded’ when it didn't

Our details on failed uploads can be extremely uninformative, and should be improved. Example from discord

Shard upload failed: {"detail":"Upload failed"}
Additional info: {"detail":"Your project has only 0 datums. Please add at least 20 datums!"}

ez2rok commented

I received the same error
Additional info: {"detail":"Your project has only 0 datums. Please add at least 20 datums!"}
when accidentally including nan values in my data. We should raise a separate error if the data contains a nan value.

iblind commented

I've also encountered the same error as @ez2rok did (re: 20 datums) but the data I'm uploading contains several thousand data points. I'm not sure how this behavior emerges/where the error actually is, but I don't have any nan values in my dataset.