nomic-ai/nomic

Drop indices on pandas dataframes

bmschmidt opened this issue · 4 comments

bmschmidt commented

Adding a dataframe gives this error:

ValueError: Underscore fields are reserved for Atlas internal use: __index_level_0__

but if we just run reset_index() before uploading it'll be fine.

hvsesha commented

hi in which file you are getting this error Kindly specify

bmschmidt commented

This is an error thrown from data_inference.py in some cases when a pandas frame is uploaded.

patschw commented

Hi, I have the same problem with the map_data function. I am passing a numpy embedddings array and a pandas df

dataset = atlas.map_data(data=df_filtered,
embeddings=df_filtered.loc[:, 'dim_0':'dim_199'].to_numpy(),
id_field='word',
description="A test map of some words from Weimar to BRD 1" )

bmschmidt commented

Thanks for the report. We'll prioritize fixing this, but I think that if you run reset_index() on your df before passing it to an Atlas method, it will hopefully resolve your issue.