defog-ai/sql-eval

what is embeddings.pkl file, and how to create it

Closed this issue · 6 comments

durgesh-rathod-sf commented

How to create emmbeddings.pkl file mentioned here

rishsriv commented

Hi there! The pruning files for our predefined schemas will automatically be created when you run ./data/init_db.sh. We have not yet released documentation for pruning new schemas, and will do that later this week.

Tokgoleo commented

Hi there! The pruning files for our predefined schemas will automatically be created when you run ./data/init_db.sh. We have not yet released documentation for pruning new schemas, and will do that later this week.

Cool!

durgesh-rathod-sf commented

Hi @rishsriv
Have we released the documentation for pruning new schemas ?

wangcong607 commented

How to create emmbeddings.pkl file mentioned here

hi, This shows how to use the embeddings.pkl file, but it doesn't explain how it is created. I want to know how this file is generated.

wongjingping commented

@wangcong607 the embeddings.pkl file is generated by another dependency we maintain, defog-data. That python library is responsible for the generation of the pickle file, and you can see more details in the generate_embeddings function.

wangcong607 commented

@wongjingping Thank you so much!