Cheukting/simple_query_app

[Enhancement] Generalize the project to take in any csv

Cheukting opened this issue · 2 comments

Cheukting commented

Now the tool only takes in csv having columns "firstname", "lastname" and "emal". The first step to make it useful is to generalize it to work on all type of csv. One way of doing it is to create a table in the DB with schema matching the columns of the csv.

Trevortds commented

I have experience with this kind of task in python. What kind of constraints do you have? Do you want it to always go into the table "users"? Do you assume the inputs to always be strings?

Cheukting commented

Hi @Trevortds thanks for interested to help. Answers to your questions:

  1. no, it does not always go into "users", actually we should let users have more than one table. They can choose which name as their table name (maybe adding an option in Clink like --name my_table) if no table name is specified, we will use the name of the csv as table name. If the table with the same name existed. Maybe prompt back to the user to confirm overwriting the table or appending the table.

  2. regarding the type fo their input from csv. I think we can use the dtype of pandas as an indicator of what data type to use.

Hope it make sense, let me know if there are more ideas or anything unclear.