[Explore]improve data type inference for drag and drop query panel
Closed this issue · 0 comments
The goal of the project is when user build query in Explore drag and drop control, it takes the data from users' dataset, detects its data type, divides it into metric and column, and in column, detect its data type, divides it into text, date, date&time, numerical, boolean, geographic, float, null, etc with icon. When users drag from the data panel, drop fields should only accept columns that has compatible data type to make sure the queries are valid.
We are going to start with postgres and mysql for now to see how it goes and then proceed with other engines once we understand the potential issues that might come.
- Map out all the use cases of postgres/mysql type conversions in one file
- Collect and match all the datatypes between sqlalchemy and postgres/mysql engines
- Make a new centralized method/s to handle all the conversions that are going to take place respectively for mentioned databases
- Start replacing old conversion methods with new ones
- Fix unit tests (if needed)
- Test!!!! It would be great idea to get all the test cases written before testing and see what's broken and what's not
- Based on results of step 6. write more unit tests if needed
One PR per DB and wouldn't break it into more smaller ones, just to keep everything clean in case we need to get back to it at some point.
Thanks @nikolagigic for taking this epic on!!