Will other DBs like SQLite be supported in the future?

[Tokgoleo]

Since we found it really hard to convert our data to conform the format of Postgres.

[wongjingping]

Hi @Tokgoleo, just to make sure I understand the issue here, are you hoping to import your own data/database into our evaluation framework to use the evaluation runners (bring your own data-style)? Or are you looking to create your own training dataset based on sqlite?

To answer your question, we are unlikely to support sqlite imports because sqlite as a dialect is too "liberal", for example allowing you to insert strings into columns created as integers, and not enforcing primary/foreign key constraints, resulting in import/export inconsistencies when importing into a stricter database like postgres. As we don't want our model to get used to these inconsistencies nor predict sql that would conform to such loose syntax, this explains the higher bar we apply to our evaluations.

[Tokgoleo]

Thx a lot! The first understanding is precise. Since you provided a wonderful evaluation framework, we were trying to incorporate it into our research experiment(instead of production environment, so we insists on using SQLite due to its flexibility).

[wongjingping]

Thanks for clarifying. Unfortunately we don't plan to support SQLite for the reasons stated above, so you would need to convert them.