SETL-Framework/setl

Implement `create` method for JDBCConnector

qxzzxq opened this issue · 1 comments

Is your feature request related to a problem? Please describe.
We'd like to be able to create a table in the database for the input DataFrame.

Describe the solution you'd like
As the connection information is given in the configuration, we can use java.sql.DriverManager to create the DB connection. Then we can generate and execute the SQL query with the connection.

The SQL query could be generated according to the schema of the DataFrame.

If possible, check also if we can retrieve the connection directly from Spark. You may need to read a bit of the Spark source code 😃 (cf. org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils)

This issue is similar to #82

Spark will create the table (without any key column) when it doesn't exist.
For instance, we don't have the need yet to have a create method. We may re-open this issue in the future