Migration with Client NodeJS: Create the generic migration tool

[grabl]

This issue was originally posted by @sorsaffari on 2018-09-03 12:57.

This task was done as a side outcome of the Migration with Client NodeJS requirement. The tool created allows developers to use a generic mechanism for migrating their data into Grakn. Actions to be taken as a part of this task: - ✅ write the skeleton for the nodejsMigrator - ✅ write the CsvMigrator - ✅ write the JsonMigrator - ✅ write the XmlMigrator - ✅ write the MySqlMigrator - ✅ refactor

[grabl]

This comment was originally posted by @sorsaffari on 2018-08-31 12:48:00+02:00.

@haikalpribadi there are two questions that I've tried to answer below. please let me know what you think of each.

Ultimately, how do we want the developers to migrate their data into Grakn?

This is what I have in mind:

• for exported data:
  • run: graql console -f ./schema.gql -k test
  • export data in JSON (could also be in CSV or XML)
  • for each data file
    • write the template in javascript (could also be in java or python)
    • run: graql migrate json --data /file.json --template /file.js
• for sql queries:
  • run: graql console -f ./schema.gql -k test
  • for each query
    • write the template in javascript (could also be in java or python)
    • write the db-config.json file to include details required for connecting to the remote database
    • run: graql migrate sql --query "..." --dbconfig /db-config.json

This approach is pretty much the same as what we have currently in the docs, with the exception of the template file being written in javascript, java or python. What happens under the hood is that, we determine what client needs to be used to run the migration based on the format of the template file provided. If we decide to take this approach, then I will write the migrators in a way that can be used dynamically, expecting any kind of data, rather than just working with an example dataset.

What SQL databases we want the migrator to support?

The SqlMigrators work differently as opposed to the other ones. This is because instead of a data file, a query is provided along with the configs for connecting to the remote database.
In case of the NodejsMigrator, since every sql database has its own node client, we need to pick the ones we want the migrator to support. Based on this article I suggest supporting, MySql, Microsoft SQL Server, Postgres and Oracle.

[grabl]

This comment was originally posted by @sorsaffari on 2018-09-03 12:57:47+02:00.

Original entity id:20668