Update marine scripts to generate SQL and load PSVs
Closed this issue · 3 comments
The old code was here:
- Create SQL files to load DB:
https://github.com/glamod/glamod-ingest/blob/master/src/populate/create-sqls-marine.sh
...which called...
https://github.com/glamod/glamod-ingest/blob/master/src/populate/create-sql-marine-year.sh
- Actually run the SQL files that were created:
https://github.com/glamod/glamod-ingest/blob/master/src/populate/load-marine-sql.sh
The new code should live in:
https://github.com/glamod/glamod-ingest/tree/master/scripts/marine
Important factors for refactoring:
-
all paths used by all scripts should be generated by the common script:
https://github.com/glamod/glamod-ingest/blob/master/scripts/glamod-config.py -
all LOTUS interactions should now use the SLURM interface instead of LSF
I have completed the alterations in my own fork so will submit a merge request so we can discuss and review. Ag - I think you mentioned you needed to do some updates before testing the actual psql input into a db?
Fixed in Pull Request. Review before closing.
Working.