podaac/SOTO

FTS Refactor/migration

Opened this issue · 2 comments

frankinspace commented

Migrate feature translation service from JPL Enterprise GH to public github.com

nikki-t commented

Top-level tasks needed to migrate FTS repos

  1. Create public GitHub repo for FTS.
  2. Move API and database repos into a single repo.
  3. Review/remove sensitive data in API and database codebases.
  4. Project file updates: README, CONTRIBUTING, LICENSE, PR templates.
  5. Push new single repo up to public GitHub repo.
  6. Create GitHub action workflows for CI/CD.
  7. Documentation for use.

Questions/items to track

  1. Do we want to use a single Docker container like Hydrocron or host two separate containers as GitHub packages and in Amazon ECR?
  2. Decide how to organize database code. Either include all files at the root of ‘db’ directory or split our by HUC and SWORD operations.
  3. Automate and move testing under GitHub actions. Should we continue to use AWS CodeBuild? https://github.com/aws-actions/aws-codebuild-run-build
    a. How much does CodeBuild cost?
  4. Host API and database documentation on GitHub pages like Hydrocron?
  5. Create a CMR service record?
nikki-t commented

Tentative repo structure: FTS

  • .github/
    • workflows/
      • build.yml
      • release-create.yml
      • build-pages.yml
      • pull_request_template.md
  • cmr/
    • fts_umm_s.json
    • ops_associations.txt
    • sit_associations.txt
    • uat_associations.txt
  • docker/
    • Readme.md
    • ftsdb.fargate.Dockerfile
    • ftsapi.Dockerfile
  • docs/
    • … include docs for API and Database.
    • examples/
      • HUC Examples/ (API)
      • Jupyter Notebooks/ (API)
      • SWOT Examples/ (API)
  • fts/
    • api/
      • controllers/
        • fts_controller.py
    • db/
      • huc/
        • README.md
        • init.py
        • create_huc_dataset.py
        • create_swot_dataset.py
        • download_data.py
        • remove_multipolygons.py
        • simplify_huc.py
      • sword/
        • setup_sword.py
        • sword_fargate.py
  • terraform/
    • api_specification_templates/
      • fts_aws_api.yml
    • bin/
      • deploy.sh
      • destroy.sh
      • setup_databases.sh (DB)
    • resources/
      • DBuserData.sh (DB)
    • tfvars/
      • ops.tfvars
      • sit.tfvars
      • uat.tfvars
    • feature-translation-service-db-iam.tf (DB)
    • feature-translation-service-db.tf (DB)
    • feature-translation-service-fargate-iam.tf (DB)
    • feature-translation-service-fargate.tf (DB)
    • feature-translation-service-lambda-iam.tf (DB)
    • feature-translation-service-lambda.tf (DB)
    • feature-translation-service-lambda-iam.tf (API)
    • feature-translation-service-main.tf (API)
    • main.tf (DB)
    • variables.tf (DB)
    • versions.tf
  • tests/
    • data/internal/SWORD/Reaches_Nodes_Test/shp/AF/ (DB)
    • test_setup_sword.py (DB)
    • FTS.postman_collection.json (API)
    • buildspec.yml (API)
    • test_huc_lambda.py (API)
    • test_rivers_lambda.py (API)
  • CHANGELOG.md
  • CONTRIBUTING.md
  • LICENSE
  • README.md
  • poetry.lock
  • pyproject.toml