medic/cht-upgrade-service

errors when calling docker compose: "variable is not set" & "Encountered errors while bringing up the project"

Closed this issue · 3 comments

mrjones-plip commented

Recently, I've been running my CHT 4 instances like this:

wget https://staging.dev.medicmobile.org/_couch/builds/medic:medic:master/docker-compose/cht-couchdb.yml && wget https://staging.dev.medicmobile.org/_couch/builds/medic:medic:master/docker-compose/cht-core.yml && wget https://raw.githubusercontent.com/medic/cht-upgrade-service/main/docker-compose.yml

CHT_COMPOSE_PROJECT_NAME=foo COUCHDB_SECRET=foo DOCKER_CONFIG_PATH=${PWD} COUCHDB_DATA=${PWD}/couchd CHT_COMPOSE_PATH=${PWD} COUCHDB_USER=medic COUCHDB_PASSWORD=password docker-compose up

This is a nice two liner, and I can then test upgrades or what ever from there.However, I see now, while everything is working, there's actually errors in the upgrade log:

compose-errors-cht-upgrade-service-1  | The COUCHDB_UUID variable is not set. Defaulting to a blank string.
compose-errors-cht-upgrade-service-1  | 
compose-errors-cht-upgrade-service-1  | The DOCKER_CONFIG_PATH variable is not set. Defaulting to a blank string.
compose-errors-cht-upgrade-service-1  | The CHT_COMPOSE_PATH variable is not set. Defaulting to a blank string.

And then at the end, this bad looking error:

compose-errors-cht-upgrade-service-1  | Encountered errors while bringing up the project.
compose-errors-cht-upgrade-service-1  | 
compose-errors-cht-upgrade-service-1  |     at ChildProcess.<anonymous> (/app/src/docker-compose-cli.js:29:25)
compose-errors-cht-upgrade-service-1  |     at ChildProcess.emit (node:events:513:28)
compose-errors-cht-upgrade-service-1  |     at ChildProcess._handle.onexit (node:internal/child_process:291:12)

Doing some testing, I see that the env vars aren't getting picked up. Putting them in an .env fixes this. Document the happy path, I opened this PR.

However, shouldn't the env vars get picked up by prepending them?

mrjones-plip commented

to test:

  1. kill all containers (to be safe): docker kill $(docker ps -q)
  2. create a new dir, cd into it; mkdir ~/23-env-vars && cd ~/23-env-vars
  3. run this to start a new CHT instance, prefacing all env vars before the docker-compose up command: 
    wget https://staging.dev.medicmobile.org/_couch/builds/medic:medic:master/docker-compose/cht-couchdb.yml && wget https://staging.dev.medicmobile.org/_couch/builds/medic:medic:master/docker-compose/cht-core.yml && wget https://raw.githubusercontent.com/medic/cht-upgrade-service/67c735a0a8f814f00d15d5cc46c4067d3c3537c4/docker-compose.yml

CHT_COMPOSE_PROJECT_NAME=foo COUCHDB_SECRET=foo DOCKER_CONFIG_PATH=${PWD} COUCHDB_DATA=${PWD}/couchd CHT_COMPOSE_PATH=${PWD} COUCHDB_USER=medic COUCHDB_PASSWORD=password docker-compose up
  4. ensure compose service has no "variable is not set" & "Encountered errors while bringing up the project" errors: docker logs 23-env-vars-cht-upgrade-service-1 and that the CHT services started OK on localhost
lorerod commented

Hi @mrjones-plip thanks for the details on how to AT.
I tested exactly as you suggested.

  • Didn't get variable is not set & Encountered errors while bringing up the project errors on the cht-upgrade-service log:
cht-upgrade-service log: 
Attaching to 23_cht-upgrade-service_1
cht-upgrade-service_1  | Listening on port 5008
cht-upgrade-service_1  | .UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8d in position 597: invalid start byte
cht-upgrade-service_1  |
cht-upgrade-service_1  | .IsADirectoryError: [Errno 21] Is a directory: '/docker-compose/cht-core'
cht-upgrade-service_1  |
cht-upgrade-service_1  | networks:
cht-upgrade-service_1  |   cht-net:
cht-upgrade-service_1  |     name: cht-net
cht-upgrade-service_1  | services:
cht-upgrade-service_1  |   api:
cht-upgrade-service_1  |     depends_on:
cht-upgrade-service_1  |       haproxy:
cht-upgrade-service_1  |         condition: service_started
cht-upgrade-service_1  |     environment:
cht-upgrade-service_1  |       BUILDS_URL: https://staging.dev.medicmobile.org/_couch/builds
cht-upgrade-service_1  |       COUCH_URL: http://medic:password@haproxy:5984/medic
cht-upgrade-service_1  |       UPGRADE_SERVICE_URL: http://cht-upgrade-service:5008
cht-upgrade-service_1  |     expose:
cht-upgrade-service_1  |     - '5988'
cht-upgrade-service_1  |     image: public.ecr.aws/s5s3h4s7/cht-api:4.0.0-alpha
cht-upgrade-service_1  |     logging:
cht-upgrade-service_1  |       driver: local
cht-upgrade-service_1  |       options:
cht-upgrade-service_1  |         max-file: '20'
cht-upgrade-service_1  |         max-size: 50m
cht-upgrade-service_1  |     networks:
cht-upgrade-service_1  |       cht-net: null
cht-upgrade-service_1  |   haproxy:
cht-upgrade-service_1  |     environment:
cht-upgrade-service_1  |       COUCHDB_PASSWORD: password
cht-upgrade-service_1  |       COUCHDB_SERVERS: couchdb
cht-upgrade-service_1  |       COUCHDB_USER: medic
cht-upgrade-service_1  |       HAPROXY_IP: haproxy
cht-upgrade-service_1  |       HAPROXY_PORT: '5984'
cht-upgrade-service_1  |       HEALTHCHECK_ADDR: healthcheck
cht-upgrade-service_1  |     expose:
cht-upgrade-service_1  |     - '5984'
cht-upgrade-service_1  |     hostname: haproxy
cht-upgrade-service_1  |     image: public.ecr.aws/s5s3h4s7/cht-haproxy:4.0.0-alpha
cht-upgrade-service_1  |     logging:
cht-upgrade-service_1  |       driver: local
cht-upgrade-service_1  |       options:
cht-upgrade-service_1  |         max-file: '20'
cht-upgrade-service_1  |         max-size: 50m
cht-upgrade-service_1  |     networks:
cht-upgrade-service_1  |       cht-net: null
cht-upgrade-service_1  |   healthcheck:
cht-upgrade-service_1  |     environment:
cht-upgrade-service_1  |       COUCHDB_PASSWORD: password
cht-upgrade-service_1  |       COUCHDB_SERVERS: couchdb
cht-upgrade-service_1  |       COUCHDB_USER: medic
cht-upgrade-service_1  |     image: public.ecr.aws/s5s3h4s7/cht-haproxy-healthcheck:4.0.0-alpha
cht-upgrade-service_1  |     logging:
cht-upgrade-service_1  |       driver: local
cht-upgrade-service_1  |       options:
cht-upgrade-service_1  |         max-file: '20'
cht-upgrade-service_1  |         max-size: 50m
cht-upgrade-service_1  |     networks:
cht-upgrade-service_1  |       cht-net: null
cht-upgrade-service_1  |   nginx:
cht-upgrade-service_1  |     depends_on:
cht-upgrade-service_1  |       api:
cht-upgrade-service_1  |         condition: service_started
cht-upgrade-service_1  |       haproxy:
cht-upgrade-service_1  |         condition: service_started
cht-upgrade-service_1  |     environment:
cht-upgrade-service_1  |       API_HOST: api
cht-upgrade-service_1  |       API_PORT: '5988'
cht-upgrade-service_1  |       CERTIFICATE_MODE: SELF_SIGNED
cht-upgrade-service_1  |       COMMON_NAME: test-nginx.dev.medicmobile.org
cht-upgrade-service_1  |       COUNTRY: US
cht-upgrade-service_1  |       DEPARTMENT: Information_Security
cht-upgrade-service_1  |       EMAIL: domains@medic.org
cht-upgrade-service_1  |       LOCALITY: San_Francisco
cht-upgrade-service_1  |       ORGANISATION: medic
cht-upgrade-service_1  |       SSL_CERT_FILE_PATH: /etc/nginx/private/cert.pem
cht-upgrade-service_1  |       SSL_KEY_FILE_PATH: /etc/nginx/private/key.pem
cht-upgrade-service_1  |       STATE: California
cht-upgrade-service_1  |     image: public.ecr.aws/s5s3h4s7/cht-nginx:4.0.0-alpha
cht-upgrade-service_1  |     logging:
cht-upgrade-service_1  |       driver: local
cht-upgrade-service_1  |       options:
cht-upgrade-service_1  |         max-file: '20'
cht-upgrade-service_1  |         max-size: 50m
cht-upgrade-service_1  |     networks:
cht-upgrade-service_1  |       cht-net: null
cht-upgrade-service_1  |     ports:
cht-upgrade-service_1  |     - published: 80
cht-upgrade-service_1  |       target: 80
cht-upgrade-service_1  |     - published: 443
cht-upgrade-service_1  |       target: 443
cht-upgrade-service_1  |     volumes:
cht-upgrade-service_1  |     - cht-ssl:/etc/nginx/private:rw
cht-upgrade-service_1  |   sentinel:
cht-upgrade-service_1  |     depends_on:
cht-upgrade-service_1  |       haproxy:
cht-upgrade-service_1  |         condition: service_started
cht-upgrade-service_1  |     environment:
cht-upgrade-service_1  |       API_HOST: api
cht-upgrade-service_1  |       COUCH_URL: http://medic:password@haproxy:5984/medic
cht-upgrade-service_1  |     image: public.ecr.aws/s5s3h4s7/cht-sentinel:4.0.0-alpha
cht-upgrade-service_1  |     logging:
cht-upgrade-service_1  |       driver: local
cht-upgrade-service_1  |       options:
cht-upgrade-service_1  |         max-file: '20'
cht-upgrade-service_1  |         max-size: 50m
cht-upgrade-service_1  |     networks:
cht-upgrade-service_1  |       cht-net: null
cht-upgrade-service_1  | version: '3.9'
cht-upgrade-service_1  | volumes:
cht-upgrade-service_1  |   cht-ssl: {}
cht-upgrade-service_1  |
cht-upgrade-service_1  |
cht-upgrade-service_1  | The COUCHDB_UUID variable is not set. Defaulting to a blank string.
cht-upgrade-service_1  |
cht-upgrade-service_1  | networks:
cht-upgrade-service_1  |   cht-net:
cht-upgrade-service_1  |     name: cht-net
cht-upgrade-service_1  | services:
cht-upgrade-service_1  |   couchdb:
cht-upgrade-service_1  |     environment:
cht-upgrade-service_1  |       COUCHDB_LOG_LEVEL: error
cht-upgrade-service_1  |       COUCHDB_PASSWORD: password
cht-upgrade-service_1  |       COUCHDB_SECRET: foo
cht-upgrade-service_1  |       COUCHDB_USER: medic
cht-upgrade-service_1  |       COUCHDB_UUID: ''
cht-upgrade-service_1  |       SVC_NAME: couchdb
cht-upgrade-service_1  |     image: public.ecr.aws/s5s3h4s7/cht-couchdb:4.0.0-alpha
cht-upgrade-service_1  |     logging:
cht-upgrade-service_1  |       driver: local
cht-upgrade-service_1  |       options:
cht-upgrade-service_1  |         max-file: '20'
cht-upgrade-service_1  |         max-size: 50m
cht-upgrade-service_1  |     networks:
cht-upgrade-service_1  |       cht-net: null
cht-upgrade-service_1  |     restart: always
cht-upgrade-service_1  |     volumes:
cht-upgrade-service_1  |     - /Users/marialorenarodriguezviruel/medic-workspace/docker-files/#23/couchd:/opt/couchdb/data:rw
cht-upgrade-service_1  |     - cht-credentials:/opt/couchdb/etc/local.d:rw
cht-upgrade-service_1  | version: '3.9'
cht-upgrade-service_1  | volumes:
cht-upgrade-service_1  |   cht-credentials: {}
cht-upgrade-service_1  |
cht-upgrade-service_1  |
cht-upgrade-service_1  | .IsADirectoryError: [Errno 21] Is a directory: '/docker-compose/cht-upgrade-service'
cht-upgrade-service_1  |
cht-upgrade-service_1  | networks:
cht-upgrade-service_1  |   cht-net:
cht-upgrade-service_1  |     name: cht-net
cht-upgrade-service_1  | services:
cht-upgrade-service_1  |   cht-upgrade-service:
cht-upgrade-service_1  |     environment:
cht-upgrade-service_1  |       BUILDS_SERVER: null
cht-upgrade-service_1  |       CERTIFICATE_MODE: null
cht-upgrade-service_1  |       CHT_COMPOSE_PATH: /Users/marialorenarodriguezviruel/medic-workspace/docker-files/#23
cht-upgrade-service_1  |       CHT_COMPOSE_PROJECT_NAME: foo
cht-upgrade-service_1  |       CHT_NETWORK: cht-net
cht-upgrade-service_1  |       COMMON_NAME: null
cht-upgrade-service_1  |       COUCHDB_DATA: /Users/marialorenarodriguezviruel/medic-workspace/docker-files/#23/couchd
cht-upgrade-service_1  |       COUCHDB_PASSWORD: password
cht-upgrade-service_1  |       COUCHDB_SECRET: foo
cht-upgrade-service_1  |       COUCHDB_SERVERS: null
cht-upgrade-service_1  |       COUCHDB_USER: medic
cht-upgrade-service_1  |       COUCHDB_UUID: null
cht-upgrade-service_1  |       COUNTRY: null
cht-upgrade-service_1  |       DEPARTMENT: null
cht-upgrade-service_1  |       DOCKER_CONFIG: /config
cht-upgrade-service_1  |       DOCKER_CONFIG_PATH: /Users/marialorenarodriguezviruel/medic-workspace/docker-files/#23
cht-upgrade-service_1  |       EMAIL: null
cht-upgrade-service_1  |       LOCALITY: null
cht-upgrade-service_1  |       MARKET_URL_READ: null
cht-upgrade-service_1  |       NGINX_HTTPS_PORT: null
cht-upgrade-service_1  |       NGINX_HTTP_PORT: null
cht-upgrade-service_1  |       ORGANISATION: null
cht-upgrade-service_1  |       SSL_CERT_FILE_PATH: null
cht-upgrade-service_1  |       SSL_KEY_FILE_PATH: null
cht-upgrade-service_1  |       SSL_VOLUME_MOUNT_PATH: null
cht-upgrade-service_1  |       STATE: null
cht-upgrade-service_1  |     image: public.ecr.aws/s5s3h4s7/cht-upgrade-service:latest
cht-upgrade-service_1  |     networks:
cht-upgrade-service_1  |       cht-net: null
cht-upgrade-service_1  |     volumes:
cht-upgrade-service_1  |     - /var/run/docker.sock:/var/run/docker.sock:rw
cht-upgrade-service_1  |     - /Users/marialorenarodriguezviruel/medic-workspace/docker-files/#23:/config:rw
cht-upgrade-service_1  |     - /Users/marialorenarodriguezviruel/medic-workspace/docker-files/#23:/docker-compose:rw
cht-upgrade-service_1  | version: '3.9'
cht-upgrade-service_1  |
cht-upgrade-service_1  |
cht-upgrade-service_1  | The COUCHDB_UUID variable is not set. Defaulting to a blank string.
cht-upgrade-service_1  |
cht-upgrade-service_1  | Creating volume "foo_cht-ssl" with default driver
cht-upgrade-service_1  |
cht-upgrade-service_1  | Creating volume "foo_cht-credentials" with default driver
cht-upgrade-service_1  |
cht-upgrade-service_1  | Pulling healthcheck (public.ecr.aws/s5s3h4s7/cht-haproxy-healthcheck:4.0.0-alpha)...
cht-upgrade-service_1  |
cht-upgrade-service_1  | 4.0.0-alpha: Pulling from s5s3h4s7/cht-haproxy-healthcheck
cht-upgrade-service_1  |
cht-upgrade-service_1  | Digest: sha256:2a8ffb31a847530a91e68a2bc4e6ccc9673005d09ebc3d2e5489ad2bbaf4c87f
cht-upgrade-service_1  |
cht-upgrade-service_1  | Status: Downloaded newer image for public.ecr.aws/s5s3h4s7/cht-haproxy-healthcheck:4.0.0-alpha
cht-upgrade-service_1  |
cht-upgrade-service_1  | Pulling couchdb (public.ecr.aws/s5s3h4s7/cht-couchdb:4.0.0-alpha)...
cht-upgrade-service_1  |
cht-upgrade-service_1  | 4.0.0-alpha: Pulling from s5s3h4s7/cht-couchdb
cht-upgrade-service_1  |
cht-upgrade-service_1  | Digest: sha256:db6543e018b31f0a3b898eca969f9f3464f124651280a717b941aeb6e0084032
cht-upgrade-service_1  |
cht-upgrade-service_1  | Status: Downloaded newer image for public.ecr.aws/s5s3h4s7/cht-couchdb:4.0.0-alpha
cht-upgrade-service_1  |
cht-upgrade-service_1  | Pulling haproxy (public.ecr.aws/s5s3h4s7/cht-haproxy:4.0.0-alpha)...
cht-upgrade-service_1  |
cht-upgrade-service_1  | 4.0.0-alpha: Pulling from s5s3h4s7/cht-haproxy
cht-upgrade-service_1  |
cht-upgrade-service_1  | Digest: sha256:2e7b75b1ed2a7ae2adeacfec4602fe383c9594e5e157baa21fb0464c1e4f1494
cht-upgrade-service_1  |
cht-upgrade-service_1  | Status: Downloaded newer image for public.ecr.aws/s5s3h4s7/cht-haproxy:4.0.0-alpha
cht-upgrade-service_1  |
cht-upgrade-service_1  | Pulling sentinel (public.ecr.aws/s5s3h4s7/cht-sentinel:4.0.0-alpha)...
cht-upgrade-service_1  |
cht-upgrade-service_1  | 4.0.0-alpha: Pulling from s5s3h4s7/cht-sentinel
cht-upgrade-service_1  |
cht-upgrade-service_1  | Digest: sha256:481d4413ff55a775a55ea8307be6f495d0d2005229a655323a3c7b949d83588a
cht-upgrade-service_1  |
cht-upgrade-service_1  | Status: Downloaded newer image for public.ecr.aws/s5s3h4s7/cht-sentinel:4.0.0-alpha
cht-upgrade-service_1  |
cht-upgrade-service_1  | Pulling api (public.ecr.aws/s5s3h4s7/cht-api:4.0.0-alpha)...
cht-upgrade-service_1  |
cht-upgrade-service_1  | 4.0.0-alpha: Pulling from s5s3h4s7/cht-api
cht-upgrade-service_1  |
cht-upgrade-service_1  | Digest: sha256:553f85fa4e6874503de21c059fa1bf59ee733631baea292cdd90f9aacdbc8646
cht-upgrade-service_1  |
cht-upgrade-service_1  | Status: Downloaded newer image for public.ecr.aws/s5s3h4s7/cht-api:4.0.0-alpha
cht-upgrade-service_1  |
cht-upgrade-service_1  | Pulling nginx (public.ecr.aws/s5s3h4s7/cht-nginx:4.0.0-alpha)...
cht-upgrade-service_1  |
cht-upgrade-service_1  | 4.0.0-alpha: Pulling from s5s3h4s7/cht-nginx
cht-upgrade-service_1  |
cht-upgrade-service_1  | Digest: sha256:f410a6321e988c5252d803e7d49755c8db38689bf4ceb3c55451d17854043ef2
cht-upgrade-service_1  |
cht-upgrade-service_1  | Status: Downloaded newer image for public.ecr.aws/s5s3h4s7/cht-nginx:4.0.0-alpha
cht-upgrade-service_1  |
cht-upgrade-service_1  | Creating foo_healthcheck_1 ...
cht-upgrade-service_1  |
cht-upgrade-service_1  | Creating foo_couchdb_1     ...
cht-upgrade-service_1  |
cht-upgrade-service_1  | Creating foo_haproxy_1     ...
cht-upgrade-service_1  |
cht-upgrade-service_1  | Creating foo_cht-upgrade-service_1 ...
cht-upgrade-service_1  |
cht-upgrade-service_1  | Creating foo_healthcheck_1         ... done
cht-upgrade-service_1  |
cht-upgrade-service_1  | Creating foo_haproxy_1             ... done
cht-upgrade-service_1  |
cht-upgrade-service_1  | Creating foo_api_1                 ...
cht-upgrade-service_1  |
cht-upgrade-service_1  | Creating foo_sentinel_1            ...
cht-upgrade-service_1  |
cht-upgrade-service_1  | Creating foo_couchdb_1             ... done
cht-upgrade-service_1  |
cht-upgrade-service_1  | Creating foo_cht-upgrade-service_1 ... done
cht-upgrade-service_1  |
cht-upgrade-service_1  | Creating foo_sentinel_1            ... done
cht-upgrade-service_1  |
cht-upgrade-service_1  | Creating foo_api_1                 ... done
cht-upgrade-service_1  |
cht-upgrade-service_1  | Creating foo_nginx_1               ...
cht-upgrade-service_1  |
cht-upgrade-service_1  | Creating foo_nginx_1               ... done
cht-upgrade-service_1  |
  • The app started ok on localhost.

However, I have an observation. I don't think this is related, please let me know if you want to take it to another issue or slack. When navigating to App Management -> Upgrades I get Error fetching available versions.

Screenshot:

Captura de Pantalla 2022-11-08 a la(s) 14 29 19

mrjones-plip commented

Yup - unrelated error! I think we're good to go. Thanks for the speedy AT!