/aws-chromadb-terraform

Deployment of chromadb into AWS resources through terraform

Primary LanguageHCLApache License 2.0Apache-2.0

aws-chromadb-terraform

The repository to deploy chromadb via terraform into aws cloud infrastructure, using API Gateway, Cloud Map, Service Discovery, NLB, EFS, ECS Fargate and VPN.

Architecture 3 Architecture 4
diagram diagram
Architecture 5
diagram

Architectures

  1. Vanilla public ec2 instance [code]
    • Translated from the cloudformation template here to terraform.
  2. Public ec2 instance with API gateway [code]
  3. Private ec2 instance with Network Load Balancer and API Gateway [code]
  4. (RECOMMENDED) Private ecs fargate with Network Load Balancer, EFS and API Gateway [code]
    • Fargate to manage docker containers.
    • Elastic File System for persistent volume of docker.
    • Cloudwatch Logs to store api gateway deployment messages and docker logs.
    • WARNING: Can take 10 mins to deploy due to VPC Link !!!
  5. (RECOMMENDED) Private ecs fargate with EFS, Cloud Map and VPN Endpoint. [code]
    • Private microservices discoverable by Cloud Map.
    • Public access only with VPN client.
    • [IMPORTANT] Read [this] to set up your vpn properly.
    • WARNING: Can take 10 mins to deploy due to VPN Networking !!!

How to deploy?

cd architectures/{architecture-directory}
terraform init
terraform plan
terraform apply -auto-approve

How to debug?

  1. Remote access to ec2 instance.
    # architecture 1,2
    ssh -i ssh-chroma.pem ec2-user@{public-chroma-ip}
    sudo docker logs `sudo docker ps | grep chroma | awk '{ print $1 }'`
    # architecture 3
    ssh -i ssh-chroma.pem ec2-user@{public-backdoor-ip}
    ssh -i ssh-chroma.pem ec2-user@{private-chroma-ip}
    sudo docker logs `sudo docker ps | grep chroma | awk '{ print $1 }'`
    # architecture 4,5 (not applicable)
  2. Curl
    # architecture 1.2
    curl http://{public-chroma-ip}:8000/api/v1/heartbeat
    # architecture 2,3,4
    curl --location --request GET \
        'https://*******.execute-api.eu-west-2.amazonaws.com/v1/api/v1/heartbeat' \
        --header 'x-api-key: ****'
    # architecture 5 (vpn on)
    curl http://test.service.internal:8000/api/v1/heartbeat
  3. Postman (architecture 2,3,4)
    • GET
      • (architecture 1,2) http://{public_chroma_ip}:8000/api/v1/heartbeat
      • (architecture 2,3,4) https://******.execute-api.eu-west-2.amazonaws.com/v1/api/v1/heartbeat
      • (architecture 5 (vpn on)) http://test.service.internal:8000/api/v1/heartbeat
    • Authorization Type: API Key (architecture 2,3,4)
      • Key: x-api-key
      • Value: {api key}
  4. Python requests
    import requests
    # architecture 1,2
    uri = 'http://{public_chroma_ip}:8000/api/v1/heartbeat'
    headers = {}
    # architecture 2,3,4
    uri = 'https://*****.execute-api.eu-west-2.amazonaws.com/v1/api/v1/heartbeat'
    headers = {"x-api-key": "****"}
    # architecture 5 (vpn on)
    uri = 'http://test.service.internal:8000/api/v1/heartbeat'
    headers = {}
    # ------
    response = requests.get(uri, headers=headers)
    print(response.text)
  5. Read Cloudwatch logs
    • from API-Gateway-Execution-Logs_xxxxxx. (architecture 2,3,4)
    • from chroma-container-logs. (architecture 4, 5) -> replace step 1
  6. Python sdk from chroma-core
    import chromadb
    
    # architecture 1,2
    hostname = '{public_chroma_ip}'
    ssl, port, headers = False, 8000, {}
    # architecture 2,3,4
    hostname = '*****.execute-api.eu-west-2.amazonaws.com/dev'
    ssl, port = True, ""
    headers = {"x-api-key": "****"}
    # architecture 5 (vpn on)
    hostname = "test.service.internal"
    ssl, port, headers = False, 8000, {}
    # ------
    
    client = chromadb.HttpClient(
        host=hostname, # don't include http or https
        ssl=ssl, port=port,
        headers=headers
    )
    print("Heartbeat: ", client.heartbeat())
    print("List collections: ", client.list_collections())