The repository to deploy chromadb via terraform into aws cloud infrastructure, using API Gateway, Cloud Map, Service Discovery, NLB, EFS, ECS Fargate and VPN.
| Architecture 3 | Architecture 4 |
|---|---|
 |
 |
| Architecture 5 | |
 |
- Vanilla public ec2 instance [code]
- Translated from the cloudformation template here to terraform.
- Public ec2 instance with API gateway [code]
- Modified from the youtube video How to run a Chroma Vector Database locally and on AWS! | EASY MODE
- Additional cloudwatch to view api gateway deployment.
- Private ec2 instance with Network Load Balancer and API Gateway [code]
- Modeified from the youtube video Deploy a PRIVATE Chroma Vector DB to AWS | Step by step tutorial | Part 2
- Additional cloudwatch to view api gateway deployment.
- Additional public ec2 to view docker logs within private ec2.
- WARNING: Can take 10 mins to deploy due to VPC Link !!!
- (RECOMMENDED) Private ecs fargate with Network Load Balancer, EFS and API Gateway [code]
- Fargate to manage docker containers.
- Elastic File System for persistent volume of docker.
- Cloudwatch Logs to store api gateway deployment messages and docker logs.
- WARNING: Can take 10 mins to deploy due to VPC Link !!!
- (RECOMMENDED) Private ecs fargate with EFS, Cloud Map and VPN Endpoint. [code]
- Private microservices discoverable by Cloud Map.
- Public access only with VPN client.
- [IMPORTANT] Read [this] to set up your vpn properly.
- WARNING: Can take 10 mins to deploy due to VPN Networking !!!
cd architectures/{architecture-directory}
terraform init
terraform plan
terraform apply -auto-approve- Remote access to ec2 instance.
# architecture 1,2 ssh -i ssh-chroma.pem ec2-user@{public-chroma-ip} sudo docker logs `sudo docker ps | grep chroma | awk '{ print $1 }'` # architecture 3 ssh -i ssh-chroma.pem ec2-user@{public-backdoor-ip} ssh -i ssh-chroma.pem ec2-user@{private-chroma-ip} sudo docker logs `sudo docker ps | grep chroma | awk '{ print $1 }'` # architecture 4,5 (not applicable)
- Curl
# architecture 1.2 curl http://{public-chroma-ip}:8000/api/v1/heartbeat # architecture 2,3,4 curl --location --request GET \ 'https://*******.execute-api.eu-west-2.amazonaws.com/v1/api/v1/heartbeat' \ --header 'x-api-key: ****' # architecture 5 (vpn on) curl http://test.service.internal:8000/api/v1/heartbeat
- Postman (architecture 2,3,4)
- GET
- (architecture 1,2)
http://{public_chroma_ip}:8000/api/v1/heartbeat - (architecture 2,3,4)
https://******.execute-api.eu-west-2.amazonaws.com/v1/api/v1/heartbeat - (architecture 5 (vpn on))
http://test.service.internal:8000/api/v1/heartbeat
- (architecture 1,2)
- Authorization Type: API Key (architecture 2,3,4)
- Key: x-api-key
- Value: {api key}
- GET
- Python requests
import requests # architecture 1,2 uri = 'http://{public_chroma_ip}:8000/api/v1/heartbeat' headers = {} # architecture 2,3,4 uri = 'https://*****.execute-api.eu-west-2.amazonaws.com/v1/api/v1/heartbeat' headers = {"x-api-key": "****"} # architecture 5 (vpn on) uri = 'http://test.service.internal:8000/api/v1/heartbeat' headers = {} # ------ response = requests.get(uri, headers=headers) print(response.text)
- Read Cloudwatch logs
- from API-Gateway-Execution-Logs_xxxxxx. (architecture 2,3,4)
- from chroma-container-logs. (architecture 4, 5) -> replace step 1
- Python sdk from chroma-core
import chromadb # architecture 1,2 hostname = '{public_chroma_ip}' ssl, port, headers = False, 8000, {} # architecture 2,3,4 hostname = '*****.execute-api.eu-west-2.amazonaws.com/dev' ssl, port = True, "" headers = {"x-api-key": "****"} # architecture 5 (vpn on) hostname = "test.service.internal" ssl, port, headers = False, 8000, {} # ------ client = chromadb.HttpClient( host=hostname, # don't include http or https ssl=ssl, port=port, headers=headers ) print("Heartbeat: ", client.heartbeat()) print("List collections: ", client.list_collections())