Quant Research using AWS Batch

Overview

This project deploys an AWS Batch infrastructure for Quant Research.

Pre-requisites

The project has been built and tested using below configurations on x86 architecture

a. Python - v3.12

# Install pyenv
curl https://pyenv.run | bash

# Install Python 3.12
pyenv install 3.12
pyenv global 3.12

b. AWS CDK - v2.1007.0

# Install NVM
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.1/install.sh | bash

# Install Node LTS
nvm install --lts

# Install AWS CDK
npm install -g aws-cdk@2.1007.0

c. Install curl and jq

sudo dnf install curl jq

d. Install Docker for building containers

e. Install AWS CLI

Deployment

Export AWS credentials via CLI for your target environment
Create a GitHub Personal Access Token for allowing AWS CodePipeline to access the repository and build the container image. Refer the GitHub documentation. You need to provide below permissions
- Read access to code, commit statuses, and metadata
- Read and Write access to repository hooks
Optional: If you enable app_with_codepipeline in parameters.json. Create an AWS Secret with the GitHub personal access token value for triggering deployment via CI/CD for your application code
```
aws secretsmanager create-secret \
--name github-token \
--description "GitHub PAT for the repository" \
--secret-string "<GITHUB_PAT>"
```

Update the .env file with the values for the infrastructure deployment. Below are the placeholder values provided in the .env file for reference.

# AWS account id and region where you need to deploy
AWS_ACCOUNT_ID=012345678901
AWS_REGION=us-east-1

# Unique identifier used as a prefix for your AWS infrastructure resources
NAMESPACE=quant-research-with-aws-batch

# GITHUB prefixed variables are optional
# Only needed if you enable `app_with_codepipeline` in parameters.json
# The secret name is from Step#3
GITHUB_OWNER=github-owner
GITHUB_REPO=github-repo
GITHUB_TOKEN_SECRET_NAME=github-token

If needed, modify the configurations provided in the parameters.json

Build the Python virtual environment

python -m venv .venv
source .venv/bin/activate
cd infrastructure
pip install -U -r requirements.txt
cdk deploy --all

Clean up

Delete all the stacks

source .venv/bin/activate
cd infrastructure
cdk destroy --all

Delete the GitHub token secret stored in AWS Secrets Manager
You may need to manually delete resources like Amazon S3, if they contain data

Security

See CONTRIBUTING for more information.

License

This library is licensed under the MIT-0 License. See the LICENSE file.