/aw-src

For fun

Primary LanguagePython

  • Create a S3 Bucket
  • Create a ECR repository

add a .env with the following credentials

BUCKET=
ECR=
AWS_ACCESS_KEY_ID=
AWS_SECRET_ACCESS_KEY=
AWS_DEFAULT_REGION=
AWS_ACCOUNT_ID=
  • Create a ECS cluster
  • Create a task for each spider and schedule it.