The main purpose of this repository is to create a cloud services infrastructure capable of creating buckets and clusters in order to run an orchestration via Airflow. Construction will be carried out in an IaC manner. The infrastructure will be coded with terraform. All environment installation and Airflow software will be via Ansible. As the OS used was windows, it became necessary to make some adjustments through WSL.
- If you decide to run with CI/CD is just push with main or dev and github actions will take care of everything about infraestructure.
- Run: terraform init
- Run: terraform plan
- Run: terraform apply --auto-approve
- Install at WSL: ubuntu 18 LTS
- Do at WSL: sudo vim /etc/wsl.config and insert "[automount] opttions="metadata"
- Certify that: WSL and sudo chmod 400 at key.pem
- Not the finnest solution, but will avoid some trouble about known hosts: rm ~/.ssh/known_hosts
- Run: ansible-playbook ./airflow/playbook.yml -u ubuntu --private-key ./infrastructure/aws/secretkey/key.pem -i ./airflow/hosts.yml
- Wait infraestructure building and Airflow deploy
- Access Airflow by: machine-ip:8080, password: airflow, user: airflow
- In order to finish all services: terraform destroy