Magento 2 auto scaling cluster with Terraform on AWS cloud

Deploy a full-scale secure and flexible e-commerce infrastructure based on Magento 2 in a matter of seconds.
Enterprise-grade solution for companies of all sizes, B2B B2C, providing the best customer experience.

AWS Graviton2 Processor - Enabling the best performance in EC2:

[?] Why we need Adobe Commerce Cloud alternative:

The biggest issue is that ACC pricing based on GMV % and AOV %, you overpay up to 80%, while the bill between Adobe and AWS remains at a minimum. With this approach, you invest money in the development of a third-party business, but not your own. Why spend so much money without having control over your business in the cloud? Configuring your own infrastructure these days is the most profitable way. You manage the resources, you have a complete overview how it works and you have full control over the money invested in your own infrastructure. At any time you can make changes to both infrastructure and application design without thinking about restrictions, 3rd party platform limitations and unforeseen costs. There are no hidden bills and payments for excess resources, which, as a result, you will not need.

Adobe Commerce Cloud has lots of technical problems due to the fact that many services compete on the same server and share the processor time, memory, network and I/O. Bad architectural solution using monolitic servers, not cloud native solution, that was not made specifically for Magento, but adapted in rush using many wrappers, with manual pseudo scaling and 48 hours to 5 days to accept and deploy new settings.

Obviously, PaaS intermediaries also use AWS Cloud. But concealing its cheap solutions with a marketing, 
trying to hook you up on a dodgy contract and making you pay 10 times more.

AWS cloud account pros:

Open source Magento
Pay as You Go
Transparent billing
No draconian contracts
No sudden surplus resources
No hardware restrictions
No services limitations
No hidden bottlenecks
No time waste for support tickets

Amazon Web Services offers an ecommerce cloud computing solutions to small and large businesses that want a flexible, secured, highly scalable infrastructure. All the technologies online retailers need to manage growth—instantly. With technologies like automatic scaling compute resources, networking, storage, content distribution, and a PCI-compliant environment, retailers can always provide great customer experiences and capitalize on growth opportunities.

The biggest benefits of using your own AWS Cloud account: Reserved Instances
Reserved Instances provide you with significant savings on your Amazon EC2 costs compared to On-Demand Instance pricing. With Savings Plans, you make a commitment to a consistent usage amount, measured in USD per hour. This provides you with the flexibility to use the instance configurations that best meet your needs and continue to save money.

[+] EC2 webstack custom configuration and management

User data templates with shell scripts. If you are familiar with shell scripting, this is the easiest and most complete way to send instructions to an instance to perform common automated configuration tasks and even run scripts after the instance starts. From default stack optimization to changing any application and service settings.

NGINX is optimized and fully supported on the latest generation of 64-bit ARM Servers utilizing the architecture. PHP using socket connection.

Debian 11 ARM 'bullseye', which will be supported for the next 5 years. Includes support for the very latest ARM-based server systems powered by certified 64-bit processors. Develop and deploy at scale. Webstack delivers top performance on ARM.

AWS Systems Manager is an AWS service that you can use to view and control your infrastructure on AWS. Using the Systems Manager console, you can view operational data from multiple AWS EC2 instances and automate operational tasks across your AWS resources. Systems Manager helps you maintain security and compliance. No SSH connections from outside, no need to track passwords and private keys.

Developer documentation to read:

https://devdocs.magento.com/
https://docs.aws.amazon.com/index.html
https://www.terraform.io/docs/
https://aws.amazon.com/cloudshell/

The terraform configuration language and all the files in this repository are intuitively simple and straightforward. They are written in simple text and functions that any beginner can understand. Terraform deployment with zero dependency, no prerequisites, no need to install additional software, no programming required.

The idea was to create something more complex, with deeper settings, that terrifies competitors at first sight.

Deployment into isolated VPC:

Login to AWS Console
Subscribe to Debian 11 ARM
Start AWS CloudShell
Install Terraform:

   sudo yum install -y yum-utils
   sudo yum-config-manager --add-repo https://rpm.releases.hashicorp.com/AmazonLinux/hashicorp.repo
   sudo yum -y install terraform

Create deployment directory:

  mkdir magento && cd magento

Clone repo:

  git clone https://github.com/magenx/Magento-2-aws-cluster-terraform.git .

Note: Right after terraform apply you will receive email from amazon to approve resources
Check all details / all files / adjust your settings, edit your domain and email in variables.tf
Run:

   terraform init
   terraform apply

to destroy infrastructure: terraform destroy
resources created outside of terraform must be deleted manually, for example CloudWatch logs

Complete setup:

5 autoscaling groups with launch templates converted from user_data
4 target groups for load balancer (varnish frontend admin staging)
1 build server to compile all the code
2 load balancers (external/internal) with listeners / rules
2 rds mariadb databases multi AZ production, single AZ staging
1 elasticsearch domain for Magento catalog search
2 redis elasticache cluster for sessions and cache
1 rabbitmq broker to manage queue messages
2 s3 buckets for [media] images and [system] files and logs (with access policy)
2 codecommit app files repository and services config files repository
1 cloudfront s3 origin distribution
1 efs file system for shared folders, with mount target per AZ
1 sns topic default subscription to receive email alerts
1 ses user access details for smtp module

resources are grouped into a virtual network, VPC dedicated to your brand
the settings initially imply a large store, and are designed for huge traffic.
services are clustered and replicated thus ready for failover.

Magento 2 development | source code:

Local provisioner copy files from https://github.com/magenx/Magento-2/tree/aws
Pickup files from your own repo @ variables.tf#L19
Files saved to AWS CloudShell /tmp directory and pushed to CodeCommit.
Later on EC2 instance user_data configured on boot to clone files from CodeCommit branch.
Right after infrastructure deployment the minimal Magento 2 package is ready to install.
Check and run SSM Document to install Magento

Replaced over 200+ useless modules. Minimal Magento 2 package can be extended anytime. Remove blacklisted components from composer.json in "replace": {} and run composer update
modules configuration here: https://github.com/magenx/Magento-2/blob/aws/composer.json

[!] Why removing bloatware modules and use Magento minimal package:

Faster backend and frontend ↓from 14% upto 50%
Better memory management ↓upto 15%
Easy deployments
Less dependencies
Zero maintenance
Low security risks

Enabled modules for business requirements:

CI/CD scenario:

Event driven
Services configuration files tracked in CodeCommit repository
Changes in CodeCommit repository triggers EventBridge rule.
SSM Document pull from CodeCommit repository and cleanup.
Change deployment logic to your needs.

Infrastructure DevOps and beyond:

Terraform AWS Provider
edit => migrate => share

TODO:

rework and simplify.

Emergency Exit:

Terraform run fails during development or other failure that messes up the account:
terraform destroy reports No changes or Nothing to destroy, but you can see some resources in AWS console.
AWS NUKE - Remove all resources from an AWS account. (runs in dry run mode to list all resources)

😻 Support the project

This takes time and research. You can use this for free. But for me its not free to create it. If you are using this project, there are few ways you can support it:

Star and sharing the project
Open an issue to help me make it better
PAYPAL - You can make one-time donation.

_{Magento 2 on the AWS Cloud: Quick Start Deployment}

vijayyadav06/Magento-2-aws-cluster-terraform