Magento 2 auto scaling cluster with Terraform on AWS cloud
Deploy a full-scale secure and flexible e-commerce infrastructure based on Magento 2 in a matter of seconds.
Enterprise-grade solution for companies of all sizes, B2B B2C, providing the best customer experience.
The biggest issue is that ACC pricing based on GMV % and AOV %, you overpay up to 80%, while the bill between Adobe and AWS remains at a minimum. With this approach, you invest money in the development of a third-party business, but not your own. Why spend so much money without having control over your business in the cloud? Configuring your own infrastructure these days is the most profitable way. You manage the resources, you have a complete overview how it works and you have full control over the money invested in your own infrastructure. At any time you can make changes to both infrastructure and application design without thinking about restrictions, 3rd party platform limitations and unforeseen costs. There are no hidden bills and payments for excess resources, which, as a result, you will not need.
Adobe Commerce Cloud has lots of technical problems due to the fact that many services compete on the same server and share the processor time, memory, network and I/O. Bad architectural solution using monolitic servers, not cloud native solution, that was not made specifically for Magento, but adapted in rush using many wrappers, with manual pseudo scaling and 48 hours to 5 days to accept and deploy new settings.
Obviously, PaaS intermediaries also use AWS Cloud. But concealing its cheap solutions with a marketing,
trying to hook you up on a dodgy contract and making you pay 10 times more.
- Open source Magento
- Pay as You Go
- Transparent billing
- No draconian contracts
- No sudden surplus resources
- No hardware restrictions
- No services limitations
- No hidden bottlenecks
- No time waste for support tickets
Amazon Web Services offers an ecommerce cloud computing solutions to small and large businesses that want a flexible, secured, highly scalable infrastructure. All the technologies online retailers need to manage growth—instantly. With technologies like automatic scaling compute resources, networking, storage, content distribution, and a PCI-compliant environment, retailers can always provide great customer experiences and capitalize on growth opportunities.
The biggest benefits of using your own AWS Cloud account: Reserved Instances
Reserved Instances provide you with significant savings on your Amazon EC2 costs compared to On-Demand Instance pricing. With Savings Plans, you make a commitment to a consistent usage amount, measured in USD per hour. This provides you with the flexibility to use the instance configurations that best meet your needs and continue to save money.
User data templates with shell scripts. If you are familiar with shell scripting, this is the easiest and most complete way to send instructions to an instance to perform common automated configuration tasks and even run scripts after the instance starts. From default stack optimization to changing any application and service settings.
NGINX is optimized and fully supported on the latest generation of 64-bit ARM Servers utilizing the architecture. PHP using socket connection.
Debian 11 ARM 'bullseye', which will be supported for the next 5 years. Includes support for the very latest ARM-based server systems powered by certified 64-bit processors. Develop and deploy at scale. Webstack delivers top performance on ARM.
AWS Systems Manager is an AWS service that you can use to view and control your infrastructure on AWS. Using the Systems Manager console, you can view operational data from multiple AWS EC2 instances and automate operational tasks across your AWS resources. Systems Manager helps you maintain security and compliance. No SSH connections from outside, no need to track passwords and private keys.
https://devdocs.magento.com/
https://docs.aws.amazon.com/index.html
https://www.terraform.io/docs/
https://aws.amazon.com/cloudshell/
The terraform configuration language and all the files in this repository are intuitively simple and straightforward. They are written in simple text and functions that any beginner can understand. Terraform deployment with zero dependency, no prerequisites, no need to install additional software, no programming required.
The idea was to create something more complex, with deeper settings, that terrifies competitors at first sight.
- Login to AWS Console
- Subscribe to Debian 11 ARM
- Start AWS CloudShell
- Install Terraform:
sudo yum install -y yum-utils
sudo yum-config-manager --add-repo https://rpm.releases.hashicorp.com/AmazonLinux/hashicorp.repo
sudo yum -y install terraform
- Create deployment directory:
mkdir magento && cd magento
- Clone repo:
git clone https://github.com/magenx/Magento-2-aws-cluster-terraform.git .
- Note: Right after
terraform apply
you will receive email from amazon to approve resources - Check all details / all files / adjust your settings, edit your domain and email in
variables.tf
- Run:
terraform init
terraform apply
to destroy infrastructure:
terraform destroy
resources created outside of terraform must be deleted manually, for example CloudWatch logs
5
autoscaling groups with launch templates converted from user_data
4
target groups for load balancer (varnish frontend admin staging)
1
build server to compile all the code
2
load balancers (external/internal) with listeners / rules
2
rds mariadb databases multi AZ production, single AZ staging
1
elasticsearch domain for Magento catalog search
2
redis elasticache cluster for sessions and cache
1
rabbitmq broker to manage queue messages
2
s3 buckets for [media] images and [system] files and logs (with access policy)
2
codecommit app files repository and services config files repository
1
cloudfront s3 origin distribution
1
efs file system for shared folders, with mount target per AZ
1
sns topic default subscription to receive email alerts
1
ses user access details for smtp module
resources are grouped into a virtual network, VPC dedicated to your brand
the settings initially imply a large store, and are designed for huge traffic.
services are clustered and replicated thus ready for failover.
- Deployment into isolated Virtual Private Cloud
- Autoscaling policy per each group, excluding
build
instance - Managed with Systems Manager agent
- Instance Profile assigned to simplify EC2 management
- Create and use ssm documents and EventBridge rules to automate tasks
- Simple Email Service authentication + SMTP Magento module
- CloudWatch agent configured to stream logs
- All Magento files managed with git only
- Live shop in production mode / read-only
- Security groups configured for every service and instances
- Enhanced security in AWS and LEMP
- AWS Inspector Assessment templates
- AWS WAF Protection rules
- Local provisioner copy files from https://github.com/magenx/Magento-2/tree/aws
- Pickup files from your own repo @ variables.tf#L19
- Files saved to AWS CloudShell /tmp directory and pushed to CodeCommit.
- Later on EC2 instance user_data configured on boot to clone files from CodeCommit branch.
- Right after infrastructure deployment the minimal Magento 2 package is ready to install.
- Check and run SSM Document to install Magento
Replaced over 200+ useless modules. Minimal Magento 2 package can be extended anytime. Remove blacklisted components from
composer.json
in"replace": {}
and runcomposer update
modules configuration here: https://github.com/magenx/Magento-2/blob/aws/composer.json
- Faster backend and frontend ↓
from 14% upto 50%
- Better memory management ↓
upto 15%
- Easy deployments
- Less dependencies
- Zero maintenance
- Low security risks
Enabled modules for business requirements:
- Stripe Payments
- Mageplaza SMTP
- Magefan Blog
- Fooman Order#=Invoice#
- Fooman Print PDF
- Fooman Email PDF
- Event driven
- Services configuration files tracked in CodeCommit repository
- Changes in CodeCommit repository triggers EventBridge rule.
- SSM Document pull from CodeCommit repository and cleanup.
- Change deployment logic to your needs.
- Terraform AWS Provider
- edit => migrate => share
- rework and simplify.
Terraform run fails during development or other failure that messes up the account:
terraform destroy
reports No changes or Nothing to destroy, but you can see some resources in AWS console.
AWS NUKE - Remove all resources from an AWS account. (runs in dry run mode to list all resources)
This takes time and research. You can use this for free. But for me its not free to create it. If you are using this project, there are few ways you can support it:
- Star and sharing the project
- Open an issue to help me make it better
- PAYPAL - You can make one-time donation.