Magento | Mage-os | PWA-Studio CI/CD using github actions:
- tests
- coding standard checks
- build
- security checks
- zero-downtime deploy
- Gitlab-CI deployer
To use this action your git repository must respect similar scaffolding to the following (its best to use our install action (s) ):
├── .github
│ └── workflows # directory where the workflows are found, see below for an example of main.yml
├── README.md
├── magento # directory where you Magento source files should go
└── pwa-studio # optional pwa-studio directory for pwa src code
Links to full usage samples using Magento official latest release or the current develop branch here.
Config Example when under magento v2.4.X
name: m2-actions-test
on: [push]
jobs:
magento2-build:
runs-on: ubuntu-latest
container: ubuntu
name: 'm2 unit tests & build'
services:
mysql:
image: docker://mysql:8.0
env:
MYSQL_ROOT_PASSWORD: magento
MYSQL_DATABASE: magento
ports:
- 3306:3306
options: --health-cmd="mysqladmin ping" --health-interval=10s --health-timeout=5s --health-retries=3
elasticsearch:
image: docker://elasticsearch:7.1.0
ports:
- 9200:9200
options: -e="discovery.type=single-node" --health-cmd="curl http://localhost:9200/_cluster/health" --health-interval=10s --health-timeout=5s --health-retries=10
steps:
- uses: actions/checkout@v2
- name: 'this step will build an magento artifact'
if: always()
uses: MAD-I-T/magento-actions@v3.18
env:
COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
with:
process: 'build'
elasticsearch: 1
Config Example when under magento 2.3 & lower
name: m2-actions-test
on: [push]
jobs:
magento2-build:
runs-on: ubuntu-latest
container: ubuntu
name: 'm2 unit tests & build'
services:
mysql:
image: docker://mysql:5.7
env:
MYSQL_ROOT_PASSWORD: magento
MYSQL_DATABASE: magento
ports:
- 3106:3306
options: --health-cmd="mysqladmin ping" --health-interval=10s --health-timeout=5s --health-retries=3
steps:
- uses: actions/checkout@v2
- name: 'this step will build an magento artifact'
if: always()
uses: MAD-I-T/magento-actions@v3.18
env:
COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
with:
process: 'build'
To use the latest experimental version of the module set the following : (uses: MAD-I-T/magento-actions@master
)
If some issues are encountered on 2.3.X version, please use the v2.0 of the action in place of v3.18
Also, in some custom cases it may be needed to force/specify the php version to use in the step. This can be done by adding php input (after with: option).
php:
[Optional] possible values (7.1, 7.2, 7.3, 7.4, 8.1)composer_version:
[Optional] possible values (1, 2)process:
option possible values ('security-scan-files','static-test', 'integration-test', 'build'...)- see all available args in the inputs section in actions.yml
To use opensearch in place of elasticsearch see following.
Example with M2 project using elasticsuite & elasticsearch here
- install magento from github actions (also supports mage-os)
- install pwa-studio from github actions
- deploy pwa-studio to prod
- Coding standard check
- Magento build
- Magento security scanners
- Unit testing
- Test your magento custom marketplace extensions
- Integration tests
- Mess detector
- Static testing
- Zero-downtime deployment
- Customize the action
- Setting the secrets
- Typical issues
- Use opensearch
- Use gitlab instead of github
- see more on the forum
To migrate from standard to zero-downtime deployment using this action. One can follow this tutorial.
This step must come after a mandatory build step.
For magento 2.4 & 2.3
- name: 'this step will deploy your build to deployment server - zero downtime'
uses: MAD-I-T/magento-actions@v3.18
env:
COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
BUCKET_COMMIT: bucket-commit-${{github.sha}}.tar.gz
MYSQL_ROOT_PASSWORD: magento
MYSQL_DATABASE: magento
HOST_DEPLOY_PATH: ${{secrets.STAGE_HOST_DEPLOY_PATH}}
HOST_DEPLOY_PATH_BUCKET: ${{secrets.STAGE_HOST_DEPLOY_PATH}}/bucket
SSH_PRIVATE_KEY: ${{secrets.STAGE_SSH_PRIVATE_KEY}}
SSH_CONFIG: ${{secrets.STAGE_SSH_CONFIG}}
WRITE_USE_SUDO: false
with:
process: 'deploy-staging'
deployer: 'no-permission-check'
- name: 'unlock php deployer if the deployment fails'
if: failure() || cancelled()
uses: MAD-I-T/magento-actions@v3.18
env:
COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
BUCKET_COMMIT: bucket-commit-${{github.sha}}.tar.gz
MYSQL_ROOT_PASSWORD: magento
MYSQL_DATABASE: magento
HOST_DEPLOY_PATH: ${{secrets.STAGE_HOST_DEPLOY_PATH}}
HOST_DEPLOY_PATH_BUCKET: ${{secrets.STAGE_HOST_DEPLOY_PATH}}/bucket
SSH_PRIVATE_KEY: ${{secrets.STAGE_SSH_PRIVATE_KEY}}
SSH_CONFIG: ${{secrets.STAGE_SSH_CONFIG}}
WRITE_USE_SUDO: false
with:
process: 'cleanup-staging'
deployer: 'no-permission-check'
Also to keep X number of build artifacts on the server after consecutive deployments. One can use keep_releases
see here.
For magento 2.3 and lower if issues with the preceding sample
- name: 'this step will deploy your build to deployment server - zero downtime'
uses: MAD-I-T/magento-actions@v2.0
env:
COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
BUCKET_COMMIT: bucket-commit-${{github.sha}}.tar.gz
MYSQL_ROOT_PASSWORD: magento
MYSQL_DATABASE: magento
HOST_DEPLOY_PATH: ${{secrets.STAGE_HOST_DEPLOY_PATH}}
HOST_DEPLOY_PATH_BUCKET: ${{secrets.STAGE_HOST_DEPLOY_PATH}}/bucket
SSH_PRIVATE_KEY: ${{secrets.STAGE_SSH_PRIVATE_KEY}}
SSH_CONFIG: ${{secrets.STAGE_SSH_CONFIG}}
WRITE_USE_SUDO: false
with:
php: '7.1'
process: 'deploy-staging'
- name: 'unlock php deployer if the deployment fails'
if: failure() || cancelled()
uses: MAD-I-T/magento-actions@v2.0
env:
COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
BUCKET_COMMIT: bucket-commit-${{github.sha}}.tar.gz
MYSQL_ROOT_PASSWORD: magento
MYSQL_DATABASE: magento
HOST_DEPLOY_PATH: ${{secrets.STAGE_HOST_DEPLOY_PATH}}
HOST_DEPLOY_PATH_BUCKET: ${{secrets.STAGE_HOST_DEPLOY_PATH}}/bucket
SSH_PRIVATE_KEY: ${{secrets.STAGE_SSH_PRIVATE_KEY}}
SSH_CONFIG: ${{secrets.STAGE_SSH_CONFIG}}
WRITE_USE_SUDO: false
with:
php: '7.1'
process: 'cleanup-staging'
The env section and values are mandatory :
COMPOSER_AUTH
: `{"http-basic":{"repo.magento.com": {"username": "xxxxxxxxxxxxxx", "password": "xxxxxxxxxxxxxx"}}}HOST_DEPLOY_PATH
:/var/www/myeshop/
HOST_DEPLOY_PATH_BUCKET
:${{secrets.STAGE_HOST_DEPLOY_PATH}}/bucket
or/var/www/myeshop/bucket/
SSH_PRIVATE_KEY
:your ssh key
SSH_CONFIG
: see more adjust the values to match your server (Host must be staging or production)Host staging //this must be staging or production User magento IdentityFile ~/.ssh/id_rsa HostName staging.server Port 12022
WRITE_USE_SUDO
: true or false, the deployer will exec commands as sudo on remote server
The first deploy will fail, unless/then you must place a valid env.php under dir HOST_DEPLOY_PATH/shared/magento/app/etc/ on the deployment endpoint.
A cleanup task must be launched if the deployment fails (see here)
Also one can limit the release history by limiting or increasing the max number of tracked releases on the server (with the keep_releases input arg)
To achieve the deployment using gitlab-ci (follow this tutorial)
One can install magento using github actions. This action will download magento source code and copy it into the github repository calling it. Make sure the repository does not contain the magento directory at the root. You will also need to specify the version. Supported versions 2.2.X, 2.3.X and 2.4.X Or you can simply clone or fork this repository and use it as a template. The use of actions/checkout@v2 is mandatory as v1 is not able to push the src to the repo. Git issue can occur depending on the linux image used (here ubuntu-latest)
name: m2-install-actions
on: [push]
jobs:
magento2-install:
runs-on: ubuntu-latest
name: 'magento install & push'
steps:
- uses: actions/checkout@v2
- name: 'install fresh magento and copy to repo'
uses: MAD-I-T/magento-actions@v3.18
env:
COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
with:
process: 'create-project'
magento_version: 2.3.0
# no_push: 1 //uncomment this to prevent files from getting pushed to repo
To set ${{secrets.COMPOSER_AUTH}}
:
- Go to
Settings>Secrets
- Create variable
COMPOSER_AUTH
- Add you composer auth as value e.g :
{"http-basic":{"repo.magento.com": {"username": "xxxxxxxxxxxxxx", "password": "xxxxxxxxxxxxxx"}}}
One can also download magento code source from the mage-os repo the advantage of this reside in the absence of the authentification requirement. No need for magento credentials. See this repository.
- name: 'install fresh magento from mage-os'
#if: ${{false}}
uses: MAD-I-T/magento-actions@master
with:
process: 'install-mage-os'
magento_version: 2.4.5 #e.g: 2.4.0, 2.4.3, 2.4.4 nightly
One can install magento using github actions. This action will download the latest release of the magento pwa-studio project source code and copy it into the github repository calling it. Or you can simply clone/fork and push the content of this repository to bootstrap a new pwa-studio project. actions/checkout@v2 is mandatory (v2 and up), there is also a video tutorial.
name: pwa-studio-install-actions
on: [push]
jobs:
magento2-install:
runs-on: ubuntu-latest
name: 'install & push pwa-studio project'
steps:
- uses: actions/checkout@v2
- name: 'install fresh pwa studio code and copy to repo'
uses: MAD-I-T/magento-actions@v3.18
with:
process: 'pwa-studio-install'
#no_push: 1 //uncomment this to prevent files from getting pushed to repo
One can also install and deploy a standalone PWA-studio website see the video below:
- name: 'install fresh pwa-studio project'
uses: MAD-I-T/magento-actions@v3.12
if: ${{false}}
with:
process: 'pwa-studio-install'
#no_push: 1
- name: 'launch magento2 build'
if: ${{false}}
#if: always()
id: build
uses: MAD-I-T/magento-actions@master
env:
COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
with:
process: 'build'
elasticsearch: 1
- name: 'launch magento2 zero downtime deploy'
if: ${{false}}
#if: always()
uses: MAD-I-T/magento-actions@master
env:
BUCKET_COMMIT: bucket-commit-${{github.sha}}.tar.gz
HOST_DEPLOY_PATH: ${{secrets.STAGE_HOST_DEPLOY_PATH}}
HOST_DEPLOY_PATH_BUCKET: ${{secrets.STAGE_HOST_DEPLOY_PATH}}/bucket
SSH_PRIVATE_KEY: ${{secrets.STAGE_SSH_PRIVATE_KEY}}
SSH_CONFIG: ${{secrets.STAGE_SSH_CONFIG}}
WRITE_USE_SUDO: false
with:
deployer: 'no-permission-check'
process: 'deploy-staging'
- name: 'unlock deployer if failure'
if: ${{false}}
#if: failure()
uses: MAD-I-T/magento-actions@master
env:
BUCKET_COMMIT: bucket-commit-${{github.sha}}.tar.gz
HOST_DEPLOY_PATH: ${{secrets.STAGE_HOST_DEPLOY_PATH}}
HOST_DEPLOY_PATH_BUCKET: ${{secrets.STAGE_HOST_DEPLOY_PATH}}/bucket
SSH_PRIVATE_KEY: ${{secrets.STAGE_SSH_PRIVATE_KEY}}
SSH_CONFIG: ${{secrets.STAGE_SSH_CONFIG}}
WRITE_USE_SUDO: false
with:
process: 'cleanup-staging'
To check some magento module or some code against Magento conding Standard, useful before marketplace submissions
For magento 2.4 and 2.3
- name: 'test some specific module code quality'
uses: MAD-I-T/magento-actions@v3.18
env:
COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
with:
process: 'phpcs-test'
extension: 'Magento/CatalogSearch'
standard: 'Magento2'
severity: 10
- extension : the module to be tested (Vendor/Name) or Path using repository scaffolding (i.e from see example here)
- standard : the standard for which the conformity must be checked 'Magento2, PSR2, PSR1, PSR12 etc...' see magento-coding-standard
For magento 2.4.x (remove elasticsearch: 1 when building with 2.3.X)
- name: 'This step will build an magento artifact'
uses: MAD-I-T/magento-actions@v3.18
env:
COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
with:
process: 'build'
elasticsearch: 1
For magento <= 2.3 if issues with preceding sample
- name: 'This step will build an magento artifact'
uses: MAD-I-T/magento-actions@v2.0
env:
COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
with:
php: '7.1'
process: 'build'
php
: 7.1, 7.2, 7.3, 7.4 or 8.1
To build static content for languages other than en_US or for multi-lang support see
Security scan actions must (in case of the modules scanner) be launched after a build step see example here as the magerun module needs the app/etc/config.php
file to be present.
To scan the magento 2 files for common vulnerabilities using mwscan, the job's step can be set up as follows
For magento 2.4.x
- name: 'This step will scan the files for security breach'
if: always()
uses: MAD-I-T/magento-actions@v3.18
env:
COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
with:
process: 'security-scan-files'
elasticsearch: 1
override_settings: 1
For magento 2.3 or lower
- name: 'This step will scan the files for security breach'
if: always()
uses: MAD-I-T/magento-actions@v2.0
env:
COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
with:
php: '7.1'
process: 'security-scan-files'
To scan the magento2 installed third parties modules for known vulnerabilities using sansecio/magevulndb, the job's step can be set up as follows:
For magento 2.4.x
- name: 'This step will check all modules for security vulnerabilities'
if: always()
uses: MAD-I-T/magento-actions@v3.18
env:
COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
with:
process: 'security-scan-modules'
elasticsearch: 1
For magento 2.3 or lower
- name: 'This step will check all modules for security vulnerabilities'
if: always()
uses: MAD-I-T/magento-actions@v2.0
env:
COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
with:
php: '7.1'
process: 'security-scan-modules'
Example of an output:
See code sample of testing your magento project here
See code sample to unit test your custom or a thrid party magento2 module here
See more about unit testing in magento2
For magento 2.4.x (remove elasticsearch 1 when building with 2.3.X)
- name: 'This step will execute all the unit tests available'
uses: MAD-I-T/magento-actions@v3.18
env:
COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
with:
process: 'unit-test'
elasticsearch: 1
Run all unit test of the magento email module
- name: 'This step will execute specific unit tests in the path dir'
uses: MAD-I-T/magento-actions@v3.18
env:
COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
with:
process: 'unit-test'
elasticsearch: 1
unit_test_subset_path: 'vendor/magento/module-email/Test/Unit'
See more about unit testing in magento2
For magento 2.3 or lower if issues with the preceding sample
- name: 'This step will execute all the unit tests available'
uses: MAD-I-T/magento-actions@v2.0
env:
COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
with:
php: '7.1'
process: 'unit-test'
Full sample, the integration test will need rabbitmq see below By default all tests in phpunit.xml will be run if the file exists. If not tests in phpunit.xml.dist will be run instead (the later should take hours to complete ^^).
See all the other integration test features here.
magento2-integration-test:
runs-on: ubuntu-latest
container: ubuntu
name: 'm2 integration test'
services:
mysql:
image: docker://mysql:8
env:
MYSQL_ROOT_PASSWORD: magento
MYSQL_DATABASE: magento
options: --health-cmd="mysqladmin ping" --health-interval=10s --health-timeout=5s --health-retries=5 -e MYSQL_ROOT_PASSWORD=magento -e MYSQL_USER=magento -e MYSQL_PASSWORD=magento -e MYSQL_DATABASE=magento --entrypoint sh mysql:8 -c "exec docker-entrypoint.sh mysqld --default-authentication-plugin=mysql_native_password"
elasticsearch:
image: docker://elasticsearch:7.1.0
ports:
- 9200:9200
options: -e="discovery.type=single-node" --health-cmd="curl http://localhost:9200/_cluster/health" --health-interval=10s --health-timeout=5s --health-retries=10
rabbitmq:
image: docker://rabbitmq:3.8-alpine
env:
RABBITMQ_DEFAULT_USER: "magento"
RABBITMQ_DEFAULT_PASS: "magento"
RABBITMQ_DEFAULT_VHOST: "/"
ports:
- 5672:5672
steps:
- uses: actions/checkout@v2
with:
submodules: recursive
- name: 'launch magento2 integration test'
if: ${{false}}
uses: MAD-I-T/magento-actions@v3.18
env:
COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
with:
php: '7.4'
process: 'integration-test'
elasticsearch: 1
Mess detection on magento2 using github actions.
ruleset
variable can be used to specify a custom rulset other than the default magento version.
- name: 'mess detector'
uses: MAD-I-T/magento-actions@master
with:
process: 'mess-detector'
md_src_path: '../Madit/Sips2/'
Checkout this sample to lean how to mess detected your magento module code.
For magento 2.3 & 2.4
- name: 'This step starts static testing the code'
uses: MAD-I-T/magento-actions@v3.18
env:
COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
with:
php: '7.2'
process: 'static-test'
For those cloning ...
Replace in action.yml
image: 'docker://mad1t/magento-actions:latest'
by
image: 'Dockerfile'
use the override_settings set it to 1 You'll also have to create scripts or config dirs in the root of your m2 project. Example of project scafolding to override the action's default configs
├── .github
│ └── workflows # directory where the workflows are found, see below for an example of main.yml
├── README.md
└── magento # directory where you Magento source files should go
└── config # the filenames must be similar to thoses of the action ex: config/integration-test-config.php
└── scripts # ex: scripts/build.sh to override the build behaviour
See more about overriding
- To not have to constantly restart php-fpm after zero-downtime deployments please configure your http servers correctly see here.
- If you're using a self-hosted runner please run it as root user using
sudo ./svc.sh install
- Do not forget to set or replace the
env.php
file in theshared
directory - Adding the ssh user to the
http-user
group ex.www-data
, also check php pool user and group setting rights - Set
WRITE_USE_SUDO
env if you want to launch the deployment script in sudo mode (not necessary in most cases) - integration test when using magento 2.4
- you will need to set mysql 8 docker with the options arg as such
options: --health-cmd="mysqladmin ping" --health-interval=10s --health-timeout=5s --health-retries=5 -e MYSQL_ROOT_PASSWORD=magento -e MYSQL_USER=magento -e MYSQL_PASSWORD=magento -e MYSQL_DATABASE=magento --entrypoint sh mysql:8 -c "exec docker-entrypoint.sh mysqld --default-authentication-plugin=mysql_native_password"
- see example here
- you will need to set mysql 8 docker with the options arg as such
Although the action automatically detects and set the appropriate composer version according to the user's codebase (magento version). One can still force the composer version to be used through the composer_version argument. Possible values (1 or 2)
It is a good practice not to set credentials like composer auth in the code source (see https://12factor.net).
So it is advised to use github secret instead of fill the value in the main.yml of your workflow.
Example for COMPOSER_AUTH
:
- Go to
Settings>Secrets
- Create variable
COMPOSER_AUTH
- Add you composer auth as value e.g :
{"http-basic":{"repo.magento.com": {"username": "xxxxxxxxxxxxxx", "password": "xxxxxxxxxxxxxx"}}}
- Use as follows
COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
in the action definition.
One may need to use opensearch instead of elasticsearch. In that case you have to replace the elasticsearch section accordingly see full sample here :
elasticsearch:
image: ghcr.io/mad-i-t/magento-opensearch:1.2
ports:
- 9200:9200
options: -e="discovery.type=single-node" -e "plugins.security.disabled=true" --health-cmd="curl http://localhost:9200/_cluster/health" --health-interval=10s --health-timeout=5s --health-retries=10