tutd9x/magento-actions

Github deployment actions for Magento 2

magento-actions

Magento and/or PWA-Studio CI/CD using github actions:

• tests
• coding standard checks
• build
• zero-downtime deploy
• (see also Gitlab-CI version)

usage

To use this action your git repository must respect similar scaffolding to the following (or you can use the install action):

├── .github
│   └── workflows # directory where the workflows are found, see below for an example of main.yml 
├── README.md 
├── magento # directory where you Magento source files should go 
└── pwa-studio # optional pwa-studio directory for pwa src code

Links to full usage samples using Magento official latest release or the current develop branch here. Don't forget to deploy your services on a container i.e (container: ubuntu below).

main.yml

Config Example when magento v2.4

name: m2-actions-test
on: [push]

jobs:
  magento2-build:
    runs-on: ubuntu-latest
    container: ubuntu
    name: 'm2 unit tests & build'
    services:
      mysql:
        image: docker://mysql:8.0
        env:
          MYSQL_ROOT_PASSWORD: magento
          MYSQL_DATABASE: magento
        ports:
          - 3306:3306
        options: --health-cmd="mysqladmin ping" --health-interval=10s --health-timeout=5s --health-retries=3
      elasticsearch:
        image: docker://elasticsearch:7.1.0
        ports:
          - 9200:9200
        options: -e="discovery.type=single-node" --health-cmd="curl http://localhost:9200/_cluster/health" --health-interval=10s --health-timeout=5s --health-retries=10
    steps:
    - uses: actions/checkout@v2 # pulls your repository, M2 src must be in a magento directory
    - name: 'this step will execute all the unit tests available'
      if: always()
      uses: MAD-I-T/magento-actions@v3.11
      env:
        COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
      with:
        process: 'unit-test'
        elasticsearch: 1
    - name: 'this step starts static testing the code'
      if: always()
      uses: MAD-I-T/magento-actions@v3.11
      env:
        COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
      with:
        process: 'static-test'
        elasticsearch: 1
    - name: 'this step will build an magento artifact'
      if: always()
      uses: MAD-I-T/magento-actions@v3.11
      env:
        COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
      with:
        process: 'build'
        elasticsearch: 1

Config Example when magento 2.3 & lower

name: m2-actions-test
on: [push]

jobs:
  magento2-build:
    runs-on: ubuntu-latest
    container: ubuntu
    name: 'm2 unit tests & build'
    services:
      mysql:
        image: docker://mysql:5.7
        env:
          MYSQL_ROOT_PASSWORD: magento
          MYSQL_DATABASE: magento
        ports:
          - 3106:3306
        options: --health-cmd="mysqladmin ping" --health-interval=10s --health-timeout=5s --health-retries=3
    steps:
    - uses: actions/checkout@v2  # pulls your repository, M2 src must be in a magento directory
    - name: 'this step will execute all the unit tests available'
      if: always()
      uses: MAD-I-T/magento-actions@v3.11
      env:
        COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
      with:
        process: 'unit-test'
     - name: 'this step starts static testing the code'
      if: always()
      uses: MAD-I-T/magento-actions@v3.11
      env:
        COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
      with:
        process: 'static-test'
    - name: 'this step will build an magento artifact'
      if: always()
      uses: MAD-I-T/magento-actions@v3.11
      env:
        COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
      with:
        process: 'build'

To use the latest experimental version of the module set the following : (uses: MAD-I-T/magento-actions@master)

If some issues are encountered on 2.3.X version, please use the v2.0 of the action in place of v3.11

Also, in some custom cases it may be needed to force/specify the php version to use in the step. This can be done by adding php input (after with: option).

options

• php: possible values (7.1, 7.2, 7.3, 7.4)
• process: option possible values ('security-scan-files','static-test', 'integration-test', 'build'...)
• see more specific args in the inputs section in actions.yml

Example with M2 project using elasticsuite & elasticsearch here

zero downtime deployment

To migrate from standard to zero-downtime deployment using this action. One can follow this tutorial.

**This step must come after a mandatory build step. **

For magento 2.4 & 2.3

- name: 'this step will deploy your build to deployment server - zero downtime'
  uses: MAD-I-T/magento-actions@v3.11
  env:
    COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
    BUCKET_COMMIT: bucket-commit-${{github.sha}}.tar.gz
    MYSQL_ROOT_PASSWORD: magento
    MYSQL_DATABASE: magento
    HOST_DEPLOY_PATH: ${{secrets.STAGE_HOST_DEPLOY_PATH}}
    HOST_DEPLOY_PATH_BUCKET: ${{secrets.STAGE_HOST_DEPLOY_PATH}}/bucket
    SSH_PRIVATE_KEY: ${{secrets.STAGE_SSH_PRIVATE_KEY}}
    SSH_CONFIG: ${{secrets.STAGE_SSH_CONFIG}}
    WRITE_USE_SUDO: false
    with:
      process: 'deploy-staging'
      deployer: 'no-permission-check'

- name: 'unlock php deployer if the deployment fails'
  if: failure() || cancelled()
  uses: MAD-I-T/magento-actions@v3.11
  env:
    COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
    BUCKET_COMMIT: bucket-commit-${{github.sha}}.tar.gz
    MYSQL_ROOT_PASSWORD: magento
    MYSQL_DATABASE: magento
    HOST_DEPLOY_PATH: ${{secrets.STAGE_HOST_DEPLOY_PATH}}
    HOST_DEPLOY_PATH_BUCKET: ${{secrets.STAGE_HOST_DEPLOY_PATH}}/bucket
    SSH_PRIVATE_KEY: ${{secrets.STAGE_SSH_PRIVATE_KEY}}
    SSH_CONFIG: ${{secrets.STAGE_SSH_CONFIG}}
    WRITE_USE_SUDO: false
  with:
    process: 'cleanup-staging'
    deployer: 'no-permission-check'

For magento 2.3 and lower if issues with the preceding sample

- name: 'this step will deploy your build to deployment server - zero downtime'
  uses: MAD-I-T/magento-actions@v2.0
  env:
    COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
    BUCKET_COMMIT: bucket-commit-${{github.sha}}.tar.gz
    MYSQL_ROOT_PASSWORD: magento
    MYSQL_DATABASE: magento
    HOST_DEPLOY_PATH: ${{secrets.STAGE_HOST_DEPLOY_PATH}}
    HOST_DEPLOY_PATH_BUCKET: ${{secrets.STAGE_HOST_DEPLOY_PATH}}/bucket
    SSH_PRIVATE_KEY: ${{secrets.STAGE_SSH_PRIVATE_KEY}}
    SSH_CONFIG: ${{secrets.STAGE_SSH_CONFIG}}
    WRITE_USE_SUDO: false
  with:
    php: '7.1'
    process: 'deploy-staging'

- name: 'unlock php deployer if the deployment fails'
  if: failure() || cancelled()
  uses: MAD-I-T/magento-actions@v2.0
  env:
    COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
    BUCKET_COMMIT: bucket-commit-${{github.sha}}.tar.gz
    MYSQL_ROOT_PASSWORD: magento
    MYSQL_DATABASE: magento
    HOST_DEPLOY_PATH: ${{secrets.STAGE_HOST_DEPLOY_PATH}}
    HOST_DEPLOY_PATH_BUCKET: ${{secrets.STAGE_HOST_DEPLOY_PATH}}/bucket
    SSH_PRIVATE_KEY: ${{secrets.STAGE_SSH_PRIVATE_KEY}}
    SSH_CONFIG: ${{secrets.STAGE_SSH_CONFIG}}
    WRITE_USE_SUDO: false
  with:
    php: '7.1'
    process: 'cleanup-staging'

The env section and values are mandatory :

• COMPOSER_AUTH: `{"http-basic":{"repo.magento.com": {"username": "xxxxxxxxxxxxxx", "password": "xxxxxxxxxxxxxx"}}}
• HOST_DEPLOY_PATH: /var/www/myeshop/
• HOST_DEPLOY_PATH_BUCKET : ${{secrets.STAGE_HOST_DEPLOY_PATH}}/bucket or /var/www/myeshop/bucket/
• SSH_PRIVATE_KEY : your ssh key
• SSH_CONFIG : see more adjust the values to match your server (Host must be staging or production)

    Host staging  //this must be staging or production
     User magento 
     IdentityFile ~/.ssh/id_rsa 
     HostName staging.server
     Port 12022
  

• WRITE_USE_SUDO: true or false, the deployer will exec commands as sudo on remote server

The first deploy will fail, unless/then you must place a valid env.php under dir HOST_DEPLOY_PATH/shared/magento/app/etc/ on the deployment endpoint.

A cleanup task must be launched if the deployment fails (see here)

To achieve the deployment using gitlab-ci (follow this tutorial)

Other processes

• install magento from github actions
• install pwa-studio from github actions
• Code quality check
• Magento build
• Magento security scanners
• Unit testing
• Integration tests
• Static testing
• Zero-downtime deployment
• Customize the module
• Setting the secrets
• see more on the forum

Install magento action

One can install magento using github actions. This action will download magento source code and copy it into the github repository calling it. Make sure the repository does not contain the magento directory at the root. You will also need to specify the version. Supported versions 2.2.X, 2.3.X and 2.4.X Or you can simply clone or fork this repository and use it as a template. The use of actions/checkout@v2 is mandatory as v1 is not able to push the src to the repo.

name: m2-install-actions
on: [push]
jobs:
  magento2-install:
    runs-on: ubuntu-latest
    name: 'magento install & push'      
    steps:
    - uses: actions/checkout@v2
    - name: 'install fresh magento and copy to repo'
      uses: MAD-I-T/magento-actions@v3.11
      env:
        COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
      with:
        process: 'create-project'
        magento_version: 2.3.0
#       no_push: 1 //uncomment this to prevent files from getting pushed to repo

Install process in video

To set ${{secrets.COMPOSER_AUTH}} :

1. Go to Settings>Secrets
2. Create variable COMPOSER_AUTH
3. Add you composer auth as value e.g : {"http-basic":{"repo.magento.com": {"username": "xxxxxxxxxxxxxx", "password": "xxxxxxxxxxxxxx"}}}

Install pwa-studio action

One can install magento using github actions. This action will download the latest release of the magento pwa-studio project source code and copy it into the github repository calling it. Or you can simply clone/fork and push the content of this repository to bootstrap a new pwa-studio project. actions/checkout@v2 is mandatory (v2 and up), there is also a video tutorial.

name: pwa-studio-install-actions
on: [push]
jobs:
  magento2-install:
    runs-on: ubuntu-latest
    name: 'install & push pwa-studio project'      
    steps:
    - uses: actions/checkout@v2
    - name: 'install fresh  pwa studio code and copy to repo'
      uses: MAD-I-T/magento-actions@v3.12
      with:
        process: 'pwa-studio-install'
        #no_push: 1 //uncomment this to prevent files from getting pushed to repo

Code quality check

To check some magento module or some code against Magento conding Standard, useful before marketplace submissions

For magento 2.4 and 2.3

- name: 'test some specific module code quality'
  uses: MAD-I-T/magento-actions@v3.11
  env:
    COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
  with:
    php: '7.2'
    process: 'phpcs-test'
    extension: 'Magento/CatalogSearch'
    standard: 'Magento2'
    severity: 10

• extension : the module to be tested (Vendor/Name) or Path using repository scaffolding (i.e from see example here)
• standard : the standard for which the conformity must be checked 'Magento2, PSR2, PSR1, PSR12 etc...' see magento-coding-standard

build an artifact

For magento 2.4.x (remove elasticsearch 1 when building with 2.3.X)

- name: 'This step will build an magento artifact'
  uses: MAD-I-T/magento-actions@v3.11
  env:
    COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
  with:
    process: 'build'
    elasticsearch: 1

For magento 2.3 or lower if issues with preceding sample

- name: 'This step will build an magento artifact'
  uses: MAD-I-T/magento-actions@v2.0
  env:
    COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
  with:
    php: '7.1'
    process: 'build'

• php : 7.1, 7.2, 7.3 or 7.4

To build static content for languagages other than en_US see (https://forum.madit.fr/t/build-magento-from-github-actions-static-deploy-with-multiple-languages/25)

Magento security scanners

Security scan actions must (in case of the modules scanner) be launched after a build step see example here as the magerun module needs the app/etc/config.php file to be present.

To scan the magento 2 files for common vulnerabilities using mwscan, the job's step can be set up as follows

For magento 2.4.x

- name: 'This step will scan the files for security breach'
  if: always()
  uses: MAD-I-T/magento-actions@v3.11
  env:
    COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
  with:
    php: '7.4'
    process: 'security-scan-files'
    elasticsearch: 1
    override_settings: 1

For magento 2.3 or lower

- name: 'This step will scan the files for security breach'
  if: always()
  uses: MAD-I-T/magento-actions@v2.0
  env:
    COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
  with:
    php: '7.1'
    process: 'security-scan-files'

To scan the magento2 installed third parties modules for known vulnerabilities using sansecio/magevulndb, the job's step can be set up as follows:

For magento 2.4.x

- name: 'This step will check all modules for security vulnerabilities'
      if: always()
      uses: MAD-I-T/magento-actions@v3.11
      env:
        COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
      with:
        process: 'security-scan-modules'
        elasticsearch: 1

For magento 2.3 or lower

- name: 'This step will check all modules for security vulnerabilities'
      if: always()
      uses: MAD-I-T/magento-actions@v2.0
      env:
        COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
      with:
        php: '7.1'
        process: 'security-scan-modules'

Example of an output:

unit testing

See code sample here

For magento 2.4.x (remove elasticsearch 1 when building with 2.3.X)

- name: 'This step will execute all the unit tests available'
  uses: MAD-I-T/magento-actions@v3.11
  env:
    COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
  with:
    process: 'unit-test'
    elasticsearch: 1

Run all unit test of the magento email module

- name: 'This step will execute specific unit tests in the path dir'
  uses: MAD-I-T/magento-actions@v3.11
  env:
    COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
  with:
    process: 'unit-test'
    elasticsearch: 1
    unit_test_subset_path: 'vendor/magento/module-email/Test/Unit'

See more about unit testing in magento2

Video tutorial

For magento 2.3 or lower if issues with the preceding sample

- name: 'This step will execute all the unit tests available'
  uses: MAD-I-T/magento-actions@v2.0
  env:
    COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
  with:
    php: '7.1'
    process: 'unit-test'

integration testing

Full sample, the integration test will need rabbitmq (this test will take a while to complete ^^)

magento2-integration-test:
runs-on: ubuntu-latest
container: ubuntu
name: 'm2 integration test'
services:
  mysql:
    image: docker://mysql:8
    env:
      MYSQL_ROOT_PASSWORD: magento
      MYSQL_DATABASE: magento
    options: --health-cmd="mysqladmin ping" --health-interval=10s --health-timeout=5s --health-retries=5 -e MYSQL_ROOT_PASSWORD=magento -e MYSQL_USER=magento -e MYSQL_PASSWORD=magento -e MYSQL_DATABASE=magento --entrypoint sh mysql:8 -c "exec docker-entrypoint.sh mysqld --default-authentication-plugin=mysql_native_password"
  elasticsearch:
    image: docker://elasticsearch:7.1.0
    ports:
      - 9200:9200
    options: -e="discovery.type=single-node" --health-cmd="curl http://localhost:9200/_cluster/health" --health-interval=10s --health-timeout=5s --health-retries=10
  rabbitmq:
    image: docker://rabbitmq:3.8-alpine
    env:
      RABBITMQ_DEFAULT_USER: "magento"
      RABBITMQ_DEFAULT_PASS: "magento"
      RABBITMQ_DEFAULT_VHOST: "/"
    ports:
      - 5672:5672

steps:
  - uses: actions/checkout@v2
    with:
      submodules: recursive
  - name: 'launch magento2 integration test'
    if: ${{false}}
    uses: MAD-I-T/magento-actions@v3.11
    env:
      COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
    with:
      php: '7.4'
      process: 'integration-test'
      elasticsearch: 1

static-test

For magento 2.3 & 2.4

- name: 'This step starts static testing the code'
  uses: MAD-I-T/magento-actions@v3.11
  env:
    COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}}
  with:
    php: '7.2'
    process: 'static-test'

Customize the action

To make all docker build on the runner (no usage of an external image)

For those cloning ...

Replace in action.yml

image: 'docker://mad1t/magento-actions:latest'

by

image: 'Dockerfile'

To override the files in default scripts and config directories without forking

use the override_settings set it to 1 You'll also have to create scripts or config dirs in the root of your m2 project. Example of project scafolding to override the action's default configs

├── .github
│   └── workflows # directory where the workflows are found, see below for an example of main.yml 
├── README.md 
└── magento # directory where you Magento source files should go
└── config # the filenames must be similar to thoses of the action ex: config/integration-test-config.php 
└── scripts #  ex: scripts/build.sh to override the build behaviour 

Video tutorial about overriding

See more about overriding

tipycal issues

• If you're using a self-hosted runner please run it as root user using sudo ./svc.sh install
• Do not forget to set or replace the env.php file in the shared directory
• Adding the ssh user to the http-user group ex. www-data , also check php pool user and group setting rights
• Set WRITE_USE_SUDO env if you want to launch the deployment script in sudo mode (not necessary in most cases)
• integration test when using magento 2.4
  • you will need to set mysql 8 docker with the options arg as such options: --health-cmd="mysqladmin ping" --health-interval=10s --health-timeout=5s --health-retries=5 -e MYSQL_ROOT_PASSWORD=magento -e MYSQL_USER=magento -e MYSQL_PASSWORD=magento -e MYSQL_DATABASE=magento --entrypoint sh mysql:8 -c "exec docker-entrypoint.sh mysqld --default-authentication-plugin=mysql_native_password"
  • see example here

Force composer version

Although the action automatically detects and set the appropriate composer version according to the user's codebase (magento version). One can still force the composer version to be used through the composer_version argument. Possible values (1 or 2)

Set secrets

It is a good practice not to set credentials like composer auth in the code source (see https://12factor.net). So it is advised to use github secret instead of fill the value in the main.yml of your workflow. Example for COMPOSER_AUTH:

1. Go to Settings>Secrets
2. Create variable COMPOSER_AUTH
3. Add you composer auth as value e.g : {"http-basic":{"repo.magento.com": {"username": "xxxxxxxxxxxxxx", "password": "xxxxxxxxxxxxxx"}}}
4. Use as follows COMPOSER_AUTH: ${{secrets.COMPOSER_AUTH}} in the action definition.