joomla-x/joomla-pythagoras

Testing

nibra opened this issue · 2 comments

nibra commented

The idea is to automatically setup the test environements using docker-compose, controlled by a Robo task (see RoboFile.php). The setup should not only serve core development, but also persue extension developers to use the testing environment to improve the quality of their products.

Unit Testing

For unit tests, there's not necessarily needed a separate environment, but it can help anyway (see Isolating PHPUnit).

  • Docker based test setup
  • Improve the current unit testing coverage
    • The overall coverage should at least be at 70%.
    • Coverage of a single package should at least be at 80%.

Integration (Functional) Testing

For integration (functional) tests, we need an installed Joomla, i.e., initialised database, mailcatcher, and so on.

  • Docker based test setup

System (Acceptance) Testing

For system (acceptance) tests, additionally the selenium grid is needed.

  • Docker based test setup for CLI
  • Docker based test setup for API
  • Docker based test setup for Browser
  • Page Objects, see #108

Security Testing

It makes sense to add a security test level using the OWASP Zed Attack Proxy and/or the Vega Vulnerability Scanner.

  • Docker based test setup
Biromain commented

We need extensions can also be tested.
With that, extensions on JED can have a status about unit test, security checked, etc.

  • 👍1
nibra commented

Updated the issue description accordingly.