Manatoko (Maori for verify, test) is a new approach to test the HAL management console. It builds on top of

The goal is that tests should be self-contained. Containers are started and stopped when necessary and tests can focus on testing the UI and verifying management model changes. The biggest advantage of this approach is that it is very easy to run UI tests in a CI environment.

TOC

Write Tests
- Operation Mode
Test Environment
- Remote
- Local
Run Tests
- Run All Tests
- Run Specific Tests
Test Report
Images
Modules
Scripts

Write Tests

Tests need to be annotated with two annotations (in this order!):

@Manatoko: This annotation activates two Junit 5 extensions:
- SystemSetupExtension: Takes care of starting / stopping singleton containers once before / after all tests.
- ArquillianExtension: Takes care about the Arquillian integration
@Testcontainers: Takes care of starting containers marked with @Container.

A simple test that tests adding a new system property looks like this:

@Manatoko
@Testcontainers
class SystemPropertyTest {

   @Container static WildFlyContainer wildFly = WildFlyContainer.standalone(DEFAULT);

   @Page SystemPropertyPage page;
   @Inject CrudOperations crud;
   TableFragment table;
   FormFragment form;

   @BeforeEach
   void prepare() {
      page.navigate();
      form = page.getForm();
      table = page.getTable();
      table.bind(form);
   }

   @Test
   void create() {
      crud.create(systemPropertyAddress(CREATE_NAME), table, form -> {
         form.text(NAME, CREATE_NAME);
         form.text(VALUE, CREATE_VALUE);
      });
   }
}

See SystemPropertyTest for the full code.

Operation Mode

The example injects a WildFly standalone container using the default configuration (standalone.xml). The enum WildFlyConfiguration lists all supported configurations for standalone mode.

If you want to use the domain mode, inject the WildFly container using

@Container static WildFlyContainer wildFly = WildFlyContainer.domain();

Tests which run in domain mode, make certain assumptions (like the default host). These assumptions rely on the properties defined in domain.properties. You can customize these properties using -D<key>=<value>.

Test Environment

Tests can be run in two modes, controlled by the system property test.environment. Valid values are either local or remote.

Remote

This is the default mode. In this mode a web driver container (with support of screen recording) is started before all tests. An Arquillian extension is registered which provides a remote web driver connected to the browser running in this container.

Local

This mode is activated by the maven profile local. In this mode a browser is started locally and Arquillian Graphene takes care of providing the web driver.

Images

By default, the tests will use the following images:

HAL: quay.io/halconsole/hal:latest
WildFly standalone: quay.io/halconsole/wildfly:latest
WildFly domain: quay.io/halconsole/wildfly-domain:latest

The image names can be customized by overriding the properties defined in container.properties. Say you want to use the latest development version of HAL instead of the latest stable release, then use

./mvnw test -P all-tests -Dhal.image=quay.io/halconsole/hal-development:latest

If you want to use a specific WildFly version for the tests, use

./mvnw test -P all-tests \
  -Dwildfly.standalone.image=quay.io/halconsole/wildfly:23.0.0.Final \
  -Dwildfly.domain.image=quay.io/halconsole/wildfly-domain:23.0.0.Final

Run Tests

Run All Tests

To run all tests, simply execute

./mvnw test -P all-tests

To run the tests with a local browser, use

./mvnw test -P all-tests,local

Run Specific Tests

If you just want to execute tests of one specific module e.g. test-configuration-systemproperty, run

./mvnw test -P all-tests --projects test-configuration-systemproperty --also-make

If you want to execute one specific test or test method, use one of the following commands:

./mvnw test -P all-tests --projects test-configuration-systemproperty --also-make \
    -Dtest=org.jboss.hal.testsuite.configuration.systemproperty.SystemPropertyTest
  
./mvnw test -P all-tests --projects test-configuration-systemproperty --also-make \
    -Dtest=org.jboss.hal.testsuite.configuration.systemproperty.SystemPropertyTest#create

If you want to debug a test, append -Dmaven.surefire.debug and attach a debugger to port 5005.

Test Report

All tests are executed every 24h and the results are aggregated by the maven-surefire-report-plugin. The report is then published to GitHub pages: https://hal.github.io/manatoko/surefire-report.html

Modules

Manatoko consists of many modules. Each module has a distinct responsibility. Here's an overview of the modules and its dependencies:

manatoko-environment: Singleton to manage the test environment (local or remote)
manatoko-management-model: Classes for working with the management model and JBoss DMR
manatoko-container: Classes to start / stop the test containers
manatoko-junit: Annotations and Junit extensions for writing and running the unit tests
manatoko-ui: Arquillian fragments and pages
manatoko-fixture: Constants and test fixtures used by the unit tests
manatoko-arquillian: Arquillian extension for the integration with Testcontainers
manatoko-command: Creaper commands to create various management resources
manatoko-test-noop: Sample test module to verify the Arquillian and Testcontainers wiring
manatoko-test-configuration-*: Configuration tests
manatoko-test-runtime-*: Runtime tests
manatoko-test-deployment-*: Deployment tests

Scripts

This repository contains various scripts to automate tasks.

`depgraph.sh`

Creates the dependency graph of the modules.

`format.sh`

Formats the codebase by applying the following maven goals:

The goals use the plugin configuration in code-parent/pom.xml and the resources in build-config/src/main/resources/manatoko.

`validate.sh`

Validates the codebase by applying the following maven goals:

The goals use the plugin configuration in code-parent/pom.xml and the resources in build-config/src/main/resources/manatoko.

`gh-test-all.sh`

Runs all tests in all test modules. This script triggers the workflow defined in test-all.yml. The tests are run in parallel, but please note that this might take some time.

The script uses the latest stable HAL release by default. If you want to use the latest snapshot, please specify --development.

The script requires GitHub CLI to be present and configured on your machine.

`gh-test-single.sh`

Runs the tests of a single test module given as an argument. This script triggers the workflow defined in test-single.yml.

The script uses the latest stable HAL release by default. If you want to use the latest snapshot, please specify --development.

The script requires GitHub CLI to be present and configured on your machine.

`tcpm.sh`

If you're using testcontainers with Podman 3.x on macOS, please start ./tcpm.sh and make sure to set the following environment variables before running the tests.

DOCKER_HOST=unix:///tmp/podman.sock
TESTCONTAINERS_CHECKS_DISABLE=true
TESTCONTAINERS_RYUK_DISABLED=true

See https://www.testcontainers.org/features/configuration/ and testcontainers/testcontainers-java#2088 (comment) for details.

For Podman 4.x, tcpm.sh is not necessary. You can either call

sudo podman-mac-helper install

or set

export DOCKER_HOST='unix:///~/.local/share/containers/podman/machine/podman-machine-default/podman.sock'

hpehl/manatoko