Kata Containers tests
This repository contains various types of tests and utilities (called "content" from now on) for testing the Kata Containers code repositories.
Getting the code
$ go get -d github.com/kata-containers/tests
Test Content
We provide several tests to ensure Kata-Containers run on different scenarios and with different container managers.
- Functional tests
- Integration tests to ensure compatibility with:
- Network tests
- Stability tests
- Metrics
Note: The unit tests of the different Kata Containers components are stored in each repository along with the source code they test.
CI Content
This repository contains a number of scripts that run from under a "CI" (Continuous Integration) system.
Centralised scripts
The CI scripts in this repository are used to test changes to the content of this repository. These scripts are also used by the other Kata Containers code repositories.
The advantages of this approach are:
-
Functionality is defined once.
- Easy to make changes affecting all code repositories centrally.
-
Assurance that all the code repositories are tested in this same way.
CI setup
WARNING:
The CI scripts perform a lot of setup before running content under a CI. Some of this setup runs as the
root
user and could break your developer's system. See Developer Mode.
Detecting a CI system
The strategy to check if the tests are running under a CI system is to see
if the CI
variable is set to the value true
. For example, in shell syntax:
if [ "$CI" = true ]; then
: # Assumed to be running in a CI environment
else
: # Assumed to NOT be running in a CI environment
fi
Breaking Compatibility
In case the patch you submit breaks the CI because it needs to be tested
together with a patch from another kata-containers
repository, you have to
specify which repository and which pull request it depends on.
Using a simple tag Depends-on:
in your commit message will allow the CI to
run properly. Notice that this tag is parsed from the latest commit of the
pull request.
For example:
Subsystem: Change summary
Detailed explanation of your changes.
Fixes: #nnn
Depends-on:github.com/kata-containers/runtime#999
Signed-off-by: <contributor@foo.com>
In this example, we tell the CI to fetch the pull request 999 from the runtime
repository and use that rather than the master
branch when testing the changes
contained in this pull request.
Developer Mode
Developers need a way to run as much test content as possible locally, but as explained in CI Setup, running all the content in this repository could be dangerous.
The recommended approach to resolve this issue is to set the following variable to any non-blank value before using any content from this repository:
export KATA_DEV_MODE=true
Setting this variable has the following effects:
- Disables content that might not be safe for developers to run locally.
- Ignores the effect of the
CI
variable being set (for extra safety).
You should be aware that setting this variable provides a safe subset of functionality; it is still possible that PRs raised for code repositories will still fail under the automated CI systems since those systems are running all possible tests.
Run the Kata Containers tests
Requirements to run Kata Containers tests
You need to install the following to run Kata Containers tests:
-
To view the versions of go known to work, see the
golang
entry in the versions database. -
make
.
Prepare an environment
The recommended method to set up Kata Containers is to use the official and latest stable release. You can find the official documentation to do this in the Kata Containers installation user guides.
To try the latest commits of Kata use the CI scripts, which build and install from the kata-containers repositories, with the following steps:
Warning: This may replace/delete packages and configuration that you already have. Please use these steps only on a testing environment.
Add the $GOPATH/bin
directory to the PATH:
$ export PATH=${GOPATH}/bin:${PATH}
Clone the kata-container/tests
repository:
$ go get -d github.com/kata-containers/tests
Go to the tests repo directory:
$ cd $GOPATH/src/github.com/kata-containers/tests
Execute the setup script:
$ .ci/setup.sh
Limitation: If the script fails for a reason and it is re-executed, it will execute all steps from the beginning and not from the failed step.
Run the tests
If you have already installed the Kata Containers packages and a container manager (i.e. Docker or Kubernetes), and you want to execute the content for all the tests, run the following:
$ export RUNTIME=kata-runtime
$ export KATA_DEV_MODE=true
$ sudo -E PATH=$PATH make test
You can also execute a single test suite. For example, if you want to execute the docker integration tests, run the following:
$ sudo -E PATH=$PATH make docker