Interacting with our test-infra setup is currently done exclusively through the releases repository. On this repository you can trigger new test runs through comments on Pull Requests.
Please be aware that all interactions with test-infra are designed with the idea in mind that a pull request will either modify or add a single release in the releases repository! Commands which are currently available are as follows:
/test cncf
runs thecncf
suite against the release./test cis
runs thecis
suite against the release./test aws
runsawscnfm
against the release - this only works for AWS.
There are multiple dashboards which can be useful to track the progress of tests:
- prow gives a rough overview of the running tests.
- tekton gives a more in depth overview of the test progress,
- kibana gives full insight of the logs emitted during your test.
Each test will generally perform the same flow currently which can be summarized as this:
- Determine release CR from PR diff.
- Create release CR on provider CP (
gaia
,godsmack
,gorgoth
). - Create tenant cluster with created release on target CP.
- Run tests against created tenant cluster.
- Report test results.
- Delete tenant cluster.
- Delete release on CP.
Feel free to reach out to team ludacris if you have further questions.
You can find guides for the creation of new tests in the docs
subfolder.
A lot of functionality of tekton is available locally - the biggest exceptions being anything requiring an Ingress of webhooks.
Tools required for local development:
- KIND
- Kustomize
- Kubectl
- Giant Swarm internal tooling for secrets
To setup tekton in a KIND cluster you can simply do the following:
- Create a cluster with
kind create cluster
- Apply all necessary files with
./tekton-kind.sh
Hack away!
Deploying test-infra
is currently a bit cumbersome as it is running as an app inside a tenant cluster.
The tenant cluster is rfjh2
on gorilla
.
To update test-infra
you will need to manually edit the app
CR on the gorilla
CP in the namespace rfjh2
.
There is currently no automatic rollout by merging to master!
The roles
and serviceaccounts
on the target CPs are currently created manually.
You can find the resources here.
Kubeconfigs to the target CPs are currently generated with a script using opsctl
and then manually updated to rfjh2
.
You can find the necessary scripts here.
This repository contains manifests for our individual test pipelines as well as the components we run to build up our testing infrastructure. The two main open source projects used are Prow and Tekton
Prow is a CI/CD system running on Kubernetes. We utilize prow to trigger tests and have a nice Github integration to run high cost conformance tests.
This is a list of components which we run:
Deck
- UI for prow jobs.Sinker
- Clean up stale prow jobs.Hook
- Handle GitHub events dispatching them to plugins.Horologium
- Start periodic jobs.Plank
- Start prow jobs.ghProxy
- Designed to reduce Github API token usage.Tide
- Automatically retest PRs & merge them if tests go green.Crier
- Reports your prowjob status changes.Pipelines
- Automatically creates Tekton pipelines for each Prow job.
Tekton is a CI/CD system which is highly integrated with Kubernetes. We utilize it to actually run tests and to structure our test stages.
This is a list of components which we run:
Pipelines
- This enables the meat of functionalities by Tekton.Dashboard
- Gives and overview of active Tekton tasks and pipelines.prow-log-aggregator
- Collects logs from Tekton tasks and makes them available to Deck.
- How to write
ProwJobs