Metrics-load-test
This document covers the steps to use opensource avalanche tool which generate lots of merics for load testing. Avalanche serves a text-based Prometheus metrics endpoint.
Avalanche binary can be run from a local server or it can be run as a container.
How to run test on K8s/Openshift cluster
Prequesite
- Must have a Kubernetes/OpenShift cluster
- Deploy dynatrace component on Kubernetes/OpenShift cluster ref doc
- Note: ActiveGate version 1.217+ running inside the Kubernetes/openshift cluster
- Connect your Kubernetes/OpenShift cluster to Dynatrace ref doc
- Configure Kubernetes cluster settings ref doc
- In Dynatrace, go to your Kubernetes cluster settings page and enable
- Monitor Kubernetes namespaces, services, workloads, and pods
- Monitor annotated Prometheus exporters
- In Dynatrace, go to your Kubernetes cluster settings page and enable
Create avalanche service
kubectl create -f https://raw.githubusercontent.com/a7vicky/metrics-load-test/main/manifest.yamlNote: Yaml file has conatiner argument and replica count which can be change to generate the count of metrics per pod and also other paramneter describe below which can be changed based on requirement.
Tweaking configuration to generate load
| Container argument | Details |
|---|---|
| --metric-count= | Number of metrics to serve. |
| --label-count= | Number of labels per-metric. |
| --series-count= | Number of series per-metric. |
| --metricname-length= | Modify length of metric names. |
| --labelname-length= | Modify length of label names |
| --value-interval=30 | Change series values every {interval} seconds. |
| --series-interval=60 | Change series_id label values every {interval} seconds. |
| --metric-interval=120 | Change name label values every {interval} seconds. |
Unknowns
This document does not talk about upper limit of ActiveGate that how much metrics can ActiveGate scrap from avalanche service and successfully forward it to Dynatrace tenant (without dropping). ActiveGate configuration tweaking might be needed.