KubeVirt must-gather

must-gather is a tool built on top of OpenShift must-gather that expands its capabilities to gather KubeVirt information.

Usage

oc adm must-gather --image=quay.io/kubevirt/must-gather

The command above will create a local directory with a dump of the KubeVirt state. Note that this command will only get data related to the KubeVirt part of the OpenShift cluster.

You will get a dump of:

The Hyperconverged Cluster Operator namespaces (and its children objects)
All namespaces (and their children objects) that belong to any KubeVirt resources
All KubeVirt CRD's definitions

By default, the VMs definitions won't be included, but only the VM Instances' custom resources.

In order to get data about other parts of the cluster (not specific to KubeVirt) you should run oc adm must-gather (without passing a custom image). Run oc adm must-gather -h to see more options.

Parallelism

Some gathering activity can be done in parallel. Collecting resources one by one may be slow, and collecting too many resources in parallel may fail. By default, 5 processes are running in parallel, and the rest of the processes are waiting for running processes to complete. It is possible to change this default number of processes by setting the PROS environment variable, but then, the default command must be specified as well, like this:

oc adm must-gather \
   --image=quay.io/kubevirt/must-gather \
   -- PROS=7 \
   /usr/bin/gather

Targeted gathering - VM information

To collect the VM information, and all the namespaces that contains VMs call directly the gather_vms_details command:

oc adm must-gather \
   --image=quay.io/kubevirt/must-gather \
   -- /usr/bin/gather_vms_details

VMs in a Namespace

The gather_vms_details command supports targeted gathering. By specifying a namespace, the command will only collect the VMs in this namespace. For example, collecting all the VM information in namespace "vm1":

oc adm must-gather \
   --image=quay.io/kubevirt/must-gather \
   -- NS=ns1 \
   /usr/bin/gather_vms_details

Specific VM

By specifying the VM name in addition to the namespace, the gather_vms_details command will only collect the specific VM information. For example, collecting the information of a specific VM called "testvm" in namespace "vm1":

oc adm must-gather \
   --image=quay.io/kubevirt/must-gather \
   -- NS=ns1 \
   VM=testvm \
   /usr/bin/gather_vms_details

Note: When collecting information for a specific VM, you must specify the namespace as well. Without the namespace, the gather_vms_details command exits and prints an error message.

List of Specific VMs

The VM environment variable can also be a comma-seperated list of VM names (without a space). For example:

oc adm must-gather \
   --image=quay.io/kubevirt/must-gather \
   -- NS=ns1 \
   VM="testvm1,testvm34,testvm52,testvm74" \
   /usr/bin/gather_vms_details

Gather VM by Regex Expression

The gather_vms_details command also support gathering VM with regex expression.

For example, suppose we have the following VMs in the cluster:

testvm1-1 testvm1-2 testvm1-3 testvm1-4 testvm1-5  
testvm1-6 testvm1-7 testvm1-8 testvm1-9 testvm1-10
testvm2-1 testvm2-2 testvm2-3 testvm2-4 testvm2-5 
testvm2-6 testvm2-7 testvm2-8 testvm2-9 testvm2-10
testvm3-1 testvm3-2 testvm3-3 testvm3-4 testvm3-5
testvm3-6 testvm3-7 testvm3-8 testvm3-9 testvm3-10
testvm4-1 testvm4-2 testvm4-3 testvm4-4 testvm4-5
testvm4-6 testvm4-7 testvm4-8 testvm4-9 testvm4-10
testvm5-1 testvm5-2 testvm5-3 testvm5-4 testvm5-5 
testvm5-6 testvm5-7 testvm5-8 testvm5-9 testvm5-10

If we want to read only VMs that starts with testvm2, testvm3 or testvm4, and that their postfix number is odd, we can use this regex expression to for that: ^testvm[2-4]-[0-9]*[1,3,5,7,9]$.

Here is how to use it in the gather_vms_details command, to search VMs by regex:

oc adm must-gather \
   --image=quay.io/kubevirt/must-gather \
   VM_EXP="^testvm[2-4]-[0-9]*[1,3,5,7,9]$" \
   /usr/bin/gather_vms_details

Here is how to use it in the gather_vms_details command, to search VMs by regex in the ns1 namespace:

oc adm must-gather \
   --image=quay.io/kubevirt/must-gather \
   -- NS=ns1 \
   VM_EXP="^testvm[2-4]-[0-9]*[1,3,5,7,9]$" \
   /usr/bin/gather_vms_details

Note: When collecting information using the VM variable, the command will ignore the VM_EPR variable. Do not use both of them together.

Targeted gathering - Images information

It is possible to collect image, image-stream and image-stream-tags information using the gather_images command:

oc adm must-gather --image=quay.io/kubevirt/must-gather -- /usr/bin/gather_images

The gather_vms_details and the gather_images commands support parallelism as well. To change the default number of processes of 5, add the PROS environment variable. This is only works when not using the NS environment variable:

oc adm must-gather --image=quay.io/kubevirt/must-gather -- PROS=7 /usr/bin/gather_vms_details

oc adm must-gather --image=quay.io/kubevirt/must-gather -- PROS=3 /usr/bin/gather_images

Development

You can build the image locally using the Dockerfile included.

A makefile is also provided. To use it, you must pass a repository via the command-line using the variable MUST_GATHER_IMAGE. You can also specify the registry using the variable IMAGE_REGISTRY (default is quay.io) and the tag via IMAGE_TAG (default is latest).

The targets for make are as follows:

build: builds the image with the supplied name and pushes it
docker-build: builds the image but does not push it
docker-push: pushes an already-built image

For example:

make build MUST_GATHER_IMAGE=kubevirt/must-gather

would build the local repository as quay.io/kubevirt/must-gather:latest and then push it.

lyarwood/must-gather