monitoring

NoReadyVirtController

Meaning

This alert fires when no available virt-controller devices have been detected for 5 minutes.

The virt-controller devices monitor the custom resource definitions of virtual machine instances (VMIs) and manage the associated pods. The devices create pods for VMIs and manage the lifecycle of the pods.

Therefore, virt-controller devices are critical for all cluster-wide virtualization functionality.

Impact

Any actions related to VM lifecycle management fail. This notably includes launching a new VMI or shutting down an existing VMI.

Diagnosis

  1. Set the NAMESPACE environment variable:

    $ export NAMESPACE="$(kubectl get kubevirt -A -o custom-columns="":.metadata.namespace)"
    
  2. Verify the number of virt-controller devices:

    $ kubectl get deployment -n $NAMESPACE virt-controller -o jsonpath='{.status.readyReplicas}'
    
  3. Check the status of the virt-controller deployment:

    $ kubectl -n $NAMESPACE get deploy virt-controller -o yaml
    
  4. Obtain the details of the virt-controller deployment to check for status conditions such as crashing pods or failure to pull images:

    $ kubectl -n $NAMESPACE describe deploy virt-controller
    
  5. Obtain the details of the virt-controller pods:

    $ get pods -n $NAMESPACE | grep virt-controller
    
  6. Check the logs of the virt-controller pods for error messages:

    $ kubectl logs -n $NAMESPACE <virt-controller>
    
  7. Check the nodes for problems, suchs as a NotReady state:

    $ kubectl get nodes
    

Mitigation

Based on the information obtained during the diagnosis procedure, try to identify the root cause and resolve the issue.

If you cannot resolve the issue, see the following resources: