monitoring

NoReadyVirtHandler

Meaning

This alert fires when no virt-handler pod in a Ready state has been detected for 10 minutes.

The virt-handler runs on every node that can schedule VMIs (as a DaemonSet). It is responsible for domain lifecycle and node-level operations for virtual machine instances.

Impact

No node has a ready virt-handler. Virtual machine instances cannot be properly managed: domain updates, migrations, and node-level operations will fail or be delayed until at least one virt-handler becomes ready.

Diagnosis

  1. Set the NAMESPACE environment variable:

    $ export NAMESPACE="$(kubectl get kubevirt -A -o custom-columns="":.metadata.namespace)"
    
  2. Check the status of the virt-handler DaemonSet and pods:

    $ kubectl -n $NAMESPACE get daemonset virt-handler
    $ kubectl -n $NAMESPACE get pods -l kubevirt.io=virt-handler -o wide
    
  3. Check DaemonSet and pod events:

    $ kubectl -n $NAMESPACE describe daemonset virt-handler
    $ kubectl -n $NAMESPACE describe pod -l kubevirt.io=virt-handler
    
  4. Review logs of any running but not ready virt-handler pod:

    $ kubectl -n $NAMESPACE logs <virt-handler-pod-name> --previous
    $ kubectl -n $NAMESPACE logs <virt-handler-pod-name>
    
  5. Check for cluster-wide node or scheduling issues:

    $ kubectl get nodes
    

Mitigation

Identify the root cause (e.g. DaemonSet not scheduling, all pods crashing or failing readiness, node or image issues) and restore at least one ready virt-handler pod on a schedulable node.

If you cannot resolve the issue, see the following resources: