monitoring

VirtHandlerDown

Meaning

No running virt-handler pod has been detected for 10 minutes..

The virt-handler runs on every node that can schedule VMIs. It is responsible for domain lifecycle, network configuration, and other node-level operations for virtual machine instances.

Impact

Virtual machine instances (VMIs) on affected nodes cannot be managed properly. New VMIs may not start on nodes without a running virt-handler, and existing VMIs may not receive updates or clean shutdowns.

Diagnosis

  1. Set the NAMESPACE environment variable:

    $ export NAMESPACE="$(kubectl get kubevirt -A -o custom-columns="":.metadata.namespace)"
    
  2. Check the status of the virt-handler DaemonSet and pods:

    $ kubectl -n $NAMESPACE get daemonset virt-handler -o yaml
    $ kubectl -n $NAMESPACE get pods -l kubevirt.io=virt-handler
    
  3. Check DaemonSet events and pod status:

    $ kubectl -n $NAMESPACE describe daemonset virt-handler
    $ kubectl -n $NAMESPACE describe pod -l kubevirt.io=virt-handler
    
  4. Check for node issues (e.g. nodes not ready or taints):

    $ kubectl get nodes
    
  5. If any virt-handler pod exists, review its logs:

    $ kubectl -n $NAMESPACE logs <virt-handler-pod-name> --previous
    $ kubectl -n $NAMESPACE logs <virt-handler-pod-name>
    

Mitigation

Identify why virt-handler pods are down (e.g. DaemonSet not scheduling, pods crashing, node issues, image pull failures) and restore the DaemonSet so virt-handler runs on schedulable nodes.

If you cannot resolve the issue, see the following resources: