Control libvirt logging for each component

Generally, cluster admins can control the log verbosity of each KubeVirt component in KubeVirt CR. For more details, please, check the KubeVirt documentation.

Nonetheless, regular users can also adjust the qemu component logging to have a finer control over it. The annotation kubevirt.io/libvirt-log-filters enables you to modify each component's log level.

Example:

apiVersion: kubevirt.io/v1
kind: VirtualMachineInstance
metadata:
  annotations:
    kubevirt.io/libvirt-log-filters: "2:qemu.qemu_monitor 3:*"
  labels:
    special: vmi-debug-tools
  name: vmi-debug-tools
spec:
  domain:
    devices:
      disks:
      - disk:
          bus: virtio
        name: containerdisk
      - disk:
          bus: virtio
        name: cloudinitdisk
      rng: {}
    resources:
      requests:
        memory: 1024M
  volumes:
  - containerDisk:
      image: registry:5000/kubevirt/fedora-with-test-tooling-container-disk:devel
    name: containerdisk
  - cloudInitNoCloud:
      userData: |-
        #cloud-config
        password: fedora
        chpasswd: { expire: False }
    name: cloudinitdisk

Then, it is possible to obtain the logs from the virt-launcher output:

$ kubectl get pods
NAME                                  READY   STATUS    RESTARTS   AGE
virt-launcher-vmi-debug-tools-fk64q   3/3     Running   0          64s
$ kubectl  logs virt-launcher-vmi-debug-tools-fk64q
[..]
{"component":"virt-launcher","level":"info","msg":"QEMU_MONITOR_RECV_EVENT: mon=0x7faa8801f5d0 event={\"timestamp\": {\"seconds\": 1698324640, \"microseconds\": 523652}, \"event\": \"NIC_RX_FILTER_CHANGED\", \"data\": {\"name\": \"ua-default\", \"path\": \"/machine/peripheral/ua-default/virtio-backend\"}}","pos":"qemuMonitorJSONIOProcessLine:205","subcomponent":"libvirt","thread":"80","timestamp":"2023-10-26T12:50:40.523000Z"}
{"component":"virt-launcher","level":"info","msg":"QEMU_MONITOR_RECV_EVENT: mon=0x7faa8801f5d0 event={\"timestamp\": {\"seconds\": 1698324644, \"microseconds\": 165626}, \"event\": \"VSERPORT_CHANGE\", \"data\": {\"open\": true, \"id\": \"channel0\"}}","pos":"qemuMonitorJSONIOProcessLine:205","subcomponent":"libvirt","thread":"80","timestamp":"2023-10-26T12:50:44.165000Z"}
[..]
{"component":"virt-launcher","level":"info","msg":"QEMU_MONITOR_RECV_EVENT: mon=0x7faa8801f5d0 event={\"timestamp\": {\"seconds\": 1698324646, \"microseconds\": 707666}, \"event\": \"RTC_CHANGE\", \"data\": {\"offset\": 0, \"qom-path\": \"/machine/unattached/device[8]\"}}","pos":"qemuMonitorJSONIOProcessLine:205","subcomponent":"libvirt","thread":"80","timestamp":"2023-10-26T12:50:46.708000Z"}
[..]

The annotation enables the filter from the container creation. However, in certain cases you might desire to change the logging level dynamically once the container and libvirt have already been started. In this case, virt-admin comes to the rescue.

Example:

$ kubectl get pods
NAME                                READY   STATUS    RESTARTS   AGE
virt-launcher-vmi-ephemeral-nqcld   3/3     Running   0          26m
$ kubectl  exec -ti virt-launcher-vmi-ephemeral-nqcld -- virt-admin  -c virtqemud:///session  daemon-log-filters  "1:libvirt 1:qemu 1:conf 1:security 3:event 3:json 3:file 3:object 1:util"
$ kubectl  exec -ti virt-launcher-vmi-ephemeral-nqcld -- virt-admin  -c virtqemud:///session  daemon-log-filters
 Logging filters: 1:*libvirt* 1:*qemu* 1:*conf* 1:*security* 3:*event* 3:*json* 3:*file* 3:*object* 1:*util*

Otherwise, if you prefer to redirect the output to a file and fetch it later, you can rely on kubectl cp to retrieve the file. In this case, we are saving the file in the /var/run/libvirt directory because the compute container has the permissions to write there.

Example:

$ kubectl get pods
NAME                                READY   STATUS    RESTARTS   AGE
virt-launcher-vmi-ephemeral-nqcld   3/3     Running   0          26m
$ kubectl  exec -ti virt-launcher-vmi-ephemeral-nqcld -- virt-admin  -c virtqemud:///session daemon-log-outputs "1:file:/var/run/libvirt/libvirtd.log"
$ kubectl cp virt-launcher-vmi-ephemeral-nqcld:/var/run/libvirt/libvirtd.log libvirt-kubevirt.log
tar: Removing leading `/' from member names