Instancetypes and preferences¶
FEATURE STATE:
v1alpha1
(Experimental) as of thev0.56.0
releasev1alpha2
(Experimental) as of thev0.58.0
release- This version now captures complete
VirtualMachine{Instancetype,ClusterInstancetype,Preference,ClusterPreference}
objects within the createdControllerRevisions
- This version is backwardly compatible with
v1alpha1
, no modifications are required to existing instancetypes, preferences or controllerrevisions.
Introduction¶
KubeVirt's VirtualMachine
API contains many advanced options for tuning the performance of a VM that goes beyond what typical users need to be aware of. Users have previously been unable to simply define the storage/network they want assigned to their VM and then declare in broad terms what quality of resources and kind of performance characteristics they need for their VM.
Instancetypes and preferences provide a way to define a set of resource, performance and other runtime characteristics, allowing users to reuse these definitions across multiple VirtualMachines
.
VirtualMachineInstancetype¶
---
apiVersion: instancetype.kubevirt.io/v1alpha2
kind: VirtualMachineInstancetype
metadata:
name: example-instancetype
spec:
cpu:
guest: 1
memory:
guest: 128Mi
KubeVirt provides two Instancetype based CRDs
, a cluster wide VirtualMachineClusterInstancetype
and a namespaced VirtualMachineInstancetype
. These CRDs
encapsulate the following resource related characteristics of a VirtualMachine
through a shared VirtualMachineInstancetypeSpec
:
- CPU : Required number of vCPUs presented to the guest
- Memory : Required amount of memory presented to the guest
- GPUs : Optional list of vGPUs to passthrough
- HostDevices : Optional list of HostDevices to passthrough
- IOThreadsPolicy : Optional IOThreadsPolicy to be used
- LaunchSecurity: Optional LaunchSecurity to be used
Anything provided within an instancetype cannot be overridden within the VirtualMachine
. For example as CPU
and Memory
are both required attributes of an instancetype if a user makes any requests for CPU
or Memory
resources within the underlying VirtualMachine
the instancetype will conflict and the request will be rejected during creation.
VirtualMachinePreference¶
---
apiVersion: instancetype.kubevirt.io/v1alpha2
kind: VirtualMachinePreference
metadata:
name: example-preference
spec:
devices:
preferredDiskBus: virtio
preferredInterfaceModel: virtio
KubeVirt also provides two further preference based CRDs
, again a cluster wide VirtualMachineClusterPreference
and namespaced VirtualMachinePreference
. These CRDs
encapsulate the preferred value of any remaining attributes of a VirtualMachine
required to run a given workload, again this is through a shared VirtualMachinePreferenceSpec
.
Unlike instancetypes preferences only represent the preferred values and as such can be overridden by values in the VirtualMachine
provided by the user.
For example as shown below, if a user has provided a VirtualMachine
with a disk bus already defined within a DiskTarget and has also selected a set of preferences with DevicePreference and preferredDiskBus
defined the users original choice within the VirtualMachine
and DiskTarget are used:
$ cat << EOF | kubectl apply -f -
---
apiVersion: instancetype.kubevirt.io/v1alpha2
kind: VirtualMachinePreference
metadata:
name: example-preference-disk-virtio
spec:
devices:
preferredDiskBus: virtio
---
apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
name: example-preference-user-override
spec:
preference:
kind: VirtualMachinePreference
name: example-preference-disk-virtio
running: false
template:
spec:
domain:
memory:
guest: 128Mi
devices:
disks:
- disk:
bus: sata
name: containerdisk
- disk: {}
name: cloudinitdisk
resources: {}
terminationGracePeriodSeconds: 0
volumes:
- containerDisk:
image: registry:5000/kubevirt/cirros-container-disk-demo:devel
name: containerdisk
- cloudInitNoCloud:
userData: |
#!/bin/sh
echo 'printed from cloud-init userdata'
name: cloudinitdisk
EOF
virtualmachinepreference.instancetype.kubevirt.io/example-preference-disk-virtio created
virtualmachine.kubevirt.io/example-preference-user-override configured
$ virtctl start example-preference-user-override
VM example-preference-user-override was scheduled to start
# We can see the original request from the user within the VirtualMachine lists `containerdisk` with a `SATA` bus
$ kubectl get vms/example-preference-user-override -o json | jq .spec.template.spec.domain.devices.disks
[
{
"disk": {
"bus": "sata"
},
"name": "containerdisk"
},
{
"disk": {},
"name": "cloudinitdisk"
}
]
# This is still the case in the VirtualMachineInstance with the remaining disk using the `preferredDiskBus` from the preference of `virtio`
$ kubectl get vmis/example-preference-user-override -o json | jq .spec.domain.devices.disks
[
{
"disk": {
"bus": "sata"
},
"name": "containerdisk"
},
{
"disk": {
"bus": "virtio"
},
"name": "cloudinitdisk"
}
]
VirtualMachine¶
---
apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
name: example-vm
spec:
instancetype:
kind: VirtualMachineInstancetype
name: example-instancetype
preference:
kind: VirtualMachinePreference
name: example-preference
The previous instancetype and preference CRDs are matched to a given VirtualMachine
through the use of a matcher. Each matcher consists of the following:
- Name (string): Name of the resource being referenced
- Kind (string): Optional, defaults to the cluster wide CRD kinds of
VirtualMachineClusterInstancetype
orVirtualMachineClusterPreference
if not provided - RevisionName (string) : Optional, name of a ControllerRevision containing a copy of the
VirtualMachineInstancetypeSpec
orVirtualMachinePreferenceSpec
taken when theVirtualMachine
is first started. See the Versioning section below for more details on how and why this is captured.
Versioning¶
Versioning of these resources is required to ensure the eventual VirtualMachineInstance
created when starting a VirtualMachine
does not change between restarts if any referenced instancetype or set of preferences are updated during the lifetime of the VirtualMachine
.
This is currently achieved by using ControllerRevision to retain a copy of the VirtualMachineInstancetype
or VirtualMachinePreference
at the time the VirtualMachine
is created. A reference to these ControllerRevisions are then retained in the VirtualMachineInstancetypeMatcher and VirtualMachinePreferenceMatcher within the VirtualMachine
for future use.
$ kubectl.sh apply -f examples/csmall.yaml -f examples/vm-cirros-csmall.yaml
virtualmachineinstancetype.instancetype.kubevirt.io/csmall created
virtualmachine.kubevirt.io/vm-cirros-csmall created
$ kubectl get vm/vm-cirros-csmall -o json | jq .spec.instancetype
{
"kind": "VirtualMachineInstancetype",
"name": "csmall",
"revisionName": "vm-cirros-csmall-csmall-72c3a35b-6e18-487d-bebf-f73c7d4f4a40-1"
}
$ kubectl get controllerrevision/vm-cirros-csmall-csmall-72c3a35b-6e18-487d-bebf-f73c7d4f4a40-1 -o json | jq .
{
"apiVersion": "apps/v1",
"data": {
"apiVersion": "instancetype.kubevirt.io/v1alpha2",
"kind": "VirtualMachineInstancetype",
"metadata": {
"creationTimestamp": "2022-09-30T12:20:19Z",
"generation": 1,
"name": "csmall",
"namespace": "default",
"resourceVersion": "10303",
"uid": "72c3a35b-6e18-487d-bebf-f73c7d4f4a40"
},
"spec": {
"cpu": {
"guest": 1
},
"memory": {
"guest": "128Mi"
}
}
},
"kind": "ControllerRevision",
"metadata": {
"creationTimestamp": "2022-09-30T12:20:19Z",
"name": "vm-cirros-csmall-csmall-72c3a35b-6e18-487d-bebf-f73c7d4f4a40-1",
"namespace": "default",
"ownerReferences": [
{
"apiVersion": "kubevirt.io/v1",
"blockOwnerDeletion": true,
"controller": true,
"kind": "VirtualMachine",
"name": "vm-cirros-csmall",
"uid": "5216527a-1d31-4637-ad3a-b640cb9949a2"
}
],
"resourceVersion": "10307",
"uid": "a7bc784b-4cea-45d7-8432-15418e1dd7d3"
},
"revision": 0
}
$ kubectl delete vm/vm-cirros-csmall
virtualmachine.kubevirt.io "vm-cirros-csmall" deleted
$ kubectl get controllerrevision/controllerrevision/vm-cirros-csmall-csmall-72c3a35b-6e18-487d-bebf-f73c7d4f4a40-1
Error from server (NotFound): controllerrevisions.apps "vm-cirros-csmall-csmall-72c3a35b-6e18-487d-bebf-f73c7d4f4a40-1" not found
Examples¶
Various examples are available within the kubevirt
repo under /examples
. The following uses an example VirtualMachine
provided by the containerdisk/fedora
repo and replaces much of the DomainSpec
with the equivalent instancetype and preferences:
cat << EOF | kubectl apply -f -
---
apiVersion: instancetype.kubevirt.io/v1alpha2
kind: VirtualMachineInstancetype
metadata:
name: cmedium
spec:
cpu:
guest: 1
memory:
guest: 1Gi
---
apiVersion: instancetype.kubevirt.io/v1alpha2
kind: VirtualMachinePreference
metadata:
name: fedora
spec:
devices:
preferredDiskBus: virtio
preferredInterfaceModel: virtio
preferredRng: {}
features:
preferredAcpi: {}
preferredSmm: {}
firmware:
preferredUseEfi: true
preferredUseSecureBoot: true
---
apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
creationTimestamp: null
name: fedora
spec:
instancetype:
name: cmedium
kind: virtualMachineInstancetype
preference:
name: fedora
kind: virtualMachinePreference
runStrategy: Always
template:
metadata:
creationTimestamp: null
spec:
domain:
devices: {}
volumes:
- containerDisk:
image: quay.io/containerdisks/fedora:latest
name: containerdisk
- cloudInitNoCloud:
userData: |-
#cloud-config
users:
- name: admin
sudo: ALL=(ALL) NOPASSWD:ALL
ssh_authorized_keys:
- ssh-rsa AAAA...
name: cloudinit
EOF