Testing

Sanity and unit tests

Sanity and unit tests can be run in two ways:

• with tox and the tox-ansible plugin (both need to be installed on the dev machine)

• with ansible-test

For development purposes tox and tox-ansible are better suited, as they allow debugging of issues in the collection on the developer’s machine.

For verification purposes of the collection the compatibility with ansible-test is ensured.

Running tests with tox-ansible

Run sanity tests with tox-ansible like so:

make test-sanity

Run unit tests with tox-ansible like so:

make test-unit

Running tests with ansible-test

In order to test changes with ansible-test, it is recommended to bind mount the repository to ~/.ansible/collections/ansible_collections/kubevirt/core if you did not check it out into this location.

This can be done with:

mkdir -p ~/.ansible/collections/ansible_collections/kubevirt/core
sudo mount --bind <project_dir>/kubevirt.core ~/.ansible/collections/ansible_collections/kubevirt/core
cd ~/.ansible/collections/ansible_collections/kubevirt/core

Run sanity tests with ansible-test like so:

ANSIBLE_TEST_PREFER_PODMAN=1 ansible-test sanity --docker

Run unit tests with ansible-test like so:

ANSIBLE_TEST_PREFER_PODMAN=1 ansible-test units --docker

Running tests with coverage analysis

Coverage analysis produces a report about the code paths covered by the unit tests.

It is required to bind mount the project as shown in section Running tests with ansible-test.

Run ansible-test with:

ANSIBLE_TEST_PREFER_PODMAN=1 ansible-test units --docker --coverage

Create the coverage report in HTML format using:

ansible-test coverage html

Or in XML format using:

ansible-test coverage xml

Multiple coverage report formats are supported such as: plain text, XML or HTML. If HTML format is used as shown above, the report can be accessed in the following location: ~/.ansible/collections/ansible_collections/kubevirt/tests/output/reports/coverage/index.html or in ~/.ansible/collections/ansible_collections/kubevirt/tests/output/reports/coverage/coverage.xml if XML format is choosen.

Integration tests

Integration tests require a working cluster and can be run with ansible-test.

Continuous integration

The hack/e2e-setup.sh script contains the steps necessary to reproduce the CI integration test environment, which relies on docker or podman, kubectl and kind.

To create the CI integration test environment on your machine run:

make cluster-up

To remove the test CI integration test environment run:

make cluster-down

Running integration tests with ansible test

Run integration tests with ansible-test like so:

ansible-test integration

Example config and playbooks

Sample playbooks and inventory configurations are provided in the examples/ directory; to run the playbooks locally, build the testing environment with the script above, then the steps are as follows:

# setup environment
pip install ansible-core
# clone the repository
git clone https://github.com/kubevirt/kubevirt.core
cd kubevirt.core
# install collection dependencies
ansible-galaxy collection install -r requirements.yml
# install collection python deps
pip install -r requirements.txt
# setup environment
hack/e2e-setup.sh
# run inventory source
ansible-inventory -i examples/inventory.kubevirt.yml
# create a virtual machine
ansible-playbook -i examples/inventory.kubevirt.yml examples/play-create-min.yml
# terminate a virtual machine
ansible-playbook -i examples/inventory.kubevirt.yml examples/play-delete.yml
# terminate the environment
hack/e2e-setup.sh --cleanup