bcachefs-tools/checks
Thomas Mühlbacher 3e50eea605 test(nix): add basic functionality testing infra
Use nixos-test scaffolding to spawn VMs and run any scripts in
`checks/*.sh` as tests inside of the VM. This should lessen the risk of
any obvious breakage in the CI but needs more and better tests to cover
more commands.

Signed-off-by: Thomas Mühlbacher <tmuehlbacher@posteo.net>
2024-06-30 13:01:18 +02:00
..
default.nix test(nix): add basic functionality testing infra 2024-06-30 13:01:18 +02:00
encrypted-multidev.sh test(nix): add basic functionality testing infra 2024-06-30 13:01:18 +02:00
encrypted-unlock.sh test(nix): add basic functionality testing infra 2024-06-30 13:01:18 +02:00
outputs.sh test(nix): add basic functionality testing infra 2024-06-30 13:01:18 +02:00
README.md test(nix): add basic functionality testing infra 2024-06-30 13:01:18 +02:00
subvolume.sh test(nix): add basic functionality testing infra 2024-06-30 13:01:18 +02:00

NixOS tests

Any *.sh file in this directory will be run in a NixOS VM for basic functionality testing as part of CI. To list all outputs, including the checks, you can use this command:

$ nix flake show

You can also run these tests locally by running nix flake check. To run one specific test you can use nix build like this:

$ nix build ".#checks.x86_64-linux.subvolume"

With the flag -L/--print-build-logs outputs are shown fully as checks are executing. Additionally, if the specific check has already been run locally, you can view the log for the check or force another run with the following:

$ nix log .#checks.x86_64-linux.subvolume
$ nix build --rebuild .#checks.x86_64-linux.subvolume

If you need any more packages inside of the VM for a test, you can add them to environment.systemPackages in default.nix. If you're unsure about the package you need, NixOS package search may be able to help.

For more information about the NixOS testing library see the testing wiki article.

Kernel version inside VM

By default linuxPackages_latest from nixpkgs is used in the testing VM. This is the latest stable kernel version available in the nixpkgs revision. Updating the nixpkgs flake input may update the used kernel. A custom-built kernel can be used as well but with added build times in CI.

Adding new tests

The easiest way to add new tests is of course to copy an existing test and adapt it accordingly. Importantly, for nix to see a file as part of the sources, the file needs to be in the git index. It doesn't have to be committed to the repo just yet but you need to git add it. If git ls-files lists the file, nix will also see it.

Interactive debugging of tests

When writing a new test or experiencing a difficult to understand test failure, an interactive login can be very handy. This can be achieved by building the driverInteractive attribute of the check, for example like this:

$ nix build .#checks.x86_64-linux.subvolume.driverInteractive

The nix build will create a symlink in your working directory called result which leads to a script that launches the VM interactively:

$ ./result/bin/nixos-test-driver

There is more information about this in the NixOS manual under running tests interactively.