Quality
How images are tested
Every image follows the same pipeline, in public CI, each time upstream releases a new version. Any failing step stops the release. This page lists what is checked and what is not checked yet.
1. Upstream source
We start from what the distribution itself publishes: its official cloud image, or its official sources for the images we build from scratch (FreeBSD, Gentoo, NixOS, Octavia). The download is verified against the checksum list the project publishes (SHA-256 or SHA-512), or against its signing key. A missing entry or a mismatch fails the build.
2. Build
Changes are kept to what a cloud image needs, the same way for every image:
- one cloud-init policy: OpenStack and ConfigDrive datasources, root login disabled, password authentication disabled;
- QEMU guest agent, serial console, security updates where the distribution expects it;
- cleanup: machine id, SSH host keys, logs and package caches are removed so that every instance starts fresh;
- sparse, compressed qcow2.
Each image documents its own adjustments in its repository, for example a workaround for a known upstream bug.
3. Boot test
Before signing, images built with libguestfs are booted in QEMU/KVM with a ConfigDrive seed: the CI logs in over SSH with a generated key and checks that cloud-init did not end in error. A failure stops the release before anything is published.
4. Signature and provenance
The image is signed with Sigstore cosign in keyless mode by the build workflow itself, and the signature is recorded in the public Rekor log. A manifest records the version, checksums (SHA-256 and SHA-512), sizes, firmware, source commit, CI run and builder. How to verify an image.
5. Publication checks
Right after the upload, the CI downloads the image back through the public CDN, like a user would, and checks the manifest fields, the checksum files, the cache headers, the size, and the cosign signature.
6. Test on a real OpenStack cloud
Each release is then booted on Infomaniak Public Cloud, in one of two regions picked at random, through the same steps as a user: Glance import, instance creation, SSH login.
| Check | Level | |
|---|---|---|
| Upload and boot | Imported into Glance, booted with a standard flavor (upsized when the image needs a bigger disk). | Required |
| SSH with the injected key | Login as the documented default user with the key from the cloud metadata. | Required |
| cloud-init finished cleanly | cloud-init status is done; on error the test prints its warnings and tracebacks. | Required |
| DNS | The guest resolves public names with the resolvers it received. | Required |
| No password login over SSH | Asked from outside: the server must not offer password authentication. | Required |
| Hostname from metadata | The guest takes the instance name given by OpenStack. | Required |
| Disk grown to the flavor | Partition and filesystem (or ZFS pool) extended to the root disk of the flavor. | Required |
| Administrator access | The default user becomes root without a password (sudo or doas). | Required |
| IPv6 | The guest configures the IPv6 address Neutron assigned to its port and reaches the IPv6 Internet. | Required |
Every check above is required: a release that fails one of them is not considered good. Waits are generous enough for a slow cloud (DHCP leases over 2 minutes were measured in one region), so a failure means the image is wrong, not that the cloud was slow.
Known limit: this test runs right after publication, so a failing release is visible for the time it takes to fix and republish it. Running it before the latest/ alias is updated is on the roadmap.
7. Afterwards
- Daily: each image watches its upstream and proposes a new release as soon as one is published.
- Weekly: an audit downloads every published image again and re-checks its manifest, checksums and signature; any problem opens a public issue.
The exact rules live in the image standards, and every run is public on GitHub.