nordisk/nornir

Viz snapshot testing — screenshot PROOF for every UI test

THE_BIG_PLAN cross-cutting LAW #4: every UI test renders an actual PNG of the success state as visual proof. This document is the design + the guide for tests/viz_snapshots.rs.

What it is

tests/viz_snapshots.rs renders the real nornir viz (nornir-ui) egui app to a committed PNG for a few key tabs, and asserts the rendered state is correct. It is the third leg of the viz test tripod:

Each snapshot test carries two proofs (LAW #1 — inject-and-assert, never no-crash):

1. Functional proof — a concrete Timeline with real, non-zero data is injected, then state_json() is asserted to carry the expected counts (releases, dep-graph edges, repos, bench points).
2. Visual proof — the frame is rendered on the GPU through egui_kittest's wgpu renderer and the PNG is written to tests/snapshots/viz_<tab>.png.

How the headless render works

egui_kittest 0.33 (built with features = ["wgpu", "snapshot"]) drives the egui app with no window:

UrdrThreadsApp::with_repos(empty warehouse, "holger", …)
   └─ inject_timeline_for_test(tab, fixture_timeline(), repos)   # real data, splash aged out
Harness::builder()
   .with_size(1400×900)
   .wgpu()                                                       # WgpuTestRenderer
   .build_state(|ctx, app| app.draw_ui(ctx), app)
harness.run_ok() ×6                                              # settle layout + texture upload
harness.render() -> image::RgbaImage                            # real pixels via wgpu
   └─ image.save("tests/snapshots/viz_<tab>.png")               # the golden / proof PNG
harness.snapshot("viz_<tab>")                                   # golden diff-compare on later runs

Harness::render() tessellates the egui output, uploads it to a wgpu texture, runs the render pass, and reads the texture back to an image::RgbaImage (egui_kittest::wgpu::WgpuTestRenderer). Harness::snapshot(name) does the same render and diffs it against the committed tests/snapshots/{name}.png (writing a .diff.png + .new.png on mismatch); on the very first run, where no golden exists yet, it reports "Missing snapshot" — which golden_compare() tolerates because save_proof_png() has just written that golden in the same run.

Why inject data instead of opening a warehouse

The local workspace_* warehouses in this checkout are empty (timeline.releases == 0 for holger/knut/skade/…): the live release / dep-graph / bench data lives on the nornir-server and is reached over gRPC via UrdrThreadsApp::with_remote. A snapshot test must not depend on a running server, so it builds a deterministic Timeline from the public nornir::viz::model types (Lane / LaneNode / DepGraphSnapshot / CrossRepoEdge / BenchHistory / BenchPoint) and injects it via the test-only UrdrThreadsApp::inject_timeline_for_test(tab, timeline, repos) (#[doc(hidden)]). The injector also:

• selects the tab by its state_json()["tab"] debug name (so the test does not depend on the private Tab enum),
• ages started_at past the 2.5 s splash so the opaque intro overlay does not cover the captured frame, and
• disables auto_refresh so the next frame does not reload the empty source warehouse over the injected data.

The fixture is a holger-flavoured cross-repo graph: lanes holger (3 releases, one failed_test), knut (2), skade (2) = 7 releases; a dep-graph snapshot holger → knut → skade with 3 cross-repo edges; and 2 bench points per repo.

GPU vs lavapipe

This box has an NVIDIA RTX 4090 (Vulkan: nvidia_icd.json) and the Mesa lavapipe software rasterizer (/usr/share/vulkan/icd.d/lvp_icd.json → libvulkan_lvp.so, reported as llvmpipe).

egui_kittest's default_wgpu_setup() prefers CPU adapters (DeviceType::Cpu => 0 in its native_adapter_selector), so the tests render on lavapipe (llvmpipe) by default — this is deliberate: a software rasterizer gives byte-stable goldens across machines. The discrete GPU is the fallback when no CPU rasterizer is present.

To force a backend / ICD:

# Force the lavapipe software path explicitly (default behaviour):
VK_ICD_FILENAMES=/usr/share/vulkan/icd.d/lvp_icd.json \
  cargo test --features viz --test viz_snapshots

# Force Vulkan backend (still CPU-preferred within Vulkan adapters):
WGPU_BACKEND=vulkan cargo test --features viz --test viz_snapshots

Both produce valid PNGs here; lavapipe is the committed-golden renderer.

Where the PNGs live + the future robot_test_results tie-in

• Goldens / proofs: tests/snapshots/viz_timeline.png, viz_depgraph.png, viz_knowledge.png, viz_bench.png (committed; each ~110–120 KB at 1400×900).
• tests/snapshots/manifest.json — one entry per test: { test, png_path, state_assertions }. This is what a future robot_test_results.screenshot blob column (C6) ingests: read each entry, load the PNG bytes into the screenshot blob, and store state_assertions alongside as the structured pass record. (The warehouse table itself is not built here — only the PNGs + manifest are produced so C6 can wire it.)
• .diff.png / .new.png (written by the snapshot API on a mismatch) are transient and should stay out of git.

Guide — run / update

Run (renders the PNGs, asserts state):

cargo test --features viz --test viz_snapshots -- --nocapture --test-threads=1

The PNGs appear under tests/snapshots/; each test prints its PROOF: <name> → <path> (<bytes>, <w>×<h>) line. The existing viz_matrix / viz_buttons tests are unaffected (cargo test --features viz runs all three).

Update the goldens after an intentional viz change:

# Re-render and overwrite the committed PNGs (save_proof_png always rewrites the
# golden), then eyeball the new images and commit them:
cargo test --features viz --test viz_snapshots
git add tests/snapshots/*.png tests/snapshots/manifest.json

# Or use egui_kittest's own update flow for the diff-compare goldens:
UPDATE_SNAPSHOTS=1 cargo test --features viz --test viz_snapshots

Add a new tab proof: add a viz_snapshot_<tab> test that calls harness_for_tab("<TabName>"), asserts the relevant state_json() slice, then save_proof_png + golden_compare + record_manifest. <TabName> is the state_json()["tab"] debug name (e.g. Timeline, DepGraph, Knowledge, Bench, Warehouse, …).