Your browser sucks, JavaScript disabled! Gunnar's console needs JavaScript and WebAssembly. Git over SSH on port 2222 does not.

nordisk/nornir

.nornir/agent-ui-test-fixing.md

Verifying a UI / gRPC / render / CLI surface — the canonical LAW

What the matrix IS. A green test-matrix row is a provable claim that the surface a user actually ships and touches works — the deployed artifact, the real data, the whole call chain, driven through the server, swept across its outcome space, asserted from emitted data with no human looking at a screen. It is not "the code compiled," not "a native build passed," not "it rendered something." When the matrix is green, the thing it names is correct and ran; when that is not true, the matrix is lying, and a lying matrix is worse than no matrix.

The creed (one line): Green ⟺ the shipped surface, on real data, through the server, across its whole outcome space, was proven RAN-and-CORRECT from emitted data — and the same oracle goes RED when it breaks.

This document is the law every UI / gRPC / render / CLI surface in the constellation obeys — nornir viz (fat + thin), the CLI (fat + --server), facett-* components, korp-poc, the MCP tools, and anything added later. It is a chapter of the docs book (docs/book.pdf) and is linked from the 🧪 Test pane (state_json["test"]["doctrine"]). Read it before you touch a red row, and read it before you claim a new surface is done.

The eight LAWS

Each law is a rule, the real failure that birthed it (so you know it is not theoretical), the concrete how (what to drive, what to assert, how to prove the RED), and a tiny assert sketch. They are not ranked by importance — they are ranked by how often they were the hole the bug crawled through.

LAW 1 — SHIPPED ≠ TESTED: the subject under test is the DEPLOYED ARTIFACT

Rule. The test subject is the artifact that actually ships — the wasm/web bundle for a web demo, the installed binary for the server, the published crate for a library. Native-build green proves the native build; it does not prove the wasm build a user loads. Therefore: require native↔wasm surface parity (the same deck/tabs/buttons/sliders exist in both), AND a headless-wasm robot-UI that loads the actual shipped bundle and asserts every expected surface is both present and ran (via its emitters). A surface that is green natively but absent or non-running in the shipped wasm is RED.

The failure that birthed it. The matrix tested the native egui_kittest build and went green: 27 facets, every button present. But the deployed wasm demo was missing / non-rendering the Sverige showcase buttons and sliders. What we tested and what we shipped were two different binaries. The native green was true and worthless.

How.

// parity: the deployed deck must equal the tested deck
assert_eq!(wasm_surface_ids(&loaded_bundle), native_surface_ids(&state_json),
    "wasm bundle is missing surfaces the native test covered");
// present AND ran: every showcase control fired its emitter in the SHIPPED build
for id in SHOWCASE_CONTROLS {
    assert!(wasm_emits.contains_key(id), "shipped wasm never ran surface {id}");
}

LAW 2 — RAGNARÖK: empty / zero data on a data-bearing surface SHALL NOT PASS

Rule. EMPTY / ZERO DATA IS NOT A PASS — IT IS RAGNARÖK. A test or surface that goes GREEN on empty / zero data is a broken test, not a passing one. Every data-bearing surface MUST assert REAL POPULATED data (count > 0, real values, real edges / rows / nodes) and go RED on zero / empty. There is no "renders empty cleanly" pass for a surface that is supposed to carry data — empty = ERROR, it shall not pass.

The failure that birthed it. The 🚇 Metro map rendered empty stubs and "passed." A blank Metro map / empty boundary landmarks counted green, which is exactly the row that would have surfaced the missing deep_scan → call_edges the instant it had been forced RED.

POPULATE-DIAGNOSTIC corollary. When a surface is empty, the test AND the agent MUST report WHY (which scan / table / config is missing) and WHAT the user must do to get data — e.g. "deep_scan = true not set → no call_edges scanned", or "the warehouse has no call_edges for this workspace → run a populate / deep scan". If the matrix cannot itself populate the data, that is not a reason to pass — it is a reason to FAIL RED and name the populate gap to the user. Never silently pass an empty surface.

Prove the RED by sensitivity. Pair every populated-state assertion with its inverse: feed the surface empty input and assert the SAME emit row goes RED. A populated-state metric that cannot go red on empty input is asserting nothing. (predicates_go_red_on_empty_or_errored_slices is the matrix-baked form of this.)

let s = drive(metro, &populated_warehouse).state_json();
assert!(s["metro_edges"].as_u64().unwrap() > 0, "RAGNARÖK: empty metro is RED");
// inverse, baked into the matrix:
let e = drive(metro, &empty_warehouse).state_json();
assert_eq!(e["metro_edges"], 0, "and the SAME oracle must go red on empty");

LAW 3 — ONE VALUE ≠ COVERED: sweep the SAMPLE SPACE, per-function and whole-chain

Rule. A surface is covered only when every function on its call chain is swept across its whole outcome space (equivalence classes + boundaries → exact expected output), AND the composed chain is swept across the same input space asserting the end-to-end contract. One value inside one partition proves nothing the next value in that partition would. (English math term: sample space, Ω — the set of all possible outcomes. "utfallsrum" is the Swedish origin name; identifiers use sample_space / SampleSpaceCoverage.)

The failure that birthed it. A single green value was counted as "covered." A function that mapped one partition correctly and the next partition wrong shipped green, because the one value the test happened to pick landed in the correct partition.

How — the facett-map3d exemplar (the canonical shape). Sample-space coverage of a chain f1 ∘ f2 ∘ … ∘ fn is BOTH of:

Make it first-class with the framework: declare the outcome space, record each class with its exact output, let the gate score it. A test that hits only the final stage, or only one input value, is not sample-space-covered.

use nornir_testmatrix::utfallsrum::Outcome; // public surface: sample_space
let mut o = Outcome::for_fn("clip_tri_near")
    .class("behind-near", "cz < near ⇒ vertex dropped")
    .class("on-near",     "cz == near ⇒ vertex kept, t==0")
    .class("front",       "cz > near ⇒ vertex unchanged");
// sweep one case per class, record the EXACT output as evidence:
o.hit("on-near", "clip_tri_near(cz=near) keeps vertex, t==0.0");
assert!(o.summary().meets_threshold(2)); // ≥K classes or the surface stays in the gap

Cross-link: the framework primitive (generic, UI-agnostic — works for a headless library with no viz/server/workspaces) lives in ../crates/nornir-testmatrix/.nornir/testmatrix-utfallsrum.md

  • its adoption guide. nornir's surface enumerator (LAW 5) is nornir applying that primitive — keep the two layers cleanly separated (the jewel is reusable; the viz/fat-thin/workspace gate is nornir's dogfood harness on top of it).

LAW 4 — END-TO-END = THROUGH THE SERVER, not the local fat path

Rule. nornir is built fat/thin client → server. "End to end" therefore means the FULL round-trip, not the in-process path:

GUI button / CLI verb  →  thin client  →  gRPC RPC  →  server handler
   →  warehouse (read/append)  →  gRPC response  →  back to client/GUI surface

The matrix MUST exercise the real server round-trip (@thin) for every workspace × every GUI-function against a running nornir-server, and assert the data that came back from the server — not a local read. @fat (embedded/in-process) is the twin and must also be covered, but the @thin cells are the ones that prove the fat/thin → server construction actually works. CLI --server parity is total: whatever a viz tab shows via the server, the CLI verb must return the same via the server (assert parity as data).

The failure that birthed it. Surfaces were proven on the local fat path and called done. The thin client → gRPC → server → warehouse construction — the thing that actually ships in remote mode — was never driven, so a thin-mode surface could be stub-degraded or warehouse-blind while the fat test stayed green.

How.

let thin = cli(["dep-graph", "--server", &srv.addr, "--json"]).run();
let fat  = cli(["dep-graph", "--json"]).run();
assert_eq!(thin.json["edges"], fat.json["edges"], "thin↔fat parity through server");
assert_eq!(thin.json["fuel_source"], "warehouse"); // proven server-backed, not stubbed

LAW 5 — WORKSPACE BLIND: the surface is workspace × {tab@{fat,thin}, cli, mcp} × chain, clean-slate→populate→verify

Rule. A green row for one workspace does not prove the surface. The completeness surface has three dimensions; cover all three or the matrix is not full:

  1. Workspace dimension. The surface is workspace × {cli_command, viz_tab@{fat,thin}, mcp_tool, grpc} × call-chain-to-end for EVERY served workspace — read the live set from workspaces_list (discover::SERVED_WORKSPACES), never a hardcoded list. The same tab that works for nornir can be blank for knut; they are DISTINCT surfaces, each keyed <workspace>/<kind:id@mode> (workspace_surface_key). CLI is a parallel END of each chain with TOTAL parity to the GUI — never GUI-only.
  2. clean-slate → populate → verify. A surface counts as covered only when, from an EMPTY warehouse — a fresh one via a throwaway $HOME or an absolute [storage].local_path pinned to a temp dir in nornir.toml (there is no NORNIR_WAREHOUSE_ROOT; path env-vars were removed — the warehouse is $HOME-derived per-workspace) — the REAL pipeline (clone → nornir map → nornir test/bench/sync) populates it AND the GUI then shows REAL data read from state_json. A pane that renders only because the test injected the rows is NOT the populate→show path — it is a seeded fixture, and a seeded fixture cannot prove the populate pipeline works.
  3. Sample-space (LAW 3) — every fn on the chain swept ≥K outcome classes.
  4. Atom granularity — enumerate DOWN TO THE ATOMS via the live AccessKit tree. A tab is not an atom. Every rendered widget — button, label, error-text — is an AccessKit node that egui_kittest walks; that walk is the only thing that sees sub-panel atoms (e.g. the 🧬 tab's CloneEvents "populate status" + Jobs panels), which coarse Tab::ALL / per-pane state_json are blind to. The verifier renders each tab × {empty-registry, populated} through a kittest::Harness, walks the tree, and asserts (a) NO error atom (is not served / Failed to load / unavailable) in the empty state, (b) the expected populated atoms present (LAW 2 RAGNARÖK). This is the mechanical "find each UI element" — the markers (button_registry → 🚇 metro) are the DECLARED truth; the atom-walk VERIFIES them and is auto-derived back into them. STOP enumerating at tab/Facet granularity; STOP hand-listing surfaces in bespoke tests — one atom-walk covers all tabs × all atoms × states, emitting surface_coverage rows autonom reads like any marker. (egui has the discovery primitive — AccessKit — so "no deterministic way to find each element" is FALSE.) See marker-matrix-plan.md §2b for the L0–L3 layering.

The failure that birthed it. The matrix proved a tab for one workspace and treated the surface as done; other workspaces' identical tabs went untested and could be blank. Separately, panes that rendered from seeded rows passed without ever proving the real populate path — rendering with seeded data is not enough; it must survive the real clean-slate→populate path.

How. For each ws in workspaces_list: fresh temp warehouse → run the real populate verb → drive every {tab@fat, tab@thin, cli, mcp} → assert real data from state_json. If a pane is empty after populate, that is a BUG to fix (LAW 2's populate-diagnostic), not a pass.

for ws in live_workspaces() {                      // from workspaces_list, not hardcoded
    let wh = TempWarehouse::empty();
    populate_real(ws, &wh);                          // clone → map → test/bench/sync
    let s = drive(viz_tab(ws, Mode::Thin), &wh).state_json();
    assert!(s["rows"].as_array().unwrap().len() > 0, "{ws}: blank after REAL populate");
}

LAW 6 — VERIFIED BY EYEBALL is not verified: every emptiable render EMITS its populated-state as DATA

Rule. Every viz render that CAN be empty MUST emit its populated-state as DATA (functional_status counts), so correctness is asserted from the EMIT, never from "does it look right?". If a render can degrade to blank / stub, it has to publish the numbers that distinguish populated from empty, and a test reads them back — headless, no human.

The failure that birthed it. An agent kept asking the human to eyeball a 3D viewer — the entire emit/harness machinery had been built precisely so that was never necessary. If you find yourself asking "does it look right?", you have skipped the doctrine.

Worked example — the 🚇 Metro map (src/viz/arch_tab.rs). A pixel/render stage whose correctness can't be read from a return value, so an in-fn functional_status emitter (gated behind --features testmatrix) is the right tool. It ships, per render:

The RAGNARÖK test (tests/viz_arch_metro.rs, metro_thin_warehouse_backed_reaches_grpc_and_emits_populated) drives the thin metro against a populated warehouse and asserts metro_edges > 0 AND metro_stub_lines == 0 AND metro_emitter_nodes > 0; its inverse (metro_thin_empty_warehouse_is_red_not_silently_green) feeds an empty warehouse and asserts the SAME row goes RED. That is the fix verified WITHOUT a human looking at pixels. Build every emptiable surface this way: emit the populated-state counts, assert them populated, prove the RED with the empty-input inverse.

LAW 7 — PESSIMISM: green ONLY when proven ran-and-passed-and-swept; an unverified surface is presumed BROKEN

Rule. Distrust every "done." A surface is presumed BROKEN until the matrix shows it RAN end-to-end, PASSED a tight oracle, and SWEPT its sample space. "✅ all tests pass" is not evidence — the evidence is the matrix row, with its data, on the shipped path. Optimism is the bug.

The failure that birthed it. "Done" claims kept being false: "✅ all tests pass" shipped while the live demo was visibly broken, because the oracle was blind or the tested path was not the shipped one. Hours burned trusting the claim over the matrix.

How. Treat a green row with an unproven/loose oracle as a RED finding until you have seen it go red on the bug (LAW 8's sensitivity proof). When a fix "works" but the symptom persists, you fixed the wrong path/scene — re-check the path (META §6), do not re-explain the fix. The matrix is only trustworthy when it is provably full: green ⟺ ran-and-passed, blank ⟺ you haven't driven it yet (so go drive it), red ⟺ real failure.

LAW 8 — WIRE ≠ HOME: the terminus is the Iceberg WAREHOUSE, via the lock-owning server

Rule. functional_status rows emit to a JSONL wire (NORNIR_TESTMATRIX_OUT); that file is a verification convenience, NEVER the destination. A green row in a /tmp JSONL is NOT "done." Done = the rows reconciled into the Iceberg warehouse via nornir test (run through the server that owns the redb write-lock) and read back FROM the warehouse (the viz Test pane / test_history). A wire-only, never-reconciled row is RED / incomplete.

The failure that birthed it. ~4491 facett graphskins call-chain rows were correctly emitted but pointed at /tmp to count them, and counted as done. A green row in a temp JSONL was waved at the human — the same failure as a blank pane.

How.

// not "rows in JSONL" — rows READ BACK from the warehouse via the lock-owner
let back = test_history(&server, ws, run_id);
assert!(back.iter().any(|r| r.test_name == "metro_thin_warehouse_backed…"),
    "row emitted to wire but never reconciled into the warehouse = RED");

The false-green checklist — run this before you ever say "done"

Every "no" is a RED finding. A surface is not done until every box is "yes."

The rest of this document is the technique the eight laws lean on — the doctrines and worked methods that have, in practice, been what actually catches the bug. They are kept because each one was paid for in burned hours.

The loop — validate as DATA, not pixels

  1. The component already emits what it renders, as readable data. Read it, don't look at it:

    • state_json() — every Facet/pane reports its live state (camera pose, counts, visible-feature count, the geometry it intends to draw).
    • nornir_testmatrix::functional_status(component, check, ok, detail) — the matrix row. Drive the component headless, assert the return value, emit one row. Green matrix ⟺ every surface works.
    • the robot-UI driver (egui_kittest / nornir-robotui) — drive real clicks / scroll / tab-switch and read the resulting state_json. So your first move is assert on emitted data, never on a screenshot.

  2. EMIT-DOCTRINE — emit only where the return value / state_json can't already prove it. If a function returns the answer, just assert it; don't add an emit. The one class data can't trivially capture is "did it render non-blank / correct?" — and even that you turn into data: emit the geometry as numbers (bbox, vertex/triangle counts, "longest projected edge", "fraction of pixels ≠ background", "triangles spanning the whole box"). A starburst, a black pane, a degenerate mesh — all become a number you can assert. Add lots of such emits; they are cheap and they are how you "see".

  3. Only if a property genuinely cannot be reduced to data, render it and read the image yourself. You can read PNGs. Render headless (egui_kittest wgpu → PNG), Read the file, judge it. This is the fallback, not the default — and if you reach for it twice, add the emit (or the pixel scanner, below) that would have caught it instead.

Walk the WHOLE call chain, start → end (the BUILD-TIME mandate)

The binary-search section below finds ONE known-red bug fast. This is the opposite and it is primary: when you build or touch a surface, you must track and verify EVERY function in its call chain, input → pixel, so a bug can never sit untested. "I tested the thing that was broken" is NOT "I verified every function I wrote" — an agent once fixed the buggy component's chain and left the other components' chains only partially covered; that is the failure this section prevents.

  1. Enumerate the chain. Write it down, input → … → pixel (e.g. a 3D map: read → project → near-clip → frustum-cull → back-face-cull → shade → depth-sort → scissor → rasterize → composite). Use the call graph to be exhaustive — knowledge_call_path / dwarf_call_path / arch. This list is the spec your tests mirror: one verified checkpoint per node, no gaps.

  2. Verify every node. For EACH function on the chain, a matrix-evaluated check: assert the return value / state_json where it proves the function correct (counts, finiteness, bbox, ordering, the resolved value); add a functional_status emit only where the return can't prove it — the pixel stages (lit_px, ink_outside_rect == 0, max_projected_extent ≤ viewport·K, byte-identical-to-reference), per EMIT-DOCTRINE. No node ships unverified.

  3. Sweep it — the sample space, not one case (LAW 3). Generate a parameter grid (input sizes, geometry/topology kinds, camera poses incl. the nasty/degenerate ones, canvas sizes, worker counts); run EVERY case through the chain, asserting at each stage + emitting one matrix row per case. One example proves nothing about the space; the grid does.

  4. Robot-drive the live surface. Use the robot-UI harness (egui_kittest / nornir-robotui) to click / scroll / navigate / orbit the REAL component, read back state_json + the emit chain, assert correct + confined + non-blank, AND scan the pixels (below). A wrong / escaped / blank frame = a RED matrix row.

  5. Emit the final picture as a PNG blob so an agent (or the viz) can see the rendered frame straight from the matrix — no human opening a window.

Outcome: the test matrix is the single source of truth for "does this work." Green ⟺ the entire call chain, across the whole sample-space sweep, driven through the real UI and the server, is proven — so the human reviews the matrix, never the screen. Do this for EVERY function you write, not only the one that was red.

THE ORACLE MUST BE ABLE TO GO RED — the threshold is the test

A green test proves NOTHING unless its oracle would turn RED on the actual bug. This is the #1 way a day gets burned: "✅ all tests pass" ships while the live demo is visibly broken — because the oracle was blind or its threshold was loose. (2026-06-21: the OSM-3D map smeared green spokes across the whole frame in the deployed wasm demo while golden_248, no_spokes, render_proof, and a 1000-case grid were ALL green. The max_extent_px oracle existed — but its cap was half_h·8, i.e. it tolerated a triangle 8× the viewport. A frame-filling smear is "inside an 8×-viewport cap," so it passed. The oracle looked rigorous and was worthless.) Before you ever say "fixed", satisfy ALL of:

  1. Define the bug as ONE numeric invariant on the LAST function that feeds pixels — the faces/mesh actually painted, not an intermediate. State it as a number: "no painted triangle bbox exceeds the widget rect", "max projected edge ≤ viewport diagonal", "fraction of fill-vs-background pixels ∈ [a,b]". NOT "non-blank", NOT "is lit", NOT "didn't panic" — those pass on a smear.

  2. The threshold = the REAL contract bound, and you must justify the number. A cap that is a loose multiple of the viewport (×4, ×8) passes the exact smear it pretends to guard. If the correct answer is "no geometry escapes the viewport", the cap is the viewport (≈ 2·half_h), full stop. Write the cap and a one-line reason next to it; a magic loose constant is a bug in the test.

  3. PROVE the test goes RED on the current/broken code BEFORE you fix it. git stash the fix (or run against the pre-fix path), watch the new assertion FAIL, restore, watch it PASS. A test you never saw fail is not evidence — it may be asserting nothing. "Sensitivity proof or it didn't happen."

  4. Same PATH + same SCENE the user actually sees. Render through the exact code path the product uses (here: the CPU egui painter the wasm demo runs, NOT a --features wgpu GPU path the demo never takes) and the exact fixture / camera pose / data. A golden that looks fine because it rendered a different path/pose than the demo is a classic false green.

  5. Then, and only then, look at ONE rendered frame to confirm the invariant actually captures "looks right" — the image is the check on the oracle, never the proof itself. If the frame looks wrong but the test is green, the oracle (not the frame) is the bug — tighten it until it goes red.

Copy-paste prompt that forces this: Define the bug as ONE tight numeric invariant on the LAST function feeding pixels. Assert it on the SAME render path + scene/pose the demo uses. PROVE the test goes RED on the current code (stash the fix → red → restore → green) before claiming a fix. The threshold must be the real bound, not a loose multiple — state and justify the number. Only after green, look at one frame to confirm the invariant means "looks right." green ⟺ correct, or the oracle is wrong.

BLACK-BOX THE CHAIN WITH KNOWN VALUES — assert EXACT expected output per stage

Driving the component via the demo (real scene in) is necessary but NOT sufficient. You ALSO must do black-box, known-value testing of the whole chain: insert a KNOWN input and assert the EXACT expected value comes out of EACH function f1 → f2 → f3 → f4 → … → pixels. Compute the expected value independently (by hand / closed form / a reference impl), not by capturing whatever the code currently emits.

One-liner: insert known values, assert exact known outputs at every f1→f2→f3→f4 boundary, THEN drive the real demo scene end-to-end and scan the pixels. Both, or it's not verified.

SCAN THE PIXELS — the image oracle must ANALYZE the frame, not assert "non-blank"

A robot-UI / render test that claims it "sees" the frame MUST run image analysis on the received pixels and assert the result is the EXPECTED structure. "non-blank" / "has lit pixels" / "wrote a PNG" is NOT seeing — it passes on a smear, a starburst, a garbage frame. (2026-06-21: robot_ui claimed to drive the OSM-3D to the failing pose and check the frame "confined + non-blank" — and was GREEN while the live frame was a green spoke smear. It never analyzed the image. That is a lie in the oracle.) If a test renders pixels, it must compute features from them and assert them:

Build these as a reusable image-probe (e.g. a facett_core image-analysis helper) so every component's render test calls the SAME scanner: scan(frame) -> {spokes, hf_ratio, coverage, centroid, …}, then assert each feature. If your render test does not compute a number FROM the pixels and assert it, it is not an image oracle — it is a lie.

EVERY EMITTER IS TWO AXES: input coverage × oracle strength

A check is assert(invariant(f(input))). It has TWO independent ways to be worthless; close BOTH for EVERY emitter / functional_status row:

Axis A — input coverage (LAW 3: MULTIPLE values, not one). Drive each function with min, max, mid, AND the boundaries/degenerate cases — derived from the function's OWN code:

Axis B — oracle strength (the threshold IS the test). Each case asserts a contract-tight, sensitivity-proven invariant (see "oracle must be able to go red"). A loose cap passes the very bug it pretends to guard.

They are ORTHOGONAL and BOTH required. The OSM-3D matrix_grid swept ~1000 poses (Axis A ✓) and was GREEN — because the oracle cap was half_h·8 (Axis B ✗). 1000 values × blind oracle = green-on-broken.

Concrete failure to never repeat (the smear, 2026-06-21)

facett-map3d shipped SCREEN_BOUND_K = 4.0 (clips only to ±4× the viewport) and a max_extent_px cap of half_h·8 (tolerates a triangle 8× the viewport). Both are loose multiples of the viewport, so near-grazing triangles survived as frame-filling green spokes that every green test missed. A screen/extent bound is the VIEWPORT (≈ 2·half_h), never a multiple of it. Any *_K / cap constant that is a multiple of the viewport is a test bug until proven otherwise.

GENERALIZED — every function's output has a CONTRACT bound; assert THAT, not a slack multiple

EVERY function has an exact set/range/relation a correct output must satisfy — its contract. The oracle asserts the contract bound; a tolerance / slack constant (×2, ×8, "close enough", "big enough") is the loophole the bug hides in:

component / fn kind input boundaries (Axis A) exact contract bound (Axis B — NOT a multiple)
render / map / graph3d near-grazing pose, camera inside geom, empty/1-tri mesh, huge/tiny scale every painted vertex inside the widget rect (= viewport, not k×it); 0 ink outside
graph layout 0/1/N nodes, disconnected, self-loop, complete graph every node+edge position inside the canvas, finite (no NaN/Inf), degree == model
table / grid 0 rows, 1 row, N rows, max col widths, unicode/empty cells rendered row count == input rows (no silent truncation); every cell value in its column domain
plot / chart empty series, single point, min==max, negative, huge range every point maps inside the axes; tick min/max bracket the data exactly
projection / transform denom→0 input, identity, extremes output range = the transformed input range; no divide-by-~0 escape
parser / codec / IPC empty, 1 byte, max size, malformed roundtrip identity; byte/elem count preserved; schema honored
search / index 0 hits, 1 hit, all-match, boundary key result set == brute-force truth on the same input

The general rule, domain-free:

  1. Derive input boundaries from the function's own branches/divides/clamps + the degenerate cases of its type. Sweep them all (Axis A).
  2. Assert the EXACT contract bound the spec implies — equality, containment, finiteness, conservation, roundtrip — never a slack multiple/tolerance (Axis B).
  3. Prove the assertion goes RED on the broken code before the fix.
  4. If you wrote a _K / cap / epsilon that is a multiple of the true bound, treat it as a bug until you justify the exact number against the contract.

One-liner: sweep the input boundaries the code itself defines, and assert the exact output contract — coverage × exactness, both proven able to go red. A tolerance constant is where the bug lives.

MULTIVARIATE + METAMORPHIC stress — the bug lives in the SPACES BETWEEN functions

Single-axis sweeps (rotate-only, zoom-only) are the amateur trap. A graphics engine is a deeply interconnected multivariate chain: day/time, zoom, camera rotation, aspect ratio, and the coordinate matrices all change at once, and the math breaks between functions where two moving variables interact — never visible when you vary one at a time. Test the JOINT space.

  1. Multivariate state vectors — the CARTESIAN product, not one-var-at-a-time. Generate a massive diverse matrix combining every axis simultaneously: day ∈ 1..365 × zoom ∈ 1..100 (incl. float extremes: tiny, huge, denormals) × azimuth 0..360 × elevation × aspect/bbox. Use proptest (shrinking finds the minimal failing vector) OR clean nested matrix loops. The bug you cannot see is the one where zoom×rotation×day coincide.

  2. Chain-of-function integrity end to end. Projection feeds facet generation feeds clipping feeds raster. Assert data integrity is preserved THROUGH the whole chain on every state vector — not just at the ends.

2b. EVERY function gets its OWN multivariate sweep — not just the last clip stage. f1 (view_space), f2 (near/frustum clip), f3 (project), f4 (screen-bound), … EACH gets the full day×zoom×rotation×aspect matrix run through IT and its output asserted (finite, topological, in-range), AND the chain. A bug in f1 at a corner of the space is invisible if you only sweep f4. (This is the per-stage half of LAW 3 — the chain_sweep.rs shape.)

  1. The NaN/Inf plague — assert NO stage produces or propagates NaN/Inf. Every divide, normalize, matrix inversion is a source. After EACH function assert all outputs .is_finite(). A single NaN silently poisons the rest of the chain and renders as a blank or a smear. The cheapest, highest-value invariant — add it everywhere.

  2. Topological sanity. Transformed facets must stay geometrically valid: every index points to a real vertex, polygons stay closed (first==last / ring intact), triangle winding doesn't unpredictably invert (a flip = inside-out face = back-face-cull eats it = hole/smear), array lengths stay consistent. Assert after each transform.

  3. Metamorphic relations — relate OUTPUTS across related INPUTS (you don't need the exact pixel, only how it must CHANGE):

    • zoom ↑ (day, pose fixed) ⇒ total rendered facet area monotonically ↓ or ==, never grows, never glitches. (Catches the smear: a smear makes area JUMP UP.)
    • rotate 360° back to start ⇒ frame identical (round-trip).
    • day/time shift ⇒ only lighting/shade changes, geometry/coverage invariant.
    • translate the whole scene ⇒ projected shape congruent (area/topology preserved).

  4. Actionable failure tracing. On any violation, panic/emit the EXACT state vector {day, zoom, azimuth, elevation, aspect, matrix} AND which function broke which invariant — reproducible from the message alone, no re-bisect. proptest shrinking

    • this trace = the minimal reproducer handed to you.

One-liner: sweep the JOINT multivariate space, assert finiteness + topology + metamorphic relations through the WHOLE chain, and on failure dump the exact state vector + stage. Single-axis green is not green.

META — what ACTUALLY catches the bug (do these FIRST, every time)

This session burned hours because the agent trusted numbers over pixels and tested the wrong path. The moves that actually worked:

  1. LOOK AT THE RENDERED IMAGE. Every time. Before claiming anything. Every number lied at least once (a loose extent cap passed an 84 000px smear; a spoke_score of 0.226 "passed" a frame-filling GREEN terrain smear because the detector only knew thin spokes). The IMAGE never lied. The oracle's only job is to AUTOMATE what the image shows — so validate the oracle against a looked-at known-good AND a looked-at known-bad frame, or the oracle is untrusted.

  2. Find the PRODUCT'S ACTUAL PATH before you test anything (LAW 1). Which feature flags does the SHIPPING build use? (wasm = --no-default-features → CPU painter, NO wgpu — so testing the native --features wgpu GPU path proves nothing about the demo.) Which DATA? (load() = real embedded OSM, not synthetic.) Which fixture/pose? (the demo's own from_geometry framing, not your tidy test grid; the close-zoom pose, not dist=3.2.) Reproduce the EXACT user scene end-to-end first.

  3. The oracle fails TWO ways — too loose AND wrong-kind. A threshold that is a multiple of the true bound passes the smear it guards (loose). A thin-spoke detector misses a filled-area smear (wrong feature). One pixel metric is never coverage — use several orthogonal ones (spoke/edge, filled-area/coverage spikes, frequency, centroid, frame-spanning-sliver) AND look.

  4. Verify the DEPLOYED artifact is the NEW build (LAW 1). Browsers cache *.wasm by its constant filename → a redeploy serves the STALE build and the "fixed" bug is still there. Cache-bust (?v=<sha>) and confirm the live bytes changed.

  5. Enumerate and instrument EVERY path (CPU+GPU, every component, every data/fixture branch), regardless of refactoring. The bug hides in the ONE path you didn't instrument. If state_json says visible_triangles: 0 but the frame has content — or vice-versa — you are reading a different path than you are rendering. Make every path emit a uniquely-named matrix row so "every enumerated path produced a row" is itself assertable.

  6. When a fix "works" but the symptom persists, you fixed the wrong path/scene. Don't re-explain the fix — re-check #2 (is this even the path the user sees?).

If you do nothing else: look at the image, on the exact path+data+pose the user runs, on the actually-deployed build. Numbers come second.

Debug by BINARY SEARCH, the way a human bisects

When a test is red and you don't know which function is wrong, do not add a debug/emit to every function top-to-bottom. Bisect:

  1. Find the midpoint of the suspect call chain (input → … → output).

  2. Add ONE emit/assert there: is the value still correct at the midpoint?

  3. Run the test, read the emitted value.

    • Correct at the midpoint → the bug is downstream; recurse into the lower half.
    • Wrong at the midpoint → the bug is upstream; recurse into the upper half.

  4. Halve again. In O(log n) steps you land on the exact faulty function. Fix it, rerun, watch the row go green, then remove the scaffolding emits (keep the one that is a real regression guard).

This is exactly how a person single-steps with a debugger — but guided, halving the interval each time instead of walking every frame.

Follow the data — drop your assumption when it's disproven

State your hypothesis ("the 3D starburst is a geometry bug"), then let an emit try to falsify it ("bbox is in-box, 0 outliers, max face 58px → geometry is clean"). If the data kills the hypothesis, change the hypothesis — do not keep fixing the thing you assumed. (Two plausible geometry fixes were wrong; the emit proved the mesh clean and pointed at the render architecture — painter's-algorithm vs a GPU z-buffer — which no geometry fix could ever address.)

Every fix earns a real matrix test

Per repo law: a fix that isn't guarded by a #[test] that fails on the bug and passes on the fix is not done. Prove the sensitivity (git stash the fix, watch the test fail, restore). Then it emits functional_status so nornir test carries it in the matrix forever — and reconciles into the warehouse (LAW 8), not a /tmp JSONL.

If a surface is EMPTY because nothing has been RUN — RUN IT, generate the data

A pane that emits empty "because no run produced data yet" is NOT a pass (LAW 2) — it is a surface you simply haven't driven. When data is missing, RUN the action that generates it (the test matrix, a bench, a knowledge scan, a populate/clone) and then assert the now-populated emit — for EVERY workspace and EVERY repo (LAW 5). "Empty until run" ⇒ run it.

Scope

Apply this to every UI/gRPC/render/CLI surface: nornir viz (fat + thin), the CLI (fat + --server), facett-* components, korp-poc, the MCP tools, and anything added later. The agent sees and validates through emitted data + the headless harness; the human reviews the matrix, not the pixels.

Status — the doctrine is LIVE, with reference implementations

This is not aspirational; the release surfaces shipped are the canonical worked examples of "drive it headless, assert the emitted state, every fix earns a matrix row." (cargo test --features viz,server --test robot_viz → 11/11, --test saga_roundtrip → 3/3.)

The static edition gate (coverage-style static_findings_only) and the sample-space-scored surface gate (covered_with_utfallsrum, LAW 3) are the pattern any new UI/gRPC/CLI surface copies.

Open debt (LAW 8). The facett graphskins call-chain rows (facett-graph / facett-syschart / facett-pipeline, ~4491 functional rows from the callchain_matrix + robot_pixels tests) were emitted but only sinked to /tmp to count them — land them in the warehouse post-merge via nornir test all on the workspace (merge → reinstall server → NORNIR_SERVER). The wire is not the home.