.nornir/harness-guide.md

The headless harness ecosystem — guide

Class: guide · audience: a techie writing or driving facett's headless tests. The what is in harness-design.md; the why in harness-reasoning.md. This file is the workflow. The base drive/snapshot state API is in elm-guide.md f3 — this builds the test-matrix, robot, runtrace, a11y and imgscan layers on top of it.

f0 — driving one component headless, end to end (the whole workflow)

The seam: hand a component data, drive it with no window, read back what it did.

use facett_core::harness;

// 1. STATE oracle (no egui, no GPU) — apply a Msg script, snapshot the Model:
let model = harness::snapshot(&mut view, [Msg::Pick(id), Msg::Clear]);
assert_eq!(model.selected, None);

// 2. PIXEL proxy — render offscreen, assert it drew and the state is right:
let r = harness::headless_render(&mut view);          // -> RenderReport
assert!(r.drew());                                     // vertices > 0
assert_eq!(r.state["nodes"], 2);

// 3. PIXEL oracle (structure, not "non-blank") — scan the real frame:
//    (needs egui_kittest + a wgpu adapter; see f6)
let scan = facett_core::imgscan::scan(img.as_raw(), img.width(), img.height(), bg);
assert!(scan.spoke_score < 0.30 && scan.coverage > 0.02);

// 4. Feed the result to the nornir matrix (feature `testmatrix`):
#[cfg(feature = "testmatrix")]
facett_core::testmatrix::emit_render("myview", r.vertices, model.nodes);

That is every axis: state, pixel-proxy, pixel-structure, matrix row. The rest of this file is each layer.

f1 — render capture & reading the trail

headless_render / render_sized / render_themed / render_themed_sized all return a RenderReport { title, state, vertices }. Use render_themed_sized when a cell crosses theme × canvas (the painter reads the active palette, so each is a distinct paint path):

let r = harness::render_themed_sized(&mut view, Theme::sci_fi(), (1280.0, 720.0));
assert!(r.drew());

To follow what a headless run did without a screenshot, read the stderr trail (one kinded, timestamped, sequenced line per render). Mirror it to a file for grepping:

FACETT_TRAIL=/tmp/run.trail cargo test -p facett-graph -- --nocapture
grep '\[CASE\]' /tmp/run.trail        # per-cell matrix summaries
grep '\[STATE\]' /tmp/run.trail       # full state_json per render

Emit your own per-cell summary from a matrix loop with harness::case_summary(component, axis, &r).

f2 — the test-matrix bridge: emitting a functional row

A leaf crate feeds the nornir test matrix one row per real check. Wire it once:

# <crate>/Cargo.toml
[features]
testmatrix = ["facett-core/testmatrix"]

Then, at each real assertion, emit the real result (never literal true):

let ok = drew && state_correct;
assert!(ok, "…");                                    // the test still fails locally
#[cfg(feature = "testmatrix")]
facett_core::testmatrix::emit("facett-graph.render_matrix", "all_cells_draw", ok,
    &format!("{cases} cells, {drew} drew, {state_correct} state-correct"));

For the two common render checks use the convenience emitters — they encode the RED-on-dead-view rule for you:

// green iff vertices>0 AND the component's own geometry>0 (a dead map = RED):
testmatrix::emit_render("facett-map", r.vertices, geometry_from_state);
// green iff content ratio clears the floor AND geometry>0 (a blank pane = RED):
testmatrix::emit_non_blank("facett-map", scan.coverage, 0.01, geometry);

Run the emitters and collect the rows:

NORNIR_TESTMATRIX_OUT=$PWD/target/nornir-functional.json \
  cargo test -p facett-graph -p facett-map --features testmatrix

A release build (feature off) has no nornir-testmatrix edge and compiles every emit out — verify with cargo tree -p facett-core | grep nornir-testmatrix (empty).

f2b — the one-liner cell for ANY pane (`probe_and_emit`)

Don't hand-compute geometry/ok for a plain pane — hand the whole Facet to the shared probe. It headless-renders, drives a scripted list of update_json messages, and emits the headless_render + responds_to_input rows for you. No concrete Msg type needed at the call site:

use facett_core::Facet;

let mut view = MyPane::demo("title");           // your local()/demo() ctor
let probe = facett_core::testmatrix::probe_and_emit(
    "facett-mypane",
    &mut view,
    &[r#"{"set_view":"tree"}"#, r#"{"run":"ci"}"#],  // serde-tagged JSON msgs
);
assert!(probe.drew() && probe.responded());          // extra pane-specific asserts
assert_eq!(probe.steps[0].state["view"], "tree");

This is BOTH the pane's functional cell AND its multi-step robot json-drive scenario, from one call (see facett-brokkrview/tests/robot.rs). Add a testmatrix = ["facett-core/testmatrix"] passthrough feature; the emit is a no-op without it (the probe + asserts still run), so cargo test -p facett-mypane --test robot passes plain and emits the cell under --features testmatrix.

f3 — the run-ledger: proving a control actually ran

Instrument each render / control handler with a stable key, then read the tally back as data — the same ledger works native and in wasm.

// in the component:
fn ui(&mut self, ui: &mut egui::Ui) {
    facett_core::runtrace::ran("PopGraphTab::ui");
    if ui.button("11M").clicked() {
        facett_core::runtrace::ran("PopGraphTab::ctl:11M");
        /* … */
    }
}
// expose the tally in state_json:
serde_json::json!({ "trace": { "ran": facett_core::runtrace::snapshot() } })

Assert it in a test (reset first — the ledger is a process-global):

facett_core::runtrace::reset();
// … drive the demo headless via egui_kittest …
assert!(facett_core::runtrace::count("PopGraphTab::ctl:11M") >= 1, "the 11M handler ran");

On wasm, the identical snapshot() object is read live from the browser via window.__facett_state() → state_json.deck.trace.ran; a control present natively but not running in the shipped bundle is the divergence to catch.

f4 — making a painted element robot-drivable (a11y)

A custom-painted node/chip/tile is invisible to get_by_label until you attach an AccessKit node. Do it in the same paint pass, keyed on a domain key:

use facett_core::a11y::{self, Semantics};
// paint the visual as usual (painter.rect_*, painter.galley, …), then:
let hit = a11y::node(
    ui, base_id, node.name,                       // stable domain key (NOT an index)
    egui::Sense::click(), node_rect,
    Semantics::button(format!("{} · {} rows", node.name, node.rows)).value(node.rows as f64),
);
if hit.clicked() { out.push(Msg::Pick(node.id)); }

Now a robot / screen reader finds it: Semantics::button → a Button-role node, .value(n) → AccessKit numeric_value, Semantics::list_item(label, selected) → a selectable carrying its selection. stable_id(base, key) gives the same id across reorder/insert/delete, so a replayed click hits the same element (FC-5).

f5 — the pixel oracle: asserting frame structure

Never assert "non-blank" — assert structure. scan returns a ScanReport you bound against a viewport-tight, sensitivity-proven threshold:

let bg = imgscan::Rgba::rgb(11, 13, 16);
let r  = imgscan::scan(img.as_raw(), img.width(), img.height(), bg);
// a clean render: low spoke/high-freq, coverage in a band, centred mass:
assert!(r.spoke_score < 0.30, "no starburst smear");
assert!(r.high_freq_ratio < 0.30);
assert!(r.coverage > 0.02 && r.coverage < 0.45);
assert!((r.centroid.0 - w as f64 / 2.0).abs() < w as f64 * 0.25);
assert!(!r.bbox.is_empty());

The smear signature (the OSM-3D green-spokes class) is high spoke_score with near-total coverage. Prove your threshold separates a good frame from the bug — draw a clean synthetic frame and a smeared one and assert a threshold splits them (the pattern in imgscan.rs's own clean_and_smeared_are_clearly_separated).

f6 — the tests/ seam: writing / running a call-chain matrix

Each crate has three test files. The pattern to add a new component to the matrix:

tests/callchain_matrix.rs — enumerate the input→pixel chain. Give each non-pixel stage an INJECT-input → ASSERT-return test (build counts, layout finiteness/in-rect, edge endpoints). Give the pixel stage the matrix: sweep your axes (topology × size × canvas × layout × theme), render each cell with harness::render_themed_sized, assert state_json == the model, count drew(), and emit one summary row. Model it on facett-graph/tests/callchain_matrix.rs.
tests/robot_callchain.rs — drive the populated component through egui_kittest::Harness, run_steps(3), render(), and assert on the pixels: non-blank AND confined to the widget rect, plus (optionally) imgscan::scan. render() returns Err with no wgpu adapter — SKIP gracefully (return), don't fail. Save the final frame under tests/snapshots/.
tests/snapshot.rs — render the populated component to a golden PNG and assert a non-trivial byte size.

Run them:

cargo test -p facett-graph --test callchain_matrix          # return-asserts + matrix
cargo test -p facett-graph --test robot_callchain -- --nocapture   # pixel legs (needs GPU)
cargo test -p facett-graph --features testmatrix --test callchain_matrix   # + emit rows

To drive the whole demo headless (every tab, every control) and read the runtrace ledger, see facett-demo/tests/pop_ran_list.rs and facett-demo/tests/robot_deck.rs; running the full mega-matrix / robot suite from a cold shell is walked keystroke-by-keystroke in harness-for-idiots.md.