nordisk/edda

nornir-robotui — design + guide (EPIC ROBOTUI)

The reusable robot-UI testing framework: auto-click around a real egui app, scroll, press buttons, type — and watch the test matrix fill up. A lean sibling crate (like nornir-testmatrix) so korp / facett-demo / mannequin / nornir's own viz can pull it without nornir's heavy deps.

1. Why a crate

Rickard's ask, verbatim: "auto-click around, scroll, press buttons, and watch the test matrix fill up." That is three concerns, each with a confirmed nordisk seam, so the framework is just the glue:

Keeping it a lean sibling (deps: egui_kittest + egui + nornir-testmatrix + serde_json + anyhow) means a leaf repo pays only for the tester, never for iceberg/arrow/tonic.

2. The three pieces

Observable (observe)

pub trait Observable {
    fn state_json(&self) -> serde_json::Value;
}

One method. If your app already has state_json() (nornir's viz, facett Facets), the impl is a one-line delegate. The robot asserts against this with RFC-6901 JSON pointers (/tab, /test/runs/0/run_id) — it reads back what the user would see, never pixels.

RobotSession (drive + assert)

Wraps an egui_kittest::Harness<A> over an A: Observable.

• actions — click_label(s), click_button(s), type_into(label, text), key(egui::Key), scroll(Vec2), drag(from, to), step()/run(n), and (feature snapshot) snapshot(name) / render().
• R6 widget-level actions — set_field(target, value) (set any text/checkbox/ slider, by label or AccessKit node id), click_by_id(id) (click by node id when labels collide/are empty), inject_events(events) (feed an arbitrary Vec<egui::Event> for one frame — the lowest-level escape hatch). See ../../.nornir/robotui-richer-livedrive.md.
• assertions — label_visible(s) -> bool, state_at(pointer) -> Option<Value>, and (R6) field_value(target) / field_checked(target) read the value/toggle straight off the rendered widget.
• every action runs through the real AccessKit tree and steps a frame, so the app actually mutates — these are not fake state pokes.

Scenario runner (emit)

Scenario { suite, steps: Vec<Step { label, action: Action, assert: Assert }> }

run_scenario(&Scenario) -> Vec<TestResultRow>: for each Step it performs the Action, evaluates the Assert against the post-action state_json, and records one row (suite = scenario.suite, test_name = step.label, aspect = "robot-ui", status = pass|fail). All rows share one run_id so they roll up as one matrix run. A failed action (missing button) or a failed assertion is a red cell — never a panic. Action/Assert are plain data (String / serde_json::Value / egui::Key / egui::Vec2), so a scenario can live in a .toml/JSON file.

This is the "drive every button → the matrix fills" call: one invocation, one row per step, ship to any TestSink.

Chained asserts ACCUMULATE (fixed 2026-09-01)

.assert(a).assert(b) means a AND b, both checked, in source order. The builder folds the chain into one Assert::All(vec![a, b]); Assert::None is the identity, and nested Alls flatten, so a chain of n clauses is always one flat All of n. Step::asserts([a, b]) is the explicit form of the same thing. check walks every clause and reports the first failure as clause k of n: <reason>, so a failure names its position in the chain.

It used to be a setter. Step::assert was self.assert = assert, so a chain silently kept only the LAST clause and every earlier one was dead code that ran nothing and reported green. Measured across /home/rickard/git on 2026-09-01: 68 chained call sites discarding 136 clauses — 129 of them in korp, 5 in ordning, 1 in facett-demo, 1 in this crate's own cmd_json test.

Consequences worth remembering:

• The watch overlay's verdict is the conjunction too (WatchDriver::verdict_of), not the last clause — otherwise the live window and the matrix would disagree.
• Any match on Assert outside this crate needs an All arm. The one in nornir-fleet's fleet_mock_server recurses; a catch-all _ arm turns an evaluatable chain into a false red.
• korp and facett consume the published nornir-robotui 0.3.x from crates.io, not this path — so their clauses stayed discarded until this crate shipped. Done: nornir-robotui 0.3.4 was published to crates.io on 2026-09-01 (via dwarves crate-release --only … --bump keep --execute; the five in-repo siblings were AlreadyPublished and correctly skipped by the source-drift set, robotui was the only upload, indexed in 66 ms).

What the consumers got, measured after the bump

korp needed zero call-site edits — all 62 sites already said the right thing; only the Cargo.lock pin moved.

facett-demo could not be reached by a version bump at all. Its tests/robot_demo.rs carries an in-crate PORT of this crate's API (a LAW 5 twin), and the port had the identical setter. It had to be fixed separately. If you need one argument for deleting a twin, this is it: the same bug, in two places, found once and fixed twice.

Liveness was proven red-first, not assumed. A guaranteed-false clause injected as the FIRST clause of all 62 korp chains:

Under 0.3.0 the suite could not tell the probes were there. The 7 chains between the source census (62) and the runtime count (55) sit behind #[ignore] or inactive cfgs — robot_demo_graph_seam.rs (3), robot_sdp_viewer.rs:211 (fluid-sph), robot_spark_viewer.rs:350 (live-pipelines), robot_sdp_on_spark.rs:130 (live tier), and one more. Those clauses have still never run.

2b. Two modes over ONE scenario — headless matrix + visible watch

Because a Scenario is plain data, the SAME steps run two ways, chosen with RunMode (see robotui-watch-mode.md for the full design + adoption):

• RunMode::Headless (default, lean) — the drive above → TestResultRows (CI).
• RunMode::Watch (feature watch) — a real eframe window that replays the scenario paced + visible: a cursor moves, clicks, types, scrolls, while synthetic RawInput events (injected via eframe::App::raw_input_hook) drive the live widgets. eframe/winit/glow are behind the watch feature so the lean matrix harness never pulls a windowing stack.

One seam gives both: run_scenario_mode(app, draw, &scenario, repo, mode, cfg) with ONE draw(ui, app, targets) closure. Mode = an API param, ROBOTUI_WATCH=1 (RunMode::from_env), or the CLI (robotui run|watch <scenario.toml>). Watch is purely additive — RobotSession::run_scenario is unchanged. Watch it move:

cargo run -p nornir-robotui --features watch --bin robotui -- \
    watch crates/nornir-robotui/.nornir/scenarios/bump-demo.toml

3. The snapshot feature split

The default build is light: drive + observe + emit, no GPU. The optional snapshot feature pulls egui_kittest's wgpu renderer (and image) so RobotSession::snapshot(name) writes golden-compared PNG proof and render() returns raw pixels. Headless CI that only asserts state_json stays GPU-free; visual-proof runs opt in with --features snapshot.

4. Reference consumer — nornir's own viz

tests/robot_viz.rs in the nornir root crate is the canonical example (inject-assert LAW #1): it builds the real UrdrThreadsApp, seeds test_results rows, wraps it in a RobotSession, then:

• drives real AccessKit clicks on the tab headers (🚀 Release, 🔗 Dep Graph, 🛡 Security, 🧪 Test) and asserts state_json()["tab"] switched;
• asserts the seeded run renders after the Test-tab click (/test/runs/0/run_id);
• runs a 6-step Scenario and asserts it returns 6 green robot-ui rows, then round-trips them through a JsonFileSink and prints the filled matrix.

The viz already exposes state_json() + apply_command; the robot reuses them.

5. Idiot guide — how ANY egui app adopts it

1. Implement Observable. If you already have state_json(), delegate:

impl Observable for MyApp {
    fn state_json(&self) -> serde_json::Value { self.state_json() }
}

(Don't own the app type? Wrap it: struct W(TheirApp); and impl on W — that's what robot_viz.rs does for the foreign UrdrThreadsApp.)

1. Build a Harness + RobotSession.

let harness = Harness::builder()
    .with_size(egui::Vec2::new(1400.0, 900.0))
    .build_state(|ctx, app: &mut MyApp| app.draw_ui(ctx), my_app);
let mut robot = RobotSession::new(harness, "korp");   // "korp" stamps each row's repo

(Single-widget UI? Harness::new_ui_state(|ui, app| …, app) instead.)

1. Make widgets addressable. Buttons label themselves with their text (click_button("Run")). A TextEdit becomes addressable by linking a label:

let lbl = ui.label("Query");
ui.text_edit_singleline(&mut self.query).labelled_by(lbl.id);   // type_into("Query", …)

1. Script + run.

let scenario = Scenario::new("korp-smoke")
    .step(Step::new("open Map tab")
        .action(Action::ClickLabel("🗺 Map".into()))
        .assert(Assert::State { pointer: "/tab".into(), expected: json!("Map") }))
    .step(Step::new("map drew").assert(Assert::Drew));
let rows = robot.run_scenario(&scenario);
JsonFileSink::new("target/robot-matrix.jsonl").append(&rows)?;   // or nornir's warehouse

That's it — the robot auto-clicks around and the matrix fills.

6. Roadmap fit

ROBOTUI in the MASTER QUEUE; the next consumers are KORP-MATRIX (drive korp's Demo/Map/Graph tabs) and the facett-demo robot-UI row (D5b2), plus nornir-viz adopting it. Each is a Scenario + an Observable delegate — no framework changes needed.