nordisk/edda

nornir-robotui — WATCH mode (one scenario, two backends)

Convention (all nordisk UI surfaces — facett, korp, nornir viz, dwarf): every UI surface ships ONE Scenario that runs in BOTH modes — the HEADLESS matrix drive (CI) and a VISIBLE watch window a human can watch the robot move through. The mode is a selectable backend over the same scenario, never a fork of the scenario format.

Why

nornir-robotui was headless-by-design: an off-screen egui_kittest Harness drives AccessKit interactions and emits nornir_testmatrix::TestResultRows (the matrix fills). Rickard's ask: "make it choosable — I want BOTH modes." Because a Scenario/Step/Action/Assert is already plain data, the watch backend is additive: same data, different driver.

The two backends

The lean default build must stay windowing-free (korp / facett-demo / mannequin pull the matrix harness without eframe). eframe/egui-serde live behind the watch feature; the lean build compiles + tests green with watch OFF.

How the watch backend injects input

egui computes pointer/keyboard interaction in begin_pass from the frame's egui::RawInput. eframe's App::raw_input_hook(ctx, &mut raw_input) is called with that RawInput before it is processed, so the driver PUSHES synthetic events there: a PointerMoved to animate a visible cursor toward the target, a PointerButton press+release to click, Text/Key to type, MouseWheel to scroll, PointerButton press→move→release to drag. Same event kinds the headless inject_events path feeds a kittest Harness — here they drive a real window.

label → widget: the consumer's draw(ui, app, targets) closure records each targetable widget's on-screen egui::Rect (t.record("Bump", resp)) — the watch-mode analogue of .labelled_by(..) in the headless path. The driver resolves an Action's label to that rect's centre and steers the cursor there. Limitation: Action::ClickById has no rect side-channel (it addresses an AccessKit node id), so watch dwells on it without a click; it still drives fine headless. Everything a closure records — the common case — animates.

The consumer seam — adopt both modes in ~10 lines

A repo writes ONE Observable, ONE draw(ui, app, targets) closure and ONE Scenario, then picks the mode:

use nornir_robotui::{run_scenario_mode, Observable, RunMode, Scenario, Step,
    Action, Assert, Targets, WatchConfig};
use serde_json::{json, Value};

struct MyApp { clicks: u32 }
impl Observable for MyApp { fn state_json(&self) -> Value { json!({ "clicks": self.clicks }) } }

let draw = |ui: &mut egui::Ui, app: &mut MyApp, t: &mut Targets| {
    if t.record("Bump", ui.button("Bump")).clicked() { app.clicks += 1; }   // record → both modes
};
let scenario = Scenario::new("smoke").step(
    Step::new("bump").action(Action::ClickButton("Bump".into()))
        .assert(Assert::State { pointer: "/clicks".into(), expected: json!(1) }));

// Headless (CI): run_scenario_mode(.., RunMode::Headless, ..) → RunOutcome::Headless(rows).
// Watch:         ROBOTUI_WATCH=1 flips RunMode::from_env() → a real window.
run_scenario_mode(MyApp { clicks: 0 }, draw, &scenario, "my-repo",
    RunMode::from_env(), WatchConfig::default())?;

Mode is chosen three ways, all landing in RunMode:

1. API — the mode: RunMode param to run_scenario_mode.
2. env — ROBOTUI_WATCH=1 via RunMode::from_env() (pace/hold knobs: ROBOTUI_WATCH_DELAY_MS, ROBOTUI_WATCH_LINGER_MS, ROBOTUI_WATCH_HOLD=1).
3. CLI — robotui run <scenario.toml> (headless) / robotui watch <scenario.toml> (window). The robotui bin is behind required-features = ["watch"]; consumer repos usually just call run_scenario_mode from their own test/bin rather than shipping this bin.

The shipped proof (demo)

src/demo.rs is a trivial component (a "Bump" button + a "Name" text field) with bump_scenario(). The SAME scenario:

• headless — tests/watch_duality.rs::demo_scenario_fills_the_matrix_headless asserts 4 green robot-ui matrix rows;
• watch — the canonical scenario is committed at .nornir/scenarios/bump-demo.toml; watch it move:

cargo run -p nornir-robotui --features watch --bin robotui -- \
    watch crates/nornir-robotui/.nornir/scenarios/bump-demo.toml
# opens a window on your display and visibly clicks "Bump" twice + types "Heimdall".
# Slower/held:  ROBOTUI_WATCH_DELAY_MS=1200 ROBOTUI_WATCH_HOLD=1 cargo run … -- watch …

Under DISPLAY=:0 this drives the real widgets to {"clicks":2,"name":"Heimdall"} with all four step verdicts green (run with ROBOTUI_WATCH_DEBUG=1 to print that final state).

The overlay verdict of a CHAINED assert (2026-09-01)

.assert(a).assert(b) accumulates into one Assert::All (see robot-ui.md). The watch overlay evaluates the conjunction, not the last clause:

• any clause the overlay CAN evaluate and that is false → red;
• else at least one evaluatable clause and all such are true → green;
• else (nothing evaluatable in watch mode — an id-addressed FieldValue, say) → neutral, exactly as a single unevaluatable clause has always been.

An unevaluatable clause therefore never drags a known-true chain down to neutral and never invents a verdict of its own. This matters because the overlay and the headless matrix must not disagree about the same step: before the fix both collapsed a chain to its final clause, so they agreed by being equally wrong.