nordisk/nornir

CLI Command Contract — the autopilot's CLI ⟺ UI correspondence

Status: design + step 1 shipped (2026-06-23). Goal (Rickard): "every UI element AND every CLI command must correspond, and test data must be sane on BOTH. CLI commands don't have a UI trait — fix that." Autopilot testing is the goal.

The asymmetry we're closing

The test matrix discovers UI surface mechanically and asserts sane data on each:

CLI had only half of this:

• discovery was shallow — Cli::command().get_subcommands().map(get_name) saw the ~25 top-level roots, NOT nested leaves (bench history, workspace ls, mimir deps-of);
• there is no uniform sannr check (truth-verdict) — each command prints ad-hoc; --json exists on some commands, inconsistently. There is no state_json equivalent the matrix can assert.

So a CLI command could rot (wrong/empty output) and the green matrix wouldn't notice — the exact blind spot the autonom gate exists to forbid.

The contract (CLI twin of Facet)

1. Discovery — recursive leaf enumeration ✅ SHIPPED

clap_command_paths() (src/bin/nornir.rs) recurses the clap tree and returns every leaf command path, space-joined ("bench history", "mimir deps-of", …). It feeds all three autonom gate sites (gate_coverage_for_repo / nornir_surface_inputs), so Gap = Surface − Covered now spans EVERY command, not just the roots. This is the CLI twin of viz_surfaces() — mechanical, never a hand list (kills the drift in tests/cli_parity.rs).

2. Sane-data check — CommandOutcome ⬜ NEXT

Every command handler returns a uniform, machine-readable outcome — the CLI's state_json:

pub struct CommandOutcome {
    pub command: String,           // canonical leaf path, e.g. "bench history"
    pub ok: bool,                  // semantic success (not just process exit 0)
    pub data: serde_json::Value,   // the READABLE DATA the command produced (the check target)
    pub human: String,             // the pretty/table rendering for a terminal
}

A global --json prints data; otherwise human. The dispatch in main() prints the outcome centrally instead of each arm printing ad-hoc. The matrix then runs each enumerated command against the same fixture workspace the viz atom-walk uses and asserts the outcome is sane: ok == true, data non-empty / expected shape, no error sentinel (ERROR_MARKERS). This is identical in spirit to the deck_no_error_atoms + state_json check — "sane data on every CLI command."

Roll-out is per-command (whole-codebase), but mechanical and gate-driven: an un-migrated command shows as an uncovered/data: null surface, so the gate names exactly what's left.

2a. CommandOutcome is presentation, NOT a transport ⚠️ READ THIS

The data path is unchanged and stays canonical:

• fat mode → direct nornir lib function calls (Rust types) — e.g. bench history calls nornir::bench::query_bench_runs(...).
• thin mode → gRPC to nornir-server (protobuf) — the server does everything via RPC.

CommandOutcome wraps the already-fetched result purely for OUTPUT: human to the terminal, data (JSON) only for --json/tooling and the autopilot check. The JSON lives at the edge — the same edge the UI's state_json lives at, and where MCP tools already speak JSON. It is never the wire. So data: serde_json::Value is a comparison/introspection shape (it lines up with the UI state_json slice for parity), not a serialization layer that competes with gRPC or the lib calls.

3. CLI ⟺ UI parity — same data, two faces ⬜ NEXT

Many commands are the CLI face of a viz tab (both read the same warehouse table):

The deepest parity is asserted at the lib / gRPC level, not by scraping JSON stdout:

• fat: the CLI command and the viz tab call the same lib fn (query_bench_runs, load_clone_events, …). The parity test calls that fn directly and asserts both faces consume it — cli_outcome.data and viz_state_json[slice] are then equal by construction.
• thin: both go through the same gRPC handler (Bench, Viz.CloneEvents, …); the test asserts the handler is the shared source.

The cli_outcome.data ≈ viz_state_json[slice] comparison is the edge check on top of that shared-source guarantee. Together they enforce the standing CLI-parity LAW ([[coolest-modern-CLI-parity]]) mechanically: same data path, two faces — if a tab shows data, its CLI twin shows the same data, and vice-versa.

Why this is the autopilot

Green matrix ⟺ (every UI atom emits sane data) AND (every CLI leaf emits sane data) AND (every twin agrees). Both surfaces are discovered from facts (clap tree + Facet registry), both are asserted on real injected data, and the completeness gate fails on any gap — no hand lists, no silent rot. That is "autopilot testing."

Build plan

1. ✅ Recursive clap_command_paths() + wire 3 gate feeders + unit test (nested leaves found).
2. ⬜ CommandOutcome type + global --json + central print in main().
3. ⬜ Migrate handlers to return CommandOutcome (per-command; gate names the remainder).
4. ⬜ Generic CLI sannr check (truth-verdict) test (run every leaf vs fixture workspace, assert data).
5. ⬜ CLI⟺UI parity tests for the twin table above.
6. ⬜ Retire the hand lists in tests/cli_parity.rs (subsumed by 1+4).