nordisk/nornir

Workspace auto-populate — design + guide (EPIC POPULATE)

When a workspace_<name> is registered/created, nornir should fully populate it with no manual steps: clone the members, build all the warehouse facts, run a light test matrix + bench smoke, and stream every step's outcome (incl. failures) to the warehouse so the viz thin client can watch it fill — green, red, or NOT RUN. Forged with Rickard, 2026-06-14.

1. Why

Today a new workspace lands half-empty:

• nordisk registered but no members materialized (empty repos: []).
• korp: git synced but current_snapshot: "" → no dep-graph / BM25 → MCP search returns nothing.
• Clone/populate failures only hit the server's stderr — a viz/CLI client sees the coarse sync_state (ok / error: …) but not which step failed or why.
• The test matrix never runs on populate, so the Test pane has nothing to show (and individual tests aren't even discovered → no NOT-RUN cells).

The bencher already proves the run→persist→view shape; populate should do the same for the whole workspace. This is the missing collection half of [[test-matrix.md]] applied at workspace-bootstrap time, persisted per [[persist-to-warehouse]] law.

2. The pipeline (current → target)

register/fetch ─▶ clone members (gix HTTPS / SSH deploy-key)   [fetch_workspace]
                      │
                      ▼  republish_with (monitor.rs) — EXISTS:
   1 dep graph  2 knowledge+security  3 BM25 index  4 iceberg snapshot  5 dep-graph persist
                      │
   ┌──────────────────┴───────── NEW (this epic) ─────────────────────┐
   6 test SURFACE discovery (nextest list)  → NOT-RUN rows
   7 LIGHT test matrix (build·unit·doctest·clippy·fmt·audit, no #[ignore]) → test_results
   8 LIGHT bench smoke (NORNIR_BENCH_QUICK, NOT the required_results corpus) → bench_runs
   └───────────────────────────────────────────────────────────────────┘
                      │
   EVERY step (1–8, + clone) emits a row to `release_events`  ──▶ viz run/event pane

Steps 1–5 already exist; 6–8 + the workspace_events stream are the work.

3. "Light" — the contract (never the heavy release path)

Light test = on by default (it's the point — see the matrix). Bench smoke = also on by default (Rickard, 2026-06-14 — adding a workspace is rare, so pay the cost once and get the full picture). Discovery always runs (cheap, and it's what makes the Test pane non-empty on a fresh workspace).

4. Failure surfacing — REUSE release_events (no new table)

Adding a workspace is rare; a dedicated table would be overkill (Rickard, 2026-06-14). release_events is already a generic per-step event stream the viz renders:

run_id · seq · ts_micros · component · repo · op · phase · status · detail · depends_on · elapsed_ms

A populate is just a run there: op ∈ {clone,dep_graph,knowledge,index, snapshot,test_discover,test,bench}, phase ∈ {start,end}, status ∈ {ok,fail,skip,running}, repo = the member ("" = ws-wide), detail = the message. record_release_event already writes this shape.

• fetch_workspace + republish_with emit these rows (keep the eprintlns for the live server log too).
• Read paths (all exist): the viz run/event pane + a CLI nornir workspace status [--server …] (CLI parity per [[coolest-modern-cli-parity]]). The registry sync_state stays the one-word summary; release_events is the drill-down.
• Zero new schema.

5. Triggers & surface

• Auto: on Workspaces.Register + the server poll loop (already call fetch_workspace→republish); extend republish with steps 6–8.
• Manual: nornir workspace populate <name> [--heavy] [--no-test] [--no-bench] — thin (RPC) + fat (local), idempotent (skip members whose HEAD SHA was already covered, like the knowledge-scan ledger). Light test + bench smoke are on by default; --heavy opts into the full corpus, --no-* skips a stage.
• Clone bootstrap: make nornir workspace fetch the single materialize verb; confirm SSH-deploy-key members clone (the fetch_workspace path) and wire the descriptor/fat resolve_sources to call it instead of only printing a hint.

6. Reuse (no new engines)

• Tests: the existing nornir-testmatrix engine + nornir test runner; persist via the existing test_results table + TestSink.
• Bench: the existing bench::api + bench_runs table, with NORNIR_BENCH_QUICK.
• Discovery: nornir_testmatrix::{parse_nextest_list, parse_cargo_test_list, listed_rows} (already exist) → NOT-RUN rows.
• Graph/index/snapshot: unchanged.

7. Test plan (LAW: every change ships matrix tests — [[implementation-gets-test-in-matrix]])

• light_test_aspects_exclude_heavy — the populate aspect set omits #[ignore]/coverage/msrv.
• populate_emits_release_events — append a clone/fail populate row, query it back (inject→assert).
• populate_persists_not_run_rows — discovery yields NOT-RUN rows for a fixture with un-run tests.
• integration: populate a tiny fixture workspace → assert release_events has clone…ok, index…ok, test…(ok|fail) rows and the Test pane reads them.
• CLI parity: nornir workspace populate + status advertised + route thin/fat.

8. Decisions (settled 2026-06-14)

1. Bench smoke — on by default (rare event, pay once). --no-bench to skip.
2. Events — reuse release_events (no new table); a populate is a run with op = the step name.

Source map

• Clone/fetch: src/monitor.rs::fetch_workspace, src/gitio.rs::clone_or_fetch.
• Populate: src/monitor.rs::republish_with (add steps 6–8 + record_release_event emits).
• Tests/bench engines: crates/nornir-testmatrix, src/bench/.
• Warehouse: existing release_events, test_results, bench_runs (no new tables).
• CLI/RPC: src/bin/nornir.rs (workspace populate/status), proto/nornir.proto.