nordisk/nornir

Always-visible test matrix — ok | fail | X, discovered build-free

Design + guide for the feature: the 🧪 test matrix is ALWAYS visible — every test shows ok | fail | X (X = not-run), populated on initial workspace load without compiling anything. The split that makes it cheap: discovery is free (the syn scan already parses every .rs), execution is expensive (a cargo build), so discovery rides initial load while execution stays gated to nornir release / nornir test run.

Builds on test-matrix.md (the C6 nornir test runner + test_results table) — read that first. This doc covers ONLY the build-free inventory + the inventory⟕results join.

1. The core idea — discover-on-load vs run-on-request

Running a test needs a cargo build (compiling the test binaries) — slow, so it must NOT happen on initial workspace load. But discovering tests is free: the syn knowledge scan (knowledge::symbols) already parses every .rs on initial load. So:

• Discover during the syn scan (no build) → every test gets an X (not-run) cell immediately on workspace add.
• Execute (which fills ok/fail) happens ONLY in nornir release (default) or on explicit nornir test run — never on initial load, never in the poll/monitor loop.
• The viz Test pane + nornir test list show, for every discovered test: ok/fail if a run result exists, else X.

initial workspace load (NO cargo build)
  knowledge syn scan ── parses every .rs ──▶ SymbolScan { symbols, calls, features, TESTS }
                                                                                  │
                                            test_inventory  ◀── append/ingest ────┘   (one row per #[test], is_heavy/is_async)

nornir release / nornir test run (the ONLY execution paths — a cargo build)
  runner ── cargo nextest run / cargo test ──▶ test_results  (ok/fail per case)

viz Test pane / nornir test list
  test_inventory  ⟕ LEFT JOIN  test_results   ON (repo, suite=crate_name, test_name)
    ⇒ ok    (latest result is pass)
      fail  (latest result is fail/stalled)
      X     (no result — discovered but never run)        + is_heavy tag

2. PART A — discover tests in the syn scan (no build)

knowledge::symbols gained a TestDefRow and a SymbolScan.tests vec alongside symbols/calls/features. In the syn walker, maybe_emit_test fires for any fn (free or impl) whose attributes mark it a test — the path's last segment is matched so namespaced forms are caught without resolution:

• #[test], #[tokio::test], #[async_std::test], #[test_case(...)], #[rstest]/#[rstest::rstest], #[proptest].
• #[ignore] → is_heavy = true (the container/corpus/matrix tests).
• async fn or #[tokio::test]/#[async_std::test] → is_async = true.

This runs on the SAME initial-load knowledge scan (holger/nordisk/…), zero extra build. It rides every scan path: scan_repo (disk walk) and scan_crate_entries / scan_file (the in-memory .crate tarball path used by deep-scan).

3. PART B — persist + the matrix join

test_inventory table (iceberg_schema::test_inventory)

Partitioned by repo (like symbol_facts/call_edges). One row per discovered test per snapshot: snapshot_id, ts_micros, repo, crate_name, module_path, test_name, file, line, is_heavy, is_async.

Write path (iceberg.rs)

scan.tests rides the existing symbol-scan writers — append_symbol_scan_async (per-scan), append_symbol_scans_async (bulk), and ingest_symbol_scans_async (the parallel deep-scan flush). The parallel path uses the SAME build_groups_parallel + skade::ingest_parallel pattern as symbol_facts (gatling worker-pool over buckets, NO rayon per nornir's pool law).

Read + join (warehouse::test_inventory)

• query_test_inventory(wh, repo) — latest-snapshot inventory for a repo (predicate-pushed on repo, max ts_micros, dedup — mirrors knowledge::query::load_latest).
• join_matrix(inventory, results) — the LEFT JOIN, keyed (repo, suite=crate_name, test_name). The newest test_results row wins; discovery- only listed rows are skipped (the inventory IS the discovery source now). A test with a pass → ok, fail/stalled → fail, anything else / no row → X.
• query_test_matrix(wh, repo) — reads both sides + joins. The CLI + viz call it.

The ok | fail | X semantics: X is not-run — the test EXISTS (discovered on load) but has never been executed. It is its own state, never conflated with a pass or a skip.

4. PART C — the viz Test pane + CLI parity

Viz (viz::test_tab)

The pane holds matrix_rows: Vec<TestMatrixRow>, built on load:

• local — opens the warehouse read-only, query_test_inventory per workspace repo + query_test_results, then join_matrix.
• remote — Viz.TestMatrix RPC returns {inventory, results}; the thin client joins them identically (the join is pure).

The always-on per-test table renders every row with a tri-state cell — ok (green) / fail (red) / X (grey not-run) — grouped repo→suite→test, with a ⚠ heavy tag. state_json()["test"]["inventory"] exposes the full matrix (every test + its ok|fail|X state + heavy flag + totals) so the harness asserts what's rendered (LAW "components expose readable data").

CLI (nornir test list)

nornir test list [<repo>]              # the ok|fail|X matrix (omit repo = all)
nornir test list <repo> --json         # raw TestMatrixRow rows
nornir test list <repo> --heavy-only   # just the #[ignore]'d heavy tests
nornir test list <repo> --not-run      # just the X (not-run) tests
nornir test list <repo> --server <url> # thin client (Viz.TestMatrix)

Same matrix the viz shows — no viz-only capability (LAW "CLI parity").

5. PART D — heavy/light gating (the rule)

HEAVY tests + the bencher run ONLY in nornir release (default) or on explicit request — NEVER on initial workspace load or in the poll/monitor loop. Audit result:

• monitor::discover_test_surface used to run a cargo test --list / cargo nextest list (which compiles the test binaries — a build) on every populate. Changed: it now reads the build-free test_inventory (written by step 1's syn scan) and just logs the per-member count to release_events. No build, no execution, on load or in the poll loop.
• The eager-register path in nornir-server.rs populates via the monitor republish → the same build-free path. No test/bench execution on register.
• EXECUTION lives only in: the Ops.RunTestMatrix RPC (the viz "Run" buttons / nornir test run thin path), the local run_test_one (nornir test run/all), and release::pipeline (heavy = !NORNIR_SKIP_HEAVY, so heavy runs by DEFAULT with --skip-heavy opt-out — unchanged).
• Reconciled NORNIR_POPULATE_SKIP_TEST + discovery: discovery is now ON by default at initial load (it's free, part of the knowledge scan); the env var only skips the per-member logging step. Only EXECUTION stays gated.

6. PART E — tests (LAW #1 inject-and-assert) + completeness

• knowledge::symbols unit (scans_test_inventory_with_heavy_and_async_flags): a fixture crate with #[test] / #[tokio::test] / #[ignore] / #[test_case] fns → asserts the scan harvests exactly them with correct is_heavy/is_async, and a plain fn is NOT a test row.
• warehouse::test_inventory units: join_left_outer_gives_x_for_not_run_and_ok_fail_for_results (no result → X, pass → ok, fail → fail, newest wins, heavy rides through, listed never counts) + inventory_roundtrips_and_joins_against_results (write a SymbolScan with tests, read it back, join against persisted results).
• tests/viz_test_pane.rs::test_pane_inventory_shows_ok_fail_x_per_test_with_heavy_flag: injects a concrete inventory + partial results, asserts state_json["test"]["inventory"] lists every test with its tri-state + heavy flag and that EVERY cell is a real ok|fail|X (no TODO/not-wired sentinels — the green-matrix completeness law).

Test scratch uses tempfile::tempdir() (never the T9 mount).