nordisk — THE BIG PLAN (single source of truth)
THE one plan — filter every idea in HERE. Rickard + collaborators add/triage ideas in this file only. Renamed from
development-plan.mdand absorbs the old untracked/run/media/rickard/T9/git/THE_BIG_PLAN.mdscratch (deleted 2026-06-14). Living roadmap for the nornir + facett + warehouse + nordisk megaproject (started 2026-06-12). Tracked innordisk/nornir, renders into the docs book. Checkboxes tick on real close (see Definition of Done).
🚀 FLEET 2026-07-08 — nornir agent session (landed + pushed to main)
One-agent-per-repo fleet pass on nornir. Shipped, each its own commit, cargo green, pushed:
- #13 static-capabilities bench seam —
BenchResultgains asource(BenchSource::{Measured,Static}) tag (serde-default, skipped-on-wire when Measured),static_capabilities()/measured()ctors +machine_capabilities()writer; the no-regression gate now skips static rows. - core-saturation warehouse-gate —
gate::core_saturation{,_run}enforces the ~11/12 LAW (peakcores_busy_max< 0.9·N fails;_st/single-core exempt);[gates].core_saturation/min_core_busy_frac; wired in the release pipeline. - #18
--autoapply-fns — the three missing AUTO executors wired:tidy-dirty(shed.claude/+*.arrowsnoise),bump-skew-join(reuseapply_skew_bumps),cut-cycle-cheap(drop the cheap MFAS edge — dev outright / optional + feature scrub). - #21 POPULATE 6-8 — new
workspace_eventsIceberg table + per-STEP stream; populate steps 7 (foldarch generateper member → historized wiring) + 8 (populate roll-up + flush); steps 3/5/6 also emit events. - #4 remote-warehouse Phase 5 (un-held) — FULLY CLOSED (read + write-back). READ:
warehouse::open{,_read_only}"remote" clones the git warehouse mirror (pure-Rust transport, per-URL cache) and opens it with the local IcebergWarehouse reader. WRITE-BACK:warehouse::republish_remotecommits the mutated mirror and fast-forward-pushes it back viagitio::push_to_local_remote(local/file remotes; gix 0.84 has no send-pack so network push stays out-of-band; non-ff fails clean). The thin-RPC coverage test was already present (stale TODO); the funnelmockgenerator is intentionally echo-only (compiler-oracle correctly rejects it) —decompose()is already general, so no fake-success mock was added.
🔄 RE-SYNC 2026-06-30 — canonical tick-list catch-up (folded from the session journal)
This canonical plan was last broadly touched 2026-06-25. The session journal (
workspace_nordisk/PLAN.md, the woven cross-repo narrative — it now defers here) carried newer state. Folded in below so nothing drifts. Inline boxes in the sections that follow are also ticked where the audit verified the work shipped.
2026-06-25 — SCIP → FUNCTIONAL pivot. The resolved SCIP knowledge index landed FAST + CORRECT + rayon-free (gatling single-pass goto_definition; 35 min → 63.5 s after the vfs_rel_path workspace-scoping fix; consumers prefer the resolved index via query.rs::load_preferred; scan_repo parallelized 3.3×). Direction then pivoted from SCIP-perf to functional work: (A) metro feed reads the resolved index, (B) the functional test-matrix / autonom completeness gate (Aspect::Functional/functional_status), (C) resume-plan leftovers. Roadmap: nornir/.nornir/scip-resolved-knowledge.md; narrative: workspace_nordisk/PLAN.md §"RESUME HERE — 2026-06-25 (SCIP perf landed → FUNCTIONAL pivot)".
2026-06-29 — four deep analyses (loki bench host + three studies). Stood up loki as a nornir bench host (nornir-server systemd user-scope :7878, 4 TB /mnt/work4t, workspace_holger populated; see memory loki-nornir-bench-host.md) and ran the heavy benches; then three deep multi-agent studies landed as planning docs (all 🔄 roadmaps):
- warehouse → "Ragnar-fast" —
workspace_nordisk/.nornir/warehouse-ragnar-fast.md. Near-free wins: bloom filters are written but never read (5–50× point lookups); the provenscan_limited35 s→0.66 s fix is wired only into preview; knowledgecallers_of/callees_ofare linearVecscans (the inverted index = the first form of the parked Ragnar static search tree). Notes the min/max bounds "GAP" is already CLOSED in code (see skade item below). - facett → game-quality rendering —
workspace_nordisk/.nornir/facett-game-quality-rendering.md. Two render worlds (GPU map3d/water/PBR vs static egui forms with motion infra built-but-unwired); Tier 1 (wire SmoothScroll/tweens/glass chrome) → Tier 2 (HDR+bloom, FXAA, backdrop-blur) → Tier 3 (SSAO/sky/CSM, graph3d GPU lane). Fastest path = FXAA + HDR/bloom on the existing map3d offscreen target first. (Tier-2 effects shipped — see below.) - docs crispification + prune —
workspace_nordisk/.nornir/docs-crispification-plan.md. The crisp nordisk README convention + a review-gated prune list (~58 DELETE / ~30 MERGE across 12 repos). Nothing deleted — review first.
Warehouse writer-side perf — DONE (reconciles the journal's stale "queued"). Both landed and are verified (nornir warehouse-maintenance+sort+zstd 5/5 + skade-bloom 4/4):
- C-perf ✅ (was
agent/warehouse-zstd) — zstd(3) on every data-file write + declared SortOrder on the big per-repo tables. - C-perf-2 ✅ (was
agent/warehouse-maintenance) —nornir warehouse compact(merge small sorted files, preserve sort/bounds) +nornir warehouse expire(--keep-last N/--keep-days Dsnapshot + orphan-file GC); both emitJobRecords. The earlier "iceberg-rust 0.9 lacks rewrite/expire" blocker (seenornir/plan.mdIceberg-efficiency notes) was worked around via redb-catalog true-replace. - The session journal still lists zstd/compaction/expiry/sort-order under "⬜ queued" (
workspace_nordisk/PLAN.md§"WAREHOUSE perf" + §C-perf/C-perf-2) — that is stale.
Carried-over open bug (from nornir/plan.md, tracked in neither canonical plan):
workspace_republishself-races on the Tantivy index lock (seen 2026-06-27, servernordisk). On republish (post-workspace_populate) twoindex_buildjobs fire at the same instant; Tantivy allows oneIndexWriterper dir → one wins (~184 ms), the loser diesFailed to acquire Lockfile: LockBusy … already an IndexWriterand its child job logsjob dropped without finish/fail. No data impact (the winner populates 2208 docs) but it litters the job ledger with falsefailedrows. Fix: serialize/dedupe the index build on the republish path (single writer, or skip if one is in-flight for the workspace). Full text:nornir/plan.md§"Open work" (now superseded by this file).
Cross-repo verified-DONE (2026-06-30 audit) — not inline-tickable here (shipped in leaf repos):
- ✅ skade data-skipping column-bounds GAP CLOSED — the FAST
ingest_parallel/ingest_pipelinedpaths now derive per-columnlower/upper_bounds+ null/value counts into the hand-builtDataFile, so those tables are file-prunable (closes the correctness-sensitive GAP flagged 2026-06-24; confirmed by warehouse-ragnar-fast). - ✅ holger read+write+security slice — npm (R2) scoped names + base URL · gem (R1) RubyGems compact-index · conda (R4)
repodata.json; write-path W1 pooled-OIDC TTL cache + W2 RON RBAC fail-closed; security H1/H2 JSON-injection → serde_json · M1 gem token sanitize · C1 decompression-bomb cap;integration_znippy_servefixed (was hanging the--workspacetest); MQ4 mannequin in the holger workspace. - ✅ znippy plugin-read contract (typed
as_npm/as_gem/as_condaviews, authoritative coords) + iceberg-seal (SEAL-IMPL, see inline tick). - ✅ znippy-zoomies osm-katana region-clip merged to master (publish still gated).
- ✅ facett HDR/bloom/FXAA/sky — the game-quality Tier-2 render effects landed on the map3d offscreen target (per the 06-29 facett-game-quality analysis; see MAP-RENDER-ENGINE).
- ✅ modgunn SBOM migration COMPLETE —
modgunn/src/sbom.rsto_cyclonedx; nornir security migrated onto it; the all-High severity fallback removed.
⚖ Cross-cutting laws (every item, every sub-agent)
- Warehouse on FAST local disk, NEVER the slow T9 external drive (decided 2026-06-13).
~/.nornirfor local,/home/nornir/.nornirfor the server — derived from the running user's$HOME, no path env-var. Git checkouts may live on T9; the warehouse must not. - Tests inject values + assert real output — never "didn't panic". (And against the LIVE/remote path, not just embedded — a green embedded test ≠ data shows up thin-mode.)
- Every feature ships
.nornir/design + guide. - All viz UI = facett components + facett theme (non-nornir-specific → facett).
- Screenshot PROOF, egui-native only —
egui_kittestwgpu snapshot (tests) + liveViewportCommand::Screenshot(app screenshots itself) → PNG inrobot_test_results. Never KDE/spectacle/compositor grabs. - Persist to the WAREHOUSE — stream, don't
/tmp. Everything durable (bench progress + CPU/mem telemetry, release events, test results, action trails) streams into the warehouse; viz reads PRIMARILY from the warehouse. Live transport = Arrow IPC record-batch stream (typed/columnar/cheap — znippy already on Arrow-IPC), NOT text/tmp; durable sink = Iceberg (write both if Iceberg snapshots are too coarse for ~1Hz live)./tmpsinks are stopgaps. - Every facett/viz component exposes what it renders as READABLE DATA — "see what the user sees" as structure, not pixels:
state_json+ typed$FACETT_TRACE/$NORNIR_VIZ_TRACEIN/OUT/END must carry the ACTUAL rendered data (rows/nodes/selected values). Tests + agents read that (LAW 1 functional proof); the screenshot (LAW 4) is the visual proof. - Containers are started/controlled in RUST — shell
*_up.share FORBIDDEN. All test/bench containers (RustFS S3, Nessie, Polaris, Spark, …) come up/down via Rust with the operational fixes baked in — skade'sbench-containers(src/containers.rs: up/down all|rustfs|nessie| polaris|spark) already replaced the oldcontainers/*.sh. Unify on the SHARED mechanism that holger / nornir-tests / znippy use (audit + converge — they should share one container harness, not per-repo copies). No new shell container scripts, ever. - GATLING + ZERO-COPY for parallel produce→save pipelines. Use znippy's GATLING no-barrier worker-pool (
znippy-zoomies, already a dep): the produce stage fans across ALL cores (e.g. encode Parquet parts on 32 cores), AND — when the sink is network/S3 — the save stage is ALSO a GATLING pool (N writes in flight), not a single sequential saver (that's only optimal for local NVMe/RAM). Zero-copy throughout (Arrow IPC /Bytes/ mmap / arrow reinterpret bridge — used in znippy + skade); never clone/serialize between stages. This is the G4 S3 fix: single-saver = 12% of NVMe; GATLING-save recovers it.
Definition of DONE (a box ticks only when ALL hold)
implemented end-to-end · injected-value auto-test · live-verified on real data/oden:7878 · .nornir design+guide · committed+pushed (+deployed if a binary).
Development graph tree
Plain-text dependency tree. Each node shows its full label and status tag ([done] = green / [wip] = amber / [todo] = grey). Indentation = a parent → child edge; where a node has more than one incoming edge it is listed under its primary parent with the extra incoming edges noted inline as (also ← X).
A · Warehouse ER + docs + gap analysis [done]
├─ B1/B2 · release_events DAG → warehouse (server + fat CLI) ✅ [done]
│ ├─ B3/B4 · viz Release view + lit dep graph ✅ [done]
│ │ └─ C3 · LiveRun hydrates from warehouse release_events ✅ [done] (also ← B1)
│ │ └─ N5 · viz action trail → warehouse viz_actions (durable sink; /tmp stays live-channel; read-back in state_json) ✅ [done] (also ← A, ← R1)
│ └─ C3 · LiveRun hydrates from warehouse release_events ✅ [done] (see above)
├─ C6 · test-matrix SPINE ✅ (nornir test + test_results + 🧪 Test pane + stall-watchdog); Bench-pane/MCP/robot-shots deferred [done] (also ← PROOF, ← D2)
│ ├─ R1 · full UI-state emission (every form/field in state_json) ✅ [done]
│ │ ├─ R2 · $NORNIR_VIZ_CMD control channel (poll+apply each frame) ✅ [done]
│ │ │ └─ R3 · viz.state + viz.click MCP tools (robot loop) ✅ [done]
│ │ │ ├─ R4 · inject-assert robot-loop test ✅ [done]
│ │ │ ├─ R5 · viz-introspection.md docs ✅ [done]
│ │ │ └─ R6 · richer live-drive (field/button/AccessKit events) ✅ [done]
│ │ └─ N5 · viz action trail → warehouse viz_actions … ✅ [done] (see above)
│ └─ H5 · 🏁 agent/model BAKE-OFF matrix → warehouse (which model+agent won?) ✅ [done] (also ← H1, ← H3)
├─ N5 · viz action trail → warehouse viz_actions … ✅ [done] (see above)
└─ E1/E2 · Funnel injection demo ✅ [done] (also ← E4)
└─ E3 · viz trigger + disk-nuke ✅ [done]
P1 · Heavy bench DONE + CPU/mem telemetry→warehouse ✅ [done]
├─ G4 · S3 read/write concurrency — 3.35× (13%→46% of NVMe) ✅ [done]
│ └─ G4b · gatling::io async-save sibling (14.06M rows/s) ✅ [done]
└─ G2 · optimize CPU vector path — AVX2 4.4× + int8 3.8× ✅ [done]
PROOF · LAW4 · screenshot-proof harness (egui_kittest wgpu → PNG) ✅ [done]
├─ D5 · facett-demo: all 16 components + snapshot proof ✅ [done] (also ← D4)
└─ C6 · test-matrix SPINE … ✅ [done] (see above)
PROOF4b · LAW4b · live-app self-screenshot (ViewportCommand::Screenshot) ✅ [done]
D1 · facett wave: caps…form…card…menu…plotters…fx…pipeline [done]
├─ D4 · theme-compliance matrix ✅ (all 12 themeable, enforcement test) [done]
│ └─ D5 · facett-demo: all 16 components + snapshot proof ✅ [done] (see above)
└─ D2 · 🏆 facett-grid SHIPPED (virtualized 1M-row · frozen · sort · select→TSV · heat · themed) ✅ [done]
├─ D3 · ALL viz UI → facett-ui module + facett theme everywhere ✅ (egui-skew: re-home pending 0.34) [done]
└─ C6 · test-matrix SPINE … ✅ [done] (see above)
C1 · About popup ✅ [done]
C2 · Knowledge: async scan (5s→~0) + rescan + re-scope ✅ [done]
C5 · DepGraph deep transitive + coloring + navigable ✅ [done]
├─ C7 · direct/transitive coloring ✅ [done] (also ← C4)
└─ I3 · graphs navigable (pan/zoom/collapse) ✅ [done] (also ← I2)
C4 · TimeTravel: text + graph views ✅ [done]
└─ C7 · direct/transitive coloring ✅ [done] (see above)
F1 · Incremental republish (affected_by_change) ✅ [done]
F2 · aegir uring · 3D fog · SBOM-from-warehouse [todo]
G1 · AMD ROCm/MIGraphX vector/embed EP + probe ✅ [done]
G3 · interchangeable embedding model ✅ [done]
H1 · MCP tool harness — functional, matrix-wired (56 tools, in/out asserted, coverage gate) ✅ [done]
├─ H4 · manual-for-idiots: local LLM + nornir MCP setup ✅ [done] (also ← H2, ← H3)
└─ H5 · 🏁 agent/model BAKE-OFF matrix … ✅ [done] (see above)
H2 · local-LLM/GPU support audit + gaps ✅ [done]
└─ H4 · manual-for-idiots … ✅ [done] (see above)
H3 · web analysis: best 2026 on-prem LLM ✅ [done]
├─ H4 · manual-for-idiots … ✅ [done] (see above)
└─ H5 · 🏁 agent/model BAKE-OFF matrix … ✅ [done] (see above)
I2 · dep graph populates (cargo metadata) ✅ [done]
├─ I1 · design-doc dep graphs VISIBLE ✅ [done]
└─ I3 · graphs navigable (pan/zoom/collapse) ✅ [done] (see above)
E4 · CLI↔server funnel store-split FIXED ✅ [done]
└─ E1/E2 · Funnel injection demo ✅ [done] (see above)
Checklist (tick on real close)
🔴🔴 P0 — WAREHOUSE LAST-MILE for test-matrix functional rows (Rickard 2026-06-21)
The home is the WAREHOUSE; the JSONL (
NORNIR_TESTMATRIX_OUT) is only the WIRE. Reinforces Law 5.
- P0.1 Enforce "JSONL-emitted ≠ done". Emitting green
functional_statusrows to a/tmpJSONL is NOT done — done is the rows reconciled into the per-workspace Iceberg warehouse vianornir test all, through the server that owns the redb write-lock (NEVER a local run whilenornir-serverholds the lock — that hitsDatabase already open), then read back FROM the warehouse. Treat a JSONL-only, never-reconciled state as a RED/incomplete signal, like a blank pane. Add this line to.nornir/agent-ui-test-fixing.md(drafted there 2026-06-21). - P0.2 Land the facett graphskins call-chain rows into the warehouse. Branch
test/graphskins-callchainadds whole call-chain matrices forfacett-graph/facett-syschart/facett-pipeline(callchain_matrix+robot_pixelstests, ~4491 functional rows, all green; PNG final-picture blobs). The agent emitted them correctly but only pointed the sink at/tmpto count → warehouse landing is the merge-time step: post-merge runnornir test allon the workspace via the lock-owning server so the rows + PNGs populate the warehouse and show in viz. - P0.3 Audit every facett crate's matrix rows for the same gap — anywhere a test/selftest points
NORNIR_TESTMATRIX_OUTat/tmpand stops, make the warehouse the terminus.
🔴 P1 — Heavy bench, live-visible
- P1.1 heavy znippy bench DONE — 28 results/29 tests → bench_runs (run 0022fe9c): ljar_scala 1854MB/s 33.9×, lzip_scala 1592MB/s, planet_pbf_to_geo 81.5GB@186MB/s/31w/36min. (skade DONE too: run bb49d14a, full leaderboard — embedded table_exists 2.04M ops/s ~577× Nessie ~2644× Polaris, skade-vs-Nessie-vs-Polaris README charts regenerated+pushed 28648ec; bench-containers RustFS port made configurable to dodge Njord :9000)
- P1.2 ✅ bencher CPU/mem telemetry (d512a4e: bench_telemetry table, [cores-busy C/N] live, stderr-streaming fixed), built INTO the bencher: sample CPU% aggregate + per-core ("how many of N cores busy?") + mem every ~1s (low-overhead /proc/stat), min/avg/max cores-busy + peak-mem → warehouse
bench_runscols; live in each[BENCH]line + Bench pane. Fix stderr-buffering so the running bench is VISIBLE. Finding: znippy decompress benches use only ~3–6/32 cores — not saturating. Stopgap:/tmp/cpu_sampler.sh. - P1.3 ✅ surface bench LIVE in the viz Bench pane (→ C6d) —
src/viz/bench_live.rs+Tab::Bench's new 📡 LIVE panel: reads the warehousebench_telemetry(P1.2) +bench_runsstraight, shows per-bench NAME + green/red status (status from the pairedtestsoutcome) + a cores-busy/N indicator (avg + peak overn_cores) + a CPU sparkline (avg→max→peak-busy-fraction) + peak-mem badge. Newest run first; re-reads every ~1.5s while open so rows that land mid-run appear live (request_repaint_after). Local mode reads the tables; remote shows a "no Bench.Telemetry RPC yet" note (same split as 🧪 Test / 🚀 Release).state_json()["bench"]["live"]+$NORNIR_VIZ_TRACEbench.liveIN/OUT/END expose what it renders (LAW #6). Inject-and-assert + wgpu snapshot intests/viz_bench_pane.rs. See.nornir/bench-live-pane.md. -
HOLGER-VENV-BENCH venv build (PARALLEL FETCH) against holger (Rickard 2026-06-21) — bench
pip installbuilding a real virtualenv against holger-as-PyPI (znippy-pythonbackend / proxy→pypi with write-back cache), measuring the parallel fetch: wheels/s + MB/s + concurrent-connections/cores busy + wall-clock, streamed to the warehouse Bencher (telemetry per LAW). Compare vs PyPI-direct AND vs Nexus/Artifactory (the B-series twin, like skade-vs-Nessie/Polaris). Three corpora:- (L) one large famous project — Apache Airflow — huge transitive dep tree (hundreds of deps) → parallel-fetch + resolver stress; ties to knut (Airflow AIP-82). Heavy-WHEEL variant for blob-byte throughput: PyTorch / transformers (multi-GB wheels) — tests holger's big-blob serve path.
- (S) one small —
requests— few deps (urllib3/idna/certifi/charset-normalizer), fast warm/cold baseline. - (M) many small — N parallel tiny venvs / top-N small pure-Python wheels — stresses holger's lock-free dispatch + per-request overhead + connection concurrency (the real test of "parallel fetch"). Inject-assert: each venv resolves + imports its top-level pkg; holger serve path is GATLING/zero-copy (B3); auth-cost variants {none/mTLS/OIDC} (B4). Extends holger pip read bench (B1). Folds into the B-series under §O.holger-fixes.
EPIC A — Warehouse docs ✅
- A1 deep-read all warehouse source + map write/read sites
- A2 document all 45 tables (cols/types/keys/relations)
- A3 ER diagram
- A4 gap analysis (release_events, test_results, telemetry, work_log, robot_test_results)
EPIC B — release_events DAG + follow-along viz
- B1
release_eventsIceberg table + append/scan helpers (7c097e4, merged + inject-assert tests + live-verified) - B2 emit from
release run(server + fat CLI) +nornir release eventsread CLI - B3 viz Release tab — release_events op-log prefixed by component (0cdfbd6)
- B4 dep graph lights up by per-component release_events status (green/red/amber-pulse)
EPIC C — Viz fixes & features
-
C1 About popup — client version + server version/endpoint + workspace + warehouse (Health.Ping)
-
C2 Knowledge: async bg scan (5s stall→~0, 60fps) + ↻ rescan button + workspace re-scope (c31b8f9, inject-assert + snapshot)
-
C3 LiveRun driven from warehouse ops, not disk — durable source is the warehouse
release_eventstable (hydrated read-only on load/reload viaquery_release_events, the same path the 🚀 Release tab +nornir release eventsCLI use), so a run survives a reboot and is visible to other clients; the/tmprelease-run-*.events.ndjsontail (local) andRelease.Progressstream (remote) remain only as a fast live-tail accelerator.state_json["live"]+$NORNIR_VIZ_TRACElive.hydrateIN/OUT expose what it rendered. Inject-and-assert + real-on-disk-warehouse tests intests/viz_live.rs. See.nornir/live-run-warehouse.md. -
C4 TimeTravel: keep text + add graph view (deps reel text⇄graph toggle, graph pinned to the cursor's release, reuses depgraph_layout's direct/transitive colouring;
.nornir/timetravel-graph.md; testviz_timetravel_graph_view_inject_assert) -
C5 DepGraph deep total-transitive explode (5ca67bc, rectangular kept, longest-path layout)
-
[~] C6 Standardized nornir test-matrix — SPINE landed (C6a/b/c/f) — see
.nornir/test-matrix.md:- C6a
nornir test run <repo>/test all/test history, CLI parity. REUSES rust's native framework —cargo nextest run --message-format libtest-jsonif nextest is on PATH, else parses plaincargo testdefault grammar; plain#[test]s ingested automatically (src/test_matrix/) - C6b results → warehouse
test_results(run_id, repo, suite, test_name, status pass|fail|ignored|stalled, duration_ms, ts, message; partitioned by repo) + append/scan insrc/warehouse/test_results.rs; inject-assert round-trip test. (robot_test_resultsscreenshot blob = follow-up) - C6c viz 🧪 Test pane (
src/viz/test_tab.rs+Tab::Test): per-repo matrix, newest run on top, green/red per case + pass/fail counts;state_json()["test"]exposes the board; wgpu snapshot proof (tests/viz_test_pane.rs) - L5 ✅ Test pane SHINES — per-repo × per-aspect matrix with health scores, trend sparklines, drill-down, and the
state_json["test"]["matrix"]shape (flat"repo/aspect"cells dict withpass,fail,skip,health,trend; backward-compatgridkept). Test:test_pane_exposes_l5_matrix_cellsinject-assert (7 passing tests + wgpu snapshot intests/viz_test_pane.rs). See.nornir/viz-test-pane-shine.md. - L6 ✅ Live data — wiring live data means running
nornir test allwhich feedstest_results; the matrix auto-populates whennornir vizopens against the warehouse. No code change needed; the read path fromquery_test_results→Matrix::build→state_json["test"]["matrix"]is end-to-end. - C6d sibling Bench pane ✅ (= P1.3): bench green/red + CPU live —
src/viz/bench_live.rs, the 📡 LIVE telemetry panel inTab::Benchreadingbench_telemetry+bench_runs. See.nornir/bench-live-pane.md. - C6e re-run znippy/nornir/skade rust matrices through the standard (deferred —
nornir test allsweep) - C6f stall-watchdog:
NORNIR_TEST_STALL_SECS(default 120) — no subprocess output for N s → kill + record a redstalledrow. (Manual/driver abort = follow-up) - C6g MCP tooling:
test_run/test_status/test_results/test_stop(+bench_run/bench_stop) (deferred) - #5
test_historyMCP tool + release test-DRIVE gate — done:mcp__nornir__test_history(src/bin/nornir-mcp.rs::test_history) returns<repo>'s per-runRunSummaryrows (run_id/ts/pass·fail·ignored·stalled·skipped/total/green), newest-first — the SAME datanornir test historyprints; embedded (warehouse) XOR client (reusesViz.TestResults— no new proto, merges clean with concurrent proto/viz work). Canonical serializerruns_to_json+RunSummary: Serialize/Deserializeinnornir-testmatrix. Release test-drive:release::gate::tests_gate(rows)(pure: red/no-run/regression-vs-last-green) +run_tests_gate(repo_root, history, aspects)(HEAVY: runsrun_full_matrix, folds history, persists fresh rows), wired intorelease_gate_all(MCP embedded + serverOps.GateAll) behind[gates].tests— release/on-request only, never on workspace load. Inject-assert:tests_gate_*+run_tests_gate_drives_matrix_*(#[ignore]heavy, scaffolds pass+fail crates) intests/release_gates.rs+src/release/gate.rs;test_historyseeded (2 holger runs, green→red) + asserted viamcp_every_tool_exercised_and_asserted(tests/support/mcp_harness.rs+mcp_seed.rs). Doc:.nornir/test-history.md.
- C6a
-
C7 direct(violet)/transitive(dashed-blue +hops) coloring + legend (5ca67bc)
-
C8 viz uses facett themable/color schema everywhere (stale duplicate — real entry w/ full detail below)
-
N5 ✅ viz action trail → warehouse
viz_actions(LAW persist-to-warehouse-stream-not-/tmp). The viz user-action log (tab/click/query/RPC/error/lifecycle) was/tmp-only ($NORNIR_VIZ_ACTIONLOG); N5 adds the durable, queryable Iceberg tableviz_actions(session_id, seq, ts_micros, kind LIFE|TAB|CLICK|QUERY|RPC|ERROR, workspace, tab, detail)partitioned bysession_id, with writer/reader insrc/warehouse/viz_actions.rs(mirrorsrelease_events.rs/test_results.rs).ActionLog::pushnow ALSO non-blocking-sends each action to a background drain thread that batches a burst into one snapshot (twin of the MCP telemetry writer) — the/tmpfile stays the fast LOCAL live-channel (write both). The viz reads its own recent history back from the warehouse (warehouse_recent_json→state_json["action_log"].{session_id, warehouse_recent}). Inject-asserttests/viz_action_log.rs(real on-disk warehouse, push 5 typed actions + context, flush, assert exact rows + read-back JSON; +/tmp-still-written check) + warehouse round-trip unit test. Doc:.nornir/viz-action-trail.md. -
C8 viz uses the facett themable/color schema EVERYWHERE — one app-wide facett palette (
UrdrThreadsApp.palette+ 🎨 top-bar picker), applied to eguiVisualsand broadcast to every pane (broadcast_palette→ each pane'sset_palette); every pane routes status/accent/zebra/hover/selection/gridline/typography through the vendoredfacett_themepalette + its semantic ramps (status_color/status_fill/health_color/heat/zebra/hover/selection/gridline), no hard-codedfrom_rgbchrome left. Palette is observable + drivable headlessly:state_json["palette"]+ each pane's nestedpalettekey (LAW 6),VizCommand { palette }over$NORNIR_VIZ_CMD, andviz.click { palette }MCP (CLI/robot parity). Inject-assert:palette_switch_reskins_every_pane(asserts every pane consumes the broadcast) +palette_drivable_via_control_command; wgpu snapshots re-rendered + committed. See.nornir/theming-and-facett-migration.md.
⭐ EPIC AUTONOM — self-discovering completeness gate (the test-matrix FRAMEWORK) 🔴🔴🔴 (started 2026-06-14)
autonom = the autonomous, self-discovering test-matrix framework. Turns C6's results-recorder into a PROVABLE completeness gate: green ⟺ every surface wired. Reusable across all nordisk (engine in nornir-testmatrix). Design+idiot guide: .nornir/autonom.md + .nornir/autonom-for-idiots.md; FC component contract: facett/.nornir/component-contract.md (authored 2026-06-14). Forged 2026-06-14 with Rickard.
-
AUT1 Always-visible matrix (NotRun) (done 2026-06-14, pushed 4d0fc77) — the 🧪 Test grid seeds axes from the EXPECTED surface (
workspace repos × Aspect::DEFAULT) so it's populated + marked not-run-yet before any run (data-driven, not a hand list).Matrix::build_with_expected+ASPECT_DEFAULTinsrc/viz/test_matrix_grid.rs. -
[~] AUT2 Self-discovery (no silent gaps) (realized 2026-06-16 via AUT5 + AUT2b + AUT2c) — the enumerators ship: viz tabs × {thin,fat}, clap subcommands (LIVE
Cli::command()),tools/list,symbol_facts − test-reachable, gRPC handlers (AUT2b), and CLI as a marker layer (AUT2c); a new uncovered surface fails the AUT5 release gate. Remaining edge: facett{impl Facet} ∩ registry()cross-repo discovery. -
AUT3 FC contract (facett) — components FC-1…FC-9 compliant (serializable
state(), AccessKit labels, injected clock,pending) so the gate's assertions are possible.facett/.nornir/component-contract.md. -
[~] AUT4 Assertions (largely realized 2026-06-16) — inject-real/assert-real is the LAW across the matrix; thin==fat parity is enforced by the shared
arch::warehouse::coverage_for_graph(fat pane == thinViz.Architecture) +viz_md_matrix(tab×mode×data); sentinel-ban + 3 layers (property/interaction/perception) via the robot-UI + snapshot suites. Remaining edge: a single formal thin==fatstate_json-equality gate across ALL tabs. -
AUT5 Coverage → warehouse + release-blocking GATE (gate/CLI/MCP done 2026-06-15, branch
feat/aut5-coverage-gate) —surface_coverageIceberg table (row per surface×mode×workspace) via schema-evolution (no history rewrite); the release-blocking completeness gate is now WIRED:[gates].coveragerunsautonom::gate_coverage_for_repo→release::gate::coverage_gate(RED iffSurface − Covered ≠ ∅OR a stale allowlist entry) insiderelease gate-all(CLIrun_gate_all+ servergate_allRPC). CLI parity:nornir test coverage(Surface vs Covered counts + lists uncovered) andnornir release gate coverage. MCP parity:test_coverage+release_gate_coverage(real, exercised bymcp_every_tool_exercised_and_asserted). Inject-assert wiring testaut5_gate_red_when_surface_uncovered_green_when_allowlisted(seed a scan with an unreached fn → gate RED naming it; allowlist it → GREEN; stale entry → RED). Design+guide../dwarves/.nornir/aut5-surface-coverage-gate.md. Follow-up DONE 2026-06-18 (feature N): see N below. -
N Source coverage → warehouse → docs/PDF + viz (maven-style) (done 2026-06-18) — the JaCoCo/maven twin of AUT5 (SOURCE line/region %, not surface-reachability).
nornir coverage <repo>(CLI parity) shellscargo llvm-cov --json→ PURE parsernornir::coverage::{parse_llvm_cov_json,classify}(demangles v0/legacy symbols via rustc-demangle; first-party files only) → historized warehouse tablescoverage(overall+crate+file rollup) +coverage_fn(per-fn, boundary-tagged), partitioned by repo. Viz 🧪 Test pane embeds 📐CoverageLive(mirrorsbench_live.rs): overall badge + per-crate bars + worst files + history sparkline + the ui/gRPC/emitter BOUNDARY view (the user's explicit ask —classifytags fns by name/path per the layer-boundary LAW;state_json["test"]["src_coverage"].boundary.{total,exercised,tally,fns}proves the hard-to-cover seams are tested). Docsnornir:gen:coveragemarker → book/PDF. Inject-assert tests in coverage/warehouse/viz/docs + a wgpu golden PNG (tests/viz_coverage_pane.rs). Validated end-to-end vs a real llvm-cov run (nornir-testmatrix 78.2%). cargo-llvm-cov 0.8.7 + llvm-tools-preview installed. Design+guide.nornir/coverage-maven.md. -
ARCH-LIVE 🏛 Architecture tab → live, clickable, multi-functionality system diagram (done 2026-06-18) — the Arch tab is now a CONTAINER with an internal sub-view selector (Deps · Live system · Wiring · Hållpunkter; deps CONSOLIDATED under Arch, not a new top-level tab). Click-to-code: a node click resolves its label → source
file:linevia the SHAREDnornir::arch::resolve_symbol(exact→leaf→substring oversymbol_facts), then (i) openscode --goto file:line(best-effort, logs if absent), (ii) an inline source-preview panel readssource_window(±6), (iii) CLI paritynornir arch open <symbol> [--open] [--all]. Hållpunkter lens draws the ui/gRPC/emitter LANDMARK fns, REUSING the coverage feature'sclassify(imported, not reinvented). Live system lens repaints-on-change (livetoggle +request_repaint_after, mirrors bench_live/coverage_live).state_json["architecture"]gains{sub_view,sub_views,live,boundaries.{ui,grpc,emitter,landmarks},last_open.{resolved,file,line,editor_opened,preview_lines}}+architecture.{subview,open}traces (LAW #6). Inject-assert tests inarch::(resolver+window) +tests/viz_arch_subviews.rs(container/click/landmarks + wgpu golden PNG) +tests/cli_parity.rs(verb registration). Design+guide.nornir/arch-tab-live.md. -
ARCH-METRO-THIN 🚇 Metro map → warehouse-backed in thin/remote mode + populated-state EMITTER (done 2026-06-22, branch
agent/nornir-metro-emitter) — Bug: the metro built lines ONLY from a local checkout scan (scan_all(repo_root)); in thin moderepo_rootis empty → every line degraded to a silent click-handler-head stub, even though the warehouse already holdscall_edges+symbol_facts. Fix: theViz.ArchitectureRPC now also ships each member'ssymbols+calls(server reads them viaknowledge::query::load_latest— the same read the knowledge tools use),ArchPayloadgainedsymbols/calls(+SymbolRow/CallEdgeRowDeserialize), andensure_loaded/ensure_symbolsseed the thin metro from the warehouse instead of degrading. Fat mode still scans the checkout. The "no checkout → stubs" degrade is replaced by a POPULATE-DIAGNOSTIC (only an unpopulated warehouse is empty, and it says "run a populate/deep scan"). EMITTER (verify-without-looking):emit_metro_status()ships atestmatrix-gatedfunctional_statusrow + trace withmetro_edges/metro_nodes/metro_emitter_nodes(emitter stations MARKED)/metro_stub_lines(==0 when populated)/fuel_source(warehouse vs checkout), also folded intostate_json["architecture"]["metro"]. RAGNARÖK tests (tests/viz_arch_metro.rs): populated thin warehouse assertsmetro_edges>0 ∧ metro_stub_lines==0 ∧ metro_emitter_nodes>0; empty warehouse asserts the SAME row goes RED (sensitivity); + a real-IcebergWarehouseround-trip throughload_latest. Prompt LAWs #1 (RAGNARÖK) + #2 (verify-without-a-human) added to.nornir/agent-ui-test-fixing.md. Design+guide.nornir/metro-warehouse-read-and-emitter.md. (Needs a human reinstall to go live.) -
[~] AUT6 Freshness — release-blocking gate runs on the WORKING TREE; thin-client staleness badge from a working-tree DIGEST (gix) vs indexed SHA (a commit SHA can't see uncommitted edits);
index_status→{sha, dirty digest, clean|dirty}. ✅ index_status primitive DONE (verified 2026-06-30 audit) — the{sha, dirty digest, clean|dirty}working-tree digest primitive shipped; remaining: wire it into the release-blocking freshness gate + the thin-client staleness badge. -
AUT7 Wiring/flow graph (the gate's visual twin) — coarsen fn-level
call_edgesto a component/gRPC/core/table graph (owner-fn quotient); persistfn→tableaccess facts; render skade-style Typst→SVG (NOT mermaid) in nornir docs (ties J4);nornir flow <entrypoint>CLI +flow_traceMCP. — done (subsumed by EPIC ARCH): owner-fn quotientsrc/arch/mod.rsbuild_graph/classify;fn→tablepersisted viasrc/warehouse/access_scan.rs→architecture_wiringtable; skade direct-emit SVGto_svg; entrypoint trace shipped asnornir arch trace <entry>CLI +arch_traceMCP (renamed fromflow/flow_trace, folded into thearchnamespace). Testtrace_from_entrypoint_keeps_only_reachable. -
[~] AUT2b gRPC backends are a counted surface (2026-06-16, branch
feat/nornir-markers) — autonom now FINDS the already-marked gRPC handlers (a tonic handler impl IS the mark) viaarch::grpc_handlers_from_symbolsand enumerates oneSurfaceKind::Grpcnode perService.verbinMode::Thin; covered iff the handler verb is test-reachable. So an RPC with no test outs itself as a red surface (theTest.Results RPC TODObug class).discover::grpc_handlers()+SurfaceInputs.grpc/grpc_labels_from_viewinenumerate_surface/gather; inject-assert tests both crates. (Remaining for full AUT2: the per-rowlayermark + the emitter datapath — see ARCH-COVERAGE below + marker-matrix-plan.md.) -
AUT2c CLI is a first-class marker layer (2026-06-16,
3979658) — completes the UI · gRPC · CLI marker triad. CLI was already a counted test-matrix surface (SurfaceKind::CliCommand, fed by LIVE clap introspectionCli::command().get_subcommands()— the anti-drift source). Now also a distinctNodeKind::Clilayer in the 🏛 Architecture board:arch::generate_with_cliseeds onecli:<verb>node per clap subcommand (terminal-green chip, off green/red coverage rings); the binary passes nornir's verbs gated to repo==nornir. Cross-check is free —cli:<verb>matches itscli_commandcoverage row → green/red. Testgenerate_with_cli_seeds_a_cli_marker_per_verb. -
ARCH-COVERAGE 🏛 Architecture tab cross-checks every marker vs the test matrix (2026-06-16, branch
feat/nornir-markers) — THE KEYSTONE: each marker (UI-component/gRPC/core-fn/table) carries itssurface_coverageverdict; chip ring = green tested / red untested / amber allowlisted (fill stays the layer).green ⟺ every marker has a covering test. Sharedarch::warehouse::coverage_for_graph(+ puremap_coverage_verdicts) used by BOTH the fat pane (opens warehouse) and the thinViz.Architectureserver handler → thin == fat; coverage rides inArchPayload.state_json.coverage{tested,untested,allowlisted,unknown,has_data}+ per-nodecov(LAW #6); no false green when no data. Inject-assert: arch_tab tally + no-data=all-unknown + puremap_coverage_verdicts(19/19 arch tests green). Design: marker-matrix-plan.md. -
[~] AUT8 All nordisk use it — (cross-repo audit 2026-06-16: 5/7 WIRED, 2 gaps) nornir + znippy + skade + facett + korp + knut + holger run autonom; results land in the warehouse. Per-repo truth:
- facett ✅ WIRED — 49
functional_statusemitters across 22+ test files; D5b1 (294-cell mega matrix) + D5b2 (45-step robot drive) both DONE; 28impl Facet(21 in demo). - holger ✅ WIRED — 29 emitters / 25 components; matrix test
xtask/tests/holger_matrix.rs::testmatrix_feature_emits_functional_rows_into_the_sinkgreen (sink read-back asserted). - skade ✅ WIRED — 33 emitters / 11 files (incl.
src/static_index.rs,src/store.rs— real production-path emits, not just tests); wiring test feeds a real emit. - knut ✅ WIRED — 5 emitters (knut-bifrost) +
emit_statushelper; wiring test green. - korp ✅ WIRED (honest caveats) — 16 emitters / 5 files; robot drives all 9 tabs (3 fully: Demo/Geo/Tables; 6 wgpu/egui-0.33-blocked → skip-cell +
functional_emit.rsrender proof); 2 gRPC handlers (Hugin StreamState / Munin ListCases). - znippy ✅ WIRED (2026-06-16; audit-corrected) — the breadth-scan undercounted: the unpublished
znippy-testscrate ALREADY has 24 real emit sites (tests/tests/integration_test.rs+ plugin/repro/wiring). The one genuinely-uncovered first-party functional test — the iceberg decompressor-correctness roundtrip — is now wired:znippy-iceberg/tests/round_trip.rsemits 3 rows (warehouse_durable_catalog / list_files_matches_fixtures / byte_exact_roundtrip_all_fixtures) behind a newznippy-icebergtestmatrixfeature. Green plain +--features testmatrix+--workspaceboth ways. - nornir ✅ SELF-EMITS (dogfood landed 2026-06-16,
670e927) —src/selftest.rs(emit+assert_emit!, EMIT-DOCTRINE) on main's buffer-drain framework;tests/self_emitters_core.rsself-tests nornir's OWN marker classifier (arch gRPC/Component/CoreFn + discover::grpc_handlers) and drains back 4 functional rows under--features testmatrix. (Deeper render-path emit-fallbacks — the can't-observe-from-outside viz cases from the salvage branch — still incremental.) - Warehouse-sweep clause: every leaf repo emits to a local JSONL; the "results land in the warehouse" sweep is delegated to
nornir testand not yet auto-run, so a single nordisk-wide matrix warehouse isn't live.
- facett ✅ WIRED — 49
-
AUT8-GAP-ZNIPPY real znippy emitters DONE (2026-06-16) —
znippy-iceberg/tests/round_trip.rsemits 3 real check rows behind atestmatrixfeature passthrough; assert-first per EMIT-DOCTRINE; green both ways. So AUT8 leaf-repo adoption = 6/6 nordisk consumers WIRED (facett·holger·skade·knut·korp·znippy); remaining AUT8 work is the warehouse-sweep clause + nornir's deeper render fallbacks. -
[~] AUT9 CLI-AUTOPILOT — every CLI command corresponds to the UI + emits sane data (started 2026-06-23, branch
agent/nornir-uiplane-autotest; Rickard: "cli commands dont have ui trait — fix that, autopilot testing is the goal") — closes the UI/CLI asymmetry: UI elements are discovered (viz_surfaces/Facet registry) AND assert sane data (state_json); CLI was discovered only SHALLOWLY (top-level roots) with NO uniform sane-data oracle. Step 1 DONE:clap_command_paths()(src/bin/nornir.rs) RECURSES the clap tree → every leaf path (bench history,workspace ls,mimir deps-of); wired into all 3 gate feeders (gate_coverage_for_repo×2 +nornir_surface_inputs) soGap = Surface − Coverednow spans every nested command, not just the ~25 roots (the CLI twin ofviz_surfaces(); kills thecli_parity.rshand lists). Unit testclap_command_paths_finds_nested_leaves(leaves > roots;mimir deps-of/diagram timelinepresent). Step 2 DONE —CommandOutcomecontract:src/cli_outcome.rsCommandOutcome{command,ok,data,human}= the CLI'sstate_json(presentation ONLY — data path unchanged: fat=lib calls, thin=gRPC;data:Valueis the edge/comparison shape, never a transport). Verdict =is_sannr()(Old Norse sannr="true" — the truth-verdict; renamed from "oracle"/"check" to NOT overload Mímir's oracle slot). RAGNARÖK: empty/null/error-marker ⇒ not sannr. Reference migrationnornir commands= the CLI's self-description (recursive leaf list, the CLI twin of MCPtools/list), emitsCommandOutcome.tests/cli_autopilot.rs: assertscommandsis sannr end-to-end AND every enumerated leaf is REAL (--helpparses — mechanical reachability floor, no hand list). Step 3 STARTED — handlers migrated (pattern = lib outcome-builder + thin CLI):bench history(nornir::bench::history_outcome, both fatquery_bench_runs+ thinBench.Historyrender via it) ·workspace ls(nornir::registry::roster_outcome+ sharedRosterRow; both fatRegistry::list+ thinWorkspaces.Listproject into it = parity by construction; added--json). Each has inject-assert unit tests (empty=RED, populated=sannr). NEXT: (3 cont.) migrate the rest per-command (gate names the remainder) —test history/docs historymirrorbench history; (4) generic CLI sannr smoke (drive migrated leaves vs a fixture); (5) CLI⟺UI parity tests at the shared-lib-fn / gRPC-handler level (bench history==Bench tab ·workspace ls==Nornir roster ·mimir deps-of==dep pane ·arch==Arch tab). Commits98f559d→65068b0. Design+guide.nornir/cli-command-contract.md.- AUT9 FINDINGS (2026-06-23, scoping the rest): (a) clean/additive migrations (no prior
--jsonto break) = the easy wins (didworkspace ls); (b) existing-raw---jsoncommands (mimir *,coverage --show,test history) already print a bareserde_json::Value— wrapping inCommandOutcomeis a shape change. Blast radius is LOW (MCP/viz consume the lib/gRPC directly, NOT CLI stdout; only the user's own scripts read CLI--json), so it's safe to proceed, but it's a deliberate call. (c) Real parity BUG found:docs historyrenders different columns fat vs thin (fatsha256/git/export_idvs thinpath) — exactly what CLI⟺UI parity tests are meant to catch; fix the thin proto/handler to match before migrating. (d)test historyhuman path uses the in-binaryrender_matrix→ lift to lib first. Recommendation: proceed wrapping the raw---jsoncommands (low risk) + fix docs-history parity, batch by namespace.
- AUT9 FINDINGS (2026-06-23, scoping the rest): (a) clean/additive migrations (no prior
-
[~] AUT8-GAP-NORNIR-SELF nornir self-emitter dogfood — FOUNDATION DONE (2026-06-16,
670e927) —selftest.rsbridge +self_emitters_core.rs(marker-classifier self-test, 4 rows drained) on main's buffer-drain framework. Remaining: the render-path emit-fallbacks (viz paint / screenshot PNG — the can't-observe-from-outside cases) from the salvage branch, ported incrementally. -
AUT8-GAP-LAYER layer DERIVED from kind (no column) (2026-06-16,
db68978) — design call: atest_resultslayercolumn is wrong (the emit site can't know its layer). InsteadSurfaceKind::layer()+arch::NodeKind::layer()derive the canonicalui/grpc/cli/core/tablefrom the kind (1:1 peer-tested), so matrix + 🏛 board group by the same strata with NO schema change; arch_tabstate_jsongains per-nodelayer. -
GAP-ROBOTUI-034
nornir-robotuion egui 0.34 — facett + korp both in-crate-port robotui (pinned egui 0.33); ship a 0.34 build so the ports become a one-line dep swap and korp's wgpu-blocked views can be driven. ✅ DONE (verified 2026-06-30 audit) —nornir-robotuiships on egui 0.34; the facett/korp in-crate ports collapse to a one-line dep swap. -
[~] AUT9 LAW 9 — UI-PLANE reachability (framework piece DONE) (2026-06-22, branch
agent/nornir-uiplane-autotest) — the catch the isolation matrix missed: a surface that exists but can't be NAVIGATED to (orphan plane / shipped-wasm differs) greened anyway (the Sverige-buttons failure). New GENERIC + UI-agnosticnornir_testmatrix::uiplane:UiPlane(tagged view-state + its surfaces) ·UiPlan(planes=nodes, transitions=edges, BFSreachable_planes/orphan_planes/bfs_route) ·PlaneDriverseam (current_plane/apply/surfaces_present/surfaces_ran— emitted data, never pixels) ·RobotPlan::walk→ oneWalkRowper (plane,surface) with the LAW 9 verdict (ReachableAndRan/ReachableNotRan/ReachableAbsent/Unreachable); RED ⟺ orphan, dangling edge, or any not-ran/absent. Reachability-COMPLETE (one row per declared surface; headless lib = empty plan = trivially green). Worked example: nornir-viz as aUiPlan(plane perTab::ALL, tab-strip click per transition) + nativeVizPlaneDriverover the liveUrdrThreadsApp(drives the REALapply_commandcontrol path, reads present/ran fromstate_json) —tests/viz_uiplane_walk.rswalks the live app (20/20 tabs reachable+ran GREEN) + 3 sensitivity tests (orphan→RED, broken-transition→RED, present-not-ran→RED). wasm/deployed driver SEAM defined (uiplane_plan::wasm::{BrowserBridge, WasmPlaneDriver}overwindow.__facett_state()/__facett_clickfor LAW 1 — facett-demo agent wires the JS hook). Composes with utfallsrum + facettNavigable(not reinvented). 13 core + 4 worked-example + 4 integration tests green;.nornir/testmatrix-uiplane{,-guide,-for-idiots}.md. Remaining: facett-demo deck wiring + the real headless-browserBrowserBridgeimpl + warehouse landing post-merge.
⭐ EPIC AIRGAP — Skíðblaðnir: 100% Rust airgap pack/move/unfold/install (crate skidbladnir, sibling repo ../Skidbladnir) 🔴 (started 2026-06-14)
RENAMED + RELOCATED (reconcile): the crate was renamed
nornir-airgap→skidbladnirand extracted out ofcrates/into its own sibling repo../Skidbladnir. nornir now depends on it as a path dep (skidbladnir = { version = "0.1.3", path = "../Skidbladnir" }in Cargo.toml) — there is nocrates/nornir-airgapin this repo any more, and nonornir-airgapcrate is published. Thecrates/nornir-airgap/src/…paths in the completed AIRGAP/INSTALL notes below are the historical locations; the code now lives under../Skidbladnir/src/…asskidbladnir::…. nornir'sairgapCLI verb +airgap_*MCP tools drive it.
Pack a product/lake into an air-gapped bundle, move it across the gap, unfold the target (Redfish · Terraform · KVM), install + configure + start every product in dependency order — no shell. Design + source prompt + prior-art map: .nornir/airgap-install.md. Prior art: dist · airgap-kubernetes-main · airgap_llm · tunnr · msi-agent · bifrost. (An earlier agent failed to filter this prompt in — now captured here + in the crate.)
- AIRGAP0 Scaffold (done 2026-06-14, pushed 4f22751) — crate
crates/nornir-airgap(lean: anyhow+serde) with module skeleton pack·archive·fetch·platform·pki·layout·product·unfold{redfish,terraform,kvm}·orchestrate·config + inject-assert tests; design doc.nornir/airgap-install.md. - AIRGAP1 pack — collect artifacts + target-identical deps (container build when a component must be 100% target-alike); VERSION file outside the bundle for easy bumping. — done:
crates/nornir-airgap/src/pack.rsStaging+pack+write_version_file; realPodmanBuild: ContainerBuild(podman build→create→cp viastd::process, no shell; renders the Containerfile, honest error when podman is absent); testspack_collects_staging_tree_and_version_file_is_outside,podman_build_renders_containerfile_and_is_honest_when_absent. - AIRGAP2 archive (znippy) — bundle with content cache + delta (next version ships only the delta); read-without-full-decompress. — DONE (real engine, branch
feat/airgap-archive-engine):crates/nornir-airgap/src/archive.rsArchivetrait +TarZstd(no-feature default) +ZnippyArchivebehind theznippyfeature: content-addressed cache (dedup by SHA-256),delta(old_manifest,new_staging)→Patch/apply(old_bundle,patch)→new_bundle(ships only changed/new blobs;apply(a,delta(a,b))is byte-identical to a fresh pack of b), and O(1)read_entry(footer→seekable index→preadone blob, never full-decompress). Self-contained znippy-faithful format (blob region + index + footer), no rayon (lean budget), does not linkznippy/znippy-cli. Wired into nornir via theairgap-znippyfeature. testspack_then_extract_roundtrip_with_manifest_checksums,tampered_manifest_fails_root_verify,znippy_content_cache_dedups_identical_entries,znippy_selective_read_does_not_decompress_whole_archive,znippy_delta_round_trip_is_byte_identical_and_ships_only_changes,znippy_full_extract_round_trips. - AIRGAP3 platform prep — per-OS specialization (e.g. SELinux toggle), fetch+verify tarball, unpack into the software dir. — done:
crates/nornir-airgap/src/platform.rsPlatform::detect/specialize/unpack+src/fetch.rsverify_on_receipt(zero network); testsos_release_detects_family_and_selinux,unpack_extracts_bundle_into_software_dir,faithful_bundle_verifies_on_receipt,tampered_manifest_is_rejected. - AIRGAP4 PKI —
PkiSource: ExistingVault | SelfGenerated | TempVault (pure-Rust or a temporary Vault). — done (SelfGeneratedreal ed25519; Vault variants areVaultClientseams):crates/nornir-airgap/src/pki.rsMachineCa/issue_leaf/verify_leaf; testself_generated_leaf_is_a_real_verifiable_cert(signature verifies + tamper rejected),vault_sources_are_explicit_seams. - AIRGAP5 layout — data dir (fast-moving data) + version-dated software dir (
software/<YYYYMMDD>/<product>). — done:crates/nornir-airgap/src/layout.rsLayout+ atomicactivate()symlink swap; testsprepare_makes_version_dated_per_product_dirs,atomic_symlink_swap_on_upgrade. - AIRGAP6 per-product install — service user + systemd/init + unpack + configure(data path) + open firewall ports. — done (generalizes
run_install_systemd):crates/nornir-airgap/src/product.rsComponentSpec+Installer/SystemdInstaller(reusessystemd::install_systemd, opens ports via firewall-cmd/ufw); testscomponent_spec_renders_layout_driven_systemd_spec,component_spec_to_product_is_consistent. - AIRGAP7 unfold backends — Redfish (existing-server BMC) · Terraform (cloud/VM) · KVM (test; reuse tunnr's pure-Rust KVM harness); Target also spans bare-metal + RDP/Windows container. — done partial: KVM qemu argv real (tunnr-identical) + MSI
guid_v5real; Redfish/Terraform provisioning + WiX build are honest seams (their plans/bodies ARE rendered).crates/nornir-airgap/src/unfold.rs+src/msi.rs; testskvm_qemu_argv_is_built_like_tunnr,terraform_resource_renders_nornir_component,external_backends_are_explicit_seams,guid_v5_is_deterministic_and_well_formed,sha1_matches_known_vector. - AIRGAP8 ordered start — dependency-ordered bring-up (services point at each other); reuse nornir dep-graph topo sort. Persist
airgap_eventsto the warehouse. — done:crates/nornir-airgap/src/orchestrate.rsKahnstart_order/start_waves+src/events.rsAirgapEvent/JsonlSink/EventEmitter; warehouse Icebergairgap_eventstablesrc/warehouse/airgap_events.rs+iceberg_schema::airgap_events(registered iniceberg.rs); teststopo_waves_from_fixture_dep_graph,cycle_is_rejected,emits_install_dag_to_warehouse_jsonl_and_reads_back,jsonl_decodes_airgap_events. - AIRGAP9 CLI + MCP parity —
nornir airgap pack|fetch|unfold|install|startmirrored as MCP tools; idiot guide. — done:nornir airgap pack|fetch|unfold|install|start|events(src/bin/nornir.rsrun_airgap) + MCPairgap_pack/airgap_verify/airgap_unfold_plan/airgap_events(src/bin/nornir-mcp.rs); idiot guide in.nornir/airgap-install.md§7; covered bymcp_every_tool_exercised_and_asserted.
⭐ EPIC ARCH — living architecture diagrams: UI buttons ↔ gRPC backends, wired (skade-style SVG) 🔴 (2026-06-14)
nornir's architecture as auto-generated, always-current wired SVG diagrams (skade circuit-board style, Typst-rendered — NOT mermaid): every UI "button"/surface wired to the gRPC backend it calls and the core fn + warehouse table behind it. The readable, regenerated picture of how the program is wired — and the visual twin of the AUTONOM completeness gate (same warehouse data: entrypoints + call_edges + fn→table). First-class for nornir. Storage candidates: per-workspace_*/ (travels with source) OR the warehouse (historized, time-travel per git sha) — DECIDE (maybe both: warehouse durable + doc-embedded).
- ARCH1 UI-button ↔ gRPC map — enumerate viz surfaces (
Facet ∩ registry()+ tab enum) and the gRPC verb each thin-mode surface calls; emit the wiring as data. — done:src/arch/mod.rsfacet_impls_from_symbols+grpc_handlers_from_symbols(testgenerate_end_to_end_derives_both_maps) - ARCH2 coarsened wiring graph — the AUT7 owner-fn quotient (UI-component · gRPC · core fn · table nodes) from
call_edges+ the newfn→tableaccess fact. — done:src/arch/mod.rsbuild_graph/classify+src/warehouse/access_scan.rs(teststest_tab_fetch_collapses_to_component_and_table,extracts_named_read_and_write_edges) - ARCH3 skade-style Typst→SVG render — regenerated by
nornir docs(ties EPIC J / J4), embedded in.nornirdesign docs; NOT mermaid. — done:src/arch/mod.rsto_svg/render_pcb_svgdirect-emit skade SVG (testsvg_is_circuit_board_with_expected_labelsasserts no mermaid). NB: direct-emit (no Typst engine) + regenerated bynornir arch(notnornir docs). - ARCH4 storage decision + persist —
workspace_*/asset vs warehousearchitecture_wiringtable (historized by git sha, schema-evolving); pick one or both. — done (warehouse):src/arch/warehouse.rs+iceberg_schema::architecture_wiring(testsrecord_then_list_and_load_round_trip,different_git_sha_makes_new_row) - ARCH5 CLI + MCP parity —
nornir arch [svg | trace <entrypoint>]+ MCP tool (reusesknowledge_call_path). — done:nornir arch generate|svg|trace|history(bin/nornir.rsrun_arch/arch_trace) +arch_svg/arch_graph/arch_traceMCP (bin/nornir-mcp.rs); teststrace_from_entrypoint_keeps_only_reachable+mcp_every_tool_exercised_and_asserted.
Subsumes/extends AUT7 + EPIC J for the UI↔gRPC angle — the "diagrams of UI-buttons and gRPC backends, wired" ask (Rickard 2026-06-14).
⭐ EPIC DOC-SEED — docbook render emits a git-committed structured seed (cold-start bootstrap) 🔴 (Rickard 2026-07-02)
Ask: when nornir docs book renders, also persist a compact structured artifact in git (via the existing gix seam) so an EMPTY warehouse can still bootstrap some data — vs. the alternative of parsing data back out of the already-rendered doc. DECISION (my call, recorded):
- Emit at render time → commit a compact JSON/NDJSON sidecar to git; do NOT parse back from the rendered PDF/typst (that output is a lossy presentation projection — re-deriving data from it is fragile + format-coupled; emit from the structured source, never the presentation).
- JSON, not Parquet, for the GIT copy — git wants diff-friendly text; a stable-sorted compact JSON diffs + packs cleanly, Parquet is a binary blob that bloats history every render. Keep Arrow/Parquet on the WAREHOUSE side (where
doc_exportsalready lives). - Warehouse stays the durable store; git artifact is the SEED. Render already historizes to Iceberg
doc_exports; this adds the cold-start path: empty warehouse → ingest the committed sidecar → populated. This is the "maybe both: warehouse durable + doc-embedded" that EPIC ARCH ↑ already flagged, applied to docs. - DOC-SEED1 define the doc-as-data schema (chapters/sources/marker-expansions/sha/git/version — the structured form the render already has in memory before typst).
- DOC-SEED2 emit the sidecar (
docs/book.seed.json, stable-sorted) as part ofbuild_book; commit it via thesrc/docs/pages.rsgix seam (extend, don't rebuild). - DOC-SEED3 bootstrap path: on an empty warehouse, ingest
book.seed.jsonintodoc_exports(+ a populate-diagnostic like the metro fix) so a cold clone isn't blank. - DOC-SEED4 round-trip test: render → sidecar → wipe warehouse → bootstrap → identical
doc_exportsrows. Gated behind the marker/arch fork landing (fork-contended nornir).
EPIC R — Robot UI tester (viz-introspection: full UI-state + viz.state/viz.click)
LAW #6 "see what the user sees as data" + the robot loop. Read-half mostly existed (state_json → $NORNIR_VIZ_STATE + action log + $NORNIR_VIZ_TRACE); this EPIC completes the emission and wraps both halves as MCP tools. See .nornir/viz-introspection.md.
- R1 Full UI-state emission — every interactive tab now emits its form/control state into
state_json()(was data-only): 🔍 Search (query + corpus + symbol/vector/call-graph fields + result counts/rows), 🚦 Gates (repo + gate report + trace), 📈 Bench (repo + metric + loaded series), 🗄 Warehouse (tables + selected + chart toggle/cols + preview shape), 📞 MCP (per-tool usage rows), 🛡 Security (scan repos + scanning + SBOM/vuln/license result).state_jsonis now a faithful mirror of every box + dropdown, not just the rendered data. Pure read, no behaviour change. - R2 Control channel
$NORNIR_VIZ_CMD(src/viz/control.rs) — a tiny consume-once command file the running viz polls each frame (poll_control_channelat the top ofdraw_ui→apply_command); switches tab (bystate_jsondebug name) and/or workspace. File-based to match the state/action-log/trace contract; round-trip + consume-once + malformed-payload unit-tested. - R3
viz.state+viz.clickMCP tools (src/bin/nornir-mcp.rs) — read the live state dump / drive tab+workspace and return the post-condition state. The robot loop: observe → act → observe, no pixels. - R4 Inject-assert proof
tests/viz_robot_loop.rs— drives the loop through the real file channel (write_command→draw_uiframe consumes + applies →state_json) and asserts the Test tab active + its seeded run/grid rows + consume-once + the drive in the action trail; a 2nd test proves aworkspacecommand re-selects the picker. - R5 Docs —
.nornir/viz-introspection.mddocuments the form-state emission, the$NORNIR_VIZ_CMDcontract, the two MCP tools, and a Claude-Code robot-loop example. - R6 Richer live-drive ✅ (merged feat/r6-richer-livedrive, d1fc936) —
nornir-robotuicrate:RobotSession::set_field/click_id/inject_event; widget-level AccessKit-backed control beyond tab/workspace; inject-assert tests intests/robot_viz.rs;.nornir/robotui-richer-livedrive.md.
EPIC D — Facett components
-
D1 finish the wave (caps · clipboard · form · card · menu · plotters · fx · pipeline)
-
D2 🏆 FLAGSHIP
facett-gridSHIPPED (db56897) — virtualized 1M-row (61.9ns window), frozen/pinned/sort/select→TSV/type-aware cells/heat/themed; 39+perf+snapshot tests, 173KB proof PNG. (drag-resize/reorder + grouping deferred, documented.)- D2a virtualized rows+cols, Arrow-backed, silky 60fps over millions of rows
- D2b frozen header + pinned leading columns + sticky totals footer
- D2c sort (multi-column shift-click, stable, indicators)
- [~] D2d columns: auto-fit/hide/show/density ✅; interactive drag-resize/reorder DEFERRED (documented)
- D2e per-column type-aware filter + global search-with-highlight
- D2f Excel-style cell selection (range/multi/row/col/all) + copy TSV/CSV + paste
- D2g keyboard nav (arrows/page/home/end/tab/enter-to-edit)
- D2h inline editing, type-aware editors + validation → change events
- D2i type-aware cells: grouped numbers · dates · enum chips · links · progress bars · booleans · tags · in-cell sparklines
- D2j conditional formatting (data bars / color scales / heat via theme ramps)
- D2k themeable via facett palettes (zebra/hover/selection/gridlines/typography), caps scalable
- [~] D2l row grouping DEFERRED (documented in facett-grid.md)
- D2m caps selectable·copyable·searchable·scalable·resizable·themeable; copy→TSV/CSV
- D2n quality gate: screenshot-proof PNG of a populated grid that out-classes Excel + scroll/sort bench
-
D3 ALL generic viz UI factored into a reusable, palette-driven module + viz consumes facett themes — the non-nornir widgets (status chips, badges, sparklines, zebra/hover rows, health pill, dep-graph node style) extracted into
src/viz/facett_ui.rs, each taking aThemeso they're already facett-ready; viz themes every pane via the vendoredfacett_themepalette (C8). True move into thefacettrepo is blocked by the egui-version skew (viz on egui 0.33 foregui-snarl0.9; facett on 0.34 — incompatibleColor32/Uitypes, same blocker as the L5facett_themevendoring); documented re-home plan in.nornir/theming-and-facett-migration.md—facett_theme+facett_uicollapse topub use facett::…re-exports once egui aligns. Computed ramps (callgraph heat/fog) deliberately left as math; legacyNodeStat::color()kept beside the themedcolor_themed. -
D4 EVERY facett component themeable + caps-compliant; maintain
facett/.nornir/compliance.md— a fat per-component × {themeable/scalable/copyable/selectable/searchable/resizable} matrix, regenerated every release; third-party-blocked gaps marked ✗ with the reason. No silent gaps. -
D5 facett-demo mega app — all 16 components + menu + effects, fake data, palette switcher, snapshot proof PNG (fbc887b)
EPIC E — Funnel demo + store-split
- E1 funnel injection test — 16-thread Arc<Mutex> hammer + reader snapshots; asserts no-lost-updates/frontier/deadlock-watchdog (94262bf)
- E2
nornir funnel demo --size N— real fake git repos (path-dep layered DAG) + mirroring funnel + manifest - E3 viz demo strip (🌱 run / 🗑 nuke w/ confirm-guard, disk-only) + funnel in state_json (LAW 6);
nornir funnel nuke --yes - E4 fix CLI↔server funnel store-split (6e70e5d: resolve_root precedence + lock-tolerant read + roundtrip test)
EPIC G — GPU acceleration + vector performance
- G1 ✅ AMD ROCm for the vector/embedding path —
embed-ort-rocmfeature wires the ort MIGraphX → ROCm → CPU EP chain intoOrtEmbedder::load_with(mirrors the CUDA-then-CPU logic, all EPsfail_silently). Newsrc/vector/rocm.rs: runtime ROCm-libdlopen(RTLD_GLOBAL)discovery +available()probe +preflight()— a box with no ROCm returnsavailable()==false, so the AMD EP isn't even requested and the path degrades to CUDA/CPU; non-AMD builds don't compile any of it. EP wiring is real (ort::ep::{MIGraphX,ROCm}, runtime dlopen); could not be executed here — no AMD GPU — but compiles under--features embed-ort-rocmand the probe test asserts the clean-CPU branch. Design:.nornir/vector-rocm.md. - G2 ✅ Optimize the CPU vector path — added an int8 + AVX-512 VNNI cosine kernel (
score_i8_batch/VectorIndex::search_i8,vpdpbusd, runtime-detected → scalar-int8 fallback) alongside the existing AVX-512F/AVX2-FMA f32 dispatch, plus batched per-query setup and the i8 matrix being 4× cache-denser.bench_kernels+nornir vector benchtime scalar vs SIMD vs int8 and compute the speedups; tests assert SIMD == scalar (<1e-5) and int8 == scalar (<4e-2). Benchmarked on Zen 2 Threadripper 3975WX (AVX2+FMA, no AVX-512/VNNI), 768-dim, 200k×30 iters: scalar 1.5 Mdps → AVX2-FMA 6.7 Mdps (4.48×) → int8-scalar 5.7 Mdps (3.79×, from memory density alone). GPU↔CPU ratio: where a CUDA/ROCm GPU embeds ~100× a scalar CPU, the SIMD/int8 CPU path closes that to ~22×; VNNI (structural-only here — no AVX-512 silicon) folds 64 int8 MACs/instr and would push the int8 path past the AVX2 f32 number, shrinking the gap further. Design:.nornir/vector-cpu-simd.md. - G4 ✅ aegir S3 read/write concurrency (462426f + nornir-bench wiring): concurrent multi-PUT writes + coalesced zero-copy reads. data_pipe_skade_s3 4.16M→13.97M rows/s (3.35×, 13%→46% of NVMe, run 93c2bf04); local stays sequential. Headroom to ~90% = commit/manifest concurrency + more files.
- G4b ✅ reuse znippy
gatlingfor the concurrent save (your ask). Done:znippy-zoomies/src/gatling_io.rs(pub mod ioingatling.rs) is the async-I/O sibling of the sync engine — a no-barrier, bounded (max_in_flight), ordered (run_orderedre-sequences by job index), zero-copyBytesasync-task pool on tokio (rununordered +run_ordered). Its module doc motivates the diff (sync OS-threads/Read-split/CPU-transform vs async tasks/job-stream/latency-bound I/O) with a side-by-side invariant table. skade'sbench/src/pipeline.rs::run_concurrent_writer_ingestdrives the concurrent S3 multi-PUT throughgatling_io::run_ordered, hitting 14.06M rows/s (shipped in skade-katalog 0.1.8, run f2f9602a). Tests:gatling::io::testsinject N ordered/unordered zero-copy jobs through a mock async sink and assert order, exact values, and that the in-flight peak both ≤ and == the cap (proving real no-barrier overlap), plus first-error short-circuit. znippy commit f732bda. • WRITEbench/src/pipeline.rs::run_single_writer_ingest— ONE writer writes files sequentially (its own comment: "Sequential, uncontended write — the single-writer payoff"); optimal for NVMe, latency-bound on S3 → 4.2M vs 34M. Pluscommit_files/fast_append writes manifests+metadata serially. • READbench/src/s3_storage.rs::S3FileRead::read— ONE blocking ranged GET per column-chunk (no read-ahead/coalesce — grep: zero FuturesUnordered/join_all/buffer_unordered). FIX: an object-store ingest/read variant with CONCURRENCY — buffer_unordered the file PUTs (N in-flight) + parallel manifest writes; concurrent + coalesced range reads (read-ahead, likeobject_store::coalesce_ranges); pair with the aegiruringfeature for the plaintext hot path. Keep the single-writer-sequential path for local NVMe/RAM (it's optimal there). Benchmark back toward ~90% of NVMe. - G3 interchangeable embedding model registry (395db58: jina-v2 default + minilm-384 + bge-768, dynamic DIM, distinct model_profile keys, inject-assert)
- G5 (#9) ✅ runtime GPU/CPU/AMD embedder selection — both backends DEFAULT. — done: a plain
cargo install nornirnow ships ONE binary carrying BOTH the tract CPU embedder AND the ort GPU embedder (CUDA + ROCm) and picks at RUNTIME. Build-safe-by-default via ort'sload-dynamic(ort/load-dynamic→ort-sys/disable-linking, pinned toort 2.0.0-rc.12): the binary LINKS NO onnxruntime, socargo buildsucceeds on a GPU-less / library-less box; ortdlopenslibonnxruntime.soat runtime only when a GPU path is chosen. New single selectorsrc/vector/select.rs::decide(&Probe)→Backend(pure, inject-assert testable): onnxruntime-present gate → NVIDIA CUDA → AMD ROCm → tract CPU fallback.cuda.rsgainedonnxruntime_dylib()/arm_onnxruntime()(find + version-loadable-probe libonnxruntime.so, setORT_DYLIB_PATH);load_embedder()/embedder_backend()route through the selector;nornir vector doctorprints the runtime choice (onnxruntime/cuda_ready/rocm_ready + selected backend + real test embed).defaultfeatures now includeembed-ort. Proven on the RTX-4090 box: (1) no libonnxruntime.so →onnxruntime=false, selector picks tract CPU, real 768-dim embed in ~50ms (no panic/link-fail despite CUDA being fully READY); (2) loadable onnxruntime stub present →arm_onnxruntimesets ORT_DYLIB_PATH, live probe seesonnxruntime=true cuda_ready=true, selector picksBackend::OrtCuda. Tests (all green): 5 pure decide() cases (CUDA / CUDA≻ROCm / ROCm-alone / CPU-fallback / no-onnxruntime-gate) +live_probe_is_self_consistent+ 2#[ignore]real-path tests (CPU embed via selector; OrtCuda arming via stub). Honest prerequisite: GPU path needs a runtimelibonnxruntime.so≥ ORT 1.24 (api-24) + CUDA/ROCm libs; absent → tract CPU. Design:.nornir/embedder-selection.md. - 🔮 G6 INVESTIGATE: CozoDB as a vector+graph QUERY-LAYER accelerator — evaluate (and write up the gains in a
.nornir/cozodb-eval.md) whether CozoDB — an embedded transactional DB with Datalog + a built-in HNSW vector index + relational/graph relations — should back the vector + knowledge-graph QUERY layer. Hypothesis / gains to measure: (1) unified semantic+structural queries — one Datalog query over BOTH embeddings (HNSW ANN) ANDcall_edges/symbol_facts(e.g. "functions semantically like X that are called by Y", recursive call-path + nearest-neighbour in one shot) — today these are two engines (vector index + petgraph) stitched in Rust; (2) faster ANN at world-scale (HNSW vs the current brute-force/SIMD cosine over a loaded blob — bench crossover point); (3) graph algorithms (transitive closure, PageRank-style heat) as Datalog rules instead of hand-rolled traversals. The LAW-it-must-respect: Iceberg stays the source of truth + the SHA-keyed time-travel store; Cozo is a derived, rebuildable INDEX/accelerator (same slot as the planned Ragnar static search tree, see render-engineRagnar stree), populated FROM the warehouse, never the other way. Cost to weigh: a second embedded engine (Cozo backends: mem/sqlite/rocksdb) + dep weight + the "everything in one warehouse" law dilution; gate behind a feature flag (cozo-index), default OFF. Deliverable: the eval doc + a small spike (feature-flagged) that builds a Cozo index fromembeddings+call_edgesand runs the unified query above, with a bench vs the current path — then a go/no-go. (Raised by Rickard 2026-06-25.) - 🔧 G7 DE-DUP the vann vector index — one engine, not two (Rickard 2026-07-02). The exact int8→f32 cosine engine exists twice: as a MODULE
znippy-zoomies/src/vann/mod.rs(~1.5k lines, part of the znippy-zoomies root crate — NOT a standalone crate) AND as a private copy insidenornir/src/vector/store.rs. Proof of the hazard: Levers A (int8 8-accumulator VNNIdot_i8_vnni) and B (search_batchcorpus-read amortization) had to be written in BOTH places — every future kernel fix is a double-edit waiting to drift. Fix: promoteznippy-zoomies/src/vann/to a realvanncrate (pubtheVectorIndexsurface nornir needs), then have nornir depend on it and delete nornir's private copy — nornir ALREADY deps the znippy-zoomies crate (znippy-zoomies = "0.1.13"/ path), so the edge exists. Keep nornir's own warehouse/Occurrencebinding insrc/vector/store.rs; only the engine core moves out. Risk: spans two repos + touchesnornir/src/vector— do it AFTER the marker/arch fork fully lands to avoid clobbering. Verify withsearch_batch_is_identical_to_per_query_search+ nornir'swired_batch_search_matches_per_queryon both sides. (Surfaced during the 2026-07-02 dedup audit alongside thezstd-sys-rsfork + the tripleznippy-zoomiestree — see those as sibling items.)
EPIC H — Local LLM / on-prem + MCP tool testing 🟠 BIG
- H1 MCP tool test harness — now functional + matrix-wired: every one of the 56 tools driven with REAL args, structured OUTPUT asserted against a seeded warehouse (LAW 1), coverage gate from the live
tools/list(RED if any tool unexercised). 56 tools → 53 pass / 3 skip (server-only: workspace_register/sync_now/index_status) / 0 fail. Shared body intests/support/mcp_harness.rs, run viacargo test --features mcp --test mcp_matrix(mcp_every_tool_functionally_asserted) AND--test mcp_tools; coverage manifesttests/mcp_tool_coverage.json. Docs:.nornir/mcp-tools.md(authoritative 56-tool table) +.nornir/mcp-from-claude-code.md(idiot guide). FIXED: seed warehouse path drift (was<root>/.nornir/warehouse; now the per-workspace<root>/.nornir/workspaces/default/buildsthe MCP actually opens). Earlier (35f795f): FIXED runtime-within-runtime hang in changed_since_last_release + regression_trace - H2 local-LLM/GPU support audit (18b73b3: CUDA-only/fixed-model/stdio gaps → G1/G2/G3) — read
nornir-mcp(stdio tools),vector::Embedder/embed-ort(CUDA)/embed-tract(CPU); what's missing to drive nornir from an on-prem LLM. Cross-ref Epic G. - H3 web analysis 2026: best on-prem LLM (Qwen3-Coder-30B default; Devstral/Gemma4 alts) for agentic coding (Claude Code/Kilo/nornir-MCP) — coding + tool-use quality, VRAM sizing (4090); ranked recommendation.
- H4 "manual for idiots" SHIPPED (.nornir/local-llm-setup.md)
nornir/.nornir/local-llm-setup.md— download model → run (llama.cpp/ollama/vLLM) → point Claude/Kilo at it → wire nornir MCP → verify via H1. - H4b general "for dummies" guide SHIPPED (
.nornir/nornir-for-dummies.md) — what nornir IS in one breath, install (cargo install nornir --features viz), the handful of verbs (test/bench/release/workspace add/viz/root), the 3 modes in two sentences,~/.nornirdata root, cross-links to data-root/viz-modes/mode-matrix/running-as-a-server. - H5 ✅ Agent/model BAKE-OFF matrix (nornir-driven, the big one) —
nornir bakeoff run --task … --agents a,b --models x,ycrosses the agent × model cross-product (claude/kilo/local-llm × any models), recording oneagent_model_runs(run_id, agent, model, task, status/ok, score, wall_ms, tokens_in/out, tokens_per_s, cost_usd, mcp_tool_calls)row per(task, agent, model)cell to the warehouse → see which agent+model won.agent/cost_usd/mcp_tool_callsadded as OPTIONAL schema fields 14–16 (migration-safe; legacy rows defaultagent="-"). Backend stays mockable (MockCallerkeyed by(agent,model)cell,--mock); a model-only run is the 1×N matrix under a single local-LLM agent.matrix_grid()→MatrixGrid{agents,models,cells}ranks cells (score→tok/s→latency→cost), grid winner == flat-leaderboard winner.nornir bakeoff demoseeds a realistic 3×3 mock matrix (one failed cell) so a fresh viz shows a populated 🏆 Leaderboard, which now renders the agent × model grid (winner gold, fails red) + the ranked cells, reads embedded or overViz.BakeoffResults, and exposes the grid instate_json.leaderboard.matrix(LAW 6).bakeoff leaderboard --matrixis the CLI twin. Inject-assert tests: 7 unit (cross-product, grid ranking+winner, demo round-trip, legacy default, failed cell) +viz_leaderboard_pane(4-cell matrix → grid+board in state_json) + remote wire round-trip (axis+economics survive, pre-matrix JSON defaults). Docs:.nornir/bakeoff.md(design+guide+idiot). Reuses H1 MCP-exercising + C6 run→persist→live shape. - F1 incremental republish (scope to changed members + dependents) —
monitor::republish*now threads the poll's change-set; the affected set =changed ∪ transitive dependents(MímirWorkspaceGraph::affected_by_change), intersected with present checkouts, scopes the index rebuild via newIndex::build_scoped. Falls back to a full rebuild when the dep graph is unavailable, on first publish (fat-modeworkspace add), on the empty-warehouse path, and on a forcedWorkspaces.Fetch. Inject-assert testmonitor::tests::affected_set_is_changed_union_dependents_and_skips_unaffected(diamond graph → asserts blast radius + that unaffected siblings/deps are skipped). Docs:../dwarves/.nornir/incremental-republish.md(design) +../dwarves/.nornir/howto-incremental-republish.md(guide). - F2 aegir uring · 3D volumetric fog · SBOM from warehouse deep-scan · bench docs
skade_*labels
EPIC #45 — Funnel local-LLM task decomposition with compiler-as-oracle ✅
- #45b ✅ funnel decompose → test matrix Aspect::FunnelDecompose + viz 🔬 trigger + results panel (state_json["decompose"]) —
nornir-testmatrix0.1.3:Aspect::FunnelDecompose(label"funnel", in ALL+DEFAULT) +run_funnel_decompose+parse_funnel_decompose(public, testable) added;FunnelTabStategainsdecompose_results/decompose_running+decompose_panel()(🔬 button + inline results: ✅/❌ per subtask + score, local-mode only) +state_json["decompose"](run_count/pass_count/rows) (LAW #6) +run_decompose_test_for_test()test hook.tests/viz_funnel_decompose.rs: 4 always-on inject-assert tests (parse pass/fail, parse_aspect round-trip, ALL+DEFAULT membership, state_json initial shape) + 2 heavy#[ignore]tests (pass_count==5, warehouse rows aspect="funnel")..nornir/funnel-decompose.md§5 updated. - #45 ✅ task decomposition + generate→judge→retry loop — a funnel task (idea/error/test) is broken into ≤20 atomic, testable subtasks in
src/funnel/decompose.rs, each RAG-anchored to a concrete file/fn: aRetrieverseam fans out over the code index (symbol_facts/call_edgesviaKnowledgeView, always-on, model-free lexical scorer) and (under#[cfg(feature="vector")]) the vector index — so it works on default features with no model. Each subtask runs a boundedGenerator → CodegenJudge → retryloop (run_subtask_loop): a localGenerator(mockfor offline/tests,--gen <spec>for a real backend) proposes a patch, theCodegenJudge(#46) is the accept/reject oracle (apply →cargo build+ test → 1.0/partial/0.0), and a reject re-feeds the compiler error for the next bounded round. Subtasks are stored as child funnel DAG nodes under the parent (store_subtasks, chained edges,state_jsonper node — LAW 6) and verdicts written back (apply_verdicts: accept→Done, reject→Failed). Offline simulationnornir funnel decompose <task> --fixture+examples/funnel_decompose.rsrun the WHOLE loop withmock/canned Generator + a REAL judge against a tiny fixture crate (no model, no network) — printing the subtask DAG + per-subtask verdicts. Matrix tests (aspectfunnel): 12 unit +tests/funnel_decompose.rsinject-assert (decompose anchors N real files; RAG ranks the relevant snippet first; loop accepts judge-1.0 / rejects judge-0.0; offline sim end-to-end with DAG reflecting verdicts) — published astest_resultsrows so they show in the matrix with the real scores as metrics. Docs:.nornir/funnel-decompose.md(design+guide). Reuses #39 Generator + #46 AnswerJudge oracle + the planner's RAG-grounded spirit.
EPIC J — Beautiful SVG flow diagrams (skade-katalog circuit-board style)
- J1 Style ✅ —
skade/.nornir/assets/read_paths_circuit.svgexists (hand-authored, circuit-board style: dark gradient #0b1c2e→#06111d, cyan/ice accents, monospace labels, left→right flow + legend). THE house style, referenced in skade.nornir/README.md. - J2 gatling flow diagram DONE (znippy a711c83 — circuit-board SVG + gatling.md)
- J2-old gatling flow diagram (znippy
.nornir/assets/gatling_flow.svg+ design doc): the no-barrier engine — Reader → split into slots → N worker threads CPU-transform (no barrier) → ordered collector → Sink, zero-copy. Plus the newgatling::ioasync sibling (job stream → bounded N-in-flight → ordered collect). In the circuit-board style. - J3 nornir flow SVGs (183f7d8: release_events_flow + warehouse_layers, embedded)
- J3b nornir flow SVGs round 2 — three more circuit-board flows authored + embedded:
deployment_modes.svg(fat/thin/server data flow → mode-matrix.md),test_matrix_pipeline.svg(run → detect_runner → parser → test_results → Test pane → test-matrix.md),viz_thin_rpc.svg(mode resolution → tab → token-bearing gRPC verb → Arrow batch → table → viz-modes.md). release_events DAG already covered byrelease_events_flow.svg(J3). - J4 every repo's
.nornir/design + guide carries its key flow as one of these SVGs (LAW 2 + visual). korp/knut/skade/facettkorp_flow.svg/knut_flow.svg/skade_flow.svg/facett_flow.svgauthored (circuit-board house style) + embedded in each.nornir/design.md+ committed + pushed (korp/knut/skade→main, facett→feat/d2h-inline-edit). - J5 nordisk constellation SVG (
assets/nordisk-constellation.svg, circuit-board style) — one diagram showing both nornir's crate deps (znippy-zoomies 0.1.8 / nornir-testmatrix 0.1.0 / skade-katalog 0.1.7 + skade 0.4.9, all verified againstCargo.toml) AND its orchestrator role (workspace descriptor + registry → git clone/fetch → republish hook → per-workspace warehouse → viz/MCP, over monitored repos holger·skade·znippy·knut·facett·korp·njord). Embedded in.nornir/design.md§"Place in the nordisk constellation".
EPIC I — Dependency graphs: more types, working, navigable, visible in design docs
- I1 design-doc graphs FIXED (df659c1: render_sources_in_place fills .nornir/*.md depgraph markers + doc-relative assets + no-silent-gap note) — ROOT CAUSE FOUND: design.md has empty
<!-- nornir:gen:start:depgraph -->/:endmarkers; renderersrc/docs/svg.rs::render_graph_svg+ adepgraph.svgasset exist, so fix = makenornir docs renderrepopulate the markers (needs docs-export build + non-emptydep_graph_edges→ ties I2). Then graphs show in znippy/nornir/skade design sections. - I2 dep graph populates from live cargo metadata (znippy design.md → 90KB SVG) — beyond cross-repo dep graph: crate-level, module/symbol (knowledge), call graph, feature graph, build-order. Verify the existing dep graph actually populates (dep_graph_edges/release_lineage) + fix where empty.
- I3 navigable — pan/zoom/click-collapse-subtree, 8 inject-assert tests + 2 snapshot proofs (5ca67bc) — pan/zoom/click-expand/transitive (ties C5/C7); improve
facett-graph/facett-graph3d("better facett graphs") — rectangular nodes, transitive explode, direct-vs-transitive coloring.
EPIC K — Bencher mashup polish (compact "absurd-best" + complete variants)
- K1 data_pipe_polaris bencher (skade 0bf9c11) — write leaderboard now skade nvme/ram/s3 vs Nessie vs Polaris — data_pipe has nvme/ram/s3/nessie but NO Polaris row; add a
DataPipePolarisbencher so the write leaderboard is complete (skade nvme/ram/s3 vs Nessie vs Polaris). - K2 compact mashup directive (c33fcf2: mode=compact + top=N, 3 inject-assert tests, documented in bencher.md) —
benches winner=rowalready bolds the best cell per column, but there's no COMPACT mode collapsing to a few standout rows (the "absurd best values per column" Rickard wants). Make it a first-class documented gen-directive (e.g.benches mode=compact top=3) insrc/docs/sections.rs/svg.rs+.nornir/bencher.md. Currently under-documented + not programmatic enough. - K3 README top best-of mashup RENDERED (top=3 reads + top=3 writes incl. Polaris, run 89bb618a, skade 4088c21)
Cross-cutting law harness
- MD-MATRIX multi-dimensional viz matrix test (tab × mode × data) —
tests/viz_md_matrix.rscrosses 10 data-bearing tabs × {fat embeddedSource::Local, thin remoteSource::Remotevs an ephemeral in-processnornir-serverwhosebuilds/warehouse is pre-seeded} × {data-present NON-EMPTY, data-absent clean-placeholder} = 40 cells, each asserting the renderedstate_jsonwith the CORRECT keys (timeline.releases,dep_graph.node_count,test.grid.rows,release.rows,knowledge.crates,funnel.plan_count,live.events,leaderboard.leaderboard, scannedrelease_eventsrows for Warehouse). 35 cells asserted (20 fat + 15 thin) — FAILS if a tab is empty-with-data in EITHER mode (catches the live remote-path emptinessviz_live_data.rs's single axis missed). 5 thin+present cells are logged#[ignore]-known-gaps (Release/Test/Leaderboard = noRelease.Events/Test.Results/Bakeoff.ResultsRPC, being wired by feat/viz-remote-rpcs; LiveRun = no remote warehouse-hydrate; Funnel = server funnel-seed out of scope) — each with a dedicated*_known_gaptest asserting the intended contract, flips to required by dropping the#[ignore]. Doc:.nornir/viz-md-matrix.md. Run:cargo test --features viz,server --test viz_md_matrix. - LAW4 test-side screenshot proof harness —
egui_kittestwgpu → PNG, 4 viz tabs, inject-and-assert,.nornir/viz-snapshot-testing.md(78b439c, merged + 4/4 verified) - LAW4b live-app self-screenshot via
ViewportCommand::Screenshot(button/keybind/CLI) → PNG —VizCommand::Screenshot+ScreenshotRequeston control channel, Ctrl+Shift+S keyboard shortcut,nornir viz --screenshot <path>CLI flag, warehouserobot_test_resultsblob persist,state_json["screenshot"]with pending/last_path,.nornir/viz-law4b-screenshot.md, 4 inject-assert tests (tests/viz_law4b_screenshot.rs)
✅ Foundation already shipped (this session)
- viz egui 0.28→0.33 bump · viz_buttons crash-hunt · 2-row toolbar
- bench-progress logging + live 🐞 action-log tail
- lock-tolerant catalog reads (CLI never wedges the live server)
- funnel red-error fix (ensure_funnel + viz error surfacing)
- facett caps/clipboard/themes/effects/raven + form/card/menu/plotters/fx/pipeline
.nornirdocs across nornir/facett/korp; Bifröst pure-Rust + pushed
⭐ EPIC N — Deployment modes: home-derived root + per-workspace + viz mode-resolution (2026-06-13)
The three modes (fat / server-monitored / thin) made coherent end-to-end. Decided + built with Rickard.
- N1 home-derived data root
<home>/.nornir, ALL path env-vars removed (home::home_dir());nornir root - N2 cross-user test — same behavior as me OR
nornirvia sudo (root_resolution.rs, PASSED) - N1b PER-WORKSPACE layout
workspaces/<name>/{git,builds(=warehouse),funnel}; no global warehouse; fat≡server (warehouse_root == server's servedbuilds) - N3 live server NUKED + repopulated onto
~/.nornir(auto-registered all 6 via the then-extantNORNIR_MONITORenv — wiring since removed,0c240f9/9ea97c4);/var/lib/nornirretired - N-viz
nornir vizmode resolution: no config→THIN (auto-connect :7878 + token from~/.nornir/token), config→FAT;--server <addr>/--token; pureviz_mode()tested across every start permutation × config present/absent - N-add eager
nornir workspace add= register+fetch+build inline (server≡user; poll loop = automated re-run) - N-stamp build stamp (C1) — git SHA + build time via build.rs → window title + ℹ About popup (green) + state_json; "is this a fresh client?" at a glance
- N-rpc viz THIN-mode remote RPCs —
Viz.ReleaseEvents(Release+LiveRun) ·Viz.BakeoffResults(Leaderboard) ·Viz.Knowledge(server-side scan; VERIFIED live: holger 45 crates). Finding: the viz was tested EMBEDDED only; the remote path was half-wired —viz_live_data.rs(catch empty tab) + the multi-dimviz_md_matrix.rs(tab×mode×data) now guard it - N5 viz logs → warehouse
viz_actionstable (durable + queryable; /tmp kept as live tail) - N-data Release/Leaderboard/Test fillable THIN — submit RPCs built (feat/n-data-submit-rpcs):
Telemetry.SubmitBakeoff/SubmitTestResults(existed) +Telemetry.SubmitReleaseEventsNEW, all mirrorBench.Submit; CLIbakeoff/test/release runroute to the server whenNORNIR_SERVERis set (persist_bakeoff/persist_test/release drop-guard). Inject-assert:tests/data_submit_rpcs.rssubmits real rows → reads back overViz.BakeoffResults/Warehouse.Scan/Viz.ReleaseEvents, asserts exact values + therelease_eventsOption round-trip. Doc:.nornir/data-submit-rpcs.md. (Human still runs the producers against oden to fill it.) - N-flip un-ignore the 2 stale
viz_md_matrixdrivers (rebase funnel cell + flip now-wired thin cells)
EPIC Holger — pluggable artifact server: deploy + docs
- HOLG1 deploy PR
feature/writable-helm-k8s: cherry-picked the helm chart + k8s/Talos manifests + Dockerfile into main; http/rest gateway + file-helm server commits DELAYED (Rickard's call: no gRPC/http changes yet) - HOLG2 whole pluggable repository-backend system documented (
.nornir/{design,guide,idiot-guide}.md) + 5 skade-style SVG flow diagrams (push/pull/registry/families/deploy) + book.pdf rendered (nornir docs-gen) and pushed
D3-followup — extract the viz top-bar as a facett component (Rickard, 2026-06-13)
- D3b ✅ (merged feat/d3b-tabbar, 6fda8a6)
facett_ui::TabBar/Toolbar— factor the viz top-bar (workspace picker + the 16 tab buttons in 2 rows + 🎨 palette picker + ℹ About + live toggle) into a reusable, themed nav/toolbar component: input =tabs:[{icon,label}]+active+ optional pickers/actions; output = selected tab (+state_json). Goes infacett_ui.rsnow (the C8/D3 module),pub use facett::TabBaronce egui 0.33↔0.34 aligns. The menu-buttons pane is a component, not bespoke chrome.
D5b — facett-demo mega matrix: + a LIVE-DRIVE row (Rickard, 2026-06-13)
- D5b1 mega matrix DONE (verified 2026-06-16) —
facett-demo/tests/mega_matrix.rs: 294 cells = 21 components × 2 data-states × 7 palettes, inject-real + assert-real (data-signal count per cell + palette consumption), emitsfacett_demo.mega_matrix/palette_gallery. Bidirectional coverage gatecoverage_gate_every_demo_component_is_in_the_matrix(LEGEND==21) +coverage_gate_catches_a_fake(proof it bites). Runcargo test -p facett-demo --test mega_matrix --features testmatrix. - D5b2 drive EVERY button/function DONE (verified 2026-06-16) —
facett-demo/tests/robot_demo.rs: 45-step scenario driving all 21 components (theme picker · toolbar · graph/3d/rabbit/depgraph · grid/table scroll · card/form · pop/plotters · map · syschart select · geomap pan/zoom · icebergview time-travel · menu palette), inject-real AccessKit action → assert-realstate_jsonread-back, oneTestResultRow(aspect=robot-ui) per step. Coverage gate: every LEGEND key driven. Red-proof guardrobot_runner_records_a_wrong_expectation_as_a_red_cell. Caveat: robotui is an in-crate port (egui_kittest 0.34) pendingnornir-robotuion egui 0.34 — one-line swap when it ships (see GAP-ROBOTUI-034).
Session 2026-06-13 — releases 0.4.14 → 0.4.20 + new epics (ALL captured here per Rickard)
EPIC RELEASES (shipped to codeberg + crates.io this session)
- 0.4.14 "Heimdall" — home-derived root
~/.nornir, per-workspace layout, 3-mode viz auto-resolve, eagerworkspace add, build stamp, viz thin RPCs, C8/D3 themes, H5 bakeoff matrix, 56-tool MCP functest, robot-UI MCP, N5/P1.3/F1, multi-dim matrix - 0.4.15 — viz fix: workspace switch re-scopes 🛡 Security tab (was missing a setter → kept prev workspace's repos); inject-assert test
- 0.4.16 — viz pre-flight popup (rustup 1-click install + 0-MCP-calls warning w/ 📋 copy) + version on splash + multi-workspace MCP (
workspaces_listreturns every workspace WITH its repos;NORNIR_WORKSPACEnow optional) + SBOM scan generate-lockfile fallback + ops:nornirservice user got rustup +~/.cargo/binon PATH (path.conf drop-in) - 0.4.17 — viz workspace×function matrix (
tests/viz_workspace_matrix.rs: per-workspace functional data over thin RPCs + switch-isolation no-leak;App::switch_workspace_for_test) + heavy tests DEFAULT-ON (--skip-heavy/NORNIR_SKIP_HEAVYopt-out) +NORNIR_TEST_FEATURESso the release gate runs the viz matrix - 0.4.18 — viz in default features: plain
cargo install nornirbuildsurdr-threads,nornir vizworks with no flag (was exec-No-such-file) - 0.4.19 — pre-flight popup non-blocking: 0-MCP + thin-mode cargo checks no longer nag over content (
Check.blocking+Preflight::run(thin)); popup only for real local-toolchain blockers in fat mode - 0.4.20 —
Telemetry.SubmitMcpCallsRPC: client-modenornir-mcpsubmits per-call telemetry to the server → served workspace'smcp_requests+ viz 📞 tab show real usage (was dropped in client mode → always 0) - 0.4.34 + the 2026-06-15 stack ship — published to crates.io: nornir 0.4.34 + nornir-airgap 0.1.0 (NEW crate, EPIC AIRGAP / Skíðblaðnir) + znippy-common 0.9.3 (lgz cargo
.cratedecompress). nornir 0.4.34 carries: deep_scan 33s→~9s (in-processCargo.lockresolver — last shell-out gone — + all-corebuild_groups_parallelflush + mimalloc; pure Rust, no FFI), always-visible test matrix (ok|fail|X, syn-discovered build-free on load; run only on release/request; backfill for unchanged members), SSH-first clones (https→scp-like, deploy key), eagerworkspace addserved-handle fix (redb single-writer), MCP auto-workspace (no more silent[]),.cargo/config.tomlTMPDIR=/home/rickard/workX. Branchautonom/plan-and-matrixmerged →main(f16537d); tagsv0.4.34,nornir-airgap-v0.1.0,znippy-common-v0.9.3pushed.
LAW (new) — Iceberg schema evolution for migrations
- LAW-MIG documented in
.nornir/warehouse-schema.md: migrations use Iceberg evolution (add/rename/reorder/widen = metadata-only, old files stay readable), never silent rewrites; new tables/cols additive so old↔new binaries interop
EPIC N-data — feed the live server warehouse (partial: benches done)
- N-data-bench — bench_runs filled on oden via
Bench.Submit: znippy, knut, skade, holger (artifact-server, 8 results). korp/njord have no harness. Recipe in memoryreference_feed_server_warehouse_benches - N-data-mcp —
Telemetry.SubmitMcpCalls(0.4.20) routes client MCP usage to the server - N-data-bakeoff/test/release — submit RPCs built (feat/n-data-submit-rpcs), all mirror
Bench.Submit. Reconciliation: bakeoff was already end-to-end (Telemetry.SubmitBakeoff+persist_bakeoff); test had the RPC + an unusedsubmit_test_remoteclient — now WIRED viapersist_testinrun_test_one; release had no submit at all — addedTelemetry.SubmitReleaseEvents+ handler +remote::submit_release_events+ a recordingReleaseEventEmitterbuffer drained by a drop-guard inrun_release_run(ships on every exit path incl.bail!).release_eventsOptionfields carried with proto presence flags (None vs empty-list vs value). Inject-asserttests/data_submit_rpcs.rs(submit→read-back over thin RPCs, exact-value asserts). Doc.nornir/data-submit-rpcs.md. (Human then runsNORNIR_SERVER=… nornir bakeoff demo/nornir test all/nornir release runto fill oden.) - #6 ✅ stream clone/populate failures so they're readable from the viz/CLI — done: 2026-06-15, branch
feat/clone-event-stream. Per-member clone/fetch failures wereeprintln!-only on the server log → a thin client saw missing data with no reason. Now FIRST-CLASS + persisted + readable remotely: new Iceberg tableclone_events(ts, workspace, member, remote, op, status, detail, elapsed_ms—src/warehouse/iceberg_schema.rs::clone_events+src/warehouse/clone_events.rs); the monitor's fetch path (src/monitor.rs::fetch_workspace, now emittingMemberOutcomeper member) is persisted viarecord_fetch_reportat every warehouse-owning call site —server_sweep+Workspaces.Fetch+ eager-register insrc/bin/nornir-server.rs, and the fat-CLIFetch/Populateinsrc/bin/nornir.rs. Exposed overViz.CloneEventsRPC (proto/nornir.proto+ handler innornir-server.rs, scoped byAppState::ws(metadata); clientremote::fetch_clone_events). Surfaced in the 🧬nornirroot pane (src/viz/nornir_tab.rs) as a per-member populate-status panel (ok/failed + error detail + when), local mode reads the warehouse (open_read_only), remote mode the RPC, exposed instate_json["populate_status"](LAW #6). CLI parity:nornir workspace events <name>(--json,--serverthin path overViz.CloneEvents). Inject-assert tests: warehouse round-trip (failed member persists + reads back, workspace-scoped) + republish-row +state_jsonsurfaces the failed member/error +Viz.CloneEventswire round-trip; all green. Docs.nornir/clone-event-stream.md. Reuses therelease_events/airgap_eventsshape + the C6 run→persist→live pattern.
⭐ EPIC POPULATE-OVERVIEW — Nornir tab = all-workspaces populate status + viz Test-tab status (Rickard, 2026-06-18)
Goal (Rickard): the 🧬 Nornir root tab must show a list of ALL the server's workspaces with a status column for populate (and friends), not just the active workspace's members. Today the populate-status panel (EPIC #6) is scoped to one workspace; we want the workspace-level roll-up so you can see, at a glance, which of holger/knut/korp/njord/nordisk/skade/znippy populated cleanly vs has a failed/stale member.
- PO1 Nornir-tab workspace roster + populate status — (✅ DONE, verified 2026-06-30 audit — the per-workspace populate roll-up renders via
Workspaces.PopulateStatus; see PF-cloneevents) — render a table of every served workspace (workspaces_list) × {members, last-synced, populate-state ok/failed/stale, last error}. Roll upclone_eventsper workspace (latest op per member → workspace verdict: green if all ok, red if any failed). Local mode reads the warehouse, remote mode an RPC (extendViz.CloneEventsto a multi-workspace roll-up, or addWorkspaces.PopulateStatus). Surface instate_json["nornir"]["workspaces"](LAW #6). CLI twin:nornir workspace lsalready lists name/members/last-synced — add apopulatecolumn (or--status). Inject-assert: seed two workspaces (one with a failed member) → assert the roster shows green/red verdicts + the failing member's error. - PO2 viz 🧪 Test tab shows status — (✅ DONE, verified 2026-06-30 audit — the Test tab renders an at-a-glance per-repo green/red/not-run status header) — the Test tab must display run status (per-repo green/red/blank + last-run verdict, not just the grid cells). Today the grid renders cells but the tab lacks an at-a-glance status (green ⟺ every repo's last matrix run passed). Add a status header/summary (passed/red/not-run counts per workspace, newest run id + when) reading
test_results/test_history; surface instate_json["test"]["status"]. The data is now PRESENT for all repos (filled 2026-06-18, see POPULATE-FINDINGS). Inject-assert: seed a red + a green run → assert the status header reports the red one.
🔴 POPULATE-FINDINGS — bugs found while generating warehouse data for every workspace (2026-06-18)
-
PF-arch ✅
nornir arch generateagainst a server crashed withredb: Database already open. Cannot acquire lock— fixed 2026-06-18. ROOT CAUSE:arch generatehad NO thin-client branch — it always scanned the LOCAL working tree + opened the LOCAL warehouse to historize, so against a running server it (a) resolved repos against the wrong (boot)nornir.toml→no [repo.X]for non-holger members, and (b) tried to open the redb the server already locks → the lock panic. FIX (mirrorstest run/bench run): newOps.RunArchRPC (proto/nornir.proto+RunArchRequest), server handlerOpsSvc::run_archinsrc/bin/nornir-server.rs(scan+coarsen off-runtime viaspawn_blocking, historize viarecord_arch_wiring_asyncunder the server's writer lock, idempotent dedup, per-repoOpTarget), and the CLIrun_archGenerate arm delegates viarun_arch_remotewhenNORNIR_SERVERis set (arch svgstays local — render only, no warehouse write). Inject-assert testarch::warehouse::tests::run_arch_async_persist_is_readable_by_merge(WRITE→READ contract: async historize + idempotent re-run dedup + theViz.Architecturemerge reads both members back).arch generate/historyfor the non-holger workspaces was previously BLOCKED by this; now routes to the server. DEPLOYED + swept 2026-06-18: built+installed releasenornir-server/nornir/nornir-mcp, restartednornir.service; then ranarch generatefor EVERY workspace member (16/16 historized: knut/korp{facett,knut,korp,skade,znippy}/njord/nordisk{facett,holger,knut,korp,nornir,ordning,skade,znippy}/skade) — verifiedskade(was the lock-crash repro) returns✓ skade nodes=310 edges=677viaOps.RunArch. The 🏛 Architecture tab now has data for all 7 workspaces, not just holger. -
PF-cloneevents
Viz.CloneEventsoverflows the 4 MiB gRPC decode cap for nordisk — ✅ DONE (verified 2026-06-30 audit): decode cap raised to 64 MiB on both ends AND a paginatedWorkspaces.PopulateStatusroll-up added (last-N, newest-first), so nordisk's big clone_events table reads back and PO1's remote roster works. Original:nornir workspace events nordisk(and the Nornir tab populate-status remote read) failedOutOfRange: decoded message length too large: ~5.6 MB, limit 4194304. The clone_events DATA was fine (poll loop every 60s × 8 members for days); the READ RPC returned the whole unpaginated table in one message. Other 6 workspaces read fine (smaller/younger). -
PF-bench-read ✅
Ops.RunBenchwrote the warehouse but reads went to the legacy JSONL →[]— fixed + DEPLOYED 2026-06-18. ROOT CAUSE (Rickard's hunch was right — JSONL vs gRPC split): the bench data path forked when it moved to the warehouse. WRITE:Ops.RunBenchpersists ONLY to the icebergbench_runstable (append_bench_run_async);Bench.Submitwrites bothbench_runsAND the on-diskbench_history.jsonlreceipt. READS:Viz.BenchTelemetry(the viz 📈 pane) readsbench_runs✅, butBench.HistoryRPC + the MCPbench_historytool + the CLIbench history-showall readbench_history.jsonl— whichOps.RunBenchnever wrote →[]. (The server comment even claimedbench_historyreadsbench_runs; the code read the file.) FIX = make ALL bench reads warehouse-backed (kill the JSONL read):Bench.Historyhandler →query_bench_runs_async(BenchFilter::for_repo)(src/bin/nornir-server.rs); MCPbench_historyembedded path →open_read_only+query_bench_runs_async(server mode already routes toBench.History) (src/bin/nornir-mcp.rs); CLIbench history-show→ server mode pulls overBench.History(newbench_history_remote+ proto→domainpb_to_bench_run), embedded reads the localbench_runs(src/bin/nornir.rs) — also FIXES the holger-only client-config-resolution bug (it no longer touchesworkspace_holger/release/nornir.tomlor any jsonl path). JSONL is now a pure receipt. Inject-assert:bench_proto_roundtrip_tests(2 tests: every proto field decodes incl.has_duration0ms-vs-None + empty-string→None). VERIFIED on the deployed server: the holger (8) + knut (11) benches that returned[]at start now read back vianornir bench history-showANDmcp bench_history. Built+installed releasenornir-server/nornir/nornir-mcp, restartednornir.service. -
PF-bench-harness znippy + skade bench examples fail server-side —
nornir bench run znippy→Permission denied (os error 13) at /home/rickard/.cache/znippy-targetsHRmVb(the znippy xtask bench derives a scratch/target path under the INVOKING user's home — wrong when the server runs as usernornir; should use a per-user/tmp cache, e.g. honor$NORNIR_WAREHOUSE_ROOT/$XDG_CACHE_HOME/the repo dir).nornir bench run skade→failed to read .../skade/git/znippy/znippy-zoomies/Cargo.toml: No such file or directory(skade's bench depends onznippy-zoomiesby path, but the skade workspace clone doesn't populate the znippy sibling at that relative path). Both are real bench-harness/clone-layout bugs, not the server restart. holger + knut benches DID run. -
PF-clone-path workspace clones miss path-dependency siblings — server journal shows recurring
cargo metadata/build failures: knut →facett/facett-pop/Cargo.toml: No such file or directory; skade →znippy/znippy-zoomies/Cargo.toml: No such file or directory. Path-dependency sibling crates aren't laid out in the per-workspacegit/clone tree the way the member's Cargo.toml expects, so dep-graph build + bench build break for those workspaces. (Manifested as knut/korp showing0 tags/neversynced after the 06:52 restart.)- ⚠️ 2026-06-22 — a SEPARATE corruption mode found + repaired: the server's holger clone was not even a git repo (no
.git, 0-byteserver/lib/Cargo.toml) — a clone abandoned mid-flight (likely guillotined by PF-server-restart; holger is private/SSH so it failed where public/HTTPS members succeeded). Re-cloned via the workingnornir@odenSSH deploy key →nornir bench run facett(facett→nornir→holger-server-libpath) now resolves. ROOT CAUSE STILL OPEN: the server's clone step isn't atomic/idempotent — an interrupted clone leaves a corrupt tree that the poll never self-heals. Make clone atomic (clone to temp dir → atomic rename) + a verify-and-reclone-on-corruption guard. Ties to PF-server-restart.
- ⚠️ 2026-06-22 — a SEPARATE corruption mode found + repaired: the server's holger clone was not even a git repo (no
-
PF-leaderboard ✅ 🏆 tab — source intact, it was the SAME write-to-server/read-from-local mismatch as PF-bench-read — fixed + DEPLOYED 2026-06-18. FIRST: confirmed NO source was lost —
src/viz/leaderboard_tab.rs(20KB, agent×model grid + score/tok-s/cost/mcp) and the GPU accel (nvidia=src/vector/cuda.rs, amd=src/vector/rocm.rs,select.rs,embed_ort.rs— the EMBEDDER backend, separate from the tab) are all present. ROOT CAUSE: the CLIbakeoff leaderboardarm had NO server branch — it ALWAYS opened the LOCAL warehouse (loaded.warehouse_root()+open_read_only), whilebakeoff demo/runwrite to the SERVER (persist_bakeoff→submit_bakeoff_remote). So the demo landed on oden butleaderboardread a stale LOCAL warehouse → the oldm1/m2(0.000) run, in every workspace. (The viz leaderboard tab itself was ALREADY correct —leaderboard_tab.rsreads Local ORremote::fetch_bakeoff_resultsper mode; only the CLI reader was broken.) FIX: CLIbakeoff leaderboardnow routes to the server overViz.BakeoffResultswhenNORNIR_SERVERis set (newbakeoff_results_remote), then applies a pureselect_bakeoff_rows(newest-run / by run_id / by model fallback — identical precedence to the embeddedBakeoffSelector); embedded mode unchanged (src/bin/nornir.rs). AddedDefaultderive toAgentModelRunRowfor testability. Inject-assert:bakeoff_select_tests(4: newest-run-only, run_id match, model fallback, empty→empty). VERIFIED on the deployed server: seeded a freshbakeoff demo(run 3911ae64, 3×3, claude/opus) andbakeoff leaderboard --matrixnow returns THAT matrix, not the stale m1/m2. Built+installed releasenornir-server/nornir/nornir-mcp, restartednornir.service. (Same class as PF-bench-read; both were "write to oden, read locally".) -
PF-server-restart
nornir.servicerestarts every ~10–50 min ("Deactivated successfully" → immediate Started) — clean stops at 05:12 / 05:21 / 06:00 / 06:52, NOT panics. Each restart drops+reacquires the redb lock and resets the registry tonever-synced until re-polled, and KILLS any in-flight heavy Ops run (my knut/skade benches at 06:52:52 hit it → transport error / connection refused; re-running after succeeded). Find what triggers the periodic restart (redeploy hook? watchdog?RuntimeMaxSec?) so long heavy runs (full test matrix, planet benches) aren't guillotined mid-flight.
EPIC HOLGER-UI — holger/for_ui_dev.md brief (for a Claude agent + human dev)
Goal: crisp/barren brief pointing the dev/agent to read holger book.pdf (codeberg link) → writing-a-ui chapter; egui via facett components preferred (facett component list); gRPC OR direct-Rust-call against holger core; circuit-diagram chapter; start mannequin (idiot-guide link); facett components are robot-testable. PUSH to codeberg when prerequisites valid.
- UIP-prereq2 holger book.pdf + flow/circuit SVGs +
writing-a-ui.md— EXIST (verified) - UIP-prereq4 facett components robot-testable —
facett-core::harness(Facet::state_json+ headlessrender_themed+RenderReport.drew()+dump_state) — framework EXISTS - UIP-prereq3a mannequin idiot guide
holger/.nornir/mannequin-idiot-guide.mdcreated - UIP-prereq3b mannequin smoke test that STARTS it (
mannequin/tests/smoke_serve.rs, heavy#[ignore], boots daemon + asserts it listens) — PASSES - UIP-prereq1 facett book.pdf — MISSING; needs docs-export render (facett has
.nornir/components.mdsource). Pragmatic fallback: point brief atcomponents.md - UIP-write
holger/for_ui_dev.mdpushed to codeberg (19a5a54)
EPIC MANNEQUIN-REVIVAL (Rickard: "go through all source, make it really work, rework UI onto facett, invent facetts if needed")
- MQ1 deep source audit DONE (all subsystems live-verified WORK; clippy fix; GUI inventory captured) (pki/oidc/auth/nexus/server/config/event_log/gui/lib) — agent running; verify each service live (PKI cert-gen, OIDC endpoint, mock registries), fix breakage. Builds today ✓, smoke-boots ✓
- MQ2 ✅ (MANNEQUIN-REVIVAL: facett-syschart::SystemChart shipped, facett 5dd326f, holger bccedf4) rework mannequin GUI (
gui.rs, currently raw egui system-map) onto facettFacetcomponents (node/edge scene for the system map, grid for the event log, …) - MQ3 invent new facett components if the system-map needs them (and ensure they're robot-testable per the harness)
- MQ4 make mannequin part of the holger workspace + test matrix (smoke test landed; add to workspace members so
cargo build/test --workspacecovers it) — ✅ DONE (verified 2026-06-30 audit): mannequin is a holger workspace member;cargo test --workspacecovers it.
EPIC HENRIK-BUGS — nornir-mcp install/first-run friction (0.4.21, reported by Henrik)
Full original report (B1–B5 + repro + suggested fixes):
.nornir/henrik-bug-nornir-mcp-install.md.
- B1
mcpadded to default features → plaincargo install nornirshipsnornir-mcp - B2
nornir-mcpclient mode (NORNIR_SERVER set) needs NO local nornir.toml — default bootstrap fallback - B3 config discovery no longer hardcodes
workspace_holger— genericnornir.tomlwalk-up + actionable error; storage-default comment de-personalized - B4
nornir-mcp --help/--versionwork before any config load - B5/minor docs: mcp-from-claude-code.md plain-install line + "Failed to connect → run nornir-mcp directly" + "NORNIR_SERVER is gRPC, don't curl"
- DOCS-REGEN holger + nornir book.pdf regenerated + pushed (0.4.22 fixed the source-vs-root image-resolver bug that blocked nornir). facett book.pdf still pending (UIP-prereq1).
DOCS-LIVE — live-server docs-export gap (Henrik, 2026-06-13, MCP against oden)
The book.pdf renders LOCALLY but not against the live oden workspaces — confirmed driving the nornir MCP while trying to docs-gen the
holgerrepo. This is the live-path face of UIP-prereq1 (facettbook.pdfMISSING) + EPIC A docs.
- DL1 redeployed the LIVE nornir-server (oden) with
--features server,docs-export,net-scansoDocs.Book/Docs.Exportwork against live workspaces (Henrik redeployed; this branch re-verified the deployed binary SHA + DR2 marker).Docs.Book holger/nornirnow return a real PDF, notUnimplemented. - DL2 green: with DL1 + DR1/DR3 landed,
docs history holgeranddocs history nornirnow return non-zerodoc_exportsrows on oden (holger book 984 KB; nornir book 10.9 MB) — a holgerdocs render+docs book(and the path-dep'd nornir) succeed end-to-end against oden.
EPIC FUNNEL-INTAKE — paste errors/ideas in (viz + CLI) + browse history (Rickard, 2026-06-13)
The funnel (ideas→plan DAG, src/funnel/) gets a human-facing intake + history UI:
- FI1 paste-in intake — viz Funnel-tab paste strip (💡 idea / 🐞 error toggle + textbox + Submit, local
StoreorFunnel.SubmitIdeaRPC) + CLI paritynornir funnel idea "…"/nornir funnel error - < report.txt(sharedfunnel_intake;-/stdin/--file). Shipped 0.4.25. - FI2 history navigation — viz history side-panel (newest-first, kind/status filter chips, click→triage→plan lineage) + CLI
nornir funnel history [--kind error|idea] [--status …] [--table]+Funnel.HistoryRPC, all over the sharedfunnel::historyprojection. Readsfunnel_events. - FI3 errors are first-class funnel items (kind
error,ItemKindonIdeaSubmitted+item_kindcol 28, migration-safe viaensure_table_schema) — triaged → a plan node with the existingfunnel planverb, exactly like an idea. Design+guide:.nornir/funnel-intake.md. inject-assert tests for derivation, round-trip, evolution, and the viz tab.
MASTER QUEUE — 2026-06-13 wave 2 (ONE place for ALL dev, incl. non-nornir repos)
Rickard: keep every thread here even if it lives in another repo. Status as of this wave.
Done (this wave)
- nornir 0.4.14→0.4.23 shipped (crates.io+codeberg); both books regenerated w/ cover + helicopter SVG; docs book image-resolver fix (0.4.22)
- HENRIK-BUGS B1-B4 (0.4.21): mcp in default features, client-mode mcp needs no local config, drop hardcoded workspace_holger discovery, --help before config
- Telemetry.SubmitMcpCalls (0.4.20) + SubmitBakeoff/SubmitTestResults RPCs (0.4.23) — fill live server data
- MANNEQUIN-REVIVAL: GUI → facett; NEW robot-testable
facett-syschart::SystemChart(facett 5dd326f, holger bccedf4) - korp demo POC end-to-end (korp d442368): simulate→skade/Iceberg→knut bifröst→FalkorDB→korp 🎬 viz + container test (live FalkorDB verified)
- tunnr test-KVM boot: UEFI/OVMF → appliance PID1 BOOT-OK + smoke test (tunnr 1a2a9c9)
- Njord: admin testmail button (local fd924bc); Caddy public-proxy on oden ready (80-217-64-156.sslip.io→:32300) — awaits router 443→192.168.1.144; Funnel deliberately NOT used (Rickard veto)
Done (2026-06-13 late wave)
- nornir-orch: bakeoff/test 16-vs-13 schema-evolution fix + EPIC FUNNEL-INTAKE (FI1/2/3) — ALL DONE (0.4.25–0.4.27)
- korp+knut(+facett): geo-locate + bidragstagare + facett MAP + DELTA-INGEST (knut e5d829e, korp 91a9bd4) — DONE 2026-06-13
- holger: holger test matrix + mannequin row + facett-syschart::SystemChart (holger bccedf4) — DONE 2026-06-13
Queue (fires as each repo frees — no two agents per repo)
- ROBOTUI: new
nornir-robotuicrate (nornir-orch) — egui_kittest (drive: click/scroll/type via AccessKit) + facettstate_json(observe, theObservabletrait) + nornir-testmatrix rows → auto-click, matrix fills. Reusable korp/facett-demo/mannequin/viz.RobotSession(click_label/click_button/type_into/key/scroll/drag/step/snapshot) +Scenario/Step/Action/Assert→run_scenarioreturnsVec<TestResultRow>(suite=scenario, aspect=robot-ui).snapshotfeature gates wgpu/PNG so the core is light. Reference testtests/robot_viz.rsdrives the REAL viz (real AccessKit tab clicks →state_json["tab"]) + fills a 6-cell green matrix via JsonFileSink. Published nornir-robotui 0.1.0. - KORP-MATRIX ✅ (korp 91a9bd4): bake korp POC into the test matrix + FULL robot-UI test (korp, after geo + ROBOTUI) — drive Demo/Map/Graph tabs, assert renders
- FACETT-ICEBERG: facett component to TIME-TRAVEL Iceberg tables via Arrow frame (facett) → wire into nornir viz Warehouse + korp
- DELTA-INGEST ✅ (knut e5d829e, skade 5406d19): fully investigate + implement Delta streaming Iceberg→FalkorDB DIRECT Rust injection (knut/bifröst/skade) — pure Rust, no Spark, CDC/incremental
- facett-demo robot-UI row (D5b2) + nornir-viz uses nornir-robotui
- korp workspace on server: fill/refresh (fetch+republish so knowledge/data lands like the others)
- Njord: IP-allowlist the friend on Caddy once Rickard gives the corporate egress IP (optional hardening)
EPIC NJORD-MAIL — isolated mail-in-out (decisions locked 2026-06-13, branch feat/njord-mail-in-out → PR for owner)
- Architecture: isolated mail-server process (tonic gRPC: Mail.Send/FetchInbox/StreamInbound) + lib (
crates/email: lettre SEND + async-imap RECEIVE poll of njord@mikronova.io + InboundHandler) + Njordcrates/webMailClient (gRPC when MAIL_SERVER set / direct-lib otherwise = nornir embedded-XOR-server). Mail crash NEVER downs Njord. - Security (very-secure app, parses untrusted mail): dedicated unpriv user + hardened systemd sandbox (NoNewPrivileges/ProtectSystem/seccomp/no-caps), gRPC bearer auth, TLS-required IMAP/SMTP, size/recipient/rate caps, NEVER render HTML, no shell-outs, secrets via env.
- STORAGE decided: skade→Iceberg lives INSIDE the isolated mail-server (quarantined — Njord-web stays postgres+aws-sdk-s3, NO iceberg dep). Object store = the SHARED MinIO Njord already runs (skade rust-s3 → MinIO endpoint; separate bucket/prefix). Bodies + attachments = blobs ref'd from Iceberg → no size limit, time-travel, exportable (scan+copy). gRPC carries refs only.
- skade enhancement: pluggable
ObjectStoretrait (default rust-s3/MinIO + embedded rustfs + aws-sdk-s3 adapter so the mail-server can use ONE S3 SDK matching Njord). [skade repo, runs first] - CRYPTO: in-flight PGP via sequoia-openpgp = VOLUNTARY (per-send/recipient toggle); at-rest = age/aes-gcm with a ONE-TIME generated key persisted in a NEW DB-backed
settingstable (Njord has NO config module today — env-only via std::env::var; create DB-backed settings in this branch; env stays for boot infra like DATABASE_URL/S3_*). Decrypt on RBAC-gated read. - mikronova.io setup documented in Njord/.nornir: MX/A/SPF/DKIM/DMARC records + relay-for-SEND (outbound :25 blocked by ISP) + router forwards (:25/:993 if self-host receive) — what Rickard does.
- CONTRADICTION resolved: Njord is postgres+S3/MinIO, no Iceberg — but process isolation quarantines skade/iceberg in the mail-server, so Njord-web is untouched.
Session 2026-06-13 — late: HONEST status reckoning + Security live-fix (NOT pushed yet)
Rickard caught me over-claiming: "you always claim test matrix is done and working and it never catches the errors … press Security on nornir … WHY??" This section is the straight answer + the fix. Honest labels from here on:
live-verified(proven against the real oden server) vswiring-tested(passes against seeded/ephemeral fixtures only).
WHY pressing 🛡 Security on nornir fails (root cause, reproduced live)
nornir/Cargo.tomlhas relative path-deps:skade = { path = "../skade/skade" },skade-katalog = { path = "../skade" }.- The server clones only
nornirinto…/<ws>/git/nornir/— noskadesibling beside it. Socargo metadataANDcargo generate-lockfileBOTH die: "failed to load manifest for workspace member". - The existing
materialize_path_dep_siblingsonly symlinks siblings already on disk — it never clones a missing one. And the server'sMimir.SecurityScanhandler calledcomponents()directly, skipping even that. → guaranteed hard error on any repo with sibling path-deps.
WHY the matrix never caught it (the gap I must own)
- Every robot-UI / workspace×function matrix runs against seeded, ephemeral fixtures and asserts
state_json(UI WIRING). None drives the live oden server against its real path-dep'd git checkouts. - So "matrix green" = the button wires up + the RPC is invoked. It does NOT mean the live scan succeeds. Calling that "done & working" was wrong — it was wiring-tested, never live-verified.
FIX A — net-scan feature: clone missing path-dep siblings (✅ coded, ⏳ uncommitted)
- New
security::clone_missing_path_dep_siblings(repo, scan_root, url_for)+ combinedprepare_path_deps(clone → then symlink). URL = workspace descriptorgit, else<NORNIR_DEP_CLONE_BASE|https://codeberg.org/nordisk>/<repo>. - Gated behind Cargo feature
net-scan(indefault+cli) so an airgapped scan host drops it:cargo build --no-default-features --features "server viz mcp"→ degrades to committed-Cargo.lock with a clear message, never a silent reach-out. (Rickard: "allow tests to clone gits … have it as a feature so we can remove things that could download stuff.") - Wired into BOTH
warm()and the serversecurity_scanhandler (clone+link beforecomponents()). - coded +
cargo build --features server,net-scangreen · [ ] unit test (local-remote clone, inject-assert) · [ ] live-verified on oden (press Security on nornir → real SBOM, not error)
FIX A also hits Docs.Render (same cargo-metadata death — Henrik, 2026-06-13, MCP against oden)
- Live
docs_render holgerfailsInternal: "cargo_metadata::MetadataCommand::exec"— the SAME missing-sibling root cause as FIX A above. The server clones only…/<ws>/git/<repo>/with no path-dep siblings, socargo metadatadies — identical to the Security/generate-lockfilefailure. (Verified NOT holger-side: localcargo metadataon~/holgersucceeds, holger has no[patch.crates-io].) - DR1
prepare_path_deps(FIX A clone→symlink) runs before the docs-gen cargo-metadata step — newintrospect::depgraph::extract_with_path_deps(repo, scan_root)(clone+symlink siblings, thenextract) is whatdocs::sections::render_depgraphnow calls (scan_root =ctx.workspace_root, thegit/dir), and theDocs.Render/Docs.Bookserver handlers also runprepare_docs_path_deps(root, workspace_root)beforerender_all. So docs-gen resolves the same siblings the SBOM scan does. - DR2 fail gracefully when a sibling is still unresolved:
security::unresolved_path_dep_siblingsdrives a clear error —docs render for <repo>: unresolved path-dep sibling(s) <dep> (<path>) — …(check the sibling out under thegit/dir, or build withnet-scan) — never the opaquecargo_metadata::MetadataCommand::exec/Internal. inject-assert tests insrc/introspect/depgraph.rs: one repo resolves the sibling (graph names thenornir→skadeedge), one (net-scan off) surfaces the named DR2 error. - DR3 live-verified on oden (redeployed nornir-server, deployed-binary SHA == freshly-built, contains the DR2 marker):
NORNIR_WORKSPACE=holger docs render holger→ok: 1 doc(s) rendered;docs book holger→ real 984 KBbook.pdf(9 sources) historized; and the path-dep'ddocs render nornir/docs book nornir(skade/skade-katalog siblings) → real render (README depgraph marker =, 26 KB SVG, NOT the no-data fallback) + 10.9 MB book.pdf historized.docs history holger/nornirnow show non-zerodoc_exportsrows.
FIX B — matrix covers ALL workspaces, live + synthetic (⏳ todo)
- (Rickard: "make the 7 workspaces a part of matrix test, period! use synthetic data as well if matrix is using that too.")
- Drive the REAL oden server: for every workspace × repo, call
Mimir.SecurityScanand assert a real, non-error result (this is the test that would have caught the bug). - Synthetic-data fallback when the live server is unreachable (so CI still runs), clearly labelled synthetic vs live.
EPIC #44 — callgraph / dep-graph performance (njord ~10s → sub-second) ✅
Full write-up: .nornir/callgraph-perf.md. Branch feat/callgraph-perf.
#44 viz callgraph/dep-graph njord ~10s
├─ PROFILE first (MEASURE, don't guess)
│ ├─ njord symbol_facts=5928 files/15GB, call_edges=5764/14GB
│ ├─ bottleneck = SCAN: read_all materializes whole table (35.4s+25.3s)
│ └─ layout NOT it: ClusterZ 97ms / MDS 265ms @160 nodes
├─ FIX dominant cost
│ ├─ scan_limited: stream Arrow + STOP at limit (0.66s+0.60s, ~50x)
│ ├─ keep cheap default ClusterZ; Spectral/MDS opt-in (off-thread unjustified by data)
│ └─ memoize draw_drilldown transitive_closure (per-frame → cache hit)
├─ adjacent: Mímir-njord build fails
│ ├─ build_graph_from_warehouse: rebuild from dep_graph_edges
│ ├─ WorkspaceCtx.name + graph() fallback on descriptor-missing
│ └─ ensure_repo/svg use has_component; seed single-repo members
└─ inject-assert tests + matrix rows
├─ callgraph layout-budget + closure-memo unit tests
├─ mimir edges-only + single-repo-seed tests
└─ tests/callgraph_perf_matrix.rs perf-aspect rows (synthetic + live njord)
- PROFILE — measured njord scan = 5.26s+4.44s (≈10s) over RPC; 35.4s+25.3s local full-read; layout sub-300ms. (
examples/profile_callgraph.rs) - scan_limited — streaming limit scan; njord per-table 35.4s→0.66s (~50×).
scan_previewuses it. - default layout stays cheap — ClusterZ default, Spectral/MDS opt-in; off-thread NOT added (measurement says unneeded).
- closure memo —
DepGraphView::closure_memo; drill-down no longer recomputes per frame. - Mímir-njord —
build_graph_from_warehousefromdep_graph_edges+ member seed;nornir mimir deps-of/svg/overview njordresolves (verified vs real njord warehouse). - tests — callgraph layout-budget, closure-memo, mimir edges-only/single-repo;
tests/callgraph_perf_matrix.rs(synthetic always-on + live#[ignore]). - deploy — rebuild+restart oden's
nornir-serveron this branch; thencallgraph_perf_matrix_live_njord --ignoredgoes green (scan <2.5s/table, deps-of njord resolves). Needs sudo systemd; not done here.
Side threads resolved/observed this session
- skade ObjectStore trait — pluggable
ObjectStore(Memory/LocalFs(rustfs)/RustS3(s3)/AwsS3(aws-s3)) + iceberg bridge; skade 0.4.10 published, codeberge6cd5a0. Unblocks the mail-server. (live-verified: build all features green; live-MinIO out-of-band.) - [~] Njord mail-in-out — committed LOCAL only (
d0a78f2, branchmain, NOT pushed). Done: isolatedemail+email-server(tonic gRPC), untrusted-MIME boundary, hardened systemd, admin inbox,.nornir/mail.md+SVGs, inject-assert tests. NOT yet done vs spec: skade→Iceberg storage of bodies/attachments, voluntary PGP, at-rest age key in DB-backed settings table. → needs a follow-up before the PR. Must move to branchfeat/njord-mail-in-out(NOT main) per owner-review rule. - [~] facett family publish — background retry loop finishing the last ~6 crates (syschart/arrow/card/osm/icebergview/umbrella) after crates.io 429.
- LAW-0 warehouse repoint — all
workspace_*/release/nornir.toml[storage].local_pathnow under/home/rickard/.nornir/workspace_*(holger/matrix use per-workspace home default). Verified.
Session 2026-06-14 — plan sync + funnel matrix + lock-spam fix
Done this session
- Plan sync — ticked R6/D3b/J1/J4/DELTA-INGEST/KORP-MATRIX/MQ2; marked Running wave as done; added this section.
- nornir-orch 0.4.27 —
viz catalog lock spamfix: server reachability takes priority over discovered config;--warehousedefault now~/.nornir/warehouse(LAW 0). Committed + pushed. - FUNNEL-DECOMPOSE test matrix —
Aspect::FunnelDecomposeadded to nornir-testmatrix (ALL + DEFAULT);funnel_tab.rs🔬 button +state_json["decompose"]+decompose_results/decompose_runningfields;tests/viz_funnel_decompose.rs(5 inject-assert + 2 heavy). Committed + pushed. - Big-plan multi-agent wave merged (N-flip, C6g MCP test_run/bench_run, D2h inline-edit, J4/J1 SVGs, L5 Test pane shining) — all branches merged → nornir-orch/main.
Open (confirmed not done, no agent running)
- F2 aegir uring + 3D fog + SBOM-from-warehouse — incremental republish extra features.
- J2-old znippy gatling sync SVG — circuit-board flow diagram for znippy GATLING pipeline.
- UIP-prereq1 facett book.pdf —
facett/.nornir/components.mdsource exists; needs docs-export render pass. - MQ3 invent new facett components for mannequin system-map if node/edge scene insufficient.
- MQ4 mannequin in holger workspace — add to workspace members so
cargo test --workspacecovers it. ✅ DONE (verified 2026-06-30 audit). - D5b2 facett-demo robot-UI row DONE (verified 2026-06-16 — see D5b2 above) —
facett-demo/tests/robot_demo.rsdrives all 21 components in 45 steps (in-crate robotui port pending GAP-ROBOTUI-034). (nornir-viz-uses-nornir-robotui tracked separately under GAP-ROBOTUI-034.) - FACETT-ICEBERG facett time-travel Iceberg table component (Arrow-backed) → wire into nornir viz Warehouse + korp.
- korp workspace on server: fill/refresh.
- Njord:
feat/njord-mail-in-outbranch + PR (skade→Iceberg storage + PGP + at-rest age key in DB-backed settings).
New epic — nornir-install: common service installer + LAKEhouse packager
⭐ RECONCILED 2026-06-15: INSTALL == EPIC AIRGAP — ONE install system, not two. This epic's
nornir-installcrate idea SHIPPED as the airgap crate — since renamedskidbladnirand extracted to the sibling repo../Skidbladnir(path depskidbladnir 0.1.3; the earliernornir-airgap 0.1.0name is retired and nonornir-airgapis published). There is no separatenornir-installcrate;skidbladnirIS it. INSTALL-2..6 below are the adoption follow-ups (CLI verbs + product specs + tests) layered onskidbladnir, not a second stack.
Rickard 2026-06-14: holger must be as installable as nornir. Shared installer crate (in nornir) — systemd unit gen, init-system detect, atomic service upgrade, LAKEhouse component registration, health-check wiring. Targets: holger, nornir-server, mail-server, tunnr, future lake components. Scope expanded 2026-06-14: also covers tunnr appliance packaging, Redfish BMC/bare-metal provisioning, Terraform resource definitions, and delta-package delivery (binary delta = znippy zero-copy patch; bsdiff/zstd diff + znippy verify). Better than Ansible; pure Rust. AIRGAP IS PRIMARY — every component must work with zero network on the target; the full lake is packed into a signed znippy bundle on a staging host, transferred by USB/relay, verified on receipt, then installed with no further network calls.
- INSTALL-1 design ✅ DONE —
.nornir/install-system.mdwritten: airgap model (two-phase: pack-lake on staging host → from-archive on target), constraints table (no DNS/HTTP/NTP/pkgmgr/registry), secure transfer options (USB/relay/age-encrypted), znippy selective read (O(1) seek, no full decompress), idempotency guards, per-machine PKI, dependency-ordered start with health-checks, Windows MSI via WiX+Wine (frommsi-agentrepo), msi-agent Perl scripts → Rust (msi.rs), Redfish, Terraform stubs, full CLI verb list. - INSTALL-2 installer crate ✅ subsumed by
nornir-airgap 0.1.0(thenornir-installcrate idea = EPIC AIRGAP, shipped to crates.io) —ComponentSpec(name/binary/env/ports/after/restart-policy) +Installer/SystemdInstaller(install/remove/status; upgrade = same atomic install path) + systemd unit render + ports. — done:crates/nornir-airgap/src/product.rsComponentSpec+Installer+SystemdInstaller,crates/nornir-airgap/src/systemd.rsSystemdServiceSpec+systemd_unit/env_file/install_systemd.DeltaPackager/znippy-delta is thearchive::Archiveseam (archive::znippy_todo,product::delta_supported);RedfishClient/Terraform =unfold.rsseams; Windows MSImsi.rsWixBuilder+guid_v5. (No separatenornir-installcrate —nornir-airgapis it.) - INSTALL-3
nornir airgap componentCLI verb ✅ —nornir airgap component add <name>(--dry-runrenders the unit),upgrade [--delta],remove <name>,status [name], dispatching tonornir_airgap::product::{registry,lookup,Installer,SystemdInstaller}. The bespokenornir install add/update/removefor nornir's own server is the canonical path (kept working) and renders the SAMEnornir-server.serviceunit. — done:src/bin/nornir.rsComponentOp::{Add,Upgrade,Remove,Status}+run_install_add/run_install_upgrade/run_install_remove/run_install_status;InstallOp::{Add,Update,Remove}for the server. (pack-deltadeferred to the znippy delta engine — see INSTALL-5.) [Originally shipped asnornir install {add,upgrade,remove,status}; the generic component verbs moved underairgap componentand the unit name converged onnornir-server.service.] - INSTALL-4 holger wired ✅ —
ComponentSpec::holger()(in the airgap crate's registry; holger is NOT a nornir code dep — described, not linked) +nornir install add holgerworks end-to-end (resolves spec → renders unit → unpacks → would-install;--dry-runproves the exact unit). — done:crates/nornir-airgap/src/product.rsComponentSpec::holger+registry/lookup; testholger_add_renders_exact_unit_text. - [~] INSTALL-5 tunnr appliance —
ComponentSpec::tunnr()+nornir install add tunnrwork (spec in registry, unit renders, appliance args + 8443 +/healthz). Delta-package from the UEFI appliance image via znippy is an honest stub (product::delta_supported()==falseuntil thearchiveznippy delta seam is wired;upgrade --deltafalls back to full re-install). — done (spec+CLI):crates/nornir-airgap/src/product.rsComponentSpec::tunnr; testtunnr_spec_renders_appliance_unit_and_delta_is_stubbed. - INSTALL-6 inject-assert tests ✅ — unit-file render round-trip (exact byte-for-byte unit text), upgrade idempotence (re-render byte-stable + version-independent via
current), registry coverage (enumerate+lookup), tunnr appliance render + delta-stub. Delta patch round-trip deferred with the znippy engine. — done:crates/nornir-airgap/src/product.rstestsregistry_enumerates_and_looks_up_components,holger_add_renders_exact_unit_text,tunnr_spec_renders_appliance_unit_and_delta_is_stubbed,upgrade_is_idempotent_at_spec_level. Adoption guide:.nornir/airgap-install.md§6 +crates/nornir-airgap/.nornir/airgap-installer.md. - INSTALL-7 products self-install via the engine ✅ (2026-06-21) — the other direction from INSTALL-4 (which is nornir installing holger top-down). A product drives the shared
nornir-airgap::systemdengine itself, no bundle/registry. First: holger —holger-server install {add | update | remove | status | render}(holger/server/cli/src/main.rs;render= no-root dry-run). holger owns its ownSystemdServiceSpec(verbstart --config, RON config) — the engine only applies it. Needed the newremove_systemd/status_systemdverbs added tocrates/nornir-airgap/src/systemd.rsso the spec layer carries the full add·update·delete·status lifecycle (previously delete/status existed only inproduct.rs, keyed byComponentSpec). Tests:remove_and_status_target_the_unit_name(nornir-airgap) +holger_unit_renders_with_start_verb(holger).
New epic — FACETT-CONTRACT: Component Contract & Agent Enforcement (FC-1..FC-9)
Rickard 2026-06-14: first phase = codebase analysis. Normative spec pasted in full — FC-1 (no hidden state) through FC-9 (pure view). Authority: where spec conflicts with existing facett code, code is wrong. Agent mandate: audit every component, produce compliance table, apply minimal refactor to full compliance. No sweeping rewrites unless no smaller path exists.
- FC-AUDIT Phase 1 — analysis: read entire facett codebase; produce compliance table
| component | FC-1..FC-9 | status |; per-violation report (id/severity/file:line/evidence/fix). No code changes yet. - FC-HOIST Phase 2 — hoist hidden state: for every
ui.memory_mut/ctx.data_mutholding observable state → hoist toModel;#[derive(Serialize,Deserialize,Clone,PartialEq)]on every State. - FC-MSG Phase 2 — writable input surface: every component gets a
Msgenum (Debug+Clone); every state-mutation path reachable viaMsg; no raw-pointer-only input paths. - FC-ACCESSKIT Phase 2 — AccessKit nodes: every interactive+value element populates an AccessKit node (role+label+value+actions); custom-painted widgets not invisible to accessibility tree.
- FC-ID Phase 2 — stable identity: explicit
egui::Idper addressable element; no ids from loop-index/timestamp/RNG; reorder-resilient. - FC-HEADLESS Phase 2 — headless render: every component renderable via offscreen/software wgpu path (egui_kittest); no real-window-only dependencies.
- FC-DETERM Phase 2 — determinism: inject clock + RNG; animations from injected clock; two renders from same state+input → identical AccessKit + frames.
- FC-QUIESCE Phase 2 — explicit quiescence: Effects as data; in-flight status in observable state; no thread::sleep; idle signal readable without timing hacks.
- FC-TESTS Phase 3 — test layers: property layer (proptest Msg sequences, invariants, round-trip); interaction layer (egui_kittest query by label/role, assert AccessKit+State); perception/snapshot layer (software-rasterizer PNG fixture).
- FC-DONE Definition of done: FC-1..FC-9 satisfied + serialize round-trip + egui_kittest label/role retrieval + headless render + determinism + idle signal + all test layers + no §4 anti-patterns.
- FC-SPEC Store spec in facett:
.nornir/component-contract.md(full FC-1..FC-9 normative text) + compliance matrix auto-generated each release.
New epic — FACETT-WASM: facett components in Apache Superset (+ browser)
Rickard 2026-06-14: embed facett egui components as Superset viz plugins. Reference:
github.com/Ignalina/superset-egui-demo— currently just a ChartPlugin skeleton with no actual WASM/Rust; proves plugin registration works.eframehas awebfeature that renders to a WebGL/WebGPU canvas.Two delivery targets: (1) Apache Superset viz plugin (
@superset-ui/coreChartPlugin — React/TypeScript shell, egui renders into a<canvas>viauseEffect); (2) standalone browser page (facett-demo served as WASM, no Superset). Superset passesprops.data(Arrow/JSON) → JS calls#[wasm_bindgen]Rust function → facett renders.Pragmatic path:
<iframe>+postMessagefor first-pass Superset integration (no npm bundling of WASM, avoids wasm-in-npm complexity); direct-canvas (wasm-pack --target web+useEffectcanvas) is the clean target once the WASM API is stable.
- WASM-1 design —
.nornir/facett-wasm.md: WASM build chain (wasm-pack/trunk,eframe --features web),#[wasm_bindgen]entry point API (init canvas, push Arrow data), Superset plugin skeleton (TSXuseEffect+canvas), iframe vs direct-canvas trade-offs, data boundary (Arrow IPC overpostMessageorwasm_bindgen-exported fn). - WASM-2
facett-wasmcrate —eframeweb build;#[wasm_bindgen] pub fn mount(canvas_id, data_json)entry; renders facett-graph (node/edge Scene) and facett-grid into the canvas. Zero server-side deps; pure client WASM. - WASM-3 Superset plugin —
superset-plugin-facettnpm package: ChartPlugin TSX withuseEffect+canvas, dynamic WASM load (import()), Arrowprops.data→ JSON →mount()call. Registered via Superset custom viz plugin mechanism. - WASM-4 iframe path —
facett-demobuilt as a static WASM page (trunk build); Superset plugin renders<iframe src="...">+postMessagefor data; simpler but slightly less integrated. - WASM-5 inject-assert tests — WASM builds headlessly (
wasm-pack test --headless --chrome); assertmount()does not panic; canvas pixel count non-zero after data push.
facett MAP components (from old osm-katana → archived) — 2D + 3D, each CPU+GPU (2026-06-16)
⛺ RENDER-ENGINE SPEC (ratified 2026-06-21): the canonical map+graph render design is
facett/.nornir/render-engine.md. Key decisions referenced here in pieces: 2 hardware categories ONLY —CPU(SIMD assumed; no-SIMD = freevello_cpuscalar fallback, not a tier; Apple/x86 always have SIMD) andCPU+GPU; build axis = native+wasm (wasm = 2 SIMD bundles + WebGL2-no-compute path). velo = Vello family (vello_cpuCPU row · facett L0 instanced-wgpu + vello L1 overlay GPU row). SIMD 3-way split: Ragnar stree = search/index · DIY kernels = projection/cull ·vello_cpu= raster (never DIY a rasterizer). Black box:(OSM + border polygons + Theme) → navigable viewer. One kernel, two skins:facett-core(L0 kernel) +facett-graphview(graph engine); map/3D/syschart/pipeline stay as thin kinds;facett-geomapfolds intofacett-map(settles MAP-GEOMAP below);facett-osm= data. galileo = design references only (Renderer-seam/Layer-trinity/Symbol/TileSchema+Lod/CRS patterns), never a dep. TEST LAW (start→pixel): instrument every render-matrix column with afunctional_statusemit checkpoint (testmatrix-gated, stripped in release); the final picture is emitted as a PNG blob so an agent inspects the render without asking; the 3D map clipping bug =clip.ink_outside_rectoracle, binary-searched + fail-on-bug/pass-on-fix matrix row. See MAP-RENDER-ENGINE + MAP-CLIP-TEST below.
-
[~] MAP-RENDER-ENGINE adopt the ratified render-engine spec (
render-engine.md) —facett-coreL0 SDF kernel (shared map+graph) +vello_cpu/velloL1 seam; runtime 2-category auto-select; DIY SIMD geo-kernels (project/cull); Ragnar stree as the viewport index; galileo-pattern Renderer/Layer/Symbol/Lod seams. Replaces the looser "each runtime-selects CPU vs GPU" wording. ✅ Tier-2 render effects DONE (verified 2026-06-30 audit): HDR + bloom, FXAA, and sky landed on the map3d offscreen target (perworkspace_nordisk/.nornir/facett-game-quality-rendering.md); remaining: the full L0 SDF kernel / vello L1 seam / Ragnar-stree viewport index. -
MAP-CLIP-TEST start→pixel emitter chain + final-picture blob (fixes the live 3D clip bug) — "clipping" is FIVE sub-steps (view-space near-clip → frustum cull → back-face cull → screen-extent → 2D
clip_poly_to_rectscissor); emitnear_clip/project/cull/shade/depth_sort/scissor/compositecheckpoints (EMIT-DOCTRINE: only can't-observe pixel stages get in-fn emitters),scissor.ink_outside_rect==0oracle, add view-space near-plane Sutherland–Hodgman clip BEFORE project (replaces brittle spoke-reject + screen-extent guard), emit the rendered PNG as a warehouse blob, drive live vianornir-robotui, RED→GREEN matrix row at the failing street-level pose (don't regress7c535a8).- ✅ M1 DONE (2026-06-21, agent a25ed7f, uncommitted) —
clip_tri_near(view-space S-H near clip) + 8-stepshade_faces+ emitter chain (testmatrix-gated) + PNG blob (/tmp/facett-map3d/map3d_clip_final_picture.png, agent Read it: confined city, no smear) + 44 tests green (sensitivity-proven) + 7 part-benchers registered. Brittle spoke-reject/screen-extent heuristics removed (kept inshade_faces_legacyas the bug-oracle diagnostic). - ✅ LIVE-WASM VERIFIED (Rickard, 2026-06-22) — facett-demo OSM 3D renders correctly in the live wasm build, no escape/smear. The days→weeks clip fight is closed; the HONEST GAP below (real escape only provable in GPU/wasm) is resolved by live confirmation. Test harness retained: emitter chain + PNG blob + 44 tests + 7 part-benchers.
- ⚠️ HONEST GAP (now closed by live verify above): agent found the CPU rect-scissor ALREADY drove ink_outside_rect=0 (legacy + fixed) across the pose sweep — near-clip's value = geometric correctness, not stopping escape. The escape Rickard sees is in the LIVE WASM / GPU path (
draw_map_gpu+ wgsl), NOT verified → see MAP-CLIP-WASM. - ✅ BENCH UNBLOCKED (2026-06-22): the arrow57/58 conflict is gone (arrow58 unified); the bench's dead out-of-repo
[patch] iceberg = path "../../iceberg-arrow58"(absent in the server clone tree) was removed (facettfe1f395), and the server's CORRUPT holger clone (0-byteserver/lib/Cargo.toml, no.git) was re-cloned via SSH deploy key. Benchers now run + LAND in the warehouse.
- ✅ M1 DONE (2026-06-21, agent a25ed7f, uncommitted) —
-
MAP-CLIP-WASM close the REAL bug — GPU/wasm clip path — verify whether
draw_map_gpu+ the wgsl pipeline apply the near-clip + rect-scissor (CPU path already does). Reproduce the live-wasm escape Rickard sees, add the clip to the GPU path (near-plane in the vertex/clip stage + wgpu scissor/viewport to the widget rect), emitscissor.ink_outside_rect==0from the GPU path too, PNG blob + RED→GREEN matrix row in BOTH native-wgpu and wasm.- ✅ DONE (2026-06-21, agent a7bc765, uncommitted→committed) — diagnosis: GPU path set NO own scissor (relied on egui's courtesy viewport) + callback used panel-wide painter (clip_rect=panel not widget). FIX:
gpu_scissor_px/set_scissor_rect(= viewport ∩ clip_rect, the GPU twin ofclip_poly_to_rect) + callback viapainter_at(rect). wgsl ALREADY near-clips correctly (real homogeneousw) → no shader change. Headless wgpu test green (used_gpu=true, 0/183600 px leaked), PNG blob Read (city contained).--features wgpu,testmatrix: lib 36 + gpu_scissor 1 pass. - ✅ LIVE-WASM VERIFIED (Rickard, 2026-06-22) — CAVEAT CLOSED. The GPU/wasm scissor fix (
gpu_scissor_px/set_scissor_rect+painter_at(rect)) holds in the actual live wasm build: facett-demo OSM 3D is contained, no escape. The "re-check against the actual wasm build when convenient" below is now done. - ⚠️ HONEST CAVEAT (now closed by live verify above): in native headless, egui's stock renderer ALREADY double-confines the blit (its scissor-from-clip_rect + courtesy viewport) → end-to-end can't distinguish our-scissor on/off; the fail-on-bug/pass-on-fix oracle is the unit-tested rect math (scissor pinned to widget not panel). Our scissor is load-bearing when the viewport courtesy is bypassed (host overrides viewport / batched passes / non-stock backend). The exact live-wasm escape was NOT reproduced here — fix is correct defense; re-check against the actual wasm build when convenient.
- ℹ️ pre-existing flaky
gpu_depth::depth_test_off_breaks_occlusion...(negative control, env adapter-dependent) — reproduced on untouched tree, NOT from this work.
- ✅ DONE (2026-06-21, agent a7bc765, uncommitted→committed) — diagnosis: GPU path set NO own scissor (relied on egui's courtesy viewport) + callback used panel-wide painter (clip_rect=panel not widget). FIX:
-
MAP-PART-BENCH part-step benchers, not just "draw map" (Rickard LAW 2026-06-21) — every render part-step gets its own Bencher entry
facett.<skin>.<step>(map3d: near_clip/project/cull/shade/depth_sort/scissor/composite; graph: layout/cull/nodes/edges/picking/composite) with injected sized input;facett.map_renderstays as the rollup. Composable arch ⇒ attributable per-stage regressions.- ✅ DONE + LANDED IN WAREHOUSE (2026-06-22, run
2d608db1): 16 part-step benchers run server-side vianornir bench run facett→Ops.RunBenchgRPC → icebergbench_runs, read back viabench history-show. Numbers: map3d near_clip 4.18M tris/s · project 12.3M · cull 3.2M · shade 4.13M · depth_sort 187M · scissor 9.88M · composite 3.74M; core sdf_marker 279M inst/s. ⚠️facett.core.gpu_uploadstillskipped—nornir bench rundoesn't pass--features wgpu(the bench Cargo.toml now HAS the feature; follow-up: route features through the runner / NORNIR_TEST_FEATURES).
- ✅ DONE + LANDED IN WAREHOUSE (2026-06-22, run
-
MAP-PROCESS-SVG GRAND skade-style SVGs per multi-part process —
facett/.nornir/assets/{map3d-clip-pipeline (✅ shipped),map-render-2d,graph-render}.svg(dark PCB/cyan/DejaVu, left→right, emit+bench tap drawn on each step), embedded in design doc + guide. -
MAP-GUIDE design + idiot-guide per new component —
render-engine.md(✅) +render-engine-for-idiots.md(✅) done; extend with per-component design+guide as facett-core kernel / facett-graphview land (LAW: every feature ships .nornir design + guide). -
MAP-DEMO-REWRITE rewrite facett-demo onto the consolidated facetts — demo uses
facett-corekernel +facett-map(absorbing geomap) +facett-graphview; OSM 2D/3D + graph as real demo tabs (not standalone bins); wasm-deployed; robot-matrix drives every tab.- 🔄 Velo L1 tab DONE (2026-06-22, facett merge
agent/velo-l1917b4eb): newfacett-corel1-vellofeature (vello 0.9 overlay — gradient/antialiased-curve/glow over the L0 SDF base, degrades to L0 when no GPU, never required) + aVelo L1demo tab (facett-demo/src/velo_l1_tab.rs) showing L0-only vs L0+L1 side-by-side with the runtime 2-category (CPU/CPU+GPU) auto-select instate_json; render-proof test + PNG blobs (l0 vs l0+l1); 161 core + 2 demo tests green. ⏳vello_hybridweb lane still DEFERRED (version skew: publishedvello_hybrid 0.0.9tracksvello_common 0.0.x, a different train than the pinnedvello 0.9/wgpu 29; + needs a wasm WebGL2/WebGPU surface). So the wasm demo still has NO GPU-vello lane (L0vello_cpuonly). Seefacett/.nornir/velo-l1-lane.md§vello_hybrid.
- 🔄 Velo L1 tab DONE (2026-06-22, facett merge
-
MAP-MODERN-FIRST modern-first doctrine (LAW) — WebGPU / wgpu-compute / AVX2-NEON / Linebender stack (Vello+parley+fearless_simd) are the targets; legacy (WebGL2-only, no-SIMD, pre-2013) served ONLY by free fallbacks (
vello_cpuscalar,vello_hybrid) else deferred; small modern stack > big compat matrix. Those stuck on 1990s hardware can fuck off. Thread resolutions: text=parley(decided), web=WebGPU+vello_hybridfallback (decided), graphs-over-maps=add shared-camera+z-layer SEAM now (feature later). -
MAP-ZEROCOPY-GATLING zero-copy/zero-alloc render + GATLING multicore (LAW) — render part-steps pass slot indices into pre-allocated reused buffers (znippy Law 1&2), geometry uploaded once + pan/zoom = uniform poke, Arrow/GeoParquet mmap-borrowed not deserialized; every parallel produce→consume stage (project Mpts / tessellate / tile-LOD build / GeoParquet decode / huge-graph layout) uses znippy-zoomies GATLING (1 reader→split→slot channel→N no-barrier workers→1 writer), verify cores-busy 1-32-32-32-1 as DATA + ship the 1→N→1 SVG. Eyes open for new gatling use-cases.
-
MAP-TEST-SWEEP MILLIONS of sub-part tests + benches (LAW) — each part-step hit by a generated grid (size sweep 1k→10M · camera-pose sweep · degenerate geometry · every OS feel), not one case; green matrix ⟺ every sub-part proven across the sweep; benches = same grid timed.
- ✅ map3d DONE (2026-06-22, facett
28ea4e9,facett-map3d/tests/partstep_sweep.rs): the FULL shipping chain (near_clip→project→cull→shade→depth_sort→scissor→composite) swept across 360° az × 25 zoom × 4 elev × 2 fixtures = 7200 poses (default arm) + the full 360×100×4 = 28800 as the#[ignore]heavy arm; oraclescissor.clipped_escapes==0asserted at EVERY pose on a giant near-plane-straddling city (100% render, 0 escapes, worst ink_outside_rect=0px); degenerate/adversarial arm (zero-area, behind-eye, grazing); per-stepfunctional_statusemitters + CPU-rasterised worst-pose PNG blob (non-blank, 0px outside rect). TODO: extend the same grid discipline to the 2D map + graph skins.
- ✅ map3d DONE (2026-06-22, facett
-
MAP-LOOKFEEL-OS look & feel per-OS (facett's whole point) (LAW) — FEEL (pan/zoom/orbit gestures, momentum+damping, dolly-to-cursor, keymap) + LOOK (nav menus, scrollbars, focus rings, spacing) differ macOS/Windows/device via
InputFeel/with_os_feel+facett_core::look::Themepresets; every component routes nav+chrome through them, never hardcoded; CLI/headless parity — OS feel selectable + instate_json, robot asserts macOS-feel ≠ Windows-feel as DATA.
facett CONSOLIDATION CHECKLIST — tick one crate at a time (2026-06-21)
Rule: render engine consolidates; domain "kinds" stay as thin skins (render-engine.md). Each box = one crate's migration onto the shared kernel, done & matrix-green independently.
-
CONS-CORE
facett-core= the shared render kernel ✅ (Phases A+B+C, facett commits fe21845/3abb1e7/e0fef64/c115ad5) — L0 SDF instanced-wgpu + CPU painter (multicore gatling raster) + sharedCamera+ LayerStack + Theme + Navigable + core::edges + labels + gpu/{types,buffer,install,sdf_pipeline,offscreen}; all 4 skins draw through it. (vello L1 overlay seam = still to add; the kernel itself is the keystone, done.)- ✅ SCOPED (2026-06-21): full impl plan =
facett/.nornir/cons-core-plan.md. Findings: core HAS look/Theme/nav/edges/labels/testmatrix but LACKS gpu/Camera/Renderer-trait/SDF/layer-stack; 3 wgpu kernels (map/map3d/graph3d) shareegui_wgpu::CallbackTraitplumbing → extract; 3 Camera types → shared Camera missing. VERSION CLASH CLEARED — egui-wgpu 0.34.3 + wgpu 29.0.3 + vello 0.9 + vello_cpu 0.0.6 already co-locked (GPU lane green); onlyparleyis a deferred risk (gate behind L1, verify skrifa). Migration: Phase A (move-and-reexport Camera/InputFeel + scissor, add gpu types/install/chunking — zero behavior change) → Phase B (new L0 SDF kernel) → Phase C (adopt in graphview→map→map3d→graph3d, behind features, fallback intact). - ✅ Phase A DONE (2026-06-21, agent a2443c3, committed) —
facett-core/src/render/{mod,camera,layer,cpu/scissor,gpu/{types,buffer,cull.wgsl,draw.wgsl}}; sharedCamera(2D+3D superset),InputFeel+clip_poly_to_rectmoved→core & re-exported from map3d (API stable),LayerStackseam, gpu install-lifecycle + chunking (fix #12) extracted verbatim, newwgpufeature (no version bumps). Zero behavior change: 123 core + map3d/map/graph3d/graphview fences green (pre-existing flakygpu_depthexcepted). NEXT: Phase B (L0 SDF kernel) → Phase C (adopt graphview→map→map3d→graph3d). - ✅ Phase B DONE (2026-06-21, agent a29b9a54, committed) — L0 SDF kernel:
render/{prim,cpu/sdf,gpu/sdf.wgsl,gpu/line.wgsl,gpu/sdf_pipeline,backend}(instanced circle/ring/square/diamond/triangle + capsule lines; CPU+GPU SAME math; Canvas/Renderer/Frame traits; CpuCanvas→vello_cpu). GPU↔CPU parity 0.003% (live wgpu); 140 core/150 wgpu tests green; fences green;vello_cpu 0.0.6no clash, no parley. Part-benchersfacett.core.*— gpu_upload 3.63M/s @1M clears the galileo ≥3.6M bar. BONUS: arrow 56→58 bumped across facett (7 crates + bench[patch] iceberg-arrow58) → bench crate now compiles+runs. NEXT: Phase C (adopt the kernel in graphview→map→map3d→graph3d). - ✅ Phase B+ MULTICORE (2026-06-21, committed: facett e0fef64, znippy 2137851 branch feat/gatling-forkjoin) — CPU SDF raster was single-threaded (1/32); now 32/32 cores-busy, ~6.3× via a NEW znippy primitive
znippy-zoomies/src/gatling_forkjoin.rs(no-barrier fork-join: sharedfetch_addself-dispatch, zero-copy row-index hand-off, zero-alloc disjoint row slices;gatling_scanlines). rayon removed (FORBIDDEN). Parallel output byte-identical to sequential (400 overlapping shapes). 98 zoomies tests + 141/151 facett-core green. - ⚠️ DEP FLAG / follow-up GATLING-EXTRACT: facett-core now path-deps the WHOLE
znippy-zoomies(heavy for a UI kernel + complicates crates.io publish). Extract the gatling primitives into a lean standalone crate (znippy-gatling) shared by zoomies + facett-core.
- ✅ SCOPED (2026-06-21): full impl plan =
-
CONS-GRAPHVIEW
facett-graphview✅ Phase C (facett commit, see Phase C note) — backend lifted tofacett-core::render::backend; opt-in L0 lane (l0.rs) routes nodes/edges through coreCanvas; defaultcpu::renderunchanged. (Engine layouts/cluster/picking = ongoing GRAPH-ENGINE work.) FOLLOW-UP: L0 lane not yet pixel-parity with cpu::render (needs rounded-rect-border SDF + cubic→line tessellation in L0). -
CONS-MAP
facett-map(2D) ✅ Phase C — chunking/types/install re-pointed tofacett-core::render::gpu(cull/draw.wgsl byte-identical); opt-in L0 point layer (l0_points.rs); default CPU painter unchanged. -
CONS-MAP3D
facett-map3d(3D) ✅ Phase C — offscreen+blit+scissor delegated to NEW sharedfacett-core/render/gpu/offscreen.rs(OffscreenColorDepth);shade_faces_clipuntouched. -
CONS-GEOMAP-FOLD
facett-geomap→ FOLD INTOfacett-mapas the slippy/scroll mode, then delete the crate (settles MAP-GEOMAP). -
CONS-GRAPH
facett-graph(2D) → thin consumer offacett-graphview(no own engine). -
CONS-GRAPH3D
facett-graph3d(3D) ✅ Phase C —LogoRendererfolded into sharedOffscreenColorDepth(realLessZ-buffer occlusion upgrade + idempotent upload fix); live GPU offscreen test green. -
CONS-SYSCHART
facett-syschart→ domain skin overfacett-graphview. -
CONS-PIPELINE
facett-pipeline→ domain skin overfacett-graphview(+ status/gantt overlay). -
CONS-OSM
facett-osm→ keep as the data layer (pure GeoParquet/WKB reader; verify it stays viewer-free — no consolidation, just confirm boundary).
Two facett map components born from the (now Codeberg-archived) mikronova/katana-osm viewer; each runtime-selects CPU vs GPU like nornir's #9 embedder selector. Real data = ways.parquet/nodes.parquet; synthetic test data = the znippy rabbit (2D + 3D).
- MAP-2D facett-map (2D) — wgpu vector basemap (
draw.wgsl/cull.wgsl/MapIndex/Mercator) ported from osm-katana; GPU mode (egui_wgpu callback) + CPU line-painter fallback. Reads real Swedenways.parquet. Merged to facettmain. - MAP-2D-STYLE 2D basemap styling (PAUSED, Rickard 2026-06-16) — current render is unfilled blue/green lines ("looks like shit"). Add polygon FILL for water/landuse/buildings (tessellate areas → filled tris) + proper OSM-tag styling through the facett
Theme(water-blue fill, roads by class, parks green, land bg, coastline), + fit-to-data initial viewport. Make it look like a real map. - MAP-3D facett-map3d (3D) — DONE 2026-06-16 (facett
main3621fbe). Extruded 3D skyline viafacett-graph3dwgpu (+ CPU fallback), znippy-rabbit synthetic data. Matrix oracle GREEN: emitted3d_map_renderedPASS (triangles=1058 tiles=5 lit_px=161704/409600) read back from JSONL, not claimed. 11 inject-assert tests + render-proof golden. - MAP-3D-NAV 3D OSM viewer — navigable orbit camera + standalone demo bin — DONE 2026-06-17 (NOT committed; on the working tree). New
facett-map3d/src/camera.rsOrbitCamera: spherical orbit (drag/WASD/arrows, pole-clamped), dolly-to-cursor zoom (wheel/pinch/+-), middle/Shift pan,ViewPresettop/iso/oblique/front hotkeys1-4,Rreset, frame-rate-independent damping (1-exp(-k·dt)), OS feel viawith_os_feel(InputFeelmacOS/Windows).shade_facesrewritten to true perspective projection (project-once buffer + near-frustum + back-face cull + depth-sort) — viewable from any angle.state_jsonnow exposes camera pose + visible-tri count + fps. Standalonefacett-demo/src/bin/osm3d_demo.rs(own[[bin]], leaves sharedmain.rsuntouched): synthetic procedural city / rabbit /$FACETT_OSM_PARQUET. 20 lib tests + render-proof + offscreen selftest GREEN (lit_px=30844); demo runs clean under wayland. Updated.nornir/facett-map3d.md. - MAP-3D-LINES (LOW PRIO, Rickard 2026-06-16) — a few lines render OUTSIDE the map bounds in the 3D view ("not high prio"). Clip/cull geometry to the map extent. Rickard: "will make a znippy rabbit of it later" → fold into the rabbit-styling pass, no rush.
- MAP-GEOMAP facett-geomap basemap (PAUSED, Rickard 2026-06-16) — geomap tab is point-only ("dead", no map). Give it a basemap (port the same vector layer) OR delegate to facett-map.
- MAP-RABBIT znippy rabbit test data — stylized rabbit as the synthetic fallback (2D in facett-map, 3D extruded logo via facett-graph3d), wgpu-snapshot-proofed. Agent
feat/znippy-rabbit-logoin flight. - MAP-PUBLISH facett 0.1.3 family publish (Rickard: yes) — publish the wgpu-map + FC-accessibility facett family so korp's Map can pull it (else korp needs a local
[patch.crates-io]). Do after the rabbit lands (bundle it).
facett GRAPH components — nodes/edges/text, 2 categories (CPU / CPU+GPU) (2026-06-21)
Same render kernel + 2 categories + SIMD split as the maps — ONE kernel, two skins. Full GRAPH render matrix + consolidation in
facett/.nornir/render-engine.md. Graphs draw lines+nodes +text (the user's ask); graph DATA comes from knut (FalkorDB/openCypher) — facett never learns Cypher. Graphs-over-maps overlay = LATER (the shared-camera+z-layer seam is added now so it's not a retrofit — see GRAPH-OVERLAY-SEAM).
-
GRAPH-ENGINE
facett-graphview= THE graph engine (the #17 spike → real engine; "beat gDotV + FalkorDB-browser") — layouts (force/Sugiyama/radial) · viewport cull · clustering→super-node · semantic zoom · GPU id-buffer picking · path-highlight, built onfacett-coreL0 (SDF instanced +vello_cpu/velloL1).facett-graph(2D)/facett-graph3d(3D)/facett-syschart/facett-pipelinebecome THIN consumers, not engines. -
GRAPH-RENDER-MATRIX adopt the 2-category graph matrix — CPU row =
vello_cpu(SDF nodes/stroke edges/text, straight-only, effects off, blit→egui); CPU+GPU row = L0 instanced nodes/edges +velloL1 (curves/bundling/animated-flow/glyph-atlas), compute cull, GPU picking,egui_wgpucomposite. Shared spine: knut adapter · facett-core model/Navigable/core::edges/camera/Theme · parley shaping. -
GRAPH-CLIP-TEST start→pixel emitter chain + final-picture blob (same discipline as MAP-CLIP-TEST) — emit
layout/cull/nodes/edges/picking/compositecheckpoints (in-fn emitters only for can't-observe pixel stages), oraclecomposite.ink_outside_rect==0+nodes.drawn==expected+edges.drawn==expected, emit rendered PNG blob, drive live vianornir-robotui, fail-on-bug/pass-on-fix matrix rows. Cover BOTH categories.- ✅ DONE (2026-06-22, facett merge
agent/graph-clip-test5d34a53,facett-graphview/src/clip.rs+tests/graph_clip.rs): the chain rendered into a widget rect ⊆ canvas with a clip fix/bug toggle;composite.ink_outside_rect==0+ exactnodes/edges.drawn;compositeemits (lit + ink_outside_rect), earlier stages return-asserted (EMIT-DOCTRINE); PNG blob read back; sensitivity-proven (bugink=7644pxRED / fix0pxGREEN); 6/6 CPU + 7/7 GPU(vello) tests green.
- ✅ DONE (2026-06-22, facett merge
-
GRAPH-PART-BENCH part-step benchers
facett.graph.{layout,cull,nodes,edges,picking,composite}(+ rollupfacett.graph_render) with injected sized graphs (e.g. 10k/100k nodes); perf bar = 60fps pan/zoom @100k nodes (render-engine.md). -
GRAPH-PROCESS-SVG
facett/.nornir/assets/graph-render.svg— skade-style flow of the graph pipeline (emit+bench tap per step), embedded in design+guide. -
GRAPH-OVERLAY-SEAM shared-camera + z-ordered layer-stack seam in
facett-core(NOW) — so a graph skin can later composite OVER a map skin without a retrofit. Just the seam (shared camera coords + layer z-index); the graphs-over-maps overlay feature itself is later. -
GRAPH-DEMO graph as a real facett-demo tab (part of MAP-DEMO-REWRITE) —
facett-graphviewdriving a synthetic + a knut-sourced graph; robot-matrix drives it; wasm-deployed.
znippy TILE container — planet-scale map data source (2026-06-21)
Idea (Rickard): a znippy archive holds millions of tiles with O(1) zero-copy access — a better PMTiles. Strategy: standard where it matters, novel only in the container. Addressing = XYZ / Web-Mercator (universal); content = MVT vector tiles (+ PNG/WebP raster); interop = read/write MBTiles (SQLite, most users) AND PMTiles (serverless, rising). znippy's value-add = O(1) Morton/Hilbert coordinate→offset index + zero-copy mmap + GATLING parallel build + airgap host. Feeds the render-engine "data scale" axis (tiles → cull/LOD → renderer). Ties to F2 (znippy geo-host).
- TILES-ZNIPPY znippy tile profile —
z/x/ypacked to a Morton/Hilbert key → dense per-zoom offset array (sparse for ocean) = O(1) lookup; tiles dedup-by-content-hash (many coords → one blob); per-tile codec (gzip/brotli/zstd); built in one GATLING pass from osm-katana→MVT. NOT a new coordinate scheme — XYZ/Web-Mercator only. - TILES-INTEROP MBTiles + PMTiles import/export — read existing
.mbtiles/.pmtilesinto the znippy tile store; emit.pmtilesfor web/MapLibre interop. CLI parity (znippy tiles import/export). Cheap on/off-ramp, no lock-in. - TILES-FACETT facett reads znippy tiles directly — mmap O(1) zero-copy tile fetch into the cull/LOD stage (device/airgap tier, NO server); MVT decode → the map skin. Holger optional tile endpoint (
GET /{z}/{x}/{y}→ znippy reader) for networked/browser clients. - TILES-BENCH part-bench the tile path —
znippy.tile_lookup(assert O(1): flat vs by-zoom-depth),tile_build(GATLING cores-busy), vs PMTiles/MBTiles lookup on the same set; inject-assert tile round-trip.
Njord (Tillsynia) — 2 tracks (2026-06-16)
- NJ-DOCS design SVGs + design.md — skade-style circuit-board SVGs (web flow, mail subsystem, doc-drop, Talos deploy) + flesh out
.nornir/design.md. Agentdocs/njord-design-svgsin flight. - NJ-DOCEXTRACT dropped-PDF auto-classify + extract — handed to a dev via
Njord/sigtuna_clade.md:DocExtractortrait, 2 impls (ClaudeDoccloud now /OnPremLlmlocal later, reusing nornir'svector::selectNVIDIA/AMD/CPU pick +generator::ollama). Replaces the brittle regex inavtal.rs. Data-governance gate (contracts → external API). - NJ-DEPLOY — Talos redeploy (rollout restart, current main) + collaborator
henrikprovisioned on oden (key + passwordless sudo + kubeconfig + tailnet).
test-matrix functional-status emitters (2026-06-16)
- EMIT-DOCTRINE (Rickard 2026-06-16 — order of preference, applies everywhere) — the in-function emitter is a FALLBACK, not the default. A test asserts on the function's return value / object status (the observable result +
state_json/observable state) as usual, as much as possible. Only when that is not enough or not possible — the result can't be observed from outside (e.g. a GPU render where "did it actually draw?" is in no return value) — do you implant thefunctional_statusemitter inside the function under test. So: prefer return-value/object-state assertions; reach for the in-function emitter only for the can't-observe-from-outside cases (map-render-non-blank is the canonical one). - EMIT-FRAMEWORK —
nornir-testmatrix 0.2.0(PUBLISHED):functional_status(component, check, ok, detail)+Aspect::Functional→ warehousetest_results→ matrix detects by status; thetestmatrixcargo feature is the ONLY-in-matrix-mode gate (release builds strip the emitters / no-op). Doccrates/nornir-testmatrix/.nornir/testmatrix-functional-status.md. - EMIT-ROLLOUT — emitters wired + existing tests updated to emit, across all 6 consumer repos (facett/znippy/korp/knut/skade/holger); robotic egui_kittest render-status for the maps (a dead map → RED functional row). Workflow
wogne3zy8in flight; then release the family + redeploy. - [~] EMIT-NORNIR-SELF dogfood landed (foundation + viz fallbacks) (2026-06-16) — nornir self-emits:
src/selftest.rs(emit/assert_emit!, EMIT-DOCTRINE) +tests/self_emitters_core.rs(marker-classifier self-test) + 4 viz render-path emit-fallbacks (draw_dep_graph/callgraph/funnel3d/screenshot) withtests/self_emitters_viz.rs(3 inject-assert, buffer-drain). Remaining: broader coverage across more of nornir's own surfaces (incremental).
AIRGAP bundle format — name + ownership (Rickard 2026-06-16, OPEN design)
The airgap installer is Skíðblaðnir (the fold-up ship; nornir-airgap codename). Its bundle is the znippy format — today a self-contained ZNIPGAP1 archive (content-addressed cache + delta + O(1) seekable read) living inside nornir-airgap/src/archive.rs.
-
AIRGAP-FORMAT — name + promote the package format. DECISION (lean / my rec, pending Rickard): ONE znippy-based format branded after the ship — Skíðblaðnir (short ext e.g.
.skid/.skb), carrying a manifestprofile: os-install | appliance(no-os)rather than splitting into two wire formats. Rationale: OS-vs-no-OS is a content/manifest concern, not a second format;tunnris the appliance consumer, not a format name (naming a format after a use-case ages badly); the hard-to-spell ship name only ever appears as a magic/extension. znippy should OWN this as a first-class named package format — promote theZNIPGAP1engine out ofnornir-airgapinto znippy proper, andnornir-airgapdepends on it. (Alt considered + set aside: two formats — Skíðblaðnir for OS-install + atunnrformat for no-OS.)- DIRECTION CONFIRMED (Rickard 2026-06-21): use real znippy as the format, drop the homegrown
ZNIPGAP1. Skidbladner packs/reads throughznippy-common, not its own engine. - ✅ znippy now has the zero-copy primitives this needs (committed
znippy@c0cc348):znippy_common::meta_sink_append::create_archive(path, &[(name,bytes)], level)builds an archive from in-memory bytes — no staging filesystem tree; newcreate_archive_to_vec(&[(name,bytes)], level) -> (Vec<u8>, report)builds one fully in memory (anonymousO_TMPFILEfd, never linked) so a release/airgap bundle can be sealed in RAM and streamed across the gap with zero disk on the build side;append_filesgrows a sealed archive in place;ZnippyArchive::open+extract_filerandom-access reads. Tests:create_archive_seeds_then_grows,create_archive_to_vec_is_filesystem_free_and_round_trips. - REWIRE nornir-airgap
archive.rsontoznippy-common— replaceZNIPGAP1'spack/read_entry/deltawithcreate_archive[_to_vec]/extract_file/append_files; keep theArchivetrait, swap the impl. Needs a customized/lean znippy facet so nornir-airgap doesn't pullznippy-cli/heavy plugins (depend onznippy-commonwith minimal features only). Then delete the homegrown engine.
- DIRECTION CONFIRMED (Rickard 2026-06-21): use real znippy as the format, drop the homegrown
AIRGAP — Go-bundler carry-overs (from airgap-kubernetes-main prior art, 2026-06-21)
Reviewed the original Go Talos bundler (git/airgap-kubernetes-main Design.md). Structural DNA already absorbed (two-phase, pure-language, version-pinned, atomic-rollback, no-shell). Open carry-overs to track:
- [~] AIRGAP-LIBVIRT — drive existing virtualization hosts. 🔄 (
nornir 9b437d4, 2026-06-21) Pure-render half DONE:unfold::libvirt::LibvirtTargetwithconnect_uri(empty host →qemu:///system, elseqemu+tcp://<host>/system) +domain_xml(q35+KVM, mem-KiB) +Target::Libvirtvariant; unit-tested, no root. Drive is a 100%-Rust SEAM, NOT avirshsubprocess (charter: nornir never shells — like gix for git): an earlier virsh-shelling version was removed. Pure-Rust libvirt RPC client DONE (nornir fd4080b— the go-libvirt equivalent, no virsh):libvirt::rpc(XDR encoder, 24-byte RPC header, 4-byte-framed messages, bodies forCONNECT_OPEN/DOMAIN_DEFINE_XML/CREATE/DESTROY/UNDEFINE) +LibvirtConn(TcpStream → CONNECT_OPEN → call/read_reply, parsesremote_nonnull_domain);unfold= define+create,teardown= destroy+undefine. 4 exact-byte wire-encoding unit tests + empty-host bail test. CONNECT_OPEN sends an empty name (libvirtd default), matching the proven go-libvirtConnect()(airgap-kubernetes-main/kvm.go). CLI wired (nornir 73b5bb3):nornir airgap unfold libvirt <name> --host h:16509 [--disk img] [--run](plan, or--rundrives the RPC). Remaining: (1) VALIDATE proc numbers + wire layout against a reallibvirtd(heavy#[ignore]libvirt_rpc_real_define_create_destroy,NORNIR_LIBVIRT_HOST=host:16509,auth_tcp="none"— no libvirtd on the dev box); (2) local unix-socket transport (qemu:///system). - AIRGAP-TEARDOWN —
destroyordering. ✅orchestrate::teardown_order(reverse topo: dependents removed before deps) +teardown_is_reverse_of_starttest;terraform::destroy(libvirt teardown rides the pure-Rust RPC client above, not virsh). CLI wired (nornir 73b5bb3):nornir airgap destroy <lakespec>walksteardown_order→SystemdInstaller::removeper component (reverse dep order, keeps data, idempotent). (--purge-datais a follow-up.) - AIRGAP-TERRAFORM-REAL. ✅
terraform::destroy(terraform destroy -auto-approve) added; theapplypath was already real behind theterraformfeature (rundrives the binary, honest error when absent). Render + absent-binary tests already present. - AIRGAP-DELTA-REAL — delta proven on real artifacts. ✅ heavy
#[ignore]znippy_delta_real_download_ships_only_changed_blob: downloads real successive.crates (anyhow shared + serde@200→@210 changed), asserts delta ships ONLY the changed blob andapplyreproduces B byte-identically. Verified run: shipped 78338 B (serde@210), reused anyhow (46741 B saved), byte-identical round-trip. Answers "does the delta work in Rust?" → yes (znippy content-addressed whole-file dedup; not the Go image-A/B model). - [~] AIRGAP-NO-SHELL — purge subprocess spawns (100% Rust charter). 🔄 Audited every
Command::newinnornir-airgap. Purged (pure-Rust, tested):curl→ureq;virsh→pure-Rust libvirt RPC;setenforce 0→fs::write(/sys/fs/selinux/enforce);useradd --system→create_system_user_in(passwd/group/shadow writer, unit-tested).systemctl→zbusDONE (nornir 6c88fb1): aSystemdstruct talksorg.freedesktop.systemd1over pure-Rustzbus(system bus, no libdbus, no subprocess) —reload/enable/disable/start/stop/reset_failed/is_active; rewiredinstall_systemd/remove_systemd/status_systemd+SystemdInstaller::{remove,status}. (Live D-Bus round-trip needs root+systemd to validate — same unvalidated status as libvirt, on holger's install path.) All 5 convertible spawns now purged. External-tool boundaries (NO pure-Rust equivalent, the Go version uses them too — not charter violations):podman(OCI build),qemu/terraform/vault(HashiCorp/external),firewall-cmd/ufw(could move to a Rust nftables crate later).
WAREHOUSE-TRAIT — make the nornir warehouse detachable (Rickard 2026-06-16, OPEN design)
The warehouse is always the nornir warehouse — never a per-feature sink choice. But it should become pluggable behind a trait so the backend can be swapped/detached later.
- WAREHOUSE-TRAIT — lift the whole nornir warehouse (read + append paths:
test_results,release_events,viz_actions, docs/SBOM, and futuresaga_events) behind aWarehousetrait. Today = the Iceberg-over-redb default impl. Future backends (e.g. DuckDB, remote) implement the SAME trait and plug in — callers depend on the trait, not the concrete store. This is what makes the warehouse "detachable." DuckDB is NOT a saga concern; it arrives here, behind this seam.
SAGA — collect user navigation for replay + bug-finding (Rickard 2026-06-16)
New nornir crate nornir-saga, production sibling of nornir-testmatrix: collect users' navigation → persist to the nornir warehouse saga_events → the matrix replays the path AND generates a robot test from it (egui = kittest/nornir-robotui; web app = Leptos+Axum via WebDriver) for bug-finding + regression (+ later AI via sigtuna OnPremLlm). Toolkit-neutral core; writes to THE nornir warehouse only (alt backends via [[WAREHOUSE-TRAIT]], not here). North star = the real-person-debugging loop: a real non-developer ("Henrik") tries Njord, hits a bug / gets lost → Saga records the session (nav + clicks + path) → a dev REPLAYS the exact steps, SEES the nav path / stuck point, and the replay becomes a standing robot regression test. "Real user hit a problem" → "dev reproduces in one click + it never regresses." Inspiration survey + the funnel/stuck/scrubber/privacy ideas: workspace_nordisk/saga-inspiration.md; full roadmap: workspace_nordisk/PLAN.md §R.
-
SAGA-CORE (DONE 2026-06-21) —
nornir-sagav0.1.0, full lean crate + workspace member (pulled WITHOUT thesagacapture feature so release pays zero cost):event.rs(SagaEvent/EventKind/SessionCtx/builders/in_session) +record.rs(record/drain_saga_events,saga-gated, OFF = no-op) +sink.rs(SagaSinktrait +JsonFileSink/NullSink) +replay.rs(ReplayPlan/ReplayStep/StepKind,for_session/all_sessions) +lib.rsre-exports; inject-assert tests across all modules. Warehouse DONE:iceberg_schema::saga_events()+src/warehouse/saga_events.rs(IcebergSagaSink,append/query/summarize/list_saga_sessions,session_replay_plan). Viz DONE: 🎬 Saga Sessions pane insrc/viz/test_tab.rs(list + replay-on-click +inject_saga_sessions_for_test). egui replay driver DONE (nornir_robotui::replay::plan_to_scenario); web replay driver SCAFFOLDED (replay_plan_to_web_scenarioinsrc/robot.rs). -
SAGA-EMITTERS — the remaining work, all toward the north-star loop (PLAN §R (a)-(f)):
- (a) Njord web capture adapter — Leptos router →
nav/click, Axum middleware → route +error(...)in the same session stream (persaga_eval.md§1; build on the nornir/workspace_njordside, Njord repo on HOLD). - (b) egui capture adapter at the facett
nav/Navigableboundary. - (c) make the WebDriver replay driver LIVE against a running Njord (
fantoccini/run_scenarios); failed step = RED matrix row namingsource_seq. - (d) Saga UX polish: one-click replay + navigation path/funnel view (PostHog) + stuck/rage detection (PostHog) + replay scrubber w/ RED failing step (Playwright Trace Viewer) + replay-as-RED-matrix-row; CLI parity (
nornir saga path <session>), never GUI-only. - (e) PII redaction default-deny — redact/allow-list
state_jsonin the adapter BEFORErecord()(Sentry Replay privacy). - (f) end-to-end "real user → dev repro" demo against
workspace_njord(the deliverable that closes the loop).
- (a) Njord web capture adapter — Leptos router →
-
SAGA-DOCS (DONE 2026-06-16) —
crates/nornir-saga/.nornir/:saga-design.md(kept current),saga-guide.md(henrik-facing app-side integration; kept in nornir, NOT pushed to Njord),saga-idiot-guide.md. Plan-side inspiration doc:workspace_nordisk/saga-inspiration.md. -
SAGA-NJORD (HOLD — do NOT touch Njord repo until Rickard says so) — hand the guide to henrik / land it in Njord only on Rickard's go.
HEAVY opt-in + bench-vs-test + chains (Rickard 2026-06-17)
Triggered by "test → run full matrix" spinning the fans for ~1 min: it was the unit aspect running heavy #[ignore]'d tests (large corpora / network), never benches. Settled model: heavy is a TEST-only opt-in; benches are explicit so never heavy-gated; bench-type tests get cloned to benches + the test downscaled.
-
HEAVY-OPTIN — heavy
#[ignore]'d arms now default OFF everywhere (release,nornir test, viz matrix); opt IN with global--heavy(NORNIR_RUN_HEAVY).--skip-heavy/NORNIR_SKIP_HEAVY= no-op (back-compat). Single sourcenornir_testmatrix::runner::heavy_enabled(); both runners read it (run_cargo_test+run_matrixadds--include-ignored/nextest--run-ignored all). Global--heavyflag added; release-local--heavyremoved. Testheavy_is_opt_in_default_offgreen; lib+bin compile. -
BENCH-NOT-HEAVY — decided + documented: benches are NOT
--heavy-gated (explicit-only; gating would gutno_regression/required_results). Bench heavy attempt reverted;Benchercontract untouched. Network/fixture benches gate vianornir.tomlnetwork_required+ fixture-skip. -
DOCS —
bencher.md(A/B/C classification + heavy-opt-in impl + benches-not-heavy), newbench-vs-test-for-idiots.md(idiot guide),bench-test-classification-audit.md(full nordisk work-list),test-bench-chains.md(chain design). nordisk_plan updated. -
CHAINS-AUDIT — ordering audited: benches ordered (
register_bench_ordered!+ prereq-check +NORNIR_BENCH_STOP_ON_ERROR); tests unordered but use lazy on-demand fixtures; znippy clean — heavy-skip causes no regression (no#[ignore]tests,ensure_jar()/assets::ensure()lazy). -
CHAINS-LAW — documented: a test MUST build inputs lazily (on-demand fixture), never depend on a sibling
#[test]running first; shared expensive steps = bench stage or fixture helper. -
[~] CLONE-DOWNSCALE — clone bucket-B tests → benches (full scale) + downscale the test twin. Per [[bench-test-classification-audit]]. Two layout patterns established + verified:
- facett — virtual-workspace → detached
bench/crate (own[workspace]+nornirpath dev-dep);facett.{graph,map,table}_renderbenches; perf.rs tests downscaled (1k,#[ignore]removed).cargo checkGREEN. - korp — single-crate-with-lib →
examples/nornir-bench.rs+nornirin[dev-dependencies];korp.graph_parsebench;bench_parse_throughputdownscaled 200k→20k.cargo checkGREEN. - znippy — [x]
tests/repro_crate→znippy.repro_cratebench (test downscaled 5000→200). [x]znippy-iceberg/round_trip→znippy.iceberg_compressbench (addedznippy-icebergdep to xtask; round_trip big.bin 25→12MB, still multi-chunk). Bothcargo checkGREEN. (znippy CLAUDE.md: never Co-Authored-By.) - knut —
knut.delta_ingestbench (delta vs full ingest rows/s; FalkorDB fixture-gated, skips if absent; BASE=20k/DELTA=5k). Addedskadeoptional dep/feature to xtask. Test twin kept#[ignore](podman/FalkorDB = heavy).cargo checkGREEN (bare +--features skade). - skade — already had a rich
bench/examples/nornir-bench.rs(~15 benchers: Nessie/Polaris compares, data_pipe_skade_{nvme,ram,s3}, osm_ingest) + nornir-testmatrix. CRITERION MIGRATED + REMOVED (Rickard's rule: convert unique / drop covered): 8 groups →skade.{append,append_partitioned,ingest,ingest_parallel_scaling,recast,read_scan,compression_append,ingest_strategies};readdropped (subset of read_scan). Deletedthroughput.rs+ criterion dev-dep +[[bench]];Cargo.lockclean. Bothcargo checkGREEN. skade is now criterion-free, fully on nornir engine. (skade_ab→ WAREHOUSE-TRAIT, see nornir-self above.) - [~] nornir self — [x]
examples/nornir-bench.rscreated withnornir.dep_graph_build+dep_graph_build_ext(#[path]-includes the test generator; l3/l4 downscaled 50→8, kept#[ignore]).cargo checkGREEN. [ ]skade_ab— DO NOT bench-ify / expose internals. It A/B's nornir's OWN Iceberg engine (append_batch/scan_all/RedbCatalog) vs skade — i.e. it's the EVIDENCE that nornir should drop its own engine and use skade as its warehouse backend ([[WAREHOUSE-TRAIT]]: lift the warehouse behind aWarehousetrait, backend = skade). Leave skade_ab as the#[ignore]A/B until that migration; delete it after (one engine = nothing to A/B). The real follow-up is warehouse→skade, an epic, not bench work.
- facett — virtual-workspace → detached
-
HEAVY-DEF — settled: heavy = large SCALE or external INFRA (container/podman, VM, model-download, network, live-server). No code change needed (
heavy_enabled()already gates all#[ignore]; container tests are already#[ignore]). Fixture-gate = secondary graceful-skip. Docs updated (bencher.md). Consequence: container/infra test twins stay#[ignore], benches stay explicit + fixture-gated.- holger — extended its EXISTING xtask harness (had 8 znippy benchers) with
holger.nexus_load+holger.latency_tail, heavy deps isolated behind anexusfeature; container fixture-gated (skip recorded, non-fatal). Downscaled test twins added + marked#[ignore](container=heavy).cargo checkGREEN (light build keeps testcontainers/tokio out). (skade = no porting, see above.) - bucket-A downscales (znippy/ljar/lzip/skade/knut). NOTE (corrected): lgz/linflate are NOT a coverage gap — the live code is
znippy-zoomies/{lgz,linflate}(standalone*-rsare the deprecated mirrors), and both ARE benched via znippy's xtask nornir-bench (lgz.guava/scalaDecompCmp,linflate.guava/scalaInflateCmp + linflate's own criterion).
- holger — extended its EXISTING xtask harness (had 8 znippy benchers) with
-
[~] RETIRE-MIKRONOVA — deprecation notices pushed (absorbed into znippy-zoomies): ljar-rs/lgz-rs/linflate-rs ✅ pushed; lzip-rs committed but push rejected (remote ahead — needs integrate, not force); katana-osm already archived on Codeberg (commit local-only). Archive-flag flip = Rickard does manually. kyokkou stays (not in zoomies).
-
AUDIT-REMAINING — bifrost, ordning, tunnr audited: NONE has a bench harness / uses nornir-fw / has any bench-type (B) test — all category C. bifrost = single bin, no tests; ordning = 13-crate ws, ~50 small unit tests; tunnr = 4
#[ignore]KVM-boot contract tests (lazy fixtures, not producer→consumer) + dev-dep onnornir-airgaponly. Nothing to port. -
CHAINS-IMPL (big job — AFTER clone-downscale, per Rickard) —
test_chainswarehouse table (syn-scan /chains.toml) +no_light_on_heavygate + heavy-skip propagation (skipped:heavy-prereq, not failed) + chain-graph viz in the 🧪 Test tab (facett node/edge Scene) +nornir test chainsCLI parity. -
VIZ-TEST-JOBS (needs skade-streaming) — move Log tab under the nornir tab + job list (1 row per test/bench trigger); "ask at close" on running jobs (detached server jobs); progress via skade streaming-on-commit (temp 1.5s polling behind same trait until skade-streaming lands).
-
JOB-ENGINE-SWEEP (done 2026-06-23) — audited every long-running/side-effecting op against the
jobs.rsledger. Gaps found: the engine had writers (release/bench/docs/knowledge/index/vector/bakeoff via the SERVER + some CLI) + aViz.JobsRPC + a viz tab, but (1) no CLI reader and (2) several local/fat CLI paths were untracked even though their server twins had jobs. Wired:nornir jobsCLI reader (remoteViz.Jobs/ local lock-tolerantJobStore::open_read_only;--kind/--running/--limit/--json, CLI⟺viz parity), and routed local coverage / test-matrix / arch-generate / airgap-pack throughstart_job(wrap-not-rewrite: start→finish on green, fail(&e) on error, Drop guard → failed). Container jobs: newJobKind::Container+jobs::run_container(spawn podman/docker, track pid/exit, capture+cap logs, done|failed) surfaced asnornir container --image <img> -- <cmd>(propagates the container exit code; works fat+thin). Tests: jobs:: container_spec/missing-engine/real-podman-echo/nonzero-exit/truncate-log (5) + tests/cli_jobs.rs jobs-reader + 2 real-podman CLI container tests. Builds green--bin nornir+--features server.
WAREHOUSE → skade (nornir uses skade as its warehouse engine) (Rickard 2026-06-17, AUTO)
Gap analysis (.nornir/warehouse-trait-skade-migration.md): nornir's "own engine" is mostly glue already over skade (append_batch→skade::append, scan_all→skade::read_all, bulk→skade::ingest_parallel). Do it via a Warehouse trait so the backend is swappable (skade now, DuckDB/remote later); scrap nornir's own engine since nothing is unique once the gaps are filled. Decisions: sync trait (block_on internally), generic Arrow core + named convenience wrappers, iceberg_schema.rs stays table-shape source of truth, Predicate backend-neutral (DuckDB-ready).
- SKADE-GAP1+2 — added to skade:
Table::read_filtered(&ScanFilter, cols)(backend-neutralScanFilter/Scalar, replaces nornir'sTable::inner()pushdown reach-through) +Table::read_limited(max_rows)(early-break, njord preview 5s→ms). 2 inject-assert tests green; both skade cratescargo checkclean. - P1 —
Warehousetrait (sync, generic Arrow + named wrappers, re-exports skadeScanFilter/Scalar) insrc/warehouse/mod.rs+ impl forIcebergWarehouse;scan_repo_filtered/scan_limitednow call skaderead_filtered/read_limited(the 2Table::inner()reach-throughs gone). Inject-assert test drives the whole trait surface;cargo checklib+bin GREEN, 15 warehouse tests pass. Zero behavior change. - P1b (DEFERRED polish — not needed to publish) — migrate the ~150 consumer sites to
&dyn Warehouse. nornir already runs on skade via IcebergWarehouse's inherent methods; this is future pluggability (swap backend), do it incrementally later. - SKADE-GAP3 —
skade::Table::ensure_schema(&mut self, &ArrowSchema)(additive, idempotent, field-id-stable — reuses current NestedFields verbatim, new cols optional above highest_field_id). Test green; both skade cratescargo checkclean. (GAP1/2/3 all done; GAP4 deferred; GAP5 wraps nornir-side.) - P2 — CONFIRMED nornir's warehouse is entirely a skade integration:
RedbCatalogIS skade_katalog, all writes =skade::append/ingest_parallel, all reads now via skade (13 MORE raw-iceberg reach-throughs routed throughread_all/read_filtered/read_limited, beyond P1's 2). NOTHING unique except GAP5 (lock-tolerant read-only = a feature over skade, not a gap).ensure_table_schemaalready on skade's catalog path (left as-is; semantics identical).cargo test --lib warehouse96 pass / 0 fail. ⇒ P4 = pure deletion, no skade gap to close. - P3+P4a — deleted
skade_ab_write_read;cargo test --workspaceGREEN (479 pass, 2 pre-existing/env fails: sandbox cargo-registry perm + viz/trace OnceLock ordering flake — neither from migration). GAP5 + its 5 lock-tolerance tests kept. - P4b-COMMIT — committed + pushed: skade
8cf8c87(branchfeat/warehouse-trait-gaps), nornir0350761(branchfeat/warehouse-skade-migration). Both via SSH to Codeberg. - P4b-RELEASE — FINDING: skade/skade-katalog/nornir are NOT on crates.io (+ skade has a vendored
iceberg-datafusionpath dep) → cargo publish is N/A; nornir builds against skade via path (^0.4.9covers local 0.4.10), so no skade publish needed. "Publish" =nornir release run --skip-publish(gate + version + tag + warehouse historize;nornir.rs:7777step 4 cargo-publish is skipped since not on crates.io). Ready to tag-release the branch; left as the final shared-infra action. - P3 — flip default to skade, verify full viz/gate/bench/test/docs matrix.
- P4 — delete nornir's
iceberg.rsengine +skade_ab; publish new nornir vianornir releasegate (only if all green).
znippy format: arrow-ipc default + skade iceberg + seal() (Rickard 2026-06-17) — funnel [[i-005]]
- arrow-ipc = znippy's DEFAULT tool format — the fast, self-contained, immutable static/airgap archive. NOT changing. (
ArrowIpcSinkdefault |IcebergSinkopt-in via--meta-format.) - znippy iceberg path uses skade as the default engine (behind znippy's iceberg trait;
IcebergSinkalready writes the index into a durable skadeWarehouse). Catalog A/B confirms skade (~8× fasterload_tablethan the old in-memory iceberg catalog) → no in-memory catalog path. - Live/dynamic = iceberg-skade (bootable: reopen + append + atomic commit — holger's incremental ingest over restarts). arrow-ipc "boot/resume" would need append-blob+footer-rewrite work (agent_idea #1); skade is the natural fit for the mutable phase.
- seal() DECISION:
seal()= finalize the current (iceberg-skade) snapshot → immutable (pre-seal open/growing, post-seal frozen). Producing a static arrow-ipc archive is a SEPARATE OPTIONAL EXPORT step (no-recompress: blobs zero-copy/byte-copy since content-addressed, index transcodes parquet→Arrow-IPC = cheap). i.e. seal ≠ auto-create arrow-ipc; export is opt-in (offer a combinedseal --export arrow-ipcconvenience). iceberg-skade→arrow-ipc conversion is cheap (blobs untouched, only the small index re-serialized). - SEAL-IMPL (i-005) — append-blob sidecar +
seal()+ optional arrow-ipc export + compaction/GC + blake3 content-addressed blob dedup. Own znippy epic, AFTER warehouse→skade. ✅ DONE (verified 2026-06-30 audit): the znippy iceberg-seal path shipped (finalize → immutable snapshot, optional arrow-ipc export, blob dedup).
MR-ROBOT-ARROWIPC — deep analysis: new/changed znippy arrow-ipc impl (Rickard 2026-06-17) → workspace_nordisk
- Deep analysis DONE →
workspace_nordisk/znippy-arrowipc-analysis.md. Bottlenecks: seal (re-materialises rows, single-threaded), random-lookup (~3 manifest reads/call, no cache), sub-index serialisation. Ideas: (B) cached mmapArchiveReaderO(log n), (A) incremental/parallel seal, (C) append+seal (agent_idea #1). Existing ArchiveBench/StreamBench fold seal into compress + never isolate index/footer/lookup. - BENCH-FIRST LAW (next, when this epic is picked up): create + baseline the 5 benches
subindex_write,footer_seal,random_lookup(cold+warm),prefix_meta,decompress_random_subsetBEFORE any impl. Then sequence: (B) reader cache → seal cleanups → (A) incremental seal → (C) append/seal. All keep the format DuckDB/Polars-readable. DEFERRED until warehouse→skade ships.
GATLING-SATURATION + FACETT-VIEWERS + AUTONOMOUS-TEST LAWS (2026-06-17/18 night)
- osm-katana convert un-regressed (was read-8GB-then-serial-sort, ~11–20% cores) → Gatling-style pass1/pass2 ~30/32, proven on PLANET 10.58B nodes / 1.18B ways. streaming RAM-bounded k-way sort, coarse PBF split (candidate offsets + worker self-seek), ring 4→8, verify 513s→2.78s, geo2arrow N-core (🔄). Ragnar stree32 u32 clone (bordlagd). Branch
fix/osm-katana-gatling-saturation, tagosm-katana-saturation-proven. Docs:znippy/.nornir/{gatling.md, gatling-guide.md, gatling-for-dummies.md, osm-katana-gatling-vs-rayon.md}. - facett OSM 2D/3D wired as demo tabs +
compliance.rs::registry(). 3D "starburst" → geometry CLEAN (proven via emits); real issue = render arch (painter's-algo vs GPU z-buffer). [ ] 🔄 GPU z-buffer path (fix #12 geom-buffer chunk + depth buffer + building LOD = real game/flight-sim tech). - LAW — autonomous test-fixing (
nornir/.nornir/agent-ui-test-fixing.md, wired into nornir docs+PDF+viz button): validate as DATA via emits (not pixels); emit ONLY where return/state_jsoncan't prove it; debug by BINARY SEARCH (halve the call chain, never slavish single-step); follow the data; every fix → fail-on-bug/pass-on-fix matrix test. APPLY to ALL UI/gRPC: nornir CLI fat+thin + viz, facett-*, korp-poc (+ keep improving the prompt). (landed 2026-06-18: LAW folded intotest-matrix.md §0+autonom-for-idiots.md+completeness-gate.md, single source =agent-ui-test-fixing.md; ships as chapter 2 of regenerateddocs/book.pdf(86 chapters, doctrine text confirmed in the PDF); 🧪 Test pane exposes a discoverable "📖 How agents fix tests" link →state_json["test"]["doctrine"]{title,doc,book,summary,opened}; matrix teststests/viz_test_pane.rs::test_pane_exposes_doctrine_surface_and_link_is_wired(fail-on-bug proven) +doctrine_doc_exists_and_is_in_the_rendered_book.) - LAW — Gatling-ify serial passages (
znippy/.nornir/agent-gatling-serial-hunter.md): find serial passages → 1 reader (reads only, 6–8 slots, zero-copy ring) → split (candidate offsets, workers self-seek) → channel → N workers (no barrier, self-dispatch) → 1 writer (saves only). EVERY super-parallel znippy process ships a skade-style SVG (1→N→1, left→right,.nornir/assets/; templateosm-katana-path-matrix.svg).
✅ RELEASE CYCLE 2026-06-18 — clean-slate merge + docs + publish across workspace_nordisk
- nornir 0.4.47 — merged feat/warehouse-skade-migration→main (arch-lock fix Ops.RunArch + warehouse bench reads + CLI bakeoff leaderboard via server), bumped 0.4.46→0.4.47, rebuilt book.pdf (14.9MB, historized 0.4.47), pushed main, published to crates.io, tagged v0.4.47, deleted merged branch.
- skade — merged feat/warehouse-trait-gaps→main (skade 0.4.11 + skade-katalog 0.1.9 already published; both match crates.io), rebuilt book.pdf, pushed, deleted 2 merged branches.
- holger — merged feat/nornir-bench-and-pip-tempdir-fix→main (real merge: resolved Cargo.lock + bench-scenarios duplicate serde_json key → non-optional; verified compiles), rebuilt book.pdf, pushed, deleted 7 merged branches. (Not a crates.io crate — server app.)
- knut — committed dirty bench-harness+delta-test work→main, rebuilt book.pdf, pushed, deleted 7 merged branches.
- korp — committed dirty bench+graph work→main, rebuilt book.pdf, pushed, deleted 1 merged branch (kept 5 unmerged: korp-map-wgpu/matrix-robotui/test-matrix/facett-adopt-look/container-tests).
- facett — merged feat/osm-viewer-tabs→main (12 commits), rebuilt book.pdf, pushed, deleted ~17 merged branches (kept facett-demo-robot-matrix + mq-mannequin-facett). facett crates are publish=false in bench/ — not published to crates.io (git-only).
- znippy — DEFAULT BRANCH = master: merged fix/osm-katana-gatling-saturation→master + origin/master (agent_idea.md), rebuilt book.pdf, pushed, deleted 7 merged branches (kept osm-region-clip). znippy 0.9.1/znippy-zoomies 0.1.10/lgz 0.3.3/znippy-common 0.9.3 == published; the merged osm-katana/native-export work is UNRELEASED (versions not bumped) — deliberate: a multi-crate znippy version bump+publish is a separate decision, not auto-done.
- NOTE — 3 nornir source files (src/arch/mod.rs, src/arch/warehouse.rs, src/viz/test_tab.rs) had UNCOMMITTED WIP from a PARALLEL session (a 🧪 Test-grid regression test + arch edits, 2026-06-18) — left untouched; the 0.4.47 release sits cleanly below them. Someone should commit/finish that WIP.
- 🏛 Architecture + 🧪 Test false-empty tabs FIXED (2026-06-18) — the LAST two false-empty viz tabs. Root cause (Arch): historized
architecture_wiringboards containNodeKind::Clinodes; any reader binary whoseNodeKindlackedCli(deployed server/viz/MCP) hit serdeunknown variant 'Cli'→load_arch_wiringerrored → member dropped →members_with_data:[],node_count:0. Fix = tolerant manualDeserializeforNodeKind(unknown variant →CoreFn, never an error; forward-compatible cross-binary reads) in src/arch/mod.rs. Root cause (Test grid): current grid-build logic was correct; the live 0/0 was the STALE urdr-threads viz client (Jun 16). Both fixed by rebuild + redeploy nornir-server + urdr-threads + nornir-mcp (/usr/local/bin + ~/.cargo/bin) +systemctl restart nornir.service. Tests (fail-on-bug/pass-on-fix, git-stash-proven):arch::tests::node_kind_decode_is_forward_tolerant(fails on derived Deserialize w/ the exact live error),arch::warehouse::tests::merge_keeps_cli_nodes_and_does_not_blank_the_board,viz::test_tab::…::real_rows_make_a_nonempty_grid_with_green_and_red. LIVE PROOF (remote viz → oden, ws=nordisk): Architecturenode_count:2217 edge_count:6379 members_with_data:8/8 empty:false; Test gridgreen:19 red:5 skip:32 repos:8/8 inventory:2125 run_count:9. - znippy release — bump+publish znippy/znippy-zoomies/lgz for the merged osm-katana region-clip + native sealed-export work (currently only on master, unreleased).