nordisk/nornir

The nornir workspace — reasoning

Class: reasoning. This file holds the why, the trade-offs, and every design we rejected. The descriptive shape we actually run is in workspace-design.md, which by rule mentions none of this; the workflow is workspace-guide.md. If you want to know "why not X?" — X is here.

Why a workspace is three parts, not one file

A workspace could have been just nornir-workspace.toml — parse it, clone, done. It is instead descriptor + registry row + data-root because each answers a different question that the others cannot:

• the descriptor answers what the members are — it is human-authored, git-tracked, and belongs in the control-plane repo;
• the registry answers what the server is currently tracking and where each member stands (sync state, snapshot id) — mutable runtime truth, one global index;
• the data-root answers where the cloned code and warehouse physically live — derived, rebuildable, never authored.

Collapsing any two would force authored config to carry runtime state, or runtime state to be re-derived from disk on every read. Keeping them apart is what lets the control plane be a tiny committed repo while gigabytes of warehouse live elsewhere.

Why the redb registry is the source of truth (not a sibling TOML)

The obvious design is: the descriptor file on disk is the truth; the registry just points at its path. Rejected. The server runs as a different user on a different host from whoever authored the descriptor. In thin mode the client's local nornir-workspace.toml path is meaningless to the server — it cannot read it. If the registry only stored the path, a monitored workspace registered from a laptop would have nothing to re-read on the next poll tick.

So the descriptor's content is stored in the redb row (Workspace.descriptor_content), and the poll loop + every republish re-read members and deep_scan straight from the row via WorkspaceDescriptor::from_content — never a file. This is the whole justification for from_content existing beside load, and for RegisterWorkspaceRequest carrying a descriptor_content field: the client reads its local file once, ships the bytes, and the server becomes self-contained.

Rejected: a sibling descriptors/<name>.toml on the server

We considered having the server write each workspace's descriptor to <home>/.nornir/descriptors/<name>.toml and read it back. Rejected because it re-introduces exactly the two-sources-of-truth problem the row was meant to kill: the row and the file could drift, and a purge would have to clean up two places. One row, one source. The registry is redb (not a pile of TOML) for the same reason the warehouse is: a single-writer, transactional, bytes-in/bytes-out index with atomic updates (set_snapshot mutates two fields in one write txn, closing the read-modify-write window where a concurrent member-SHA write could be clobbered).

Why register is eager by default (server ≡ user)

Early on, register was record-only and population was deferred to the first poll tick. That left new workspaces half-empty and confusing: nordisk registered with repos: [], korp synced but with current_snapshot: "" so MCP search returned nothing, and clone failures only hit the server's stderr. A user ran register, saw success, and had an empty workspace.

We flipped the default to eager (decided with Rickard 2026-06-12; made the register default, not just add, 2026-06-14): record and fetch and build inline — the same record → fetch → build pipeline the poll loop runs on a change. The principle is server ≡ user: a fat user and the daemon do the same thing; the poll loop is just the automated re-run of what add does by hand. The lazy primitive survives as --lazy — an explicit opt-out for scripted/daemon flows that genuinely want record-only — but it is the escape hatch, not the norm.

Why add still exists as an alias

add is kept as a habit-name for the eager path; it does exactly what the default register now does. There is no behavioural difference. We didn't delete it because "add a workspace" is what people type, and breaking muscle memory to save one enum arm isn't worth it.

Why bench smoke and the light test matrix run on populate

They cost time on a bootstrap that is rare. We turned them on by default anyway (Rickard, 2026-06-14): adding a workspace happens seldom, so pay the cost once and get the full picture — a Test pane that isn't empty, a bench baseline, and NOT-RUN rows for every discovered-but-unrun test. The alternative (an empty Test pane on every fresh workspace, populated only on a later manual run) was the exact gap that made new workspaces feel broken. --heavy opts into the full corpus; --no-test / --no-bench skip a stage for the impatient.

Rejected: a dedicated workspace_events table

The populate pipeline needs to stream per-step outcomes (clone, dep_graph, index, snapshot, test, bench — start/end, ok/fail/skip) so the viz can watch a workspace fill. The tidy design is a purpose-built table. Rejected (Rickard, 2026-06-14): adding a workspace is rare, and release_events is already a generic per-step event stream the viz renders (run_id · seq · ts · component · repo · op · phase · status · detail · …). A populate is just a run there, with op = the step name. Zero new schema, and the viz run/event pane reads it for free. The registry's one-word sync_state stays the summary; release_events (surfaced as clone_events for the CLI) is the drill-down.

Why the descriptor supports path, git, and both

Three source shapes exist because three real situations do:

• path only — the steady-state fat case: the code is already checked out beside the descriptor and must be used as-is, offline, with no network. A missing path is a hard error precisely so a typo never silently pulls from the internet.
• git only — a monitored member the server owns; there is no local checkout to respect.
• both — a fat member you usually have on disk but want auto-materialized when absent (a fresh machine, a CI runner). We rejected forcing "choose one": that made the same repo need two different descriptors for laptop vs CI. Instead path + git means "use the checkout if present (PathPlan::Use), else clone the fallback into it (PathPlan::Materialize)". The decision is factored into the pure plan_path so it's unit-testable without touching disk or network.

Why an empty directory doesn't count as "present"

path_is_present treats an empty dir as absent. A failed clone leaves an empty directory behind; if we counted that as present, we'd be stuck — never re-materializing, never erroring usefully. Requiring at least one entry lets a partial clone be retried cleanly.

Why materialize clones pure-Rust (no git subprocess)

materialize_member uses gitio::clone_or_fetch (gix over HTTPS, russh over SSH), not a shell-out to git. Reasons: the server may run in a minimal container with no git binary; shelling out means parsing another tool's stderr for auth failures; and the whole codebase already commits to pure-Rust transport for reproducibility. The best-effort branch checkout after clone is non-fatal on purpose — a stale or renamed branch should leave you on the default branch with a warning, never block the member entirely.

Why the data root is $HOME-derived with no env-vars

There used to be NORNIR_ROOT, NORNIR_WAREHOUSE_ROOT, and NORNIR_FUNNEL_ROOT. They are gone. The failure mode of path env-vars is that they must be kept in sync across every process, and one forgotten export silently splits a workspace's data across two roots. The home rule subsumes all three: a process inherits its user's $HOME, so there is exactly one place data lives per user and nothing to forget.

The deeper payoff is the server ≡ user invariant again: because the same code (nornir_home() → the home crate) computes the root for both a login user and the nornir system user, fat and server modes are byte-for-byte the same layout. There is no server special case to drift. Containers get the right path the standard way — set $HOME — with no nornir-specific knob.

What we kept configurable, and why only that

An absolute [storage].local_path still pins the embedded CLI/MCP warehouse to an explicit path — deliberately, so a user can put a hot warehouse on a fast disk. That is a per-config override, not a filesystem-root env-var, so it can't silently desynchronize two processes the way NORNIR_WAREHOUSE_ROOT did. The server has no warehouse override at all — its warehouse is always $HOME-derived per-workspace, so there is exactly one thing that decides it.

Why per-workspace folders instead of a global warehouse

Breadth could have lived in one global warehouse with a workspace column on every row. Rejected: a workspace is meant to be a self-contained unit you can register, inspect, purge, or move as a whole. One folder per workspace makes rm --purge a single remove_dir_all, makes a workspace's MCP scope obvious, and lets the same repo belong to several workspaces (each with its own warehouse) without a cross-workspace query having to filter a shared pile. A workspace_<name> bundles as many repos as you want, so breadth lives inside a workspace, not in a global heap.

Why the workspace name is validated so aggressively

validate_ws_name rejects empty, ./.., path separators, and absolute/multi-segment names — and remove(.., purge=true) re-canonicalizes the resolved builds/ and asserts it still lives under the registry root before wiping. This is defensive to the point of paranoia because the name is both a redb key and an on-disk directory segment handed to remove_dir_all. A name like ../etc or /tmp could make the purge escape the registry root and delete an arbitrary directory. This guard is the scar tissue from the nordisk wipe — the data-loss lesson that taught us to validate before ever touching the filesystem, and to run the escape-check even when no registry row matched (an orphan builds/ from an earlier non-purge rm must still be cleaned up, not used as a bypass).

How we got here (the road, briefly)

The workspace started as a pure descriptor parser (src/workspace/ only parses the TOML; the dep-graph it feeds lives in warehouse::dep_graph, co-located with its persistence because graphs are historical artefacts). The registry came next as the server grew a poll loop and needed a persistent index. The eager-register flip (2026-06-12/14) and the EPIC POPULATE work (2026-06-14) followed once "registered but empty" proved to be the dominant new-workspace failure. The descriptor_content-in-the- row change closed the last gap: it made a thin-registered monitored workspace fully self-contained, so the server never depends on a client path it cannot read. The path+git combined member and pure-Rust auto-materialize were the final piece of "fat does what the server does" — for cloning, and for auth (one deploy key per identity: the server's from install, the user's from key generate).