nordisk/nornir

The nornir workspace — design

Class: design · Function axis: f0 (the workspace as one system) + f1 descriptor · f2 registry · f3 data-root · f4 register/autopopulate · f5 the workspace CLI verbs. Strict: this file states only the design that is — architecture, laws, data-flow, source pointers. Every why, every rejected alternative, lives in workspace-reasoning.md. How to drive it is in workspace-guide.md; the cold-shell walkthrough is workspace-for-idiots.md.

f0 — the workspace as one system (the whole)

A nornir workspace is the appointed set of N independent repositories the release/build/warehouse pipeline treats as one unit. It is not a directory of code — it is three co-operating parts that together name, index, and store that set:

Two planes, one workspace:

• Control plane — the workspace_<name>/ repo: nornir-workspace.toml, release/nornir.toml, SPEC.md, WORKSPACE.md. Config + descriptor only, no code, no data.
• Data plane — <home>/.nornir/workspaces/<name>/{git,builds/warehouse,funnel}. The members are cloned into git/; every warehouse fact for the workspace lands in builds/warehouse/.

The server ≡ user law. Fat (a user on their laptop) and thin (the daemon on host oden, running as system user nornir) are the identical layout, computed by the same code (config::nornir_home() reads $HOME). The only difference is who fills git/: you check it out, or the server clones it. There is no server-special path (data-root.md).

Reuse map (what the workspace subsystem uses). The descriptor is parsed by src/workspace/; the dep-graph built from it lives in crate::warehouse::dep_graph (graphs are historical artefacts Urðr remembers, so analysis + persistence are co-located there). Register/populate drives monitor::{fetch_workspace, republish}, which drive the warehouse, code index, and nornir-testmatrix/bench engines. The registry row is read by nornir workspace ls, the Workspaces.* gRPC service, and the viz Nornir-tab roster.

f1 — the descriptor (nornir-workspace.toml)

A single TOML file lists every member by name and gives its source. Parsed into WorkspaceDescriptor (descriptor.rs:29).

[workspace]
name = "holger-suite"
# deep_scan = true   # opt-in: republish also scans the FULL transitive dep closure

[repos.holger]
path = "../holger"                               # a local (fat) checkout

[repos.znippy]
git = "https://codeberg.org/nordisk/znippy"      # a git member
# branch = "main"                                # optional pinned ref

[repos.korp]
path = "../korp"                                 # BOTH: local checkout…
git  = "git@codeberg.org:nordisk/korp.git"       # …with a remote to auto-materialize from

Source resolution laws (RepoSpec::source, descriptor.rs:82):

• path present (with or without git) → RepoSource::Path. A relative path joins the descriptor's parent dir; an absolute path is honored verbatim. A git alongside path becomes the fallback remote — a fat member auto-materializes from it when the checkout is absent.
• git only → RepoSource::Git(GitRef) carrying url + optional branch.
• neither → hard error ("repo spec has neither path nor git").

Loading. WorkspaceDescriptor::load(path) reads + parses the file and canonicalizes descriptor_dir (descriptor.rs:104). from_content(&str) parses raw TOML with no file — the seam that lets the registry store the descriptor in redb and re-read members/deep_scan from the row (descriptor.rs:123). sources() returns (name, RepoSource) in deterministic BTreeMap order (descriptor.rs:129).

Discovery. workspace::discover_descriptor(start) walks up from start (cwd → parent → filesystem root) returning the first nornir-workspace.toml found, so a user standing in a workspace_<name>/ dir needs no --descriptor flag (mod.rs:29).

Materialize plan (resolve.rs). resolve_sources(desc) maps every member to a concrete local path (resolve.rs:86). For a Path member the pure decision is plan_path(present, fallback) -> PathPlan (resolve.rs:47):

A checkout is "present" iff path is a non-empty directory (path_is_present, resolve.rs:60) — an empty dir from a failed clone is re-materialized. Materialize clones pure-Rust via gitio::clone_or_fetch (HTTPS via gix, SSH via russh — no shell-out), then best-effort checks out branch (materialize_member, resolve.rs:150). A git-only member resolves to a deterministic cache dir under $NORNIR_CACHE_DIR (git_cache_dir, resolve.rs:71).

f2 — the registry (redb source-of-truth)

A single redb file at <root>/registry.redb is the persistent index of every workspace the server tracks. One table, TABLE: &str → &[u8] — workspace_name → serde_json(Workspace); bytes in, bytes out; the server is the sole writer (registry.rs:24).

The row (Workspace, registry.rs:114):

descriptor_content is the source of truth. The poll loop and every later republish re-read the descriptor TOML straight from the row (WorkspaceDescriptor::from_content(&ws.descriptor_content)), never a sibling descriptors/*.toml on disk. Workspace::new reads a local descriptor file and captures its content into the row (registry.rs:142); Workspace::from_content is the thin-mode seam where the client ships the TOML over RPC and the server stores it with no disk touch (registry.rs:169).

Mode (registry.rs:56):

Mode::parse maps any unrecognized string to Pushed (registry.rs:77).

MemberState (registry.rs:87): name, remote (mirrors RepoSpec.git), git_ref (empty ⇒ main), last_seen_sha, last_synced, sync_state ("ok"/"fetching"/"error: …"/""), and the worktree-freshness pair worktree_digest + worktree_dirty (sees uncommitted edits the SHA cannot; equals gitio::CLEAN_WORKTREE_DIGEST when clean).

Registry API (registry.rs:229):

Name safety. validate_ws_name (registry.rs:35) rejects empty, ./.., anything with / or \, and absolute/multi-component names before the name is joined onto the root and handed to remove_dir_all. remove(.., purge=true) additionally canonicalizes the resolved builds/ and asserts it still starts with the registry root before the irreversible wipe, and runs the wipe even when no row exists (orphan cleanup) (registry.rs:343).

The roster projection. RosterRow (registry.rs:521) is the shape both the fat (registry Workspace) and thin (proto) faces map into so workspace ls and the viz Nornir-tab render identically: name · mode · members · last_synced plus the optional --status roll-up (populate ∈ green/red/stale, failing_member, last_error). roster_outcome(rows) renders it as a uniform CommandOutcome (registry.rs:547); an empty registry is RED for the autopilot.

f3 — the data-root layout

nornir keeps everything under <home>/.nornir, one folder per workspace, no path env-vars (data-root.md):

<home>/.nornir/
  workspaces/
    registry.redb             # the ONLY global file (the index)
    <name>/                   # one folder per workspace
      git/                    # member source: server clones here / fat = your checkout
      builds/
        warehouse/            # THIS workspace's Iceberg warehouse (catalog.redb, jobs.redb beside it)
      funnel/                 # THIS workspace's idea→plan funnel

The data root is derived from the running user's $HOME — one rule for everyone. Canonical resolvers in src/config.rs:

home::home_dir() reads $HOME (falling back to getpwuid), so the same code yields /home/rickard/.nornir for a login user and /home/nornir/.nornir for the systemd service. An absolute [storage].local_path in nornir.toml still pins the warehouse to an explicit path (a deliberate per-config override, config.rs:274); a relative local_path joins workspace_root. nornir root prints the resolved home.

f4 — the register / autopopulate flow

register is EAGER by default (workspace-autopopulate.md, creating-a-workspace.md): it records the row and runs the same record → fetch → build pipeline the poll loop runs on a change, inline, so the workspace is populated now. Handler WorkspaceOp::Register (nornir.rs:5794):

① registered  <name> (N member(s))     # reg.upsert — seed members from the descriptor
② fetching    N member(s)…             # monitor::fetch_workspace → clone into <root>/<name>/git/
③ building    warehouse…               # monitor::republish → dep graph + knowledge + index + snapshot
✓ registered  <name> — populated now (snapshot …) at <root>/<name>/builds

• add is the explicit eager alias — identical behaviour (WorkspaceOp::Add, nornir.rs:5866).
• --lazy (or any non-monitored mode) records the row only and defers population to the poll loop. A silent-empty guard rejects an eager monitored register that resolves 0 members (nornir.rs:5828).
• --descriptor is optional — omitted, it is discovered via discover_descriptor walking up from the cwd (resolve_descriptor_arg, nornir.rs).

Thin vs fat. In thin mode (NORNIR_SERVER) the register + fetch route over RPC and the server republishes from the fetched change. A discovered local descriptor's content is read client-side and shipped in RegisterWorkspaceRequest.descriptor_content (proto field 6), because the server is a different host and cannot read the client's path; a --descriptor that is a git URL or an already-server-readable path is passed through with empty content (descriptor_is_local_file / resolve_descriptor_content, nornir.rs / nornir-server.rs).

The poll loop is the automated register. A monitored server re-fetches + rebuilds on a git HEAD change (monitor.rs). Nothing differs in kind from the manual button — only who pushes it.

Autopopulate (EPIC POPULATE). On register + on each poll tick, republish_with runs the full member-bootstrap pipeline (workspace-autopopulate.md): (1) dep graph, (2) knowledge + security scan, (3) BM25 index, (4) Iceberg snapshot, (5) dep-graph persist, then the light half — (6) test-surface discovery → NOT-RUN rows, (7) a light test matrix (build·unit·doctest·clippy·fmt·audit, NORNIR_SKIP_HEAVY=1), (8) a bench smoke (NORNIR_BENCH_QUICK=1). Every step (plus clone) emits a release_events row (run_id·seq·ts·component·repo·op·phase·status·detail·…) the viz run/event pane renders — no new table. Sources: monitor::{fetch_workspace, republish_with}, crates/nornir-testmatrix, src/bench/, warehouse tables release_events / test_results / bench_runs.

Auth (the one setup difference). Member sources clone pure-Rust; SSH remotes read a per-user deploy key — server → /home/nornir/.ssh/id_ed25519 (created by nornir install), fat user → ~/.nornir/ssh (created by nornir key generate, its public key registered as a codeberg deploy key). HTTPS-public members need no key.

f5 — the nornir workspace CLI verbs

WorkspaceOp (nornir.rs:515); each verb routes fat (local Registry) or thin (RPC).

--monitored and --external are mutually exclusive (nornir.rs:5802).