nordisk/nornir

Running nornir as a server

nornir runs embedded (the CLI opens the warehouse in-process — the default) or as a long-running server that the CLI talks to over gRPC. Same verbs either way. The two are mutually exclusive per warehouse — one process owns the redb/iceberg write lock (see design.md).

Verified end-to-end on 2026-06-07 against workspace_holger (3 repos).

Install / build

server is a default feature, so a plain install is functional out of the box:

cargo install nornir                 # installs `nornir` + `nornir-server`; client mode works
cargo install nornir --features cli  # + MCP (nornir-mcp) + docs PDF/HTML (docs-export)
cargo install nornir --features cli,embed-ort  # + semantic search embedder, CUDA→CPU
                                               #   (GPU needs cuDNN discoverable — see vector.md)

From source: cargo build already includes the server (→ target/debug/{nornir, nornir-server}). Library/bench consumers that want nornir lean use nornir = { …, default-features = false }.

The server is just an authenticated port (for idiots)

Read this first; the rest is detail.

• The server has exactly two settings: a bearer token (auth) and a bind address (which port to listen on). That is the entire server config surface.
• It has no boot config file — no nornir.toml, no --config, no list of workspaces baked in at launch. You can start it on a totally fresh box with zero workspaces; serving 0 is a valid resting state.
• Everything the server knows lives in one redb file: the registry at <home>/.nornir/workspaces/registry.redb. The registry is the source of truth — it even stores each workspace's descriptor TOML inside the row, so it can rebuild a workspace from nothing but its own git URLs. Wipe the registry → the server forgets everything; re-add a workspace → it re-clones and re-scans it.
• You add workspaces while the server is running with nornir workspace register …. The server then clones the members, scans them, indexes them, and snapshots a warehouse — no restart needed.

The whole flow at a glance

Start the server

One command — nornir serve execs nornir-server for you. The only things it needs are a token and (optionally) a bind address:

NORNIR_SERVER_TOKEN=<≥16-char-secret> \
  nornir-server --addr 127.0.0.1:7878

The server has no boot config file. It does NOT read a nornir.toml, and there is no --config / NORNIR_CONFIG / NORNIR_WORKSPACES / --monitor / --poll / --warehouse ingress (all removed). Workspaces come ONLY from the redb registry at <home>/.nornir/workspaces/registry.redb, mutated at runtime via nornir workspace register (→ the Workspaces.Register RPC). A server with an empty registry (zero workspaces) is a valid resting state: it starts cleanly and serves 0.

On a fresh box it prints:

nornir-server: serving 0 workspace(s) (empty registry — register one with `nornir workspace register <name> --descriptor <toml>`)
nornir-server: listening on 127.0.0.1:7878

…and once you've registered workspaces (which persist in the registry across restarts) it re-opens and serves them:

nornir-server: serving monitored workspace `njord` (builds/ warehouse)
nornir-server: serving 1 workspace(s); default `njord`
nornir-server: listening on 127.0.0.1:7878

The entire server config surface is two flags, each with a matching env-var fallback — precedence --flag > env > default — so a deployed systemd unit (env) and a hand-run binary (flags) both work:

That is all the server reads at boot — bind + auth, nothing else. There is no workspace/warehouse/poll boot flag. The warehouse is $HOME-derived per-workspace at <home>/.nornir/workspaces/<name>/builds/warehouse (no override flag), and the poll cadence is the per-workspace registry poll field set at register time (the server's monitor loop ticks at a fixed 60s default).

• bind address — to reach it from other machines pass --addr 0.0.0.0:7878 or set NORNIR_SERVER_ADDR=0.0.0.0:7878, then restart. Symptom of loopback-only: nornir viz --server http://<host>:7878 → Connection refused (os error 111) while http://127.0.0.1:7878 works. (Historical note: before the flag-precedence fix the env was read unconditionally, so a --addr in ExecStart was silently ignored when the unit also exported the env — set the env in that case.)
• --token / NORNIR_SERVER_TOKEN — required, ≥16 chars; the bearer every RPC must carry. If neither is set, falls back to user-readable <home>/.nornir/token.

Add a workspace at runtime (this is how the server learns anything)

The server is launched empty; you populate it while it runs:

# Against a running server (NORNIR_SERVER + token set), register a monitored
# workspace from its descriptor. The descriptor's TOML is stored IN the registry
# row — there is no `descriptors/*.toml` sibling file the server depends on.
export NORNIR_SERVER=http://127.0.0.1:7878
export NORNIR_SERVER_TOKEN=<the-secret>

# `--descriptor` is EITHER a git ssh URL (the server clones the workspace repo and
# reads its nornir-workspace.toml) OR a local path to an already-checked-out repo
# (its dir, or the .toml itself). Either way the content lands in the redb row.
nornir workspace register nordisk \
  --descriptor git@codeberg.org:nordisk/workspace_nordisk.git \
  --monitored --poll 60s

# …or from a local checkout:
nornir workspace register njord \
  --descriptor /path/to/workspace_njord/nornir-workspace.toml \
  --monitored --poll 60s

What happens:

• --descriptor resolves a git ssh URL (clone the workspace repo) or a local path; the resolved TOML is stored IN the redb row — so the registry, not a file, is the source of truth, and a re-add re-populates from the row's git URLs.
• Workspaces.Register is an idempotent upsert. Re-registering the same name updates its settings (e.g. --poll) in place — it never duplicates.
• The descriptor content is stored in the redb row (descriptor_content, the raw TOML). The registry is self-contained: members (name + git URL) and the deep_scan flag both derive from the stored content, so a registry wipe is total and a re-add re-populates from the row's git URLs alone. (Thin clients that can't hand the server a readable path ship the content directly; the server persists it the same way.)
• Registering is EAGER by default: the call clones the members, knowledge-scans them, builds the code index, and snapshots the warehouse at once — the populated members + new snapshot come back from the call, no waiting for a poll tick. Pass --lazy to defer population to the next poll tick (record-only).
• No restart. The freshly-registered workspace is served immediately and shows up in nornir workspace ls and the viz top-bar picker.

After register, the server's consumer loop keeps it current: on each tick (fixed 60s default) it fetches each monitored member's git HEAD (the poll field you set at register is that fetch's per-workspace cadence) and consumes every commit a member is behind into the workspace's code room — see The code room.

An empty registry is fine. A server with zero workspaces is healthy, not broken — it is an authenticated port waiting for you to register something. You never have to seed workspaces at launch.

The code room (Mímir v2): what the server builds from git

Since Mímir v2 the server does not "populate" a warehouse with scans. It consumes each member's git history into a code graph — the workspace's code room — one atomic release per commit:

<root>/<ws>/git/<member>/                      the server's clone (the fetch door)
<root>/<ws>/builds/mimir_v2/lake/              the event log: its OWN skade catalog
<root>/<ws>/builds/mimir_v2/clones/<member>/   the consumer's own clone
<root>/<ws>/builds/mimir_v2/target/<member>/   the nightly driver's target dir
<root>/<ws>/builds/mimir_v2/out/<member>/      driver dumps (removed once committed)
<root>/<ws>/builds/mimir_v2/open-set.bin       cache, rebuilt from the lake
<root>/<ws>/builds/mimir_v2/std/<toolchain>/   the toolchain's std dump (once per toolchain)
<root>/<ws>/builds/mimir_v2/artifacts/<member>/<sha>/<profile>/<bin>
                                               linked artifacts of a commit — put there by
                                               whoever BUILT it; the server only reads them

Every 60 s the consumer loop asks, per monitored workspace: is a member behind? (No offset in the lake, or the offset is not the commit its clone is at.) Boot, a wiped lake and a server that was away for a week are that same question. What is behind is consumed, at most 8 commits per member per tick.

• A member that was never consumed starts at its tip — one release holding the whole tree, not its history. History on request: nornir workspace populate <ws> --from <sha|YYYY-MM-DD> [--member <m>] — see History: a backfill is a background job.
• The FIRST run of a workspace also brings the universe ring in: every dependency crate the driver compiles is written once per crate@version (that is part of the first commit's release), and the toolchain's own crates — std, core, alloc and std's private dependencies — are dumped once per toolchain, before the members (a cargo check -Zbuild-std of an empty crate under the driver: minutes, once; nornir jobs shows it as std@<toolchain>). The ring is what a path from the workspace's code down to an extern "C" declaration walks through. A std dump that fails is said and does not stop the members; repopulate tries it again.
• After a member's commit landed, the feeds run for that (repo, sha), once: git_heat, sbom, dep_graph, code_index, binary, embeddings. Each writes a row in the lake's consumer_feed_runs ledger (ok / skipped with the reason / error / no_artifact). nornir workspace fetch <ws> --force (or repopulate) retries the ones that are not ok.
• embeddings does not run INSIDE the consumer's run: the run only reads the member's sources, and the embed itself — minutes per member on a CPU embedder — is a job of its own on the workspace's background lane, holding nothing. A commit never waits for a vector.
• Readers never wait for it: a query opens a room on the last release that landed, also while a cargo check of the next commit is running.
• A member's stream is the first-parent history of ONE branch, and that branch is NAMED, never guessed: branch = "…" for the member in the workspace descriptor, else the branch its clone under git/ is on, else the default branch of its remote (the one the clone was made on). A clone on a detached HEAD with none of the three — or a named branch the clone does not have — ends the member's turn as no_branch with the sentence that says what to set; it is not a failed run, and the other members are consumed.

History: a backfill is a background job

nornir workspace populate <ws> --from <sha|YYYY-MM-DD> [--member <m>]   # over --server
#   ✓ znippy-zoomies: backfill from 8242a30f1c — queued, 20 commit(s) to walk, at seq 1 @ 66a2891d0e
#   ✓ `ws`: 1 backfill(s) queued on the server — one commit per turn, in the background
nornir workspace repopulate <ws>
#   · in flight: znippy-zoomies: backfill from 8242a30f1c — 7 of 20 commit(s), at seq 8 @ 5c2033e9aa

The call returns at once. What it left is ONE row in the lake (backfill_runs, requested), and the server walks it on the workspace's background lane:

• one commit per turn, each turn its own hold of the workspace's gate, and between two turns whoever waits for the gate goes first — a fetch, the sweep consuming another member's tip. Nothing waits for a backfill longer than the one commit it is on;
• one at a time per workspace (a second member's backfill queues behind), one open backfill per member (a second request is refused and says where the first is);
• the member belongs to its backfill until it is done: the sweep leaves it out. A commit pushed to it meanwhile is the new end of the same walk and is consumed after the commits before it;
• a restart resumes it from the lake: the request and the member's offset are all the state there is — the next tick queues the walk again and it goes on with the commit after the offset. A turn that fails leaves the request open; it is queued again after 10 minutes;
• for a member that already has an offset this replays from that commit: its ordinals keep counting, nothing is deleted — and until the walk is back at the tip a reader sees the member at a commit of the past.

Progress: the unit's own row in nornir jobs (workspace_populate, backfill:<member>: commits done / total, the commit it is on; one child run per turn), and repopulate's in flight lines, read from the lake.

The request rides Telemetry.SubmitJobs — a queued workspace_populate job row with {"backfill": {"from": …, "member": …}} — because the workspace RPCs have no field for from (the proto is edda's nornir-wire). Without a server (nornir workspace populate --from on the box, server stopped) the walk runs inline, as before.

The binary hook: linked artifacts are READ, never built

The binary half of the graph (linked symbols, sizes, monomorphised instances — what generic_cost and binary_drift read) comes from the artifacts of a commit. The server does not build them. After a member's commit landed the binary hook looks in

<root>/<ws>/builds/mimir_v2/artifacts/<member>/<sha>/<profile>/<bin>
<root>/<ws>/builds/mimir_v2/artifacts/<member>/<sha>/<profile>/deps/   (optional: exact crate ids)

— a cargo target dir of exactly that commit, e.g. what cargo build --release --target-dir <root>/<ws>/builds/mimir_v2/artifacts/<member>/<sha> leaves. The sha in the path is the claim that the artifact is of that commit. Every executable there that is a bin the workspace declares is joined against the lake's own source rows of the commit (+ the ring) and written as one release of the binary stream.

Nothing there: the ledger row is no_artifact and names the directory, the binary lenses say the same in their evidence, and the hook looks again whenever the consumer asks what is behind — put the artifacts there later and the same commit gets its binary half, no new commit needed. Artifacts of an EARLIER commit stay in the room until newer ones arrive (the lenses say that the two halves are of two commits).

The driver — and what "driver not available" means

The graph comes from a rustc driver on ONE pinned nightly (crates/mimir-driver/rust-toolchain.toml). The stable workspace never builds it; a box that should consume needs the pair cargo-mimir-dump + mimir-driver and that toolchain with rustc-dev.

The server probes the driver at boot and before every run: the binaries, the toolchain, its compiler libraries, and the driver itself started against it. When that fails nothing is consumed and nothing pretends: the boot log says NOT CONSUMING [driver_not_available]: … with the step, nornir workspace repopulate <ws> fails with the same sentence, Workspaces.Fetch carries it in errors, readers answer not_consumed. The feeds ride the consumer's offsets, so they do not run either.

Build it on a dev box: cd crates/mimir-driver && cargo build --release --bins (rustup installs the pinned nightly from the toolchain file). For a server box: podman build -t nordisk-mimir-driver -f containers/mimir-driver/Containerfile . from the repo root — the image holds the pinned nightly (rustc-dev, rust-src), a linker and the two binaries built against it, and asserts in its own build that the driver starts. Run it by digest.

Asking what state it is in

nornir workspace repopulate <ws>            # over --server / NORNIR_SERVER
#   ✓ behind, consuming: skade: seq 41 @ 0b07d30f37 → head 9c1e22aa01 (ok)
#   · in flight: znippy seq 12 @ 77aa01bc9d (41 s, 3 behind)
#   ✓ repopulate `nordisk` — lake 3f9a1c2e · skade@41 0b07d30f37 · znippy@11 … — 1 member(s) behind, 1 in flight
nornir workspace repopulate <ws> --purge    # also drops the orphaned v1 tables (once)
nornir workspace ls --status                # red when the consumer could not consume a member
nornir jobs                                 # one row per (member, commit) and per feed run;
                                            # a backfill's unit with its progress; the std dump

The registry's current_snapshot is the lake's stamp: member@seq,….

One process holds a lake

A lake is a single-process skade catalog (a redb file lock). While the server runs, no other process can open it — not the fat CLI, not an embedded MCP: they answer lake_unavailable and are told to go through the server's RPCs. There is no copy-aside read of a held lake. Inside the server any number of readers share the consumer's handle.

A new schema: the lake is evolved in place

The lake is derived from git, but it is not thrown away for a new column. When a new build ADDS nullable columns to the graph's schema (the usual mimir-schema bump), the server evolves the lake when it opens it: one metadata commit per table adds the columns (Iceberg schema evolution, no data file is rewritten) and moves the table's mimir.schema stamp. Nothing to do, nothing is lost:

• rows written before read the new columns as null (= not known yet);
• the next consumed commit of a member dumps every unit of that member again once (the driver's kept dumps were taken at the old schema), and writes an event only for a row the producer now says something new about — a row whose new columns stay null is not written again;
• until that commit the new columns are null for that member. The dependency ring (@universe) is dumped once per crate version and keeps null there.

Any OTHER difference is still refused on open — a column this build does not have (dropped or renamed, or the lake was written by a NEWER build), a column of another type, another sort order — and the server says what to do:

code room of `nordisk`: this lake was written with another schema: mimir_v2.code_node_events
has […] this build does not, and lacks […] — only added nullable columns are evolved in
place; a lake that differs otherwise is not migrated: delete the lake directory and the
consumer rebuilds it from each repo's tip. Stop the server, remove
<root>/nordisk/builds/mimir_v2/lake and start it again (the clones and the driver's
target dir beside it can stay)

Build / runtime requirement: SuiteSparse:GraphBLAS 7.x

The code room's engine links the system GraphBLAS. To build anything that serves a code room: apt-get install libgraphblas-dev. To run it: libgraphblas7 (+ libgomp1) in the runtime image. The gates that link the workspace check first and fail with one line (SuiteSparse:GraphBLAS 7.x not found — apt-get install libgraphblas-dev). The thin GUI atom (nornir-ui) does not link it.

• The embeddings feed runs only where there is hardware for it: an NVIDIA GPU, or an AMD GPU that holds the model in its DEDICATED VRAM (an on-chip AMD GPU shares the CPU's memory and does not count — the selector of nornir vector doctor). On any other box its ledger row is skipped and says so: an embed on the CPU is minutes per member with the workspace gate held, and nobody asked for it. Two ways to get vectors there anyway: nornir workspace embed <ws> (now, on what the box has), or the pin the row names — NORNIR_EMBED_BACKEND=ort-cpu, tract-cpu where there is no onnxruntime — on the server (the feed then embeds on the CPU).

Use the CLI against it (client mode)

Watch it in the viz — one line, from any machine:

nornir viz --server http://<host>:7878 --token <the-secret>

--server makes the viz a thin client (shows all the server's workspaces). The token also resolves from $NORNIR_SERVER_TOKEN or a user-readable ~/.nornir/token — drop the secret there once and bare nornir viz auto-connects. Full flow + token resolution: viz-modes.md.

For the CLI verbs: set NORNIR_SERVER (+ the same token) and any verb routes to the server:

export NORNIR_SERVER=http://127.0.0.1:7878
export NORNIR_SERVER_TOKEN=<same secret>

nornir repos                 # the server's configured repos
nornir guard status          # the server's guard report
nornir index stats           # the server's index
nornir index search "query"
nornir knowledge query <repo> lookup <name>
nornir release gate all <repo>
nornir bench run <repo>      # runs locally, ships the BenchRun via Bench.Submit
nornir introspect symbol-lookup <bin> <pat>   # `<bin>` resolves on the SERVER
nornir docs render <repo>    # rendered on the server

Unset NORNIR_SERVER and the exact same commands run embedded against the local warehouse.

What to expect (the lock rule, by design)

• Server up + CLI client (NORNIR_SERVER set): every verb works over gRPC.
• Server up + CLI embedded (NORNIR_SERVER unset) for a warehouse verb: fails with redb … Database already open. Cannot acquire lock — correct: the server owns the lock. Config-only verbs (repos) still work.
• Server down + CLI embedded: everything works; the CLI owns the lock.
• Wrong/absent token → Unauthenticated: missing or invalid bearer token.

Coverage

Every everyday verb is wired CLI→server: repos, guard, bench run, index {stats,search,snapshot,upload}, knowledge query, release gate, introspect {symbols,lookup,defined-in,callers,callees,path}, docs {init,render,check,history}. index upload streams a locally-built index up to the server (client-streaming Index.UploadSnapshot), the live-refresh path for a server's index — the server reconstructs the blobs under its write lock and snapshots them. Notes: introspect's binary path and docs rendering resolve on the server's filesystem.

Install as a systemd service

Two scopes — pick by who runs it

The lifecycle is three verbs — add (install), update (swap in a new binary), remove (tear down) — across two scopes, auto-picked by privilege (override with --scope user|system):

One law for both: there is no /etc/nornir and no env file. The bearer token is the 0600 <home>/.nornir/token (read by the server and the CLI client); the bind addr is an inline Environment= line in the unit.

No monitor/poll at install time (done). The --monitor/--poll CLI flags are removed, and the systemd-unit generator (render_server_unit) writes no NORNIR_MONITOR/NORNIR_POLL Environment= lines — unit tests assert the generated unit carries neither. The unit's only env lines are HOME and NORNIR_SERVER_ADDR (the server reads just --token/NORNIR_SERVER_TOKEN and --addr/NORNIR_SERVER_ADDR). Workspaces are added at runtime with nornir workspace register — never seeded from the unit. (--config remains an optional explicit pin that, only when passed, emits a single NORNIR_CONFIG line. The unit renderer still accepts a warehouse pin, but no install flag passes one — both live install paths hand it None, so no NORNIR_WAREHOUSE line is ever written.)

TL;DR — developer (user scope, no root)

cargo install nornir --features viz                  # → nornir, nornir-server, nornir-ui
nornir install add                                   # NO sudo → user scope (token + unit only)
systemctl --user enable --now nornir-server          # start now (serves 0 workspaces yet)
loginctl enable-linger "$USER"                        # (optional) start at boot, no login session

# Add the workspace at RUNTIME (the server learns it; persists in the registry):
export NORNIR_SERVER=http://127.0.0.1:7878
export NORNIR_SERVER_TOKEN="$(cat ~/.nornir/token)"
nornir workspace register njord \
  --descriptor /path/to/workspace_njord/nornir-workspace.toml --monitored --poll 60s

# Same-box client just works — token + endpoint live in your ~/.nornir, no sync:
nornir --server 127.0.0.1:7878 --workspace njord repos
# Stop = fat again; start = served. One warehouse (~/.nornir/warehouse), flip with:
systemctl --user stop nornir-server
nornir install update                                # restart after `cargo install` (no sudo)
nornir install remove                                # UNINSTALL: unit + ~/.nornir DATA erased; ssh key + token KEPT

TL;DR — server box (system scope, root)

cargo install nornir --features viz
sudo nornir install add \                            # sudo → system scope (the `nornir` user)
  --addr 0.0.0.0:7878                                # 0.0.0.0 = reachable over the tailnet/LAN
sudo systemctl enable --now nornir-server            # serves 0 workspaces until you register

# Register the workspace at RUNTIME against the running server (token from nornir's home):
export NORNIR_SERVER=http://127.0.0.1:7878
export NORNIR_SERVER_TOKEN="$(sudo cat /home/nornir/.nornir/token)"
nornir workspace register njord \
  --descriptor /abs/path/workspace_njord/nornir-workspace.toml --monitored --poll 60s
# (descriptor must be readable by the SERVER process, or ship its content via thin mode)

# UPDATE on a new release. Run YOUR build BY FULL PATH: bare `sudo nornir` hits
# sudo's secure_path → the already-installed /usr/local/bin/nornir (a no-op).
cargo install nornir --features viz
sudo "$(command -v nornir)" install update           # copies your bins → /usr/local/bin, restarts

# The token is in the `nornir` user's home, not /etc. Read it / hand it to clients:
sudo cat /home/nornir/.nornir/token
# A client on THIS box has a DIFFERENT home, so copy the token across once:
sudo cat /home/nornir/.nornir/token > ~/.nornir/token

sudo nornir install remove                           # UNINSTALL: unit + bins + DATA erased; ssh key + token KEPT

install update never touches a registry. It does not run cargo install and does not fetch from crates.io / holger. It is a pure local binary swap: you build/cargo install the new binaries, then (system scope) update copies them onto the box and restarts; (user scope) cargo install already replaced ~/.cargo/bin and update just restarts. Producing the new binary is your step; update only moves it the last mile.

nornir install add writes a hardened nornir-server.service unit. In system scope (root) it also creates the nornir system user; in user scope it writes ~/.config/systemd/user/nornir-server.service and needs no root:

# Install the service — it needs nothing but (optionally) a bind addr. The server
# starts EMPTY and you register workspaces against it at runtime (see above).
sudo nornir install add --addr 0.0.0.0:7878

Re-running install add is idempotent (refreshes binaries, preserves token + key, restarts). Use it when you change --addr; use install update when you only shipped a new binary.

No workspace flags are needed (or used) at install time. The server is launched with just bind + auth and learns workspaces at runtime via nornir workspace register (which persists them in the registry across restarts). View a registered workspace with nornir viz --workspace njord (no separate nornir-ui invocation needed; it's a thin wrapper).

No workspace flags at install time (done, see above): install add has no --monitor/--poll flags and writes no NORNIR_MONITOR/NORNIR_POLL env lines. Register workspaces at runtime instead.

The installer copies the binaries onto the system — nornir-server (and the nornir CLI + nornir-ui viz when present) → /usr/local/bin/ — and the unit's ExecStart points there. The service never execs from the build tree (a removable drive the nornir user can't execute from gives 203/EXEC). Re-run the install after a rebuild to refresh the installed binaries (atomic replace).

This binary-copy is system scope only. In user scope the service runs as you and execs your ~/.cargo/bin/nornir-server in place — nothing is copied, so update is just a restart after cargo install.

In system scope the service runs as the nornir system user, whose home is /home/nornir. That user can't read arbitrary paths (e.g. a removable drive mounted 0700 for your login user) — so do not change --user. What the install sets up for that user:

• creates the nornir system user (idempotent) and its home /home/nornir,
• creates the registry/workspaces root /home/nornir/.nornir/workspaces (home-derived) — where the registry redb and each workspace's git/ + builds/ live once you register them,
• writes the 0600 /home/nornir/.nornir/token, chowns everything to nornir, and the unit gets WorkingDirectory=/home/nornir + Environment=HOME=/home/nornir.

There is no descriptor staging anymore. You do not drop a descriptor file into a /home/nornir/descriptors/ dir — that staging model is gone. Instead, register the workspace at runtime with nornir workspace register <name> --descriptor <toml>; the server reads the descriptor at register time and stores its content inside the redb registry row (descriptor_content), so the registry is fully self-contained (a re-add re-populates from the row's git URLs, no sibling file needed). The descriptor source must be readable by the server process at register time, or — for a thin client whose path the server can't see — its content is shipped over the wire and persisted the same way.

Install behavior: install add has no --monitor/--poll flags and writes no NORNIR_MONITOR/NORNIR_POLL env lines — workspaces come from the registry (added at runtime), and the poll cadence is per-workspace. (--root is deprecated/ignored — the registry root is home-derived as <home>/.nornir/workspaces; relocate by setting the service user's $HOME.) Re-running install add preserves token + key and restarts. (Shipped only a new binary? install update is the lighter path — no flags.)

A private monitored repo is cloned/fetched over SSH with the server's deploy key — use a git@host:owner/repo.git (or ssh://) URL in the descriptor and register sudo nornir key show as a deploy key on the repo. The SSH transport is pure-Rust (russh runs git-upload-pack; gix drives the wire protocol + pack indexing over it — no ssh/git subprocess). In server mode the transport is effectively SSH-only: the installed server always holds a deploy key, so it tries SSH FIRST for every member — an https:// descriptor URL is coerced to its scp-like SSH form (git@host:owner/repo.git) before each fetch (gitio::clone_or_fetch). So every monitored member, public or private, needs the server's key (nornir key show) registered as a deploy key, or it won't sync — a public https:// URL does not exempt it. (The keyless HTTPS path only applies to the FAT/no-key client, which has no key to try first.) (SSH push / git-receive-pack is not implemented yet — monitoring only fetches.)

It (system scope): creates the nornir system user (idempotent), writes the 0600 /home/nornir/.nornir/token (a generated bearer secret, owned by nornir), and writes /etc/systemd/system/nornir-server.service (User=nornir, inline Environment= lines, NoNewPrivileges, Restart=on-failure), then runs systemctl daemon-reload. There is no /etc/nornir and no env file — the token is the file, the rest are Environment= lines in the unit. Then:

sudo systemctl enable --now nornir-server     # start at boot + now
journalctl -u nornir-server -f                # follow logs

Want the service as YOU instead of the nornir user? Drop sudo (user scope) — see the developer TL;DR above. --scope user|system forces either explicitly.

Remove:

sudo nornir install remove            # system scope. NO FLAGS NEEDED: stop+disable,
                                      #   remove the unit AND the /usr/local/bin binaries
                                      #   AND erase the service account's data — but SPARE
                                      #   its .ssh deploy key, the token, and the `nornir`
                                      #   account that owns them
nornir install remove                 # user scope (no sudo): remove the --user unit and
                                      #   erase ~/.nornir's data, KEEPING ssh/ + token

sudo nornir install remove --wipe-identity   # ⚠ the one opt-in: ALSO destroys the deploy
                                      #   key, the token, the home and the `nornir` user

(Everything the service writes lives under its own home, home-derived from HOME=/home/nornir: the warehouse at /home/nornir/.nornir/warehouse, the workspace registry at /home/nornir/.nornir/workspaces, and the Tantivy index cache under /home/nornir/.nornir/cache — so the service writes only inside /home/nornir.)

It also creates the server's git SSH identity: an ed25519 keypair at /home/nornir/.ssh/id_ed25519 (mode 600, owned by the nornir user), generated in pure Rust (the ssh-key crate — no ssh-keygen subprocess). Idempotent and preserved across re-installs like the token. This is the identity the server uses to fetch/push member repos in server-monitored mode. The state home /home/nornir doubles as the workspace root (<root>/workspaces/<name>/{git,builds} — see mode-matrix.md).

Print the public key to register as a deploy key on each monitored repo:

nornir key show       # public key (paste as deploy key)
nornir key path       # path to the .pub
nornir key generate   # for non-systemd/dev: writes ~/.nornir/ssh, never ~/.ssh

THE UNINSTALL LAW — always kill the data, always keep the key

Suite-wide and non-negotiable (workspace_nordisk/.nornir/suite-uninstall-law.md):

uninstall ALWAYS kills the DATA and ALWAYS keeps the SSH KEYS. Not behind a flag. Not opt-in. Not --purge. Plain install remove does both halves, every time.

So nornir install remove — with no flags at all — removes the unit, the installed binaries (system scope) and everything under the state root: warehouses, workspace registry, caches, funnel, jobs. A reinstall gets a genuine clean slate. What SURVIVES, byte-identical, is the install identity: the ssh/ (system scope: .ssh) deploy key that the git remotes have already registered, and the server bearer token. System scope also keeps the nornir account, because that account owns the spared key — an uninstall must never orphan or destroy the key as a side effect of erasing data.

Destroying the identity is the one explicit opt-in:

After --wipe-identity a reinstall mints a brand-new keypair, which you must register again as a deploy key on every monitored remote before the server can fetch or push.

Two more rules the law spells out, both of them from real incidents:

• Scope-aware state root. The root is resolved, never assumed — the invoking user's $HOME/.nornir under user scope, the service account's passwd home under system scope (printed as home: … (resolved from passwd, not assumed)). Two sibling apps erased a hardcoded /home/<app> regardless of scope, so on a per-user install they wiped nothing and printed success anyway.
• Honest reporting. The output names what was erased and that the key was KEPT. An uninstall that removed nothing prints erased: NOTHING REMOVED, never a success line.

Each individual delete goes through skidbladnir's guarded primitive (skidbladnir::systemd::guarded_remove_dir_all, the authority behind purge_service_data_keep_keys), which refuses any path containing a .ssh component, a real user's home, a system directory, or a symlink pointing out of the tree, and pins the delete to the resolved state root. Proof: src/bin/nornir.rs::purge_law_tests — a fake data home is erased by a FLAGLESS uninstall and the deploy key must come out byte-identical.

Update an existing install by simply re-running the same command — it's overwrite/idempotent: keeps the nornir user, preserves the existing token (so configured clients keep working), rewrites the unit (bind addr + binary path), daemon-reloads, and try-restarts the service if it's running (so the update takes effect; a no-op when stopped). Output says updated (and token preserved) vs installed.

Where compute happens (today) — important

No source code crosses the wire (a remote client never uploads its tree). The server operates on repos on its own filesystem: path= local, or git= which it clones/fetches itself — pure-Rust, SSH-only in practice for a server (it holds a deploy key, so it tries SSH first for every remote, https:// URLs included; see "Git transport" in design.md). Facts are Iceberg rows keyed by git SHA read from those checkouts; the warehouse never stores source.

Consequence: heavy indexing/embedding happens where the source is — on the server for monitored workspaces (it cloned them), or client/local for the embedded path. A live server's warehouse is kept current by the monitor poll loop, and can also be fed via Bench.Submit / Index.UploadSnapshot. Paths you pass in client mode (introspect <bin>, docs) resolve on the server's filesystem, not yours.

Done since: network git-clone (HTTPS + SSH) — monitored workspaces already clone + produce server-side. Future (plan.md §"Nornir-as-a-service"): expose it as on-demand, client-triggered build repo@<sha> producer RPCs (Index.Build, Vector.Index, Knowledge.Scan, Introspect.Persist) so a thin client can request a build of any SHA without local sources. Bench stays client-side (real hardware).

Monitor a git repo + watch it live (server-monitored + viz)

A server can monitor git repos and republish their warehouse on change, and the nornir-ui viz can watch that warehouse live over gRPC — all Rust binaries, no scripts.

# Server: launch it empty (token + addr only). It seeds NO workspaces from env.
NORNIR_SERVER_TOKEN=<≥16 chars>  nornir-server --addr 127.0.0.1:7878

# Add the monitored workspace at RUNTIME (descriptor = a server-readable
# nornir-workspace.toml or its members' git URLs). Its content is stored in the
# registry row; the per-workspace poll loop fetches + republishes on git change.
export NORNIR_SERVER=http://127.0.0.1:7878
export NORNIR_SERVER_TOKEN=<same>
nornir workspace register njord \
  --descriptor /path/to/workspace_njord/nornir-workspace.toml --monitored --poll 60s

# Viz: pick the workspace from the top-bar dropdown (it lists the server's
# registered workspaces and switches live); auto-refresh is on by default.
NORNIR_SERVER=http://127.0.0.1:7878 NORNIR_SERVER_TOKEN=<same>  nornir viz

How it fits together:

• Each monitored workspace lives at <home>/.nornir/workspaces/<name>/{git,builds} (home-derived; no path env-var). The server serves its builds/ warehouse as a gRPC workspace, and the poll loop republishes through that same warehouse handle (redb is single-writer), so reads (viz) and writes (republish) share one owner — no lock war.
• The viz sends nornir-workspace: <name> (from the top-bar workspace picker; switch workspaces live) on Viz.Timeline / Warehouse.{Tables,Scan}, and auto-refreshes (the live toggle, ~5s) — the 🗄 Warehouse tab shows the tables update as the server republishes.
• Inspect from the CLI (embedded or NORNIR_SERVER-routed): nornir workspace {ls,show <name>,fetch <name>}.
• Registering at runtime is EAGER by default. nornir workspace register <name> --monitored … (and the Workspaces.Register RPC / MCP workspace_register tool behind it) records the workspace AND clones its members + builds the warehouse at once — the populated members + snapshot come back from the call, no waiting for a poll tick. The poll loop then handles UPDATES on a git HEAD change. Pass --lazy (CLI) / lazy: true (RPC + MCP) to defer population to the poll loop (the old record-only behaviour). (There is no boot-time auto-registration — the server seeds nothing from env/flags; every workspace is added at runtime via register, and persisted workspaces are re-served from the registry on the next boot.)
• Deep-scan (opt-in): deep_scan = true under [workspace] in the descriptor makes each republish also knowledge-scan the members' entire transitive dependency closure — cargo fetch materializes every dep source, cargo metadata enumerates the closure, and znippy's gatling engine fans the per-crate syn scans across all cores (src/deepscan.rs). Rows land under repo deps (one crate entry per dependency in the viz 🕸 Call Graph and the MCP knowledge_* tools). Needs cargo on the service PATH — drop a toolchain cargo into /opt/nornir/bin and add a systemd drop-in Environment=PATH=/opt/nornir/bin:… (this also un-skips the dep-graph step).

Worked example: [../../workspace_njord/MONITOR-DEMO.md] (monitors the private Njord repo over SSH). See mode-matrix.md for the full self-sync loop.

Embed the holger crates cache in the server (the marriage)

The daemon can run holger's /cache (crates.io mirror) + /sparring (release rehearsal registry) in-process — no separate holger-server. It's linked in via the default-on embed-holger feature; turn it on at boot with an env var:

# in the unit, or your shell before `nornir serve`:
NORNIR_HOLGER=1                    # enable it (presence of NORNIR_HOLGER_HTTP also enables)
NORNIR_HOLGER_HTTP=127.0.0.1:18464 # sparse HTTP cargo talks to (default)
NORNIR_HOLGER_GRPC=127.0.0.1:18443 # control plane (default)
NORNIR_HOLGER_DATA=<dir>           # znippy archives (default <home>/.nornir/holger-cache)

The listeners run on the server's own tokio runtime (same process, owned by the service user). Non-fatal: if holger can't bind, the nornir server still comes up. The everyday foreground equivalent is nornir cache up. See nornir-for-dummies.md.

Not yet

The three deployment modes (fat / server-monitored / thin) and the verb×mode test matrix that the items below feed into are specified in mode-matrix.md. Server-monitored = the server playing thin client against itself, fed by a git poll loop; the single server-user is nornir.

• nornir server {stop,status} — manage via systemctl for now.
• Other install targets (container, Redfish/IPMI, Terraform) — plan.md.

Now works (was "not yet"): multiple monitored workspaces from one server, selected per-request by the nornir-workspace header (CLI/MCP) or the viz's top-bar picker — verified with njord/holger/knut/nornir-catalog live. And pure-Rust SSH clone/fetch for private repos (see "Git transport" above).