.nornir/gunnar-quickstart.md · nordisk/gunnar

gunnar — fast setup

New here? Start here below: clone the source, build the server with one cargo line, run it on your own computer and push to it. Nothing else is needed for that. The five numbered lists after it are the appliance route (a bootable medium, a twin, the console, the metered medium) — the advanced path.

Addresses are examples (RFC 5737): 192.0.2.10 is the main, 198.51.100.10 the twin, 203.0.113.10 the browse replica. Substitute yours.

Every command below was run on one laptop before it was written down: §1 and §3 against three appliances installed from the medium §4 builds and booted in KVM (cargo xtask install-test pair), §2 against the console in podman, pointed at one of them.

Start here — clone, build, run

The first way, and the simplest: Gunnar is a Rust program. You build it from its source and run it on any Linux computer — no installer, no medium, no virtual machine.

What you need: a Linux computer (x86-64), about 10 GB of free disk and 20–40 minutes for the first build.

 1  sudo apt install git build-essential cmake pkg-config protobuf-compiler

 2  curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
    . "$HOME/.cargo/env"

 3  mkdir ~/nordisk && cd ~/nordisk

 4  export GIT_LFS_SKIP_SMUDGE=1
    git clone ssh://git.gunnar.rs:2222/nordisk/gunnar
    git clone ssh://git.gunnar.rs:2222/nordisk/gunnar-ui
    git clone ssh://git.gunnar.rs:2222/nordisk/monetize

 5  git clone ssh://git.gunnar.rs:2222/nordisk/edda
    git clone ssh://git.gunnar.rs:2222/nordisk/Skidbladnir
    git clone ssh://git.gunnar.rs:2222/nordisk/znippy
    git clone ssh://git.gunnar.rs:2222/nordisk/znippy-zoomies
    git clone ssh://git.gunnar.rs:2222/mikronova/draupnir

 6  cd gunnar
    cargo build --release -p gunnar-server -p gunnar-cli

 7  ./target/release/gunnar-server serve

Leave 7 running. Open a second terminal:

 8  cd ~/nordisk/gunnar && mkdir -p ~/gunnar

 9  ssh-keygen -t ed25519 -N "" -f ~/gunnar/admin

10  ./target/release/gunnar-cli control status --at 127.0.0.1:50051

11  ./target/release/gunnar-cli control bootstrap alice --key @$HOME/gunnar/admin.pub \
        -i ~/gunnar/admin --at 127.0.0.1:50051

12  ./target/release/gunnar-cli control repo create alice/hello \
        -i ~/gunnar/admin --at 127.0.0.1:50051

13  export GIT_SSH_COMMAND="ssh -i ~/gunnar/admin -o IdentitiesOnly=yes"

14  cd ~ && git clone ssh://git@127.0.0.1:2222/alice/hello.git

15  cd hello && git commit --allow-empty -m first && git push origin HEAD:refs/heads/main

What the lines mean, one clause each:

1 installs the tools the build calls: a C compiler, cmake, and protoc (the protobuf compiler that turns Gunnar's .proto files into Rust). The line is for Debian and Ubuntu; on Fedora it is sudo dnf install git gcc gcc-c++ cmake pkgconf protobuf-compiler.
2 installs Rust (rustup, the stable toolchain). Accept the defaults.
GIT_LFS_SKIP_SMUDGE=1 (in 4): the rendered PDFs under docs/ are Git LFS objects, and the build needs none of them. The clones take their small pointer files instead, so a clone works whether or not git-lfs is installed and whatever the server holds of them.
4 and 5 put eight checkouts side by side in one directory. Gunnar's workspace names the other seven by path (../edda/…), so they must sit beside it with exactly these names. They are compiled from source, not downloaded as crates. cargo xtask one-clone (inside gunnar/) names any sibling that is missing.
6 is the whole build. It writes target/release/gunnar-server (the server) and target/release/gunnar-cli (the client you talk to it with). The first build compiles about 800 crates; later builds are quick.
7 starts the server with its defaults: git over SSH on port 2222 (every interface), the control plane on 127.0.0.1:50051 (this computer only), and the data in ~/.local/share/gunnar (--data-root DIR puts it elsewhere). Stop it with Ctrl-C; start it again with the same line, and your data is still there.
10 asks the new server who it belongs to. It answers bootstrap OPEN until 11.
11 makes alice the administrator, with the key from 9. It works once.
12–15 create a repository, clone it and push the first commit to it.
To let other computers push, they need port 2222 of this one. The control plane stays on 127.0.0.1. From another computer, reach it with ssh -L 50051:127.0.0.1:50051 you@this-computer.

When this works, the numbered lists below turn the same server into an appliance of its own: a bootable medium (§4), a twin that takes over (§3), and the web console (§2).

1. One gunnar server

 1  cargo xtask iso-installer --variant gunnar-installer-open \
        --opt dhcp=on --opt wireguard=off

 2  cargo xtask install-test pair --members main --keep

 3  ssh-keygen -t ed25519 -N "" -f ~/gunnar/admin

 4  gunnar-cli control status --at 192.0.2.10:50051

 5  gunnar-cli control bootstrap alice --key @~/gunnar/admin.pub \
        -i ~/gunnar/admin --at 192.0.2.10:50051

 6  gunnar-cli control user add bob --key @~/gunnar/bob.pub \
        -i ~/gunnar/admin --at 192.0.2.10:50051

 7  gunnar-cli control repo create alice/hello \
        -i ~/gunnar/admin --at 192.0.2.10:50051

 8  export GIT_SSH_COMMAND="ssh -i ~/gunnar/admin -o IdentitiesOnly=yes"

 9  git clone ssh://git@192.0.2.10:2222/alice/hello.git

10  cd hello && git commit --allow-empty -m first && git push origin HEAD:refs/heads/main

What the lines mean, one clause each:

1 writes .iso-out/gunnar-installer-open.iso. Boot it on the machine (dd it to a stick, or attach it as boot media) and the machine is a gunnar appliance; 2 does that in KVM on this box instead and prints the forwarded ports. On real hardware, drop 2 and use the machine's own address. Use --build kernel instead of --opt wireguard=off when the pinned kernel toolchain image is on the build box — see §4.
4 needs no credential and is the one question an unclaimed server answers. It says bootstrap OPEN until 5, and closed for ever after.
5 works once, ever. It prints enrolled alice and role gunnar-admin.
7 creates sha256 repositories. Stock git init is sha1 — clone the empty repository as in 9 rather than pushing an existing sha1 one, or create it with --hash sha1.
--at has an environment twin: export GUNNAR_CONTROL_ADDR=192.0.2.10:50051 and drop it from every line below.
The control plane binds loopback by default. The appliance medium binds it to the private interface (--grpc-bind eth1:50051); on a host you configured yourself, reach it with ssh -L 50051:127.0.0.1:50051 192.0.2.10.

2. The console beside it

 1  cd ../gunnar-ui
 2  cargo build --release -p gunnar-ui-server --target x86_64-unknown-linux-musl
 3  (cd crates/gunnar-ui-web && trunk build --release)

 4  ssh-keygen -t ed25519 -N "" -f ~/gunnar/ui-id
 5  ssh-keygen -t ed25519 -N "" -f ~/gunnar/ui-guest

 6  GUNNAR_UI_CONTROL=192.0.2.10:50051 GUNNAR_UI_IDENTITY=~/gunnar/ui-id \
    GUNNAR_UI_STATE=~/gunnar/ui-state.redb \
      target/x86_64-unknown-linux-musl/release/gunnar-ui-server \
      enroll --admin-key ~/gunnar/admin --name gunnar-ui

 7  gunnar-cli control user add gunnar-ui-guest --key @~/gunnar/ui-guest.pub \
        --kind machine -i ~/gunnar/admin --at 192.0.2.10:50051

 8  mkdir -p ~/gunnar/console && cd ~/gunnar/console
 9  cp ~/git/gunnar-ui/target/x86_64-unknown-linux-musl/release/gunnar-ui-server .
10  cp -r ~/git/gunnar-ui/crates/gunnar-ui-web/dist .
11  printf 'FROM scratch\nCOPY gunnar-ui-server /gunnar-ui-server\nCOPY dist /dist\nENTRYPOINT ["/gunnar-ui-server"]\n' > Containerfile
12  podman build -t gunnar-console:local .

13  export TOKEN=$(openssl rand -hex 16)
14  podman run -d --name gunnar-console --network=host \
        -v ~/gunnar:/keys:Z \
        -e GUNNAR_UI_CONTROL=192.0.2.10:50051 \
        -e GUNNAR_UI_IDENTITY=/keys/ui-id \
        -e GUNNAR_UI_GUEST_IDENTITY=/keys/ui-guest \
        -e GUNNAR_UI_WEBAUTHN_RP_ID=localhost \
        -e GUNNAR_UI_WEBAUTHN_ORIGIN=http://localhost:3210 \
        -e GUNNAR_UI_ADMIN_TOKEN=$TOKEN \
        -e GUNNAR_UI_BIND=127.0.0.1:3210 \
        -e GUNNAR_UI_DIR=/dist \
        -e GUNNAR_UI_STATE=/keys/ui-state.redb \
        -e GUNNAR_UI_GIT_URL=ssh://192.0.2.10:2222 \
        gunnar-console:local

15  curl -s http://127.0.0.1:3210/auth/status

16  curl -s -H "Authorization: Bearer $TOKEN" \
        "http://127.0.0.1:3210/auth/passkey/register/start?user=alice"

17  open http://localhost:3210/login   # finish the passkey in the browser

18  curl -s -X POST -H "Authorization: Bearer $TOKEN" \
        -H 'content-type: application/json' \
        -d '{"role":"operator","user":"bob"}' \
        http://127.0.0.1:3210/auth/passkey/invites

The six that matter, and what each is:

variable	what it takes
`GUNNAR_UI_CONTROL`	the server's control plane, `host:port`
`GUNNAR_UI_IDENTITY`	path to an ed25519 private key; enrolled by `5` as a `Machine` principal with a server-wide admin grant
`GUNNAR_UI_GUEST_IDENTITY`	a second key, enrolled with no grant, for visitors with no session. Unset means nothing is served anonymously. Never the key above
`GUNNAR_UI_WEBAUTHN_RP_ID`	a domain. An IP address is refused at start-up; in development the domain is `localhost`
`GUNNAR_UI_WEBAUTHN_ORIGIN`	the exact browser origin, scheme and port included. It is also what invite links are built from
`GUNNAR_UI_ADMIN_TOKEN`	break-glass bearer for passkey number one. It stops working the moment an admin passkey exists

6 is the enrolment: it makes three calls with your admin key (AddPrincipal{Machine} → AddKey → Grant{server-wide admin}) and prints enrolled gunnar-ui and the console's fingerprint.
11–12 produce an 18.6 MB image on scratch — the musl binary and the built page, no base image, no shell. roles for 18 are viewer, operator, admin; anything else is refused by name.
15 answers {"authenticated":false,…,"open":false,"passkey_enabled":true,…}. "open":true there means the console is running with no credential at all — which it refuses to do on any non-loopback bind.
16 returns 200 and a WebAuthn challenge. The ceremony itself needs a browser; that is 17.
Browse localhost:3210, not 127.0.0.1:3210: the RP ID has to match.
GUNNAR_UI_DIR unset gives you a console that answers its API and serves no page, with one warn line to say so. GET / is 401 until you have a session; GET /login is the page.

3. A twin, and the two flips

§2 is unchanged and applies to the pair: the console is one process pointed at one control plane, and the only thing a failover changes about it is GUNNAR_UI_CONTROL.

3a. Install the twin and start replicating

 1  cargo xtask install-test pair --members main,twin,spare --keep

 2  gunnar-cli control bootstrap alice --key @~/gunnar/admin.pub \
        -i ~/gunnar/admin --at 198.51.100.10:50051

 3  gunnar-cli control replication standby -i ~/gunnar/admin --at 198.51.100.10:50051

 4  gunnar-cli control replication add twin ssh://git@198.51.100.10:2222 \
        --control 198.51.100.10:50051 -i ~/gunnar/admin --at 192.0.2.10:50051

 5  gunnar-cli control user add primary --key '<the line step 4 printed>' \
        --kind machine -i ~/gunnar/admin --at 198.51.100.10:50051

 6  gunnar-cli control grant primary admin --namespace alice \
        -i ~/gunnar/admin --at 198.51.100.10:50051

 7  gunnar-cli control replication resync twin -i ~/gunnar/admin --at 192.0.2.10:50051

 8  gunnar-cli control replication list -i ~/gunnar/admin --at 192.0.2.10:50051

3 makes the twin read-only to people: it takes its primary's replication push and refuses a git push from anyone else with "push to the primary".
4 mints the main's own deploy key if it has none — /data/replication_key on an appliance, and replication list names the path — keeps the private half on the box for ever, and prints the public line and the two commands to run on the twin. 5 and 6 are those commands. admin, not write: the twin has to create each repository. You never place a key by hand; a box with no shell could not be handed one.
7 is not optional. Adding a standby copies nothing; replication rides the push bus, so without a backfill the twin holds only what is pushed from now on. --repo <ns>/<repo> does one; --except <ns>/<repo> (repeatable) does all but.
8 is the proof. Before 7 it reads

twin  pending 0  failures 0  last success NEVER
  NO COPY  alice/hello  [twin]

and after it, once the queue drains,

replication    1 target(s) · synced 1 · failed 0 · bus gaps 0
twin  pending 0  failures 0  last success <ms> ms  idle
  copied   alice/hello  [twin]  1 refs there

last success NEVER with pending 0 means the twin has never taken a byte and nothing is asking it to — usually 7 not run. failures climbing means 5/6 not run, or run with write instead of admin.

3b. The twin becomes the writable main

 1  gunnar-cli control replication promote -i ~/gunnar/admin --at 198.51.100.10:50051

 2  gunnar-cli control replication demote -i ~/gunnar/admin --at 192.0.2.10:50051

 3  gunnar-cli control replication standby -i ~/gunnar/admin --at 192.0.2.10:50051

 4  git remote set-url origin ssh://git@198.51.100.10:2222/alice/hello.git

 5  git push origin HEAD:refs/heads/main

1 on the twin: promoted — this box is now a writable primary and takes client writes.
2 and 3 on the old main, and they are two different things. 2 clears its standby roster — without it a recovered old primary resumes pushing its stale state and force-clobbers the new primary's writes — and says so:

demoted — 1 standby target(s) cleared; this box ships no second copy

  It STILL TAKES CLIENT WRITES. If this is a failover and another box is
  the primary now, freeze this one too or it will accept a push nobody
  replicates:

      gunnar-cli control replication standby

3 is that freeze:

standby — this box is now read-only to clients; it accepts only its primary's replication push
role           standby (read-only to clients)

A client still pointed at it is then refused by name — fatal: remote error: this box is a standby; push to the primary — instead of leaving a commit nobody replicates. demote alone leaves the box writable, and says so.

A planned hand-off is one command on the healthy old main instead of 1–3, and it does not lose a write — freeze, flush every standby, demote, then promote the named box:

    gunnar-cli control replication abdicate --promote 198.51.100.10:50051 \
        -i ~/gunnar/admin --at 192.0.2.10:50051

Then move the traffic, which is infrastructure and not gunnar's to move: DNS (or your load balancer) to 198.51.100.10, and the console's GUNNAR_UI_CONTROL=198.51.100.10:50051.

3c. The main was nuked

 1  cargo xtask iso-installer --variant gunnar-installer-open \
        --opt dhcp=on --opt wireguard=off

 2  gunnar-cli control bootstrap alice --key @~/gunnar/admin.pub \
        -i ~/gunnar/admin --at 192.0.2.10:50051

 3  gunnar-cli control replication standby -i ~/gunnar/admin --at 192.0.2.10:50051

 4  gunnar-cli control replication add rebuilt ssh://git@192.0.2.10:2222 \
        --control 192.0.2.10:50051 -i ~/gunnar/admin --at 198.51.100.10:50051

 5  gunnar-cli control user add survivor --key '<the line step 4 printed>' \
        --kind machine -i ~/gunnar/admin --at 192.0.2.10:50051

 6  gunnar-cli control grant survivor admin --namespace alice \
        -i ~/gunnar/admin --at 192.0.2.10:50051

 7  gunnar-cli control replication resync rebuilt -i ~/gunnar/admin --at 198.51.100.10:50051

 8  gunnar-cli control replication list -i ~/gunnar/admin --at 198.51.100.10:50051

Every --at 198.51.100.10:50051 line runs against the survivor, which is the primary now; every --at 192.0.2.10:50051 line runs against the fresh box. 8 ends at synced 1 · failed 0 and copied alice/hello [rebuilt], and a git clone off the rebuilt box returns the history with fsck --full --strict clean.
The surviving twin is the main now; the fresh box is its standby. That is the finished state — the pair is whole again, replicating the other way.
To put the rebuilt box back in charge afterwards, that is §3b with the addresses swapped.
Install the fresh box the same way you installed the first: 1, then boot it. The rig in 3a step 1 lays down a third appliance up front (spare) so the drill can be run without waiting for an install.

3d. A browse replica, in one line

 1  gunnar-cli control replication add public ssh://git@203.0.113.10:2222 \
        --scope public-only --control 203.0.113.10:50051 \
        -i ~/gunnar/admin --at 192.0.2.10:50051

 2  gunnar-cli control user add primary --key '<the line step 1 printed>' \
        --kind machine -i ~/gunnar/admin --at 203.0.113.10:50051

 3  gunnar-cli control grant primary admin --namespace alice \
        -i ~/gunnar/admin --at 203.0.113.10:50051

 4  gunnar-cli control replication browse -i ~/gunnar/admin --at 203.0.113.10:50051

4 comes last, and that is not a preference. A browse replica refuses every control-plane mutator, itself included:

this box is a browse replica: it holds public repositories read-only and cannot be
promoted or changed here — Access.AddPrincipal is refused; run it against the primary

so 2 and 3 are unrunnable on a box that has already been flipped. Authorise the key first, flip second. replication standby is the way back.

Public repositories only; one that goes private is deleted there, which is why --scope public-only also needs --control. It cannot be promoted — not by promote, not by abdicate --promote, not by the passive heartbeat.

4. Building gunnar

The advanced route. The plain cargo build in Start here gives you the server as a program. The lines below build the appliance medium instead: an installer image that turns a whole machine into a Gunnar server, composed by the Build Thing in a pinned container.

 1  dwarves buildthing gunnar --variant gunnar-installer

 2  dwarves buildthing gunnar --variant gunnar-installer-open --opt dhcp=on

 3  dwarves buildthing gunnar --variant gunnar-installer-metered

 4  cd ../gunnar-ui && cargo build --release -p gunnar-ui-server \
        && (cd crates/gunnar-ui-web && trunk build --release)

and the one equivalent from inside the gunnar checkout, which finds the dwarves sibling and the kernel for you:

    cargo xtask iso-installer --variant gunnar-installer-open --opt dhcp=on

Run 1–3 from the directory that holds the checkouts (dwarves resolves gunnar as a child of the working directory, then $NORDISK_ROOT/gunnar, then ../gunnar, and prints which rule it used). Add --list to any of them to resolve the variant and build nothing.
1 is the keyed appliance medium, 2 the open (trust-on-first-use) one, 3 the metered one. All three declare target_disk_gib = 64 and require UEFI.
--opt <name>=on|off only accepts a toggle the variant declares; an undeclared name is refused by name. --list-options prints every one.
--build kernel compiles the appliance kernel from source in the pinned container instead of carrying one off the build box, and it is the way the law wants. Without that flag the medium takes $KORP_KERNEL (or $TUNNR_KERNEL) — a kernel you built and can name. With nothing declared the verb REFUSES by name ([host-kernel-not-declared]) rather than sealing whatever this box happens to be running into a medium that installs an appliance somewhere else; GUNNAR_HOST_KERNEL=1 is the named exception that takes the newest readable /boot/vmlinuz*, and the verb then says CARRIED OFF THIS BOX on every line that mentions the kernel. This is not theory: the O1 run in open-build-capacity.md composed with no KORP_KERNEL, silently carried /boot/vmlinuz-7.0.0-30-generic, and the installed appliance stopped at md: personality for level 0 is not loaded! because that kernel has raid0 and xfs as modules. Whichever kernel is named, it is checked against every symbol the variant's options require before a byte is composed. A carried kernel with CONFIG_TUN=n is refused by name for the wireguard option (boringtun is a userspace data plane and opens /dev/net/tun); --opt wireguard=off drops the requirement along with the feature, and --build kernel satisfies it properly.
4 is the console, and it is the one that is not a declared build variant: gunnar-ui carries no .nornir/build-variants.toml, so there is no dwarves buildthing line for it and these two commands are the build.

5. The metered medium, and the money half

Skip this section entirely unless you built gunnar-installer-metered (§4, line 3). The open medium has no money code in it at all: no plans, no quotas, no buying, storage unlimited. Everything below describes the OTHER medium.

gunnar never takes money. It does three things and stops:

reports what each namespace uses, over Usage.Read / Usage.Activity;
accepts a signed entitlement and enforces it — Entitlement.Set, verified against the Ed25519 key named by --monetize-pubkey, and refused if the signature does not check;
mints an actor ticket on request, Billing.MintActorTicket, which is the appliance vouching that a console session really is a namespace admin.

The money half is a separate product in a separate repository, monetize, running as its own binary and talking to this one over that signed seam. It is not part of this checkout and is not built by §4. An appliance from the metered medium with no monetize server in front of it enforces the entitlements it was last given and issues no new ones.

 1  gunnar control appliance-identity

 2  gunnar control plan show <namespace>

 3  gunnar control plan list --expiring

1 prints the appliance's ticket public key — the value monetize must be configured with before it will believe a ticket this box minted.
2 and 3 are read-only. There is no gunnar control entitlement set: the only way a cap changes is a fact signed by monetize's private key, which this box does not hold and must never hold.
Entitlement and Billing exist only in the metered build. On an open appliance the symbols are not compiled in and the services are not mounted.

Growing the data set while it serves

This part is gunnar's own, and it works the same whoever asks for it.

 4  gunnar control capacity

 5  gunnar control grow status

4 answers total / servable / used, and says whether this boot stack can grow at all. A medium that reports growth=SEALED will take the data and refuse the resize — only a medium built with the runtime grow engine says growth=AT RUNTIME · hotplug=yes.
A growth is ordered when the set crosses 80 % used, and it grows down to 65 % — one growth, sized to the low mark, not one per customer. There is a minimum of one hour between growths unless the set is over 90 %.
The sequence drains the gate, waits for in-flight transfers to finish, unmounts /data, waits for the members to come back larger and equal, grows the array and remounts. No reboot. Pushes wait; nothing is killed.
It will not start unless a replication twin, if one is configured, is caught up — and it sizes that twin to what the primary is about to hold, before the primary grows. A twin too small to hold the primary can never catch up, and every future growth would then be refused.
Nothing here resizes a cloud volume. The appliance asks for members that are already bigger; making them bigger is the outside actor's job, and on a real cloud that actor is monetize with an explicitly enabled write credential.