nordisk/holger

RULE: Never add Co-Authored-By or any AI attribution to commits. Ever.

holger

Secure artifact server for airgap environments backed by znippy archives. Repository: https://git.gunnar.rs/nordisk/holger

Workspace layout

traits/                    shared types, ZnippyPlugin discriminants (→ znippy-common)
server/lib/                gRPC service, wiring engine, auth, storage
server/cli/                `holger`: the server (lib `holger::run` + bin); one of the four products
holger-downstream/         `holger-downstream`: the airgapped downstream, a thin crate over server/cli
holger-cli/                `holger-cli`: the command-line tool (console actions, bundle, verify, agent)
server/repository/         one crate per package ecosystem:
  znippy-{maven,python,rust,npm,nuget,go,gem,deb,rpm,helm,docker,conda,composer,package}
  ⚠ THE CRATE NAME IS OFTEN NOT THE NAME YOU SEARCHED FOR. A capability carries up
    to four names and the docs' name is usually not the directory's:
      PyPI (docs/UI) = "pip" (ron_repo_type) = ArtifactFormat::Pip = znippy-python/ (crate)
      Docker (enum)  = "oci" (alias)         also file-oci/
      Znippy (enum)  = "znippy"              but the crate is znippy-package/
      Rust           znippy-rust/ is package holger-rust-znippy-repository (words swapped)
    `grep -ri pypi server/` finds NOTHING in the implementation. Use
    `ArtifactFormat::home()` (traits/src/lib.rs) — the format→crate map, kept honest
    by `every_format_names_a_crate_that_exists`.
agent/lib/                 airgap / export / push logic
agent/cli/                 the courier as a library (`holger-cli agent`)
agent/connector/nexus/     Nexus connector (basic auth)
agent/connector/world/     external registry connector (crates.io, PyPI, …)
agent/connector/artifactory/  Artifactory connector
mannequin/                 mock server: PKI + OIDC + Nexus + auth for dev/test (GUI behind `gui`)
plugin/                    wasm package-handler + meter lane: abi, host, handlers, guests
holger-front/              holger.rs public front page (mounted by server/lib)
holger-estate-front/       the estate front (own workspace)
estate/                    THE ESTATE WORKSPACE (H1): holger-fleet (the ONE monetize pull channel,
                           server-lib `metered`), holger-live; members outside it by `package.workspace`:
                           xtask, the build declaration, crates/holger-monetize-guest, estate-tests
crates/                    holger-monetize-guest (an estate/ member)
holger-recipe/ holger-build-thing/ build-space/   the build declaration (estate/ members): one product list, .nornir/SUITES.md
update-bundle/ errcode/    signed update bundles (and `holger-cli bundle`/`verify`); the frozen error-code registry
provenance/                the offline provenance verifier (DSSE/in-toto/SLSA, Sigstore, PEP 740, npm, cosign)
bench-scenarios/           bench arms (dev-dependency only)
distribution/              Dockerfile
xtask/                     cargo xtask verbs (test, gate, build, bench, gen-proto, …; `cargo xtask` lists them)

Design

Vision

Holger replaces Nexus/Artifactory in environments where security, auditability, and airgap compliance are primary requirements:

• Airgap transfer — move packages across network boundaries as immutable znippy archives
• Strong authentication — OIDC and mTLS, not just username/password
• Immutable storage — append-only, content-addressed, auditable
• Multi-format — Rust, Python, Java, Go, npm, NuGet, Gem, Deb, Helm, Docker from one server

Security model

Authentication:

Client ──TLS──▶ holger-server
  Option A: mTLS cert  → extract CN as identity (CA-rooted trust)
  Option B: Bearer token → validate via OIDC userinfo endpoint

With ron_tls.client_ca the client certificate is also checked against ron_tls.crl (a PEM/DER CRL file or a directory, lane HOL-NCSC): a revoked certificate fails the handshake at the gRPC and HTTP doors; a CRL that is unreadable, not signed by the client CA or past its nextUpdate refuses the start (holger-tls-1), and a changed CRL is read at the next handshake. security_profile: Critical requires a CRL wherever client_ca is set. The twin's own mutual-TLS door (nordisk-twin, its own issuer) revokes by superseding a principal's leaf, not by CRL.

Authorization (prototype → production):

Separation of duties (the intake chain, server/lib/src/promote/duties.rs): the same auth block grants duties / group_duties — Developer (reads AVAILABLE only), Importer (ingests), SecurityApprover (approves, quarantines, releases, blocks), RepositoryAdmin (configures, never approves), Auditor (reads audit + evidence, changes nothing). The importer never approves its own import; promotion.intake.approvals: N needs N distinct approvers (steps: asks for 2 unless it names a number, lane HOL-NCSC); each approval is an attested statement in the approval evidence slot. Wire verb: gRPC PromotionService.AdvanceArtifact / GetArtifactState, HTTP POST /-/intake/advance, GET /-/intake/state, CLI holger-server intake. Stored evidence is read back (W5): S9's PromotionService.GetArtifactEvidence (by coordinate; HTTP GET /-/intake/evidence?sha256= and GET /-/intake/sbom?sha256= resolve the coordinate holding the bytes and call the same duties::evidence_as; CLI holger-server intake evidence --name) also returns every stored record, its DSSE envelope, the signer certificate, the stale reason and the SBOM link, under the read duty of GetArtifactState. SCANNED and the osv gate reuse a FRESH stored record (same scanner ids and versions, intel sequence not newer) and audit it as a scanned step with reused=true; missing or stale ⇒ scan again. One quarantine (W6): HOLGER_QUARANTINE_PROPERTY is retired and refuses the boot (holger-intake-14); the hold is the intake QUARANTINED state. npm through the chain (lane P5, server/lib/src/promote/npm.rs): promotion.intake.npm names an archive-less ron_repo_type: "npm" repository (the estate's npm). npm publish there is INGESTED into incoming at the tarball's registry path (<package>/-/<file>.tgz, the coordinate the walk names); the packument (each manifest read from the SEALED tarball, sha512 integrity), the tarballs and dist-tags are a view of the AVAILABLE set; unpublish and rewrites are refused by policy (holger-npm-1..5). Pull-through into the chain (server/lib/src/pull_intake): python, oci, npm, maven, crates and apk (with url) sources fetch on a client's request (one jera PullIntake job per want), INGEST as pull-through:<source>, walk SCANNED (scanners + holger-provenance) and POLICY_CHECKED, wait for approval (or auto_approve), and are served from the sealed store once AVAILABLE; until then each client gets its ecosystem's "not yet" (503 + Retry-After, then 404 / 403), and every index (PEP 503, tag list, packument, maven-metadata.xml, sparse index file, APKINDEX) lists only AVAILABLE items. Paths: /production/<source>/… (OCI: /v2/production/<source>/…). Scanned pushes (lane HOL-OCI-EVIDENCE, server/lib/src/promote/oci_push.rs): on a Holger with an intake chain every ron_repo_type: "oci" repository is a SCANNED REGISTRY unless promotion.intake.oci_push (or steps.oci_push) says (repository: "<name>", scan: off); an opt-out where the policy requires a scan (Critical, or scan.required naming a kind) is holger-oci-1, an unknown registry holger-oci-2. Its store only STAGES: a manifest pushed by TAG takes the image into the chain whole (read back with the pull-through's registry client: every platform, the .sig/.att/ .sbom tags and referrers already pushed, so a signature pushed before the tag is verified on intake), INGESTED as the pusher at <registry>/<image>/sha256-<hex>.oci.tar (origin = the registry), walked SCANNED and POLICY_CHECKED, then the approvals; a refusal fails the push (403 DENIED with the reason). Reads serve only AVAILABLE images (the first read after the approvals seals it), their blobs, tags, later attachments and Holger's evidence; writes need the Importer duty; DELETE is holger-oci-3. The image SBOM (server/lib/src/scanner/image_sbom.rs, holger-image-sbom): a CycloneDX 1.5 document of an image layout made in the scan worker from its layers (holger_inspect::image::catalog: apk, dpkg, rpm, Python, npm, Java archives' pom.properties, gem specifications, Composer, .NET deps.json, Go build info, cargo-auditable), one purl per package; signed as a CycloneDX statement about the layout and its image digests and served as a referrer and in the .att tag. The evidence key door (server/lib/src/evidence_key.rs): GET /-/evidence-key (and .pem) is the public half of attestation.signer_cert as PEM, .jwks a JWK Set (kid = the DSSE keyid), .crt the certificate; open on every box, for cosign verify-attestation --key https://<holger>/-/evidence-key. Tests: pushed_images_go_through_the_intake_chain.rs, an_image_carries_holgers_signed_cyclonedx_sbom.rs, the_evidence_key_door_serves_the_signer_key.rs, keyless_cosign_offline_through_holger.rs (keyless cosign with an offline test root on intake; real cosign 2/3 through Holger, ignored) and the real-client additions in real_signing_clients_verify_through_holger.rs (syft, Kyverno over Holger's evidence).

Provenance of what comes in (provenance/, server/lib/src/scanner/provenance.rs, lane G3a): the provenance scanner slot (id holger-provenance) verifies, OFFLINE, the attestations that came with an artifact (<data_root>/holger-provenance/<sha256>/): DSSE + in-toto bound to the sha256/sha512, SLSA v1/v0.2 (builder, build type, invocation, resolved dependencies into the evidence), keys (Ed25519, ECDSA P-256) and keyless Sigstore (Fulcio chain to a pinned root, issuer + subject identity, SCT, Rekor SET / inclusion proof against a signed checkpoint) — for Sigstore bundles, PyPI PEP 740, npm attestations and cosign .sig/.att layers (Chainguard's images). Policy: promotion.intake.scan.provenance: (rules: <floor>, sources: {name: require(...) | prefer | off}); a source only tightens the floor. The trust root is a signed holger.sigstore-trust/v1 bundle (rules.trust_root, verified with promotion.intake.trust_policy). An airgap export carries the material; the import verifies it again under the secure side's policy (gate airgap). Codes holger-provenance-1..13. APK provenance (lane G3, provenance/src/apk.rs): Wolfi and Chainguard packages are signed only through APKINDEX.tar.gz. A package verifies when the index's .SIGN.RSA256.<key> (or .SIGN.RSA., SHA-1) verifies over the raw index stream with a PINNED apk key (before the index is decompressed), the index entry for its pkgname/pkgver names its control segment's hash (C:Q1/Q2) and size, and .PKGINFO's datahash is the data segment's sha256 (hashed, never unpacked). It is one provenance candidate (apk-index:<file>, builder apk-key:<name>, no SLSA level, so slsa_min > 0 refuses it). The keys ride in the holger.sigstore-trust/v1 bundle as apk-keys/<name> payloads (TrustBundleOptions.apk_keys, TrustRoot.apk_keys) and reach the scan worker in the job (ProvenanceJob.apk_keys). sources: [(name: "wolfi", kind: apk)] is an upload source whose artifacts also get <inbox>/apk/<source>/* as material (exported with the airgap bundle); require(apk_keys: [...]) narrows the keys. APK pull-through (lane T8, server/lib/src/pull_intake/apk.rs): a kind: apk source WITH url (e.g. https://packages.wolfi.dev/os) pulls <arch>/-/pull/<name> (the newest version and its dependency closure) or one <arch>/<file>.apk. The upstream APKINDEX.tar.gz is verified against a pinned key (pull_through.rules.apk_keys or the trust bundle's apk-keys/) before a package is read from it; each .apk is held to that signed index before it is INGESTED. Holger serves its OWN APKINDEX.tar.gz, re-signed with Holger's per-source key (/production/<source>/-/apk-key-name, then /production/<source>/<key> into the client's /etc/apk/keys), listing only AVAILABLE packages with the upstream's stanzas. Tests: server/lib/tests/pull_through_apk.rs (suite-check trust:apk-pull) and its ignored real_client_apk_add_through_holger, a real apk add jq in cgr.dev/chainguard/wolfi-base against the real Wolfi repository (the demo's row, cargo xtask demo tests; the client reaches Holger through pasta -T <port>, never --network host). Maven PGP provenance (lane T9, provenance/src/pgp.rs + pgp/rpgp.rs, rpgp pgp =0.21.0, pure Rust, no default features): Maven Central's detached .asc beside a file verifies OFFLINE when its issuer fingerprint (or key id) names a PINNED OpenPGP key's primary or signing subkey and the signature (binary or text, SHA-224 or stronger) verifies over the exact bytes. An unpinned signer is "unverified" (holger-provenance-4), never a pass. It is one provenance candidate (pgp-signature:<file>, builder pgp-key:<name>, no SLSA level). The keys ride in the holger.sigstore-trust/v1 bundle as pgp-keys/<name> payloads (TrustBundleOptions.pgp_keys, TrustRoot.pgp_keys, the job's ProvenanceJob.pgp_keys) or in the floor's keys as alg: openpgp; require(pgp_keys: [...]) narrows them per source (intake sources and pull_through sources). The Maven pull-through fetches <file>.asc beside every POM and JAR. It is OPT-IN (T9b): holger-provenance pgp, server-lib and CLI maven-pgp, carried by the CLI's online and full; the default holger does not have it (rpgp is about 96 crates; the default tree is bounded at 345). server-lib's own online does not carry it (a weak feature would put rpgp in every nested lock, cargo#10801): test the Maven .asc with --features jobs,online,supply-chain,maven-pgp. Without it every pin FAILS CLOSED by name, [provenance-pgp-not-built]: a trust bundle's pgp-keys/<name> and a floor key of alg: openpgp are refused at load, so is a source's require(pgp_keys: [...]), and an .asc is unverified (holger-provenance-4). Tested both ways: provenance/tests/pgp_build_feature.rs, suite-check trust:maven-pgp, trust:maven-pgp-intake, trust:pgp-not-built, trust:pgp-intake-not-built.

Secrets in what comes in (server/lib/src/scanner/secrets.rs, lane HOLGER-G3): the secrets slot (id holger-secrets) runs in the scan worker and walks the artifact with the INGEST walk's own limits (holger_inspect::Inspector::visit: zip, tar, gzip, bzip2, xz, zstd, ar (.deb), rpm (its cpio payload), conda (.conda and .tar.bz2), nested archives; a 7z, or any archive the walk cannot open, BLOCKS as not-unpacked (HOL-NCSC), every image layer, deleted files included) against a rule set that is DATA (holger.secret-rules/v1 RON; built-in scanner/secrets-rules.ron, or the operator's file at promotion.intake.secrets.rules / steps.secrets.rules, read and checked at boot and handed to the worker in the job): private keys, AWS/GCP/Azure credentials, GitHub/GitLab/Slack/Stripe/npm/ PyPI/crates.io tokens, .env files, kubeconfigs, high-entropy strings with context. A finding is secret:<rule id> with the path and line, never the value. The scanner warns; the policy decides: rules.secrets: block | warn | off (default block) and a source's gates.secrets (tighten only); block refuses high and critical findings at SCANNED by name.

Every scan runs in a container (lane SANDBOX-SCAN, owner 2026-10-23; server/sandbox, server/scan-worker, server/lib/src/sandbox): the server never unpacks, parses or scans an artifact itself. The INGEST walk (bombs, nesting, traversal, MIME, executables), YARA, the model, licence, wheel-SBOM and image-SBOM scanners, the osv gate's licence read, an image's package catalog (apk, dpkg, rpm sqlite, python, npm, Go build info, cargo-auditable; server/inspect/src/image.rs) and holger_provenance::verify_artifact (DSSE, Sigstore, DER/X.509, Rekor) run in holger-scan-worker, one hardened rootless podman container per artifact (--network none, the artifact read-only, a capped noexec tmpfs, a read-only root, --cap-drop all, no-new-privileges, the default seccomp, uid 65532, CPU/memory/pids limits, a wall-clock deadline). The worker answers one JSON report on stdout, capped and validated field by field; the server keeps the advisory match (an image's CVEs, each with its severity, so cve_max holds), the evidence and the signatures. ClamAV is clamd in its own sandboxed container (scripts/clamd-sandbox.sh); the server streams bytes to its socket. A real build (not sandbox-off) takes ONLY that unix socket (NCSC P1, lane HOL-NCSC): a TCP clamd, a host clamd's socket, clamscan or no socket refuses the start (holger-sandbox-15, sandbox::check_clamav); the sandboxed clamd says AlertExceedsMax yes (sandbox/clamd.conf), so what it cannot scan whole is FOUND and blocked, never passed unscanned. promotion.intake.rules: (sandbox: required) is the default; off only in a test build (holger-sandbox-6); no engine refuses every scan (holger-sandbox-1), never an in-process fallback. The image is minted FROM scratch (cargo xtask scan-worker image) and crosses the airgap as an OCI archive with its sha256 (scan-worker save); the secure side never pulls. Engine: the podman program, or podman's API socket (sandbox: (url: Some("unix://…"))) for a Holger that runs in a container. The image is pinned by digest (NCSC P1, lane HOL-NCSC): a local tag is resolved ONCE, at the engine's first answer, and every run names <repository>@sha256:…, never the tag (holger_sandbox::pinned_ref; an image with no sha256 digest is holger-sandbox-8). Codes holger-sandbox-1..15; container arms cargo xtask scan-worker tests. On the box (lane HOLGER-MEDIA): the main and twin media carry the sandbox in the app archive (holger-recipe sandbox_payload_entries: sandbox/{manifest.toml,crun,holger-scan-worker.rootfs.tar, clamav.rootfs.tar}, staged by cargo xtask scan-worker stage into .sandbox-out/, packed by the forge beside the app, so the installer, the keep medium and the live app all carry it). The OS-less appliance has no podman: the engine there is the static crun (1.9 MB, sha256-pinned) with Holger writing the OCI bundle (holger_sandbox::crun: a user namespace, uid 65532 mapped to host 100000+, a new empty network namespace, the same limits), over the worker rootfs laid under /data/holger/sandbox and RE-HASHED AT EVERY START before the server serves (NCSC P1, lane HOL-NCSC; not only at the first lay: each medium entry against the manifest, then every laid file's sha256, mode, owner and link against its entry; a laid tree that differs is laid again from the medium and said by name, holger-sandbox-14, and one that still differs refuses the scans; server-cli sandbox_boot, server-lib sandbox::media). HOLGER_SANDBOX_MEDIA_DIR is IGNORED on a box korp-start booted (a warning names it): the appliance lays from its signed app archive only; clamd runs the same way (sandbox::clamd, uid 100, 2 GiB, supervised). The estate declaration says sandbox: Some((engine: crun, …)) and clamav: Some((…required: true)); GET /-/sandbox/probe (admin-read) is the worker's probe from inside, the install test's sandbox row. The kernel rows every holger medium adds for it (container_kernel_requires): MULTIUSER (the appliance-boot contract's one-user default, lifted by name), the namespaces, cgroup v2 with memory, pids and cpu, seccomp.

The airgap (server/lib/src/promote/airgap.rs, docs/v1/demo-airgap.md): holger-server export takes the AVAILABLE set of the intake chain's production repository with its stored evidence, signed attestations, SBOMs and intake records as a bundle candidate whose verdict is COMPUTED from that evidence (an artifact without passing evidence is refused by name). It carries ONLY what decides (server/lib/src/export_profile.rs, lane HOL-NCSC): bytes and digests, the signed evidence, SBOM, approvers, provenance, sequence and expiry; the intake record crosses as its decision lines (no importer or admin name, reason, policy words, timing, repository or source; free-text references as their SHA-256), holger-export.json without exported_by/at/production, core.json without fetched_at/source_repository, sync invalidations without issued_by; every exported field is classified in PROFILE, guarded by server/lib/tests/the_export_carries_only_what_decides.rs; holger-cli bundle approve/finalize sign it; holger-server bundle import on the secure side verifies every payload hash and evidence attestation and lands each artifact AVAILABLE, audit first, served through a PEP 503 index (/production/simple/). network: Offline (server/lib/src/egress.rs) refuses a config that needs the network and every outbound dial; every socket asks egress::check first (SIEM, a test build's TCP clamd, the off-site S3 copy, the WireGuard tunnels too; held by server/lib/tests/every_refusal_leaves_an_audit_row.rs), and Offline implies the SIEM's local_only (egress::check_site_local). Every refusal leaves an audit row (lane HOL-NCSC, server/lib/src/refusals.rs): a blocked dial, a refused import (signature, floor, core.json, intel, inbox), an unauthorised importer and a refused TLS handshake (at most 60 rows a minute). scripts/demo-airgap.sh runs the whole 1.0 demo.

The time floor (lane HOL-NCSC, server/lib/src/time_floor.rs): the import door judges every expiry, maximum age and certificate window at max(clock, floor); the floor (<data_root>/holger-time/floor) rises only with trusted time (a verified estate clock, each accepted bundle's signed created_at, an intel bundle's signed snapshot, a signed time token) and never moves back; an unverified host clock never raises it. A clock behind it by more than 5 min is judged at the floor and audited (holger-time-1); an unreadable floor refuses (holger-time-3). The time token is an ordinary signed bundle of product holger.time-token/v1 (holger time-token make --signer-dir D --trust-policy-out P on the online side, carried in by holger bundle import or the inbox, no first-import grant; holger-time-2); holger time-token status --config C. Tests: server/lib/tests/the_time_floor_never_moves_back.rs, server/cli/tests/bundle_time.rs.

The sync protocol (Holger v2.0 H4, sync/ + server/lib/src/sync.rs, feature sync, in the server's default; docs/sync.md): every sync from an upstream Holger (a pull, a stick, or the evidence/holger-sync.json an export carries) is a signed manifest (Ed25519; generation, packages added, every invalidation: coordinate, sha256, cve-after|malware|pulled|policy) verified under the sync: block's trust list, anti-rollback. The downstream holds each invalidated package at once through IntakeChain::invalidate (QUARANTINED from any state, bytes kept, audited first with the invalidated trail step, never released, every index drops it) and signs an ack (outbox + --ack-out stick; pull: POST /-/sync/ack). The upstream's ledger keeps per downstream the generation applied, the lag and the invalidations reached; GET /-/sync/board is the board's data; an invalidation not acked within ack_deadline_secs raises one ALARM (audit, board flag, the holger::alarm log line), cleared by the ack. CLI holger-server sync key|invalidate|manifest|apply|ack|board|pull; the sync.tick jera job. Codes holger-sync-1..10.

The box's import doors (server/lib/src/import_door.rs, lane W4): a box has no shell, so intel and update/airgap bundles also arrive over gRPC AdminService.ImportIntelBundle / ImportBundle (client streams), HTTP POST /-/intel/import / POST /-/bundle/import, and the <data_root>/intel.inbox / bundle.inbox drop directories (swept by the holger_import_inbox jera job, then archive/ or refused/ + .why.txt; an absent floor is set only by an admin's allow-first-import grant, and a FIRST-IMPORT marker is refused). One body of code with the CLI verbs: verify, floor, audit first, fail closed; Repository Admin or Security Approver only. Bundles land AVAILABLE through IntakeChain::land_imported (trust policy promotion.intake.trust_policy); the old bundle import --repo raw-directory write is refused by name (holger-import-1). An online box (intel.offline: false, the sold estate's default) runs the intel.refresh jera job once at boot and daily: it fetches OSV, NVD and KEV (or intel.feed_mirror), signs the set with the box's own refresh signer and lands it through the same import (ImportDoor::refresh_intel); server-lib intel-refresh, named only by the two metered MVP rows (server-cli intel-fetch); the airgap tiers keep zero egress. A scan never dials; it reads the local database.

Scanner signatures as a signed bundle (lane HOL-NCSC, intel/src/sigs.rs): ClamAV's .cvd/.cld (and hotfix .ndb/.hdb/.ldb, YARA .yar) cross the airgap as a holger.update-bundle/v1 with product holger.scanner-sigs/v1: holger scanner-sigs build signs them, holger bundle import (every import door) verifies, holds the floor, audits first and installs them into <data_root>/sandbox/signatures/clamav/, mounted read-only over the sandboxed clamd's DatabaseDirectory, then asks clamd to RELOAD. A member that is no signature file refuses the bundle (holger-sigs-1). A hotfix is the next sequence with one file, laid over the set. Operator steps: docs/v1/scanner-signature-bundles.md. Codes holger-sigs-1..3.

Reads are open by default (airgap-mirror posture: the network is the perimeter, and unmodified pip/cargo/docker carry no credential). Set require_auth_for_reads: true in the RON auth block to gate them behind reader — opt-in, fail-closed, health probes stay open. See .nornir/auth-rbac-design.md §R3.

Safe defaults (lane HOL-NCSC, NCSC P1; Holger::validate_listeners, server/lib/src/listen.rs): a Holger with no auth.methods serves loopback only (holger-listen-1 for any listener that is not localhost/127/8/::1 by name); a listener off loopback needs ron_tls or the endpoint's explicit insecure: true (holger-listen-2; the estate and appliance RONs write it, their wire is WireGuard or the LAN until a certificate ships). Configured authentication implies RBAC: with no role policy the server serves with default_role: Reader (Holger::serving_auth), never "every signed-in user may do everything"; security_profile: Critical needs the policy written in the file. holger-cli takes a bare --at host:port as https:// unless it names this machine. A bundle that would set an absent anti-rollback floor is refused (holger-import-9) until an admin grants that vendor/product/sequence once: holger bundle allow-first-import, holger-cli bundle allow-first-import (BundleService.AllowFirstImport), POST /-/bundle/first-import, or the console's Admin → Sync form. An invite that makes a Security Approver is minted by a person's stepped-up passkey session only, never by the console's machine token (holger-auth-6; an admin is never invited). Key files (lane HOL-NCSC, NCSC P3, server/lib/src/keyfile.rs): every private key or secret Holger loads from a file (a door's TLS key, the evidence signer, a mirror's client key, the sync key, the export-push signer, the apk key, a static token file, a passkey session key, the WireGuard identity, the holger CLI's signing key) is refused by name when its mode grants group or others any bit or another user owns it (holger-key-1); the one other shape accepted is a Kubernetes secret volume (root:fsGroup, defaultMode: 0440, which the chart and deploy/k8s set). Holger writes its keys and the estate it restores 0600. The evidence signer can keep its key in a hardware token instead (--features pkcs11, default off, server/lib/src/pkcs11_signer.rs): attestation.signer_hsm: Some(Pkcs11(module, token_label, key_label, pin_file)) beside signer_cert signs DSSE evidence and audit checkpoints through the PKCS#11 module (ECDSA P-256 or Ed25519; the PIN file is a key file); a build without pkcs11, a missing token or key, or both a file key and a token refuse by name (holger-key-2). Tested against SoftHSM2 (server/lib/tests/the_evidence_signer_lives_in_an_hsm.rs, HOLGER_PKCS11_REQUIRED=1 in its container).

The admin listener (lane HOL-NCSC, NCSC P2; server/lib/src/listen.rs): an endpoint with ron_admin_url (gRPC) and/or ron_admin_http_url (HTTP) serves the admin plane there only: AdminService, PromotionService, AuditService, TokenService, the IdentityService, jobs, admission, migration and the thin client's doors; over HTTP /-/auth, /-/bundle, /-/intel, /-/intake, /-/audit, /-/flow, /-/twin, /-/sandbox and the sync administration doors. The data listener keeps the package doors and answers the other plane's doors holger-listen-3 (and the admin one the packages). ron_admin_allow: ["10.0.0.0/8", ..] (CIDRs) is required (holger-listen-5); any other peer is dropped and audited (holger-listen-4). Without an admin listener one door serves all.

Repository tokens (server/lib/src/repo_token.rs, lane P3): what cargo, pip, npm and maven carry instead of a passkey. A person signed in with a passkey (stepped up) mints one over gRPC holger.token.v1.TokenService (console page /tokens, CLI holger-server token mint|list|revoke) for named repositories, read or read-write, 1 to 366 days. The secret (holger_rt_…) is shown once; holger-passkeys/repository-tokens.json keeps blake3 hashes, the last use, and revoked rows. Only the repository door takes a token (auth::validate_request_for; every other door refuses it): it acts as its owner as they stand now, narrowed to its repositories and scope. Refusals holger-token-1..11 (revoked, expired, owner gone, out of scope, read only, …); an unknown token is the plain 401. A gated rust repository says "auth-required": true in config.json, so cargo sends the token on reads.

Roles mapped from OIDC token claims or mTLS CN → role config.

Transport: gRPC over HTTP/2 (tonic). No HTTP/REST API surface.

Server architecture

Config is a directed graph of three node types:

ExposedEndpoint ←── Repository ──→ StorageEndpoint
    (gRPC)               │              (disk / znippy archive)
                         ▼
                   Repository (upstream proxy)

The wiring engine (wire_holger) resolves in multiple passes: index nodes → wire forward refs (repo→storage, repo→exposed) → wire reverse links (exposed→repos) → attach FastRoutes.

Request flow (gRPC):

gRPC request (tonic)
  → HolgerGrpc service
  → FastRoutes.lookup(repo_name)
  → RepositoryBackendTrait.fetch() / .put()
      ├── RustRepoFile      (disk)
      ├── RustRepoZnippy    (znippy archive)
      └── MavenRepoZnippy   (znippy archive)

Direct Rust bindings (no serialization for in-process callers):

holger.fetch("repo-name", &artifact_id)  // returns Option<Vec<u8>>

Agent architecture

The agent moves artifacts between systems:

Source ──▶ [Agent Logic] ──▶ Target
Sources: crates.io, Nexus, Holger, .znippy archive, directory
Targets: .znippy archive, directory, Nexus, Holger

All sources/targets implement ConnectorTrait:

async fn list_repositories() -> Result<Vec<RemoteRepository>>
async fn list_assets(repo) -> Result<Vec<RemoteAsset>>
async fn download_asset(asset) -> Result<Vec<u8>>
async fn upload_asset(repo, path, data) -> Result<()>

Auth per connector:

enum ConnectorAuth { None, Basic { username, password }, OidcToken { token }, Mtls }

Storage

• znippy archives — immutable, content-addressed, blake3 checksums. One archive per repository version. Carriable across airgap on USB/optical.
• Filesystem backend — simple {repo}/{path} tree, used for dev/testing.
• Promote workflow — DEV (mutable, live ingest) → promote → PROD (immutable). The seal is content-addressed (blake3) into <data_root>/holger-prod-store; the ref IS the seal and is written create_new, so a sealed coordinate can never be overwritten. Signing 🟡.

Mannequin (test infrastructure)

mannequin/ mocks the entire external world for dev and integration tests:

holger-mannequin
  ├── PKI    — CA mgmt, cert issuance, regen
  ├── OIDC   — /token, /userinfo, discovery endpoint
  ├── Nexus  — proxy + upload (replaces real Nexus in bench)
  └── Auth   — users, roles, validate, issue
       │
       └── egui GUI (--gui flag) — live system map + request logs

Used in-process (integration tests, no network) or as a standalone server (mannequin serve). Also used for Nexus benchmark comparison — mannequin impersonates Nexus so holger and Nexus run against the same mock backend.

What's production-ready vs prototype

Key design decisions

1. RON config — config is a graph (repos → endpoints → storage), not a flat list. RON handles typed enums and nested refs naturally.
2. gRPC for control — streaming, strong typing, efficient binary. Package clients use the native HTTP format doors (and the OCI door).
3. Dual access — gRPC for network; direct Rust calls for in-process (agent, tests).
4. Separate agent and server — airgap requires offline operation. Agent runs internet-side, server runs secure-side. No direct connectivity needed.
5. Trait-based backends — RepositoryBackendTrait + ConnectorTrait add new formats without touching core logic.
6. Mannequin — mock the whole external world in one process. Replaces Keycloak + Nexus + CA for dev.

Local dev setup

A lone clone builds from crates.io: no committed manifest or [patch] points out of the repository (xtask/tests/lone_clone.rs). For estate work, check out edda, Skidbladnir, monetize, tunnr, draupnir, modgunn and facett next to holger/ and run cargo xtask siblings once: it copies .cargo/estate-siblings.toml (every sibling patch, ONE table) to the uncommitted .cargo/siblings.toml that .cargo/config.toml includes as optional, and every other cargo xtask verb refreshes it. cargo xtask always starts: the alias runs xtask/boot, a detached launcher with no dependency, which refreshes the copy and hands the verb to the real xtask (run it from the repository root). Add a sibling dependency by VERSION in the member's manifest and its path to .cargo/estate-siblings.toml, never path = "../<sibling>". estate-tests/ (the tests against the real ONE box) is a detached workspace that reaches the siblings by path. znippy comes from crates.io.

Build & test

Build and test in the nordisk-cargo-toolchain:live container. Run git lfs install --local && git lfs pull first (the front's picture and the boot logo are LFS; a test names the pointer file if you forget).

cargo xtask test                # the wasm guests, then default + metered suites
cargo xtask gate                # rows, builds, tests and the declared arms

Tests that load a wasm guest build it on demand (traits::test_support::wasm_guest: host cargo, --target wasm32-unknown-unknown, into <target>/wasm-guests); no container and no manual step. Shared test helpers (fstatus, MemReader, CountingReader, the race-free free_port lease, the LFS check) live in traits::test_support behind the test-support feature, a dev-dependency only.

Benchmarks: measure THROUGH the door that persists

cargo xtask bench                       # measure + prove the row landed
cargo xtask bench --heavy               # …including the #[ignore]'d container arms
NORNIR_BENCH_ONLY=holger.blake3 cargo xtask bench    # one arm
cargo xtask bench-landed                # the guard: newest receipt has a warehouse row

cargo xtask bench is a THIN door onto nornir bench run holger — it owns no arm, no scale and no feature (those live in xtask/examples/nornir-bench.rs and edda/crates/nornir-bench). It resolves the absolute workspace_nordisk/release/nornir.toml, passes NORNIR_BENCH_ONLY, NORNIR_BENCH_SKIP, NORNIR_BENCH_FEATURES, NORNIR_MACHINE and HOLGER_BENCH_SCALE through, and then counts bench_runs before and after: no new row is RED, whatever the exit code said. Running the example directly and redirecting stdout — which is how every holger measurement was taken until 2026-09-20 — writes no row anywhere.

Every arm here, no nornir: cargo xtask bench --local (2026-10-23). It runs the same example, writes bench-results/receipt-<run id>.json, latest.json and latest.txt (one line per arm), appends the BenchRun to bench_history.jsonl, and checks the tracked metrics (ops_per_sec, mb_per_sec, every *_p50_ms) against the baseline of the same NORNIR_MACHINE (bench_baseline.<machine>.json; the legacy bench_baseline.json is oden's): worse than its threshold_pct is red. --only <prefix>, --fetch-fixtures (the large intake fixtures, once, online side), --update-baseline (refused without NORNIR_MACHINE), --runs N (judge, or take a baseline from, the per-metric median of N runs), --no-regress (report the verdict, never fail). The bench machine is builder-64 (2026-10-25; oden is retired): judge a phase with NORNIR_MACHINE=builder-64 cargo xtask bench --local --runs 3, without --no-regress. In the toolchain container set HOLGER_BENCH_FIXTURES=/target/bench-fixtures (the default is <workspace>/target, not $CARGO_TARGET_DIR): without the fixture cache (270k OSV advisories, the large packages) the CVE arm and every *_l/oci_layer metric are missing and the "real DB" arms measure an empty one. The baseline lists noisy metrics (they moved more than the threshold between its own three quiet runs: the upstream fetches, the large-file seals, concurrent_upload, secure_update_*), and medians under 1 µs (MIN_JUDGED_MS) are below the timer's resolution; both are reported, never judged. The holger.intake_* arms (bench-scenarios intake.rs) time every step of the intake chain per package type over real packages (bench-scenarios/fixtures/intake): hash, inspect, malware (HOLGER_BENCH_CLAMD=host:port for ClamAV), CVE against the real OSV dumps, licence, SBOM, provenance (holger-provenance and the primitives), the gate catalog and the source gates, seal, the whole chain for one package and N in parallel, and a cold znippy fetch from HOLGER_BENCH_SLOW_DIR. holger.sandbox_scan times the scan sandbox (the per-package overhead of the worker's container against in-process, the chain in the container, chainguard/static's CVE scan, EICAR through clamd's sandbox); scripts/bench-sandbox.sh brings up the podman API socket and clamd's sandbox and runs it. Threshold 50% on both bench machines (oden: two identical runs differed by up to 44%; builder-64: three quiet runs differed by a median of 3.2%, the noisy metrics aside; see each baseline's threshold_why).

The README's speed table: cargo xtask bench-readme (lane README-BENCH, 2026-10-31; the owner's rule: a speed claim comes only from a bencher run on REAL packages). fetch --seeds <dir> (host: pip3, npm, curl, helm, podman) fills <dir> with the pinned real sets of holperf/sets/ (pip, npm, Maven, Helm, apt, dnf, OCI); run --seeds <dir> --machine <name> [--holperf <bin>] runs holperf over them (HOLPERF_REAL: each set's own sealed store and the estate's intake chain) and writes holperf/results/<date>-<machine>-holperf.json (schema nordisk-bench-result/1, shared with Gunnar); write renders the table between <!-- bench-readme:begin/end --> in .nornir/README.md and README.md; check (and xtask/tests/bench_readme.rs) is red when they differ from the newest result; compare --threshold <pct> is red on a p50 or peak-RSS regression against the previous run of the same machine.

LOCAL vs CENTRAL is NORNIR_SERVER (or --server <url>): unset, nornir writes the iceberg row here; set, the SERVER runs the bench and appends the row, because a thin client cannot hold the single-writer redb catalog lock. The door prints which mode it used and where the row landed.

One run writes the value twice: the bench_runs row and the bench_history.jsonl receipt at the repo root. The receipt is committed — it is the backup the doc mashup renders from when the warehouse cannot be opened — and .gitattributes gives it merge=union so two branches' runs both survive a merge.

Release procedure

1. cargo audit and cargo deny check (policy in deny.toml). This step is first because it is the one that decayed unnoticed: the 2026-06-14 review recorded 0 advisories, the 2026-09-11 review found 29, and nothing ran either tool in between. .forgejo/workflows/security.yml runs both on every push and daily (the root builds from crates.io, so a lone clone resolves the graph), and the full set of cargo xtask ci has the deny step. ONE ignore list: deny.toml (with reasons) and .cargo/audit.toml (its ids), held equal by xtask/tests/audit_follows_deny.rs (lane HOL-NCSC).
2. Remove all [patch.crates-io] path overrides
3. cargo xtask bench — holger vs Nexus via mannequin
4. Gate: holger ops_sec ≥ Nexus for every artifact type
5. Gate: no type drops >10% vs last same-machine bench_history.jsonl entry
6. Gate: agent push → store → pull round-trip passes (rust, jar, wheel)
7. Update CLAUDE.md / README.md if architecture or numbers changed
8. Append result to bench_history.jsonl, commit
9. git tag vX.Y.Z && git push --tags
10. Publish: traits → server/repository/* → server/lib → agent/lib → agent/cli, update-bundle → server/cli (holger) → holger-downstream, holger-cli; holger-ui from its repository
11. SLSA Build L2 (not L3: the release key is the release manager's, not the platform's; docs/v1/SLSA.md says what is missing): the crates.io release goes through dwarf's release runner (dwarves release-build: build, the owner signs on the YubiKey with sign-release.sh, verify against slsa/release-signing-key.pub, publish the attested bytes only), and the bundle lands in attestations/ (docs/v2/07-build-thing.md). The media, each app archive and every product binary get their own statement, cargo xtask release-attest --release holger-<version> (unsigned into target/release-attest/, a missing artifact refused by name; --check <statement> before signing), signed the same way into attestations/<release>-media.intoto.jsonl.