nordisk/holger

Holger — design (the whole system)

Class: design · Function axis: f0 (the whole) + registry · serve · storage · auth · agent · proxy · embed. Strict: this file states only the system we ship. Every why, every rejected alternative and every not-yet-built direction lives in the reasoning book (reasoning.md, registry-reasoning.md, promotion-design.md, static-dynamic-profile-design.md, …).

f0 — one engine, three build variants

Holger is a secure, airgap-first artifact server: it serves Maven, Python, Rust, npm, NuGet, Go, Gem, Deb, RPM, Helm, Docker, Conda, Composer — and znippy's own native package — from immutable, content-addressed znippy archives. It replaces Nexus/Artifactory where auditability and airgap compliance are the point: move packages across a network boundary as one signed .znippy file, then serve every ecosystem's native protocol out of it.

The system is one serve engine wired seven ways. Each capability is a ## section below; here is how they fit:

The reuse graph. serve uses registry (dispatch) and storage (extract). auth gates serve's write path. proxy is a registry backend that uses storage (write-back) and an upstream. embed exposes registry + storage with no serve. agent is a sibling process that produces what storage serves. traits/ is the floor all of them stand on.

The three build variants (the split nornir's build-thing models)

The same engine ships three ways; the boundary is the crate graph, not a runtime flag.

The companion airgap-transfer binary holger-agent (agent/cli) is a sibling to the server, not a server variant — it runs internet-side (see agent). What each variant compiles in, and why the crate boundary (not #[cfg]) is the airgap guarantee, is the reasoning book's territory.

These three tiers are declared as data and consumed through nornir's build-thing. The holger-build-space crate (build-space/src/lib.rs) fills nornir-build-thing's BuildVariantSet/BuildSurface with holger's real (package × bin × features × profile × target) points — grounded in the actual server/lib + server/cli Cargo features (sign, testmatrix, served, gui), no invented UI. The bytes are the on-disk .nornir/build-variants.toml, include_str!-embedded so descriptor and file can never drift; holger_build_space::build_variant_set() parses them and build_surface() projects the three rows (each with its materialized cargo command and a nornir build --variant <tier> action) a Build tab/CLI renders.

holger is the first real consumer of build-thing (task #36), and it presses the button end-to-end: holger_build_space::press_tier(name, repo_root, outcome, artifacts) (build-space/src/lib.rs:press_tier) validates a tier against the set, then captures a content-addressed build-provenance Attestation binding the verbatim tier descriptor, the materialized argv, the VCS rev + working-tree freshness, the detected toolchain + host (cores/memory), the outcome, and the SHA-256 of every produced artifact — folded to a stable digest. The record is unsealed (is_sealed() is false) — holger never fabricates a signature; the real seal is znippy's detached-CMS signer applied behind nornir's build-provenance feature. This is the dep-light describe/attest seam (it does not itself shell out to cargo); the heavy end-to-end press — spawn cargo build --message-format=json, hash the executables, attest in one shot — is nornir_build_thing::build_variant, driven by nornir build --variant <tier>. The worked demo build-space/examples/press_build_thing.rs (cargo run -p holger-build-space --example press_build_thing) runs the full describe → press → ledger round-trip: reflect the surface, press full, persist to a dep-light JsonFileSink ledger, reload, and prove the digest still verifies.

registry — one trait, many backends

A repository backend is a plugin. Each maps an ArtifactId (namespace / name / version) to a path inside a znippy archive, and speaks its ecosystem's registry protocol over HTTP so a stock cargo/pip/helm/docker client talks to holger unmodified. Every backend implements one trait — RepositoryBackendTrait (traits/src/lib.rs):

The engine holds backends only as Arc<dyn RepositoryBackendTrait> and dispatches through the vtable — it never matches on the concrete type. ArtifactFormat::znippy_type_id() pins each format to znippy's DenseUnion discriminant; a test in traits/src/lib.rs asserts the numbers never drift (znippy is the single source of truth).

Wiring. Config is a directed graph in RON (holger.ron): Repository nodes reference StorageEndpoint and ExposedEndpoint nodes by name. At startup wire_holger() (server/lib/src/lib.rs) resolves it in passes — index nodes → instantiate_backends() (backend_from_config() in server/lib/src/repository.rs matches ron_repo_type to a concrete backend and boxes it; this match is the registry) → wire each repo's IO to storage + exposed endpoints → reverse-link → build a FastRoutes table per exposed endpoint. FastRoutes (server/lib/src/exposed/fast_routes.rs) buckets routes by the first byte of the repo name into two [usize; 256] arrays, so lookup(name) is O(1) and lock-free on the hot path. The same FastRoutes is shared by the gRPC service and LocalHolger.

backend families

All backends share the trait and the storage floor; the only per-family code is the ArtifactId→path mapping and the protocol framing.

• A · real registry protocol — handle_http2_request synthesizes the index on demand: znippy-{python,npm,nuget,go,docker,helm,composer,rust,gem} (Gem = real RubyGems compact-index; Conda = real generated repodata.json).
• B · path passthrough — the client knows the layout; holger serves bytes verbatim: znippy-{maven,deb,rpm,conda}.
• C · native — znippy-package serves znippy's own {name}-{ver}.znippy.

file-rust and file-oci are the only writable backends (the "dynamic"/development shape); every znippy-* backend is read-only by construction (the "static"/drift shape).

• D · virtual aggregate — GroupBackend (server/lib/src/group.rs:GroupBackend) is a read-only "group repo" (the Nexus/Artifactory concept): it owns no storage and resolves each request across an ordered list of MEMBER backends, first hit wins, so a team exposes e.g. maven-all = [maven-releases, maven-snapshots, central-proxy] as one URL. Config declares it with ron_repo_type: "group" + ron_members (member repo names); wire_holger resolves the members in a second pass and backend_from_config() (server/lib/src/repository.rs) boxes the GroupBackend. It is distinct from a proxy — no writable primary, caches nothing, put is refused; format is the members' shared format. Tests: fetch_returns_first_member_hit_in_order, group_is_read_only_and_takes_member_format (server/lib/src/group.rs).

Maven metadata + checksums are synthesized (not just passed through). MavenRepoZnippy (server/repository/znippy-maven/src/lib.rs) synthesizes the native maven-metadata.xml on demand from the index columns the packer extracted — both the GA-level version listing and, for a -SNAPSHOT GAV, version-level snapshot metadata (synthesize_snapshot_metadata → build_snapshot_metadata). holger is an immutable airgap mirror storing snapshots under their literal -SNAPSHOT name (no deploy timestamp), so it advertises a non-unique ("local") snapshot (<snapshot><localCopy>true</localCopy>). Every served path also answers .sha1/.md5 requests by hashing exactly the bytes it serves (sha1_hex/md5_hex), so stock Maven clients verify downloads. Passthrough still wins when the archive already carries upstream metadata. Tests: snapshot_metadata_declares_local_copy_and_lists_classifiers, maven_backend_synthesizes_checksums_and_snapshot_metadata (server/repository/znippy-maven/).

serve — two doors, one backend

There are two front doors onto the same backend:

• handle_http2_request(suburl, body) (server/lib/src/exposed/http.rs) — the native-protocol door. A stock cargo/pip/npm/docker client hits the HTTP gateway; the backend parses the sub-URL into a protocol verb and returns (status, headers, bytes).
• RepositoryService (gRPC, server/lib/src/grpc.rs) — the holger-native door. FetchArtifact returns the whole blob; StreamArtifact chunks it at 64 KB; ListArtifacts / PutArtifact round out the surface. Two more services — ArchiveService and AdminService — live in proto/holger.proto.

Both bottom out in backend.fetch(id), which maps the ArtifactId to an archive path and calls ZnippyReader::extract_file(path) — random-access extraction, no full-archive decompress, blake3-verified. The proto Rust is generated without protoc (pure-Rust protox via cargo xtask gen-proto, committed to server/lib/src/generated/holger.v1.rs).

storage — immutable znippy + writable file backends

• znippy archives — immutable, content-addressed (blake3 per chunk, verified on read), one archive per repository version, built internet-side and carried across the airgap, served read-only via ZnippyReader::extract_file (O(1) random access, no full unpack).
• filesystem — a {repo}/{path} tree for file-rust (writable Cargo sparse registry) and file-oci (writable OCI v2 CAS).

StorageEndpoint (server/lib/src/storage/mod.rs) names a location (znippy | rocksdb) and a path; repos reference it in ron_in / ron_out.

file-rust (server/repository/file-rust/src/lib.rs, RustRepoFile) serves a real Cargo sparse registry out of a {ns}/{name}/{version} disk tree via three routes: /{repo}/index/config.json (JSON, advertising the dl URL), /{repo}/index/{p1}/{p2}/{crate} (NDJSON, one line per version, text/plain), /crates/{crate}/{version}/download (the exact stored bytes). The dl URL is the ron_base_url config knob — there is no 127.0.0.1 fallback. Checksums are the live SHA-256, streamed in 64 KiB chunks (crate_sha256_hex_streaming) and memoized by (len, mtime); stored artifacts are immutable (atomic write-then-rename), so a matching stamp reuses the cached digest.

auth — authenticate, authorize, audit (write path)

A write goes: agent → auth gate → route by repo name → backend.put() → storage. Reads are not auth-gated.

• Authenticate (server/lib/src/auth.rs, validate_request()): an mTLS CN (chain verified once at the TLS handshake by rustls WebPkiClientVerifier, exposed/tls.rs; per request only a leaf-CN parse) or an OIDC bearer token (validated against the issuer's /userinfo through one pooled reqwest::Client + a TTL token cache keyed by (issuer, blake3(token)), default 120 s). Empty method list = open; configured-but-invalid = 401.

• Authorize (RBAC): both write paths run a role check — gRPC put_artifact → authorize_write (grpc.rs) and the HTTP/OCI gateway → authorize_http_write (exposed/http.rs). Role = Reader | Writer | Admin; DELETE needs admin, other writes need writer-or-admin; fail-closed. RBAC is opt-in — off until a roles/default_role policy is set (rbac_enabled()); policy lives in RON, not redb.

• Ingest cap: put payloads over ron_max_body_bytes (default 1 GiB, one knob for gRPC put and the HTTP gateway) are rejected (413 / ResourceExhausted) before the backend sees them.

• Audit (on by default): an append-only Arrow-IPC log records identity (sub/CN) + repo + artifact + action + source IP + timestamp; replay skips a corrupt segment. How much it records is audit.mode in RON, and each setting gives up something named:

  • Everything (default, and what holger always did) — one row per call, blake3-chained to its predecessor and flushed to the OS before the response returns. The chain covers every call.
  • Aggregated — identical calls fold into (ident, action, repo, artifact, ip, status) -> count, bytes, first_seen, last_seen, written every audit.flush_secs (default 30). MEASURED on a 64-address / 16-artifact mirror workload: 19.5× fewer rows and 19.0× fewer segment bytes for 20 000 fetches. The chain then covers the FOLDED rows, and whatever has not been flushed is unwitnessed process memory, lost on a crash. The per-principal and per-IP totals (audit sent-to) stay exact — the fold key keeps every column that tally groups on — but the per-call timeline does not survive, and a --since/--until window cuts at fold boundaries rather than at calls.
  • Off — nothing is recorded and there is no tamper-evidence at all; the chain claims above do not apply. audit query and audit sent-to then REFUSE (holger-audit-8) instead of printing an empty result, on both the gRPC and the CLI door, because an empty answer that means "not recorded" reads as "nothing happened". SecurityProfile::Critical refuses to boot on it.

The UI is not audited separately: the SERVER audits the call, whoever made it — the console, or a plain cargo / pip / docker.

• Path-traversal guards: check_component on the file-* paths rejects empty / . / .. / / / \ / NUL; the agent download path uses safe_join.

agent — the airgap producer

The agent runs internet-side and never needs the server to be reachable. It moves artifacts between systems through the ConnectorTrait seam (traits/src/lib.rs): list_repositories, list_assets, download_asset, upload_asset. Sources/targets: crates.io, Nexus, Holger, a .znippy archive, a directory. Each connector carries a ConnectorAuth (None | Basic | OidcToken | Mtls). Concrete connectors: agent/connector/nexus (Basic auth) and agent/connector/world (crates.io / PyPI, https host allow-list). The agent packs the downloaded set into one signed .znippy and the operator carries it across the boundary on USB/optical; the server serves it read-only. Binary holger-agent (agent/cli).

proxy — pull-through remote cache

ProxyBackend (server/lib/src/proxy.rs) is a registry backend that tries primary then each upstream and write-through-caches a hit. The upstream is pluggable via the RemoteUpstream trait (traits/src/lib.rs) — a read-oriented backend carrying an upstream base URL + the shared UpstreamAuth enum. Concrete impls: NexusUpstream and ArtifactoryUpstream (server/lib/src/upstream/{nexus,artifactory}.rs) alongside the crates.io-shaped RustRegistryUpstream, wired through the nexus-remote / artifactory-remote repository types (ron_base_url + ron_upstream_repo, server/lib/src/repository.rs). So holger targets holger / Nexus / Artifactory interchangeably as a cache backend. Proxy is a full-variant capability (it links egress crates).

embed — the in-process functional API

The parsed RON config is the Holger value (server/lib/src/lib.rs); it carries direct method bindings so a linking consumer drives a live registry with no gRPC socket and no serialization:

These call straight through Arc<dyn RepositoryBackendTrait>. A consumer wanting a repo's contents reaches holger.repositories[i].backend_repository (the same trait object) and calls list / archive_files / fetch; the trait + ArtifactEntry / ArtifactId types are re-exported from holger-server-lib so no separate holger-traits dep is needed. Holger::dev_pair_ron emits the RON for the dev release pair — /cache (a writable-znippy crates.io mirror that fills write-through on miss) and /sparring (a writable release-rehearsal registry). The reference consumer is nornir's embed-holger feature (which lives on the nornir side, not here).

Which recipe reader this repo uses, and why (lane AJ, 2026-08-30)

Reader: nornir-build-thing 0.1.12, PATH (three entries: build-space, xtask, holger-build).

Why: in-constellation. The pin said 0.1.0 while the lockfile compiled 0.1.8 — a version claim no build ever tested. holger's recipe is schema 1. NOTE the open gap: holger's graph still holds a SECOND, registry-sourced copy pulled by published transitives (modgunn, nornir, nornir-config); that closes on the release cascade, not in this manifest.

The rule itself — [nornir-build-thing] schema = N, a refusal that names the version to upgrade to, and a loader that refuses to DROP a key it cannot read — lives once, in edda .nornir/build-design.md, "THE COMPATIBILITY RULE". The cross-version guard that would have caught the 2026-08-30 skew is edda/nornir-build-thing-skew.