nordisk/Skidbladnir

Upstream-registry federation — the ship between harbours

Status: BUILT. skidbladnir::federation (feature federation, + upstream-azure / upstream-aws) ships the UpstreamRegistry trait and five harbour clients — Artifactory, Nexus, Azure Artifacts, AWS CodeArtifact, and OCI Distribution (GHCR/Harbor/Quay/Docker Hub) — proven by offline tests (cargo test --features federation,upstream-azure,upstream-aws). This doc is the architecture decision: where the federation lives and who consumes it.

The metaphor — Skíðblaðnir is the ship

In the myth Skíðblaðnir is Freyr's expert vessel: it always finds a fair wind and folds away when not in use. Here it is the ship that sails between harbours.

• The harbours are the external registries and clouds — JFrog Artifactory, Sonatype Nexus, Azure Artifacts (Azure DevOps feeds), AWS CodeArtifact, and any OCI Distribution v2 registry (GHCR, Harbor, Quay, Docker Hub). Each has its own docks (URL layout) and harbourmaster (auth).
• The ship is skidbladnir::federation: the reusable library expert that knows how to dock at every harbour, show its papers (auth), take on cargo (fetch), and run a full voyage (mirror). It owns the mechanism.
• holger is a harbour that HIRES the ship. holger does not own docks or sailors; when it needs to bring artifacts across from an upstream, it charters Skíðblaðnir. holger owns the voyage orders — the orchestration/policy: which feeds to pull, scheduling, dedup-on-store, and compression via znippy.

            hires the ship for a voyage
 holger  ───────────────────────────────▶  Skíðblaðnir (the ship)
 (harbour / consumer)                        skidbladnir::federation
   • which feeds, scheduling                   • UpstreamRegistry trait
   • pull-through cache, dedup-on-store         • auth · list · fetch · mirror
   • compression via znippy                     │
                                                ▼   sails between harbours
              ┌──────────┬────────┬───────────────┬────────────────┬──────────────────┐
           Artifactory  Nexus   Azure Artifacts  AWS CodeArtifact  OCI Distribution
                                                                   (GHCR/Harbor/Quay)

The contract — UpstreamRegistry

One trait (src/federation/mod.rs), five impls. The shared vocabulary (UpstreamAuth, ArtifactMeta, ArtifactCoord, RegistryFormat, upstream_path) lives once in the module so every harbour shares one auth enum, one metadata shape, and one coordinate→layout mapping.

The impls: ArtifactoryRegistry, NexusRegistry, AzureArtifactsRegistry (feature upstream-azure), AwsCodeArtifactRegistry (feature upstream-aws), and OciDistributionRegistry (base federation).

The OCI Distribution harbour (a different protocol, not a twin)

The four file-path harbours GET a per-ecosystem artifact path (upstream_path). OciDistributionRegistry speaks the OCI Distribution Spec v2 instead — a content-addressed image store, so it is genuinely non-twin coverage:

• fetch(id) GETs the image manifest at {base}/v2/{repository}/manifests/{reference} (id.name = repository, e.g. nordisk/holger; id.version = a tag or sha256:… digest), sending the OCI Accept header so the registry returns an image manifest/index.
• metadata(id) HEADs the manifest (size + Docker-Content-Digest + media type).
• fetch_blob(repository, digest) pulls a config/layer blob at {base}/v2/{repository}/blobs/{digest} and verifies it content-addresses to its digest — a protocol-native integrity guarantee (a tampered layer is a hard error), reusing the shared verify_sha256.
• Auth — token auto-handshake. On a 401 carrying a WWW-Authenticate: Bearer challenge, the harbour parses it (parse_bearer_challenge — realm/service/scope; a malformed challenge or missing realm is a hard error), mints a token at the realm's token endpoint (echoing service/scope, carrying the operator's Basic/Bearer creds to the realm origin), and RETRIES the request once with Authorization: Bearer <token> — the standard Distribution flow, all three of fetch/metadata/fetch_blob routed through it. The token creds are still supplied out-of-band (gh auth token / docker login / a PAT); only the challenge→token→retry negotiation is automated. Convenience constructors: OciDistributionRegistry::ghcr(auth) and harbor(base, auth).

The mock server can emit response headers

The federation test mock (testutil::MockServer) gained start_with — a responder that sees the whole request (so it can branch on Authorization) and returns a MockResponse with arbitrary response headers. start is a thin wrapper (header-less fixtures unchanged, L2). This unblocked two things that were previously untestable: the OCI token handshake above (fixture emits the WWW-Authenticate challenge + serves its own /token), and HEAD-checksum assertions — the shared head_meta now has a test proving it reads X-Checksum-Sha256 + Content-Type (covering all four file-path harbours, which share it), OCI's metadata reads Docker-Content-Digest, and Artifactory has an end-to-end test that a mismatched advertised checksum makes fetch hard-error (the cache-poisoning guard, proven end-to-end).

Pure-Rust by charter — no async cloud SDK, no C toolchain

Every harbour speaks its documented HTTP endpoint over a shared SYNC ureq client (rustls/ring). We deliberately do not vendor aws-sdk-codeartifact, aws-config, or the async azure_* crates:

• their default TLS stack is aws-lc-rs, which pulls a C/cmake/nasm build toolchain — a direct violation of the nordisk pure-Rust charter (this very crate's Cargo.toml pins ring, "not aws_lc_rs", "no C/cmake build");
• the one thing the SDK is genuinely needed for — SigV4-signing the CodeArtifact GetAuthorizationToken STS call, or minting an Azure AAD token — is done out-of-band by the operator's aws / az CLI, exactly as npm/pip/cargo obtain the same token. The resulting bearer token / PAT is provisioned via config, and the ship carries it as UpstreamAuth::Bearer / PAT-over-Basic.

So no AWS/Azure crypto is hand-rolled here — the ship is a faithful HTTP client of each harbour's package-manager-compatible endpoint. This keeps full-tier holger green with zero C dependencies (verified: cargo tree shows no aws-lc, aws-sdk, azure_*, or openssl).

The boundary (the architecture decision)

• Federation mechanism lives HERE, in Skidbladnir — one home for the registry clients. Artifactory, Nexus, Azure and AWS all sit behind UpstreamRegistry.
• holger is a CONSUMER (the top-level feature). It keeps the user-facing capability + orchestration/policy ("mirror upstream X into holger", pull-through caching, dedup-on-store, compression via znippy) and calls the library through the trait. holger's full tier depends on skidbladnir/{federation,upstream-azure,upstream-aws}; its mini/embedded tiers link none of it (see holger's .nornir/holger-tiers.md → Deployment modes). holger must not reimplement registry clients — that would duplicate the ship.
• znippy stays pack-format / format-export ONLY — no cloud, no registry, no clone knowledge. holger calls znippy for compression of what the ship fetched.
• znippy-zoomies is unrelated (scan/compression research) — carries no registry code.

Cross-reference — the nordisk constellation dedup law

This split is the constellation dedup law applied: a capability's mechanism lives in exactly ONE repo, and every other repo depends on it rather than re-implementing it (the same law that moves korp's lineage/segmentation into skade, and jera's job handler into edda — "no dup code"). Registry-client code has one home: Skidbladnir. holger (and any future consumer — nornir, tunnr) hires the ship; it does not build its own.

Feature map