nordisk/holger

5. The intake chain

Every package walks the same chain before it is served. The chain has four repositories: incoming → quarantine → approved → production. Only what passed is served.

States

A package's states, in order (server/lib/src/promote/intake.rs, ArtifactState): UNTRUSTED, INGESTED, SCANNED, POLICY_CHECKED, APPROVED, SEALED, PROMOTED, AVAILABLE.

• AVAILABLE is the only state that is downloadable. Every index (PEP 503, the packument, maven-metadata.xml, the sparse index, the OCI tag list, APKINDEX, the apt and dnf metadata) lists only AVAILABLE packages, apart from the held lines described below.
• Holds. BLOCKED is final. QUARANTINED keeps the bytes; a release puts the package back where it was held.
• A refusal names its reason and is audited.

The console's Security page and holger-cli intake list --queue <q> show the queues (intake.rs, the queue table; an unknown name is refused and the known ones listed):

Duties and approvals

Who may move a package is a duty (server/lib/src/promote/duties.rs):

• steps: (approvals: N) needs N distinct approvers. The default is 2 (server/lib/src/sources.rs, Steps; test steps_ask_for_two_approvers_by_default in server/lib/tests/safe_defaults.rs). The old spelling promotion.intake.approvals defaults to 1. approvals: 0 is refused at load.
• The importer never approves its own package (holger-duty-1). One person approves once (holger-duty-3). anonymous never approves (holger-duty-5). Each approval is an in-toto statement in the package's evidence. The proof is server/lib/tests/intake_separation_of_duties.rs (a_wheel_needs_two_distinct_approvers_who_never_include_its_importer).

Walk a package by hand

Upload as an Importer, then move it one state at a time:

curl -fsS -X PUT -H "Authorization: Bearer $IMPORTER_TOKEN" \
  --data-binary @demo-1.0.txt "$HOLGER/incoming/demo-1.0.txt"
holger-cli --at "$HOLGER_GRPC" --token-file importer.token intake advance --name demo-1.0.txt --reason scan
holger-cli --at "$HOLGER_GRPC" --token-file importer.token intake advance --name demo-1.0.txt --reason policy
holger-cli --at "$HOLGER_GRPC" --token-file approver1.token intake advance --name demo-1.0.txt --reason approved
holger-cli --at "$HOLGER_GRPC" --token-file approver2.token intake advance --name demo-1.0.txt --reason approved
for s in sealed promoted available; do
  holger-cli --at "$HOLGER_GRPC" --token-file admin.token intake advance --name demo-1.0.txt --reason "$s"
done
holger-cli --at "$HOLGER_GRPC" --token-file auditor.token intake state --name demo-1.0.txt

The same over HTTP: POST /-/intake/advance?name=<file>&verb=advance&reason=<why> and GET /-/intake/state?name=<file>; the evidence at GET /-/intake/evidence?sha256=<hex> (server/lib/src/exposed/http.rs). Over gRPC: PromotionService.AdvanceArtifact, GetArtifactState and ListIntakeArtifacts. The README guide walks a wheel exactly so (chapter 3).

Every package is scanned in its ecosystem

The vulnerability scan matches a package against the advisories of its own ecosystem.

• Pulled packages are scanned as their source's ecosystem: every Python file (sdist or wheel) as pip, npm and Maven with their coordinates, crates as cargo, and an APK by its repository's distribution (Chainguard, Wolfi or Alpine) (server/lib/src/pull_intake/mod.rs, scan_target, apk_distro; test a_pulled_sdist_is_scanned_as_its_python_package_and_a_known_cve_refuses_it in server/lib/tests/pull_through_runs_the_intake_chain.rs).
• Uploads under a chain view are scanned as the view's format, with the coordinates the path names: deb/… as deb, rpm/… as rpm, a maven3 view as Maven with group:artifact, helm/… as Helm (intake.rs, view_target).
• A package file uploaded as a raw file is held, never passed. A .jar .war .ear .pom .deb .rpm .crate .gem .nupkg .apk .conda that reaches the scan as raw is blocked with the finding holger-no-ecosystem: "… is a … package, but no ecosystem names it here (it came in as a raw file), so no advisory can be matched; upload it under its format's repository" (server/lib/src/scanner/mod.rs).

The proof is server/lib/tests/an_upload_is_scanned_as_its_format.rs: the real log4j-core 2.14.1 POM uploaded under the Maven view is refused for Log4Shell (a_maven_upload_is_scanned_as_maven_and_log4shell_refuses_it), and a .pom in the raw root is held (a_package_file_in_the_raw_root_is_held_not_passed_as_raw).

No intel, no pass

The vulnerability scan reads only the local advisory database (chapter 4). Without one, or with one that is too old, it holds every package. It never passes one with "0 findings" (intel/src/resolve.rs, fold_staleness):

The age counts from the bundle's snapshot date, not from the import. These words are not counted as advisory findings in the scan's sentence (server/lib/src/intel.rs). Tests: with_no_intel_imported_the_cve_scan_holds_and_says_why and with_intel_past_its_block_limit_the_cve_scan_holds_and_says_why in an_upload_is_scanned_as_its_format.rs.

Fill or refresh the database with a signed intel bundle:

holger intel build --feeds-dir feeds --policy intel-trust.ron \
  --release-cert releaser.der --release-key releaser.pk8 \
  --approve "security-reviewer=reviewer.der,reviewer.pk8" --out intel.holger.znippy
# the first intel bundle on a box needs an admin's grant, once
holger bundle allow-first-import --config holger.ron --from-bundle intel.holger.znippy --policy intel-trust.ron
holger intel import intel.holger.znippy --config holger.ron

Over HTTP the bundle goes to POST /-/intel/import (RepositoryAdmin or SecurityApprover); the first-import grant is POST /-/bundle/first-import (RepositoryAdmin). Without the grant the import is refused, holger-import-9 (server/lib/src/import_door.rs; test an_intel_bundle_posted_over_http_is_imported_and_the_scan_reads_it in server/lib/tests/the_box_imports_bundles_without_a_shell.rs).

The scan worker must be this server's build

Every scan runs in holger-scan-worker, in a container (chapter 8). The worker must be the same build as the server:

• The default image is localhost/holger-scan-worker:<Holger release> (:3.1.0 for Holger 3.1.0) (server/sandbox/src/lib.rs, DEFAULT_IMAGE over RELEASE; test the_default_worker_image_names_this_version). The tag is Holger's release, not the version of the holger-sandbox crate.
• Every scan report names the worker's version and scanners (server/lib/src/sandbox/wire.rs, WorkerBuild).
• The server refuses a report that names no build, another version or fewer scanners, as holger-sandbox-5, with the fix in the message: "rebuild the worker from this server's tree (cargo xtask scan-worker image for podman, cargo xtask scan-worker stage for a medium's rootfs)" (wire.rs, check_build; test a_worker_older_than_this_server_is_refused_by_name_with_the_rebuild_command in server/lib/src/sandbox/tests.rs).

After every upgrade, rebuild the worker image on each host: cargo xtask scan-worker image (podman), or the README's recipe with the new version as the tag. A RON that names an old tag in sandbox: (image: …) must name the new one.

A scan that could not run is retried

When the scanners could not judge a pulled package (the sandbox refused, the worker is not this build, clamd did not answer), it is recorded as scan-failed, not refused:

• Nothing is stored and nothing is served (fail closed). The client gets a 404 that says the scanners could not judge it.
• After pull_through.retry_after_secs (default 2 s) the next request pulls and scans it again, once the scanners can run.
• A refusal that judged the bytes (malware, a gate, a known CVE) stays a refusal.

(server/lib/src/pull_intake/mod.rs, ScanFailed; test a_pull_the_scanners_could_not_judge_is_scanned_again_once_they_can in server/lib/tests/pull_through_runs_the_intake_chain.rs.)

The pull-through

A client asks Holger; Holger pulls the package from the source named in pull_through: (chapter 4), and the package walks the chain like an upload (server/lib/src/pull_intake/door.rs):

Nothing is approved automatically by default, so run the client again after the approvals. A client that resolves dependencies one level at a time (pip, Maven) needs one run per level.

Pinned versions through the pull-through

A pinned version that is not the newest pulls through. The index names the versions that are not AVAILABLE in the way each client skips unless it is pinned exactly:

The reason a client prints is "not in Holger yet: … pulls it into Holger's intake chain, where it waits for approval", or "held by Holger's intake chain (STATE), waiting for approval" (server/lib/src/pull_intake/mod.rs, ecosystems.rs). Blocked and quarantined files are never listed.

Maven

• Plugin prefixes resolve: the group-level maven-metadata.xml (for example org/apache/maven/plugins/) is served as the source's plugin list (names and prefixes; the plugin jars still walk the chain) (test a_plugin_groups_metadata_names_its_prefixes_for_maven).
• A Maven source waits up to wait_secs (default 30 s) for its pull, then answers 404. Maven caches a 404, so run Maven with -U after the approvals.

Go

@v/list and @latest name only AVAILABLE versions. Holger serves no checksum database and does not proxy sum.golang.org (server/lib/src/pull_intake/go.rs), so the client turns the checksum database off or uses one the site runs (chapter 3).

A pull is approved as one dependency set (3.1.1)

When a client asks for a package, Holger pulls its whole dependency closure at once: for Maven, every POM Maven reads (parents, imported BOMs, every version the graph visits) and the JARs of the versions that win mediation; for npm, cargo and Python, the dependencies their registries' metadata names. Every file walks the chain on its own (inspection, malware, the scanners and provenance), and the approvers see ONE set:

• GET /-/intake/sets and GET /-/intake/set?id= show each set with its members, states, votes, and the members that refuse or hold it. holger-ui shows them on the Security page.
• POST /-/intake/set/approve?id=&reason= casts the caller's vote on every member waiting for approval. Each member still needs its distinct approvers, never its importer.
• POST /-/intake/set/refuse?id=&reason= blocks every waiting member.
• A member with a refusal reason refuses the set, named (holger-depset-2). A member that could not be resolved or scanned holds the set (holger-depset-3). Nothing in a refused or held set is approved.
• From the command line, over gRPC (PromotionService.ListDependencySets, GetDependencySet, ApproveDependencySet, RefuseDependencySet, the same code as the doors): holger-cli intake sets [--status awaiting-approval], holger-cli intake set <id>, holger-cli intake set approve <id> [--reason <why>] and holger-cli intake set refuse <id> --reason <why> (holger-cli/tests/dependency_sets_from_the_cli.rs).

A Maven POM's advisories are a warning (security-policy change, 3.1.1)

A .pom carries no code. Maven reads POMs to build the dependency graph, and it reads the POMs of old versions that dependency mediation then drops, without fetching their JARs. A real mvn dependency:get of commons-lang3 reads the POMs of commons-collections 3.2.1, log4j 1.2.12 and dom4j 1.1 under maven-dependency-plugin 3.7.0. Holger used to refuse those POMs by their CVEs, and Maven fails on a refused descriptor, so the plugin could never resolve.

Since 3.1.1, the vulnerability scan of a Maven .pom records its advisories as a WARN: they are named in the evidence, in the dependency set and in the console, and the rule cve_max does not refuse on them. The JAR, WAR, EAR or AAR of the same coordinates is matched and refused as before. Malware, licence, provenance, secrets and the inspection gates still block a POM. The rule applies to the Maven ecosystem and the .pom suffix only (server/lib/src/scanner/mod.rs, advisories_warn_only).

Scanned pushes to an oci repository

On a Holger with an intake chain, every oci repository is a scanned registry (server/lib/src/promote/oci_push.rs):

• A push by tag enters the chain as the pusher, walks SCANNED and POLICY_CHECKED, then waits for its approvals, never the pusher's. Nothing is served before the image is AVAILABLE.
• A refused push fails the push itself with 403 DENIED and the reason.
• Writes need the Importer duty; a DELETE is refused (holger-oci-3).
• The opt-out is promotion.intake.oci_push: [(repository: "<name>", scan: off)] (also steps.oci_push). It is refused where a scan is required (security_profile: Critical, or scan.required; holger-oci-1); an unknown registry name is holger-oci-2.

Still in v1

• v1/secure-update-gateway-mvp.md: the bundle format and its lifecycle (the current text is chapter 8).