nordisk/heimdall

Seccomp profiles: heimdall-seccomp and heimdall-probe

Heimdall is a defensive sandbox. This document covers milestone 3 (spec §3): the profile types and BPF builder (heimdall-seccomp), the learning-mode probe and generator (heimdall-probe), and heimdall profile probe|gen|check.

heimdall-seccomp

Rule semantics (the same in Profile::evaluate, the BPF program and the interpreter): the rules of one syscall are tried in order; a rule matches when its when is empty or any alternative matches; an alternative matches when every ArgCmp holds: (arg & mask) == value, on the low 32 bits for W32 (int parameters, whose upper half the kernel ignores). The first match wins; no match gives the mode's default. In every mode, a foreign arch and (on x86_64) every x32 call (nr >= 0x40000000) are killed.

The launcher. After fork the child puts the program's O_PATH fd at fd 1022 and (with a listener) a socket at fd 1021, both close-on-exec, sets no_new_privs, installs the filter, sends the listener over fd 1021 (SCM_RIGHTS) and calls execveat(1022, "", argv, envp, AT_EMPTY_PATH). Every filter built by the launcher starts with a two-rule prelude that allows exactly that execveat and that sendmsg; a worker's profile needs no bare execve (a test shows the worker's own execve is killed).

The USER_NOTIF API (for heimdall-broker)

use heimdall_seccomp::notif::{AddFd, Listener, Response};
let mut child = Launch::new(prog).profile(&p, &Mode::Enforce).listener(true).spawn()?;
let l: Listener = child.take_listener().unwrap();
Listener::check_kernel()?;                       // struct sizes match this crate
loop {
    match l.wait(None)? { Readiness::Hangup => break, _ => {} }
    let n = l.recv()?;                           // n.id, n.pid, n.data.{nr, arch, args}
    // read the target's memory, decide, open the file yourself...
    if !l.id_valid(n.id)? { continue; }          // the target died / the call was interrupted
    l.add_fd(n.id, file.as_fd(), AddFd { cloexec, ..AddFd::send() })?;  // inject + answer
    // or: l.respond(n.id, Response::Errno(libc::EACCES))?;
}

• Response::Continue lets the kernel run the call on the target's own arguments: only for decisions on registers alone (the probe), never after checking memory the target can change (spec §4 TOCTOU).
• recv returns ENOENT when the target died before the receive; respond returns ENOENT when the call is gone. Neither is a supervisor error.
• send_fd / recv_fd move a listener over a unix socket from a process that installed its own filter.
• Kernel: USER_NOTIF 5.0, CONTINUE 5.5, ADDFD 5.9, ADDFD_FLAG_SEND 5.14.

heimdall-probe

1. Learning mode (learn): the filter's default is SECCOMP_RET_USER_NOTIF; the probe records every call and answers CONTINUE. No strace, no ptrace (no ptrace fallback is built: every kernel Heimdall targets has 5.5). clone3 is answered ENOSYS by the filter itself, so the libc falls back to clone, whose flags a filter can check; generated profiles keep that rule.
2. Argument facts (facts::SPECS): socket/socketpair domain, type and protocol; open/openat flags; ioctl request; mmap/mprotect/pkey_mprotect prot; clone flags; prctl option; fcntl command. Everything else is recorded as the bare call.
3. Merge and minimise (generate): exact tuples while at most exact_limit (16) distinct, else a reported bare allow; flag fields (openat, clone) become "no bit outside the observed union"; prot values are exact, and PROT_WRITE|PROT_EXEC together is reported and left out unless the workload sets allow_wx.
4. Review: near-miss calls (seen only in error_path runs; kept in the profile, listed), W|X findings, bare allows, unknown calls, fixed rules, the vDSO note. In the .json (review) and the .rs header.
5. Codegen: profiles/<workload>/<arch>-<libc>.rs (a const StaticProfile, fully qualified paths, include!-able), .json, .oci.json. No dates; sorted; byte-identical for the same observations.
6. Drift check (check): a fresh probe must be a subset of the checked-in .json; each failure reads new call sysinfo (kill_process), made by test basic_run. .rs and .oci.json must match what .json generates.

workload.toml

workload = "heimdall-wasm-host"
timeout_secs = 60     # per run
repeat = 3            # every run this many times; the union is kept
exact_limit = 16
allow_wx = false
[binary]
glibc = "{target}/release/heimdall"
musl = "{target}/x86_64-unknown-linux-musl/release/heimdall"
[[run]]
name = "good_input_is_accepted"         # the test name check reports
args = ["run", "--job", "wasm", "--module", "{corpus}/good.wasm", "--out", "{tmp}/out"]
expect_exit = 0                         # a broken corpus fails the probe
[[run]]
name = "tampered_input_is_refused"
error_path = true
args = ["..."]
expect_exit = 4

Placeholders: {dir}, {corpus} (<dir>/corpus), {tmp} (a fresh dir per run, also its cwd), {target} ($CARGO_TARGET_DIR or <profiles>/../target). Runs start from an empty environment plus PATH=/usr/bin:/bin and [env].

Tests and where they run

• cargo test -p nordisk-heimdall-seccomp -p nordisk-heimdall-probe: builder == evaluate (proptest through the interpreter), modes, OCI, minimise, codegen, check. No kernel needed.
• scripts/seccomp-tests.sh: the kernel tests (--test kernel of both crates, harness = false, the test binary is its own worker) in a privileged rootless podman container, then the drift check of every checked-in profile.

Checked-in profiles

Profiles a program's own repository checks in (measured with heimdall profile probe-exec where the program really runs, generated with heimdall profile gen, MIT-0; lane prof, 2026-11-01, oden kernel 7.0):

Both were run green under their profile on a real kernel, and a call outside each was killed (the scan worker: signed heimdall-seccomp-1; clamd: SIGSYS, exit 159, clamd still serving); the READMEs there list the corpus and how to measure again.

The crate nordisk-heimdall-profiles (profiles/; publishable from 0.3.0, because heimdall-container and heimdall-cli enforce its tables at runtime; the package holds only the generated .rs/.json/.oci.json and workload.toml, not the corpus) compiles every .rs and holds it to its .json and .oci.json; its enforce test runs each profile in enforcing mode on the real release binary over the whole corpus. The corpus guests are written by crates/heimdall-cli/tests/profile_corpus.rs (small WAT guests, corpus/) and guests/verify-bundle/tests/probe_corpus.rs (the REAL verify-bundle guest, single and split with N = 1 and N = 8 decoder instances, over good stored, good OpenZL, many-chunk OpenZL, tampered and two bomb bundles: verify-bundle/ and the marked block of workload.toml). With the real guest the rule count stayed the same: wasmtime's own threads already needed every clone, futex, madvise and mmap tuple the decoder workers use; mremap moved from near-miss to a normal path (a good bundle grows the guest's memory).

A generated .rs starts with // SPDX-License-Identifier: MIT-0 and says it is generated output of Heimdall, not covered by Heimdall's AGPL: a profile is data about the profiled program, so MIT, Apache and proprietary repos can check it in. MIT-0 rather than CC0-1.0: it is OSI-approved and accepted where code licences are checked (some policies refuse CC0 for code, for its patent clause); it asks for no attribution. The .json forms carry no licence text.

Enforcing the profiles at runtime

• Native jobs (heimdall-container): a job's policy.seccomp (or the runner's HeimdallConfig::seccomp) names a profile by workload + digest or as an OCI JSON file pinned by sha256; the executor installs it on top of the fixed profile right before the job's program starts. Details, refusals and the fallback: API.md, "Generated seccomp profiles". heimdall_seccomp::oci::from_oci reads an enforcing OCI JSON back into a Profile (fails closed on anything it cannot represent exactly).
• Enforcing the wasm host profile (heimdall run --job wasm): wasmtime is the only native code of a wasm job, so the smallest place for the filter is the process that runs the guest. The parent heimdall run (no filter, holds the per-boot key) starts itself again as a worker through heimdall_seccomp::launch::Launch with the heimdall-wasm-host profile of its arch and libc: the same prelude and start the probe and profiles/tests/enforce.rs use, a clean environment (PATH, RUST_BACKTRACE=0). The worker loads and runs the guest and prints the unsigned result; the parent re-checks the output files, signs, and states Generated { profile_digest }. SIGSYS = signed Refused heimdall-seccomp-1. Debug builds default to --seccomp none (a debug binary makes calls the release profile does not have, e.g. std's fcntl(F_GETFD) assertions); the probe corpus sets HEIMDALL_SECCOMP = "none" so it profiles the whole host in one process.

What is not built

• aarch64: the syscall table and the BPF builder support it (tests run the aarch64 filter through the interpreter); no aarch64 box probed a workload yet, so there is no aarch64 profile and no aarch64 kernel test.
• The ptrace fallback for kernels before 5.5.