nordisk/heimdall

Seccomp profiles: heimdall-seccomp and heimdall-probe

Heimdall is a defensive sandbox. This document covers milestone 3 (spec §3): the profile types and BPF builder (heimdall-seccomp), the learning-mode probe and generator (heimdall-probe), and heimdall profile probe|gen|check.

heimdall-seccomp

Rule semantics (the same in Profile::evaluate, the BPF program and the interpreter): the rules of one syscall are tried in order; a rule matches when its when is empty or any alternative matches; an alternative matches when every ArgCmp holds: (arg & mask) == value, on the low 32 bits for W32 (int parameters, whose upper half the kernel ignores). The first match wins; no match gives the mode's default. In every mode, a foreign arch and (on x86_64) every x32 call (nr >= 0x40000000) are killed.

The launcher. After fork the child puts the program's O_PATH fd at fd 1022 and (with a listener) a socket at fd 1021, both close-on-exec, sets no_new_privs, installs the filter, sends the listener over fd 1021 (SCM_RIGHTS) and calls execveat(1022, "", argv, envp, AT_EMPTY_PATH). Every filter built by the launcher starts with a two-rule prelude that allows exactly that execveat and that sendmsg; a worker's profile needs no bare execve (a test shows the worker's own execve is killed).

The USER_NOTIF API (for heimdall-broker)

use heimdall_seccomp::notif::{AddFd, Listener, Response};
let mut child = Launch::new(prog).profile(&p, &Mode::Enforce).listener(true).spawn()?;
let l: Listener = child.take_listener().unwrap();
Listener::check_kernel()?;                       // struct sizes match this crate
loop {
    match l.wait(None)? { Readiness::Hangup => break, _ => {} }
    let n = l.recv()?;                           // n.id, n.pid, n.data.{nr, arch, args}
    // read the target's memory, decide, open the file yourself...
    if !l.id_valid(n.id)? { continue; }          // the target died / the call was interrupted
    l.add_fd(n.id, file.as_fd(), AddFd { cloexec, ..AddFd::send() })?;  // inject + answer
    // or: l.respond(n.id, Response::Errno(libc::EACCES))?;
}

• Response::Continue lets the kernel run the call on the target's own arguments: only for decisions on registers alone (the probe), never after checking memory the target can change (spec §4 TOCTOU).
• recv returns ENOENT when the target died before the receive; respond returns ENOENT when the call is gone. Neither is a supervisor error.
• send_fd / recv_fd move a listener over a unix socket from a process that installed its own filter.
• Kernel: USER_NOTIF 5.0, CONTINUE 5.5, ADDFD 5.9, ADDFD_FLAG_SEND 5.14.

heimdall-probe

1. Learning mode (learn): the filter's default is SECCOMP_RET_USER_NOTIF; the probe records every call and answers CONTINUE. No strace, no ptrace (no ptrace fallback is built: every kernel Heimdall targets has 5.5). clone3 is answered ENOSYS by the filter itself, so the libc falls back to clone, whose flags a filter can check; generated profiles keep that rule.
2. Argument facts (facts::SPECS): socket/socketpair domain, type and protocol; open/openat flags; ioctl request; mmap/mprotect/pkey_mprotect prot; clone flags; prctl option; fcntl command. Everything else is recorded as the bare call.
3. Merge and minimise (generate): exact tuples while at most exact_limit (16) distinct, else a reported bare allow; flag fields (openat, clone) become "no bit outside the observed union"; prot values are exact, and PROT_WRITE|PROT_EXEC together is reported and left out unless the workload sets allow_wx.
4. Review: near-miss calls (seen only in error_path runs; kept in the profile, listed), W|X findings, bare allows, unknown calls, fixed rules, the vDSO note. In the .json (review) and the .rs header.
5. Codegen: profiles/<workload>/<arch>-<libc>.rs (a const StaticProfile, fully qualified paths, include!-able), .json, .oci.json. No dates; sorted; byte-identical for the same observations.
6. Drift check (check): a fresh probe must be a subset of the checked-in .json; each failure reads new call sysinfo (kill_process), made by test basic_run. .rs and .oci.json must match what .json generates.

Learning where a program really runs

heimdall profile probe-exec and probe-container (CLI.md) learn ONE program while something else drives it, with the same filter and prelude as learn, and merge into an evidence file gen can read (heimdall_probe::live::{Evidence, record}):

• probe-exec: the program runs on the host as it would anyway (a daemon on a booted node, a plugin its host runs).
• probe-container: the program runs as a HeimdallRunner native job inside Heimdall's own container. The runtime installs no seccomp profile; the executor, inside the job after Landlock, installs the learning filter with NEW_LISTENER (ExecSelf::install_listener, no TSYNC: the job's init is single-threaded), sends the listener over the prelude's sendmsg on HANDOFF_FD, reports on its report fd and execveats the program. The shim hands the listener and the program's pid to the runner over an abstract unix socket (the runner accepts only its shim's pid, SO_PEERCRED); the per-boot key never leaves the runner, and no result is signed. The calls the prelude allows (execveat on EXEC_FD, sendmsg on HANDOFF_FD, write on the report fd) are not recorded, exactly as with Launch, so a profile generated from this evidence fits genprof's enforcement unchanged.

A run of a daemon (Postgres under TPC-DS, a web server under HTTP load) is error-path free unless the driver makes it fail; name such runs --error-path so their calls show as near-misses in the review.

workload.toml

workload = "heimdall-wasm-host"
timeout_secs = 60     # per run
repeat = 3            # every run this many times; the union is kept
exact_limit = 16
allow_wx = false
[binary]
glibc = "{target}/release/heimdall"
musl = "{target}/x86_64-unknown-linux-musl/release/heimdall"
[[run]]
name = "good_input_is_accepted"         # the test name check reports
args = ["run", "--job", "wasm", "--module", "{corpus}/good.wasm", "--out", "{tmp}/out"]
expect_exit = 0                         # a broken corpus fails the probe
[[run]]
name = "tampered_input_is_refused"
error_path = true
args = ["..."]
expect_exit = 4

Placeholders: {dir}, {corpus} (<dir>/corpus), {tmp} (a fresh dir per run, also its cwd), {target} ($CARGO_TARGET_DIR or <profiles>/../target). Runs start from an empty environment plus PATH=/usr/bin:/bin and [env].

Tests and where they run

• cargo test -p nordisk-heimdall-seccomp -p nordisk-heimdall-probe: builder == evaluate (proptest through the interpreter), modes, OCI, minimise, codegen, check. No kernel needed.
• scripts/seccomp-tests.sh: the kernel tests (--test kernel of both crates, harness = false, the test binary is its own worker) in a privileged rootless podman container, then the drift check of every checked-in profile.

Checked-in profiles

Profiles a program's own repository checks in (measured with heimdall profile probe-exec where the program really runs, generated with heimdall profile gen, MIT-0; lane prof, 2026-11-01, oden kernel 7.0):

Both were run green under their profile on a real kernel, and a call outside each was killed (the scan worker: signed heimdall-seccomp-1; clamd: SIGSYS, exit 159, clamd still serving); the READMEs there list the corpus and how to measure again.

The crate nordisk-heimdall-profiles (profiles/; publishable from 0.3.0, because heimdall-container and heimdall-cli enforce its tables at runtime; the package holds only the generated .rs/.json/.oci.json and workload.toml, not the corpus) compiles every .rs and holds it to its .json and .oci.json; its enforce test runs each profile in enforcing mode on the real release binary over the whole corpus. The corpus guests are written by crates/heimdall-cli/tests/profile_corpus.rs (small WAT guests, corpus/) and guests/verify-bundle/tests/probe_corpus.rs (the REAL verify-bundle guest, single and split with N = 1 and N = 8 decoder instances, over good stored, good OpenZL, many-chunk OpenZL, tampered and two bomb bundles: verify-bundle/ and the marked block of workload.toml). With the real guest the rule count stayed the same: wasmtime's own threads already needed every clone, futex, madvise and mmap tuple the decoder workers use; mremap moved from near-miss to a normal path (a good bundle grows the guest's memory).

A generated .rs starts with // SPDX-License-Identifier: MIT-0 and says it is generated output of Heimdall, not covered by Heimdall's AGPL: a profile is data about the profiled program, so MIT, Apache and proprietary repos can check it in. MIT-0 rather than CC0-1.0: it is OSI-approved and accepted where code licences are checked (some policies refuse CC0 for code, for its patent clause); it asks for no attribution. The .json forms carry no licence text.

Enforcing the profiles at runtime

• Native jobs (heimdall-container): a job's policy.seccomp (or the runner's HeimdallConfig::seccomp) names a profile by workload + digest or as an OCI JSON file pinned by sha256; the executor installs it on top of the fixed profile right before the job's program starts. Details, refusals and the fallback: API.md, "Generated seccomp profiles". heimdall_seccomp::oci::from_oci reads an enforcing OCI JSON back into a Profile (fails closed on anything it cannot represent exactly).
• Enforcing the wasm host profile (heimdall run --job wasm): wasmtime is the only native code of a wasm job, so the smallest place for the filter is the process that runs the guest. The parent heimdall run (no filter, holds the per-boot key) starts itself again as a worker through heimdall_seccomp::launch::Launch with the heimdall-wasm-host profile of its arch and libc: the same prelude and start the probe and profiles/tests/enforce.rs use, a clean environment (PATH, RUST_BACKTRACE=0). The worker loads and runs the guest and prints the unsigned result; the parent re-checks the output files, signs, and states Generated { profile_digest }. SIGSYS = signed Refused heimdall-seccomp-1. Debug builds default to --seccomp none (a debug binary makes calls the release profile does not have, e.g. std's fcntl(F_GETFD) assertions); the probe corpus sets HEIMDALL_SECCOMP = "none" so it profiles the whole host in one process.

What is not built

• aarch64: the syscall table and the BPF builder support it (tests run the aarch64 filter through the interpreter); no aarch64 box probed a workload yet, so there is no aarch64 profile and no aarch64 kernel test.
• The ptrace fallback for kernels before 5.5.