nordisk/holger

Proto codegen via xtask — no protoc, no shell-out

Design + guide for cargo xtask gen-proto. Status: shipped on branch agent/holger-noprotoc.

Why (the bug it fixes)

server/lib/build.rs used to call tonic_build::compile_protos("proto/holger.proto"). That shells out to the protoc binary at every cargo build. On checkouts without protoc in PATH (notably the nornir server user) the build-script fails, which:

• kills holger-server-lib's build/doctest,
• takes down everything path-dep'd on it (the viz test arms — 4 red rows, run 26806bc4 — and holger/skade/nornir benches that path-dep nornir → holger).

Rickard's directive: "use an xtask, avoid shelling." Generate the proto Rust once, commit it, and build against the committed code.

Design

proto/holger.proto
        │
        │  cargo xtask gen-proto         (developer / CI step, NOT build time)
        ▼
  ┌───────────────────────────────────────────────────────────────┐
  │ protox::compile([proto], [proto_dir])   ← PURE RUST, no protoc │
  │     → prost_types::FileDescriptorSet                           │
  │ tonic_build::configure()                                       │
  │     .build_server(true).build_client(true)                     │
  │     .out_dir(server/lib/src/generated)                         │
  │     .skip_protoc_run()           ← guarantees protoc unused    │
  │     .compile_fds(fds)                                          │
  └───────────────────────────────────────────────────────────────┘
        │
        ▼
  server/lib/src/generated/holger.v1.rs      ← COMMITTED to the repo
        │
        │  include!("generated/holger.v1.rs")  (src/grpc.rs)
        ▼
  normal `cargo build`  →  NO protoc, NO subprocess

Key choices

1. protox (pure-Rust protobuf compiler) does the parse step — there is no protoc binary and no subprocess anywhere in the pipeline, matching holger's "100% Rust" ethos. Its FileDescriptorSet output feeds tonic-build's compile_fds, so we keep tonic's exact server/client codegen.
2. skip_protoc_run() on the tonic-build Builder is belt-and-suspenders: even though we hand over a ready descriptor set, this asserts the binary is never invoked.
3. Generated code is committed at server/lib/src/generated/holger.v1.rs with a // @generated … DO NOT EDIT banner. src/grpc.rs does include!("generated/holger.v1.rs") instead of the old tonic::include_proto!("holger.v1") (which read from OUT_DIR).
4. build.rs no longer compiles protos. It is reduced to a single cheap cargo:rerun-if-changed=proto/holger.proto hint (no subprocess). The [build-dependencies] tonic-build entry was removed.
5. Public surface unchanged. The module path stays holger_proto::* (i.e. server_lib::grpc::holger_proto::RepositoryServiceServer, ArtifactId, …), so dependents compile untouched.

The codegen deps (protox, tonic-build, prost-types) live in xtask's [dependencies] — they are compiled only when a developer runs the xtask, never on a normal build of the workspace's product crates.

Guide — regenerate the proto code

You only need this after editing proto/holger.proto:

cargo xtask gen-proto

It rewrites server/lib/src/generated/holger.v1.rs. The step is idempotent: running it twice on an unchanged .proto produces a byte-identical file. Commit the regenerated file together with the .proto change.

You do not need protoc installed. A normal build/test never runs the xtask:

cargo build --workspace      # no protoc required
cargo test  --workspace      # no protoc required

Proof it needs no protoc

Build with a PATH that contains no protoc:

env PATH=/path/with/no/protoc cargo build -p holger-server-lib   # green

The committed-code round-trip is guarded by a real test: server/lib/tests/unit_tests.rs::generated_proto_types_roundtrip constructs the generated ListArtifactsResponse/ArtifactEntry/ArtifactId messages, encodes them via prost, decodes them back, and asserts every field survives — proving the generated module is real, not a stub.

Files