nordisk/nornir

G1 — AMD ROCm / MIGraphX execution-provider path for the ort embedder

Status: ✅ EP wiring + runtime probe landed (feat/g-series). Structural on CI/dev (no AMD GPU on the build box) — compiles under --features embed-ort-rocm, the probe degrades cleanly to CPU, and the EP registration is real (ort::ep::MIGraphX / ort::ep::ROCm, runtime dlopen).

Problem

The embed-ort backend (src/vector/embed_ort.rs) runs the jina ONNX model on ONNX Runtime. It requested only the NVIDIA CUDAExecutionProvider, falling back to CPU. On an AMD box that means CPU always — the GPU sits idle. AMD's ROCm stack exposes the same model through ONNX Runtime's MIGraphXExecutionProvider (graph-compiled) and ROCmExecutionProvider (generic), but nornir never asked for them.

Design

Mirror the existing CUDA-then-CPU logic exactly, one vendor deeper:

load_with(prefer_gpu=true)
  ├─ embed-ort-rocm compiled AND rocm::available()  →  request [MIGraphX, ROCm]  (fail_silently)
  │                                                     gpu_vendor = "rocm"
  ├─ otherwise                                       →  request [CUDA]           (fail_silently)
  │                                                     gpu_vendor = "cuda"
  └─ prefer_gpu=false                                →  CPU EP only
                                                        gpu_vendor = "cpu"
commit_from_file(model.onnx)  →  ORT picks the first registered EP that loads,
                                  else CPU.  Every EP is fail_silently, so a
                                  missing GPU runtime never errors `load`.

Three honesty guarantees

1. Feature-gated. embed-ort-rocm = ["embed-ort", "ort/migraphx", "ort/rocm"]. A build without it compiles none of the ROCm code (#[cfg(feature = "embed-ort-rocm")] on the module + the load branch), so non-AMD builds are byte-for-byte unaffected. Default cargo build is untouched.

2. Runtime probe, not a build-time assumption. src/vector/rocm.rs dlopens the ROCm runtime libs (libamdhip64, libMIOpen, librocblas, librocfft, libmigraphx_c) with RTLD_GLOBAL — the same trick cuda.rs uses so the provider's NEEDED sonames resolve against already-resident globals (glibc fixes the loader path at process start, so an in-process LD_LIBRARY_PATH is ignored). available() returns true only when the AMD driver is present and HIP + MIOpen loaded. When it's false the AMD EP is not even requested → the code flows straight into the CUDA-then-CPU path. A missing ROCm lib degrades to CPU, never errors.

3. fail_silently() on every EP. Even if available() is true but a provider can't actually register (driver/lib mismatch), ORT logs and falls through to the next EP, ending at CPU. The old CUDA path used ? on a hard error; G1 switched all GPU EPs to fail_silently() so the embedder never dies on a flaky GPU runtime.

Discovery order (rocm.rs candidate_dirs)

1. NORNIR_ROCM_LIBS (explicit colon-separated dirs).
2. $ROCM_PATH / $HIP_PATH → …/lib, …/lib64.
3. System: /opt/nornir/rocm, /opt/rocm*/lib, /usr/lib/x86_64-linux-gnu.

preflight() returns (gpu_ready, human_report) mirroring cuda::preflight(), ready to drive a future nornir vector doctor --rocm.

What is benchmarked vs structural-only

• Structural-only here: end-to-end AMD-GPU embedding throughput — the build box (Threadripper 3975WX) has no AMD GPU, so the EP can't be executed. The ROCm code compiles, the probe runs, and the no-GPU branch is asserted.
• Verified: cargo build --features embed-ort-rocm compiles; the probe_degrades_to_cpu_without_rocm test asserts available() == false, driver_present() == false, and a CPU fallback verdict on this box (and the inverse invariants if an AMD GPU ever IS present).

Files

• src/vector/rocm.rs — discovery, available(), driver_present(), preflight(), + inject-assert test.
• src/vector/embed_ort.rs — load_with EP chain, gpu_vendor(), rocm_ready().
• src/vector/mod.rs — #[cfg(feature = "embed-ort-rocm")] pub mod rocm;
• Cargo.toml — embed-ort-rocm feature.