Your browser sucks, JavaScript disabled! Gunnar's console needs JavaScript and WebAssembly. Git over SSH on port 2222 does not.

nordisk/korp-collectors

.nornir/collectors/trafikverket-trainpos.md

Collector: trafikverket-trainpos (adopted — WAVE 2, batch 6 · the POST-body daemon)

The Trafikverket real-time train-position daemon. Unlike every other warehouse-sink daemon (which GETs a URL), the Trafikinfo API answers a query POSTed as an XML <REQUEST> body carrying the authentication key — so this is the one daemon that uses the Fetcher POST verb. It parses the JSON reply's /RESPONSE/RESULT/0/TrainPosition array into a moving-map position table. Adopted to the daemon architecture (../collectors-daemon-design.md, §11) as a warehouse-sink (Iceberg) daemon. Implementor: src/iceberg_daemons.rs (TrafikverketDaemon, parse_trafikverket_trainpos), behind --features daemon. It reuses the existing trafikverket::build_request (body builder) + trafikverket::TrafikverketCollector::train_positions field map + rest::parse_records, and rides the SAME shared reconcile_iceberg / daemon_run_loop as the GET daemons — only its Source is a Post.

Config trait (Settings ▸ Collectors ▸ trafikverket-trainpos)

enabled:    bool
url:        String   // Trafikinfo API   (default https://api.trafikinfo.trafikverket.se/v2/data.json)
objecttype: String   // result array     (default TrainPosition)
key:        String   // API key — a Settings field; empty ⇒ read from key_env (never persisted as a secret)
key_env:    String   // env var holding the key (default TRAFIKVERKET_API_KEY)
query:      String   // optional <QUERY …>…</QUERY> (default: a minimal query for the objecttype)
table:      String   // Iceberg table    (default trafikverket_trainpos)

State machine (State trait)

Offline → Downloading(POST %) → Indexing(parse JSON) → Online     // position rows landed
        → Failed{reason}                                          // empty-is-RED / fetch/parse error

The State Facet renders sink=iceberg, phase, table, rows_written, last_run_unix, error. Green only when ≥1 position row is landed.

Storage — Iceberg (warehouse-sink)

Rows land through CollectorCtx.warehouse into trafikverket_trainpos (columns: train_id, wgs84, timestamp, bearing, speed, status_active) — the moving-map position table facett-map3d overlays + animates. No S3 bucket.

Acquisition — the fetch seam (POST) + the secret seam

On each tick the daemon resolves its key through the injected CollectorCtx.secret [SecretStore] — an explicit config key override wins, otherwise store.secret(key_env) (key_env default TRAFIKVERKET_API_KEY). No daemon calls std::env::var directly, so a VaultSecretStore (nornir-vault / OpenBao) plugs in later with no code change here — see the // TODO(vault): at SecretStore in src/daemon.rs. The default EnvSecretStore reads the env var (or an inline literal), so today's behavior is unchanged.

A resolved key is a SecretString (Debug/Display redact to ***redacted***, zeroized on drop); it is expose()d ONLY into the <REQUEST><LOGIN authenticationkey=…/><QUERY …/></REQUEST> body and never logged. The body is POSTed as text/xml through CollectorCtx.fetch.post(...) (korp = rustls HTTP core; tests = a canned fetcher). A missing secret (store returns None, or an empty value) short-circuits to a loud Failed naming the secret — the daemon never POSTs without its key. Fetch + parse run off the async reactor via spawn_blocking.

empty-is-RED

A valid envelope with an empty TrainPosition array parses to 0 rows → Failed{…empty-is-RED} (red), never a silent green.

Tests (src/iceberg_daemons.rs + tests/collector_daemons_register.rs)

Credential note

Needs a live credential. The offline sample proves the whole POST → parse → sink → Online path (the injected fake key reaches the request body; the fixture returns the canned reply), but a live standalone run needs a real Trafikverket authentication key resolvable through the SecretStore — config key, the TRAFIKVERKET_API_KEY env var (default EnvSecretStore), or a future VaultSecretStore. Without one the store returns None and the daemon reports Failed (never a silent green). It runs live the instant a real key is provided to the SecretStore. Both RoadGeometry (NVDB ways) and the richer trains() arm are the same POST skeleton with a different objecttype/field map (follow-on).