.nornir/collectors/pxweb.md

Collector: `pxweb` (adopted — WAVE 1)

The PxWeb / JSON-stat2 statistics daemon. One connector covers SCB, Energimyndigheten, BRÅ, Folkhälsomyndigheten and most Swedish statistics agencies (thousands of CC0 cubes) — same client, different url per agency. Adopted to the daemon architecture (../collectors-daemon-design.md, §11) as a warehouse-sink (Iceberg) daemon. Implementor: src/iceberg_daemons.rs (PxWebDaemon), behind --features daemon. Rides the existing pure cube shaper src/pxweb.rs::parse_jsonstat2.

Config trait (Settings ▸ Collectors ▸ pxweb)

enabled:   bool
url:       String   // PxWeb 2.0 table endpoint   (default SCB api.scb.se/OV0104/…/ssd)
selection: String   // optional JSON cell selection (blank ⇒ GET the whole cube)
table:     String   // Iceberg table              (default pxweb_cube)

State machine (State trait)

Offline → Downloading(fetch %) → Indexing(parse cube) → Online     // cube cells landed
        → Failed{reason}                                           // empty-is-RED / fetch/parse error

The cube is normalised to one Utf8 code column per dimension (in id order) + a nullable value column, row-major (last dimension varies fastest). The State Facet renders sink=iceberg, phase, table, rows_written, last_run_unix, error.

Storage — Iceberg (warehouse-sink)

Cells land through CollectorCtx.warehouse into pxweb_cube (skade Table in korp, MemWarehouse in tests).

Acquisition — the fetch seam

GET the cube (or POST the selection) through CollectorCtx.fetch (rustls HTTP core in korp; canned fixture in tests). SCB is gentle-rate-limited (≤30 calls/10s) — the live korp fetcher throttles; the daemon loop's refresh_secs (default 3600) paces re-pulls.

empty-is-RED

An enabled cube with 0 cells ⇒ Failed{reason:"…(empty-is-RED)"} (red), never silent green.

Tests (`src/iceberg_daemons.rs`)

pxweb_daemon_reaches_online_and_sinks_cube — the synthetic 2×2×1 JSON-stat2 cube drives fetch→parse→append→Online, exactly 4 cells in pxweb_cube, BIOS tuned in.
daemon_loop_is_stoppable — the never-ending loop reconciles once to Online then stops.

Follow-on (mechanical)

Per-agency presets (a config list of (url, table) pairs) → one daemon per cube, or a multi-cube reconcile in a single daemon; dimension-schema pre-declaration for stable tables.

Collector: pxweb (adopted — WAVE 1)