crates/gunnar-http/tests/admission.rs
//! **The smart-HTTP front door has an admission gate, and it REFUSES**.
//!
//! # What was measured on 2026-08-31, and why this test is shaped the way it is
//!
//! The 2026-08-31 `vs_forge_concurrent` run found that `gunnar-http` had *no*
//! admission gate at all — no `Admission`, no `try_admit`, no permit — so 64
//! concurrent clients got 64 slots and **nothing was ever refused**. The gate
//! lived entirely in `gunnar-ssh`; an HTTP-fronted deployment (the `--http-bind`
//! token path) therefore had zero backpressure and no counter to see it on.
//!
//! Measure, never assert is the whole point of this file. A test that fires N requests at a
//! server and sees N succeed proves *nothing*: it is exactly the green a server
//! with no gate produces. So the two tests here are a matched pair, and the
//! first is the **red**:
//!
//! * [`an_unlimited_gate_admits_the_whole_flood`] — the gate present but with no
//! ceiling. Six concurrent transfers, six admitted, **zero refused**. This is
//! the finding reproduced: with nothing to bind, the flood all lands.
//! * [`a_low_ceiling_refuses_the_excess_with_429`] — the SAME flood against a
//! server-wide ceiling of three. Now only the **anonymous allowance** lands
//! and the rest are **refused with `429`**, the in-flight gauge stands at the
//! allowance, and the gate's refusal counter reads the excess. The refusal is
//! asserted on the applied output — the real HTTP status codes off the wire
//! and the gate's own emitted counts — not on a permit object existing.
//! * [`a_full_server_wide_gate_refuses_with_429_too`] — the same flood, the same
//! ceiling, with the anonymous door switched off, so what binds is the plain
//! server-wide limit and the refusal is `Refusal::AtCapacity`. That refusal is
//! the one the gRPC control plane and the console see, and `429` (never `413`)
//! is what the console's `http_status` is written against.
//!
//! Between the first two, the only thing that changed is the ceiling, so the
//! refusals are the ceiling's doing and nothing else's.
//!
//! # Every refusal here is a `429`, and that is a contract with the console
//!
//! `gunnar-http` answers `TOO_MANY_REQUESTS` for **every** `Refusal`
//! (`service::refuse_admission`); `413` on this listener belongs to LFS and to
//! an object that is too large, never to a full gate. `gunnar-grpc` maps the
//! same refusals to `ResourceExhausted`, which the console splits back apart by
//! gunnar's own words (`gunnar-ui-server`'s `grpc.rs`: *"a full admission gate
//! is a 429, not a 413"*). So the status code is asserted on every refused
//! request below, not just counted.
//!
//! # Why the transport parks
//!
//! [`gunnar_ssh::seam::BannerTransport::holding`] serves each admitted transfer
//! and then does not return until the test releases it. That is what keeps the
//! admitted transfers *in flight* — their admission permits are held for the
//! whole park — so a later request meets a gate that is genuinely full rather
//! than one that has already drained. Nothing is released until the assertions
//! are done.
use std::net::SocketAddr;
use std::sync::Arc;
use std::time::Duration;
use gunnar_core::{memory_control_store, AuthTable, NewRepo, RepoCatalog, RepoRef, Visibility};
use gunnar_http::{GunnarHttp, HttpConfig};
use gunnar_ssh::seam::BannerTransport;
use gunnar_ssh::server::Dispatch;
use gunnar_ssh::{Admission, Control};
use tokio::io::{AsyncReadExt as _, AsyncWriteExt as _};
use tokio::net::TcpStream;
/// One Public repository, `team/project`, anonymous read permitted — the whole
/// control plane a smart-HTTP advertisement needs to be authorised.
///
/// A real in-memory `RedbControlStore`, not a mock: the request goes through
/// the real `authorise_service` before it reaches the gate, so the gate is
/// metering a transfer the server actually would have served.
fn control() -> Control {
let store = memory_control_store().expect("open the control store");
let auth = AuthTable::new(Arc::clone(&store)).expect("open the identity table");
let catalog = RepoCatalog::new(store).expect("open the catalog");
catalog
.create_namespace("team")
.expect("create the namespace");
let mut spec = NewRepo::new(RepoRef::new("team", "project").expect("a legal repo path"));
// Public, so the anonymous principal every flood request carries may read
// it — that is what lets the request reach the gate at all (rule 0).
spec.visibility = Visibility::Public;
catalog.create(spec).expect("create the repository");
Control::new(auth, catalog)
}
/// A listener whose HTTP front routes through `gate`, serving a transport that
/// parks every admitted transfer until `hold` is released.
///
/// Returns the bound address and the hold, and drives the accept loop on a
/// background task. `allow_anonymous` is the default (`true`), which is what
/// makes a keyless `GET /info/refs` reach the gate.
async fn serve(gate: Arc<Admission>) -> (SocketAddr, gunnar_ssh::seam::TransportHold) {
let (transport, hold) = BannerTransport::holding();
let bound = GunnarHttp::new(
HttpConfig {
// Port 0: the kernel picks, and `local_addr` reads it back.
listen: "127.0.0.1:0".parse().expect("a literal address"),
..HttpConfig::default()
},
Dispatch::new(control()),
Arc::new(transport),
)
.with_admission(gate)
.bind()
.await
.expect("bind the smart-HTTP listener");
let addr = bound.local_addr();
tokio::spawn(async move {
let _ = bound.serve().await;
});
(addr, hold)
}
/// The advertisement request for `team/project`, sent raw and answered with its
/// HTTP status code **and its still-open socket**.
///
/// The socket is returned, not dropped, and that is load-bearing: an admitted
/// `200` is a transfer parked *while it holds the client's connection*, and
/// closing that connection is exactly what tells the server the client went
/// away, which tears the transfer down and frees its admission permit. Holding
/// the socket open until the assertions are done is what keeps the transfer
/// genuinely in flight — which is the whole thing an in-flight gauge measures.
///
/// Only the status line is read: an admitted `200` streams a body that never
/// ends (the transport is parked), so waiting for end-of-body would hang. The
/// status line is the first thing on the wire either way, and it is the applied
/// output the assertion is about.
async fn advertise_status(addr: SocketAddr) -> (u16, TcpStream) {
let mut socket = TcpStream::connect(addr).await.expect("connect");
let request = "GET /team/project.git/info/refs?service=git-upload-pack HTTP/1.1\r\n\
Host: localhost\r\n\r\n";
socket
.write_all(request.as_bytes())
.await
.expect("write the request");
// Read until the end of the status line, with a bound so a parked `200`
// (whose body never arrives) cannot hang the test.
let mut buf = Vec::new();
let mut chunk = [0u8; 256];
let status_line = tokio::time::timeout(Duration::from_secs(10), async {
loop {
let n = socket.read(&mut chunk).await.expect("read the response");
if n == 0 {
break; // connection closed (a refusal does this)
}
buf.extend_from_slice(&chunk[..n]);
if let Some(end) = buf.windows(2).position(|w| w == b"\r\n") {
return String::from_utf8_lossy(&buf[..end]).into_owned();
}
}
String::from_utf8_lossy(&buf).into_owned()
})
.await
.expect("a status line arrived before the timeout");
// "HTTP/1.1 429 Too Many Requests" → 429
let code = status_line
.split_whitespace()
.nth(1)
.and_then(|code| code.parse().ok())
.unwrap_or_else(|| panic!("no status code in {status_line:?}"));
(code, socket)
}
/// Fire `count` advertisement requests at `addr` at once and return their
/// status codes, **keeping every connection open** so the admitted transfers
/// stay in flight while the caller reads the gate's counters.
async fn flood(addr: SocketAddr, count: usize) -> (Vec<u16>, Vec<TcpStream>) {
let mut handles = Vec::with_capacity(count);
for _ in 0..count {
handles.push(tokio::spawn(advertise_status(addr)));
}
let mut codes = Vec::with_capacity(count);
let mut sockets = Vec::with_capacity(count);
for handle in handles {
let (code, socket) = handle.await.expect("a flood request task");
codes.push(code);
sockets.push(socket);
}
// A beat for the admitted transfers to be picked off the standing fan and
// reach their park (where the permit is held) before the caller samples the
// gauge; the refusals were already counted synchronously, before their
// `429` was even written.
tokio::time::sleep(Duration::from_millis(200)).await;
(codes, sockets)
}
const FLOOD: usize = 6;
/// **RED.** With no ceiling the gate binds on nothing: the whole flood is
/// admitted and not one request is refused. This is the finding exactly —
/// and the proof that the green in the next test is the ceiling's doing, not the
/// harness always saying "refused".
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
async fn an_unlimited_gate_admits_the_whole_flood() {
let gate = Arc::new(Admission::unlimited());
let (addr, hold) = serve(Arc::clone(&gate)).await;
let (codes, sockets) = flood(addr, FLOOD).await;
let admitted = codes.iter().filter(|&&c| c == 200).count();
let refused = codes.iter().filter(|&&c| c == 429).count();
assert_eq!(
admitted, FLOOD,
"an unlimited gate must admit the whole flood; got {codes:?}"
);
assert_eq!(
refused, 0,
"an unlimited gate must refuse nothing; got {codes:?}"
);
// The sockets are still open, so all six transfers are genuinely parked.
let stats = gate.stats();
assert_eq!(stats.refused(), 0, "no refusal counter may have moved");
assert_eq!(
stats.in_flight, FLOOD as u64,
"every admitted transfer is still parked, so all {FLOOD} are in flight"
);
drop(sockets);
hold.release();
}
/// **GREEN, the fix.** The SAME flood against a server-wide ceiling of three:
/// the anonymous allowance lands, the excess is refused with a real `429`, the
/// in-flight gauge stands at the allowance, and the gate's refusal counter reads
/// the excess.
///
/// The ceiling is deterministic: `Admission` builds the governor's band as
/// `[3, 3]` from a configured limit of three (`LimitBand::for_budget`), and the
/// band cannot move, so exactly three permits exist. The parked transport never
/// releases them during the assertions, so which requests won is irrelevant —
/// the counts are fixed.
///
/// # 🔴 What this asserted until 2026-09-08, and why it was wrong
///
/// It asserted `admitted == CEILING` on the grounds, written into its own
/// comment, that *"no per-principal cap and a huge memory budget, so the ONLY
/// thing that can bind is the server-wide limit"*. Measured red:
///
/// ```text
/// assertion `left == right` failed: exactly the ceiling may be admitted;
/// got [429, 429, 200, 429, 200, 429]
/// left: 2
/// right: 3
/// ```
///
/// The flood is **anonymous** — a keyless `GET /info/refs` on a public
/// repository is the whole point of this listener — and since `e0db08f` (D23b,
/// the keyed reserve) the top of the box is keyed-only:
/// [`gunnar_ssh::Admission::keyed_reserve`] slots above
/// `CONTENTION_PERCENT` of the governed limit are refused to a caller with no
/// key, instantly, as [`gunnar_ssh::Refusal::KeyedReserve`]. At a limit of three
/// that reserve is one, so an anonymous flood can reach **two** and structurally
/// never three. The old number was not a sentence the code drifted away from; it
/// was a promise gunnar deliberately stopped making, and the test was still
/// asserting it.
///
/// So the expectation is **derived from the gate** rather than restated — and
/// then pinned, so that a change to `CONTENTION_PERCENT` or to the anonymous
/// ceiling fails here by name instead of being silently re-derived. The refusal
/// counter moved from `refused_at_capacity` to `refused_keyed_reserve` for the
/// same reason: it is the counter that names the rule that actually fired, and
/// the two demand different operator actions. Nothing about the **status code**
/// changed — `429`, on every refused request, asserted below one by one.
/// `a_full_server_wide_gate_refuses_with_429_too` keeps the `AtCapacity` arm.
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
async fn a_low_ceiling_refuses_the_excess_with_429() {
const CEILING: usize = 3;
// Server-wide ceiling of three; no per-principal cap and a huge memory
// budget, so the only two things that can bind are the server-wide limit and
// the keyed reserve carved out of it — and the reserve is what an anonymous
// caller meets first.
let gate = Arc::new(Admission::with_memory_budget(
Some(CEILING),
None,
gunnar_ssh::governor::DEFAULT_MEMORY_BUDGET_BYTES,
));
// What an anonymous flood may hold, read off the gate: the governed limit
// less the keyed reserve, and never more than the anonymous ceiling.
let limit = gate.governed_limit().expect("a governed limit");
let reserve = gate.keyed_reserve();
let allowance = (limit - reserve).min(gate.max_anonymous().unwrap_or(usize::MAX));
assert_eq!(
(limit, reserve, allowance),
(CEILING, 1, 2),
"the arithmetic this test is written against: a ceiling of {CEILING} keeps its top slot \
for callers with a key, so a stranger may hold {}",
CEILING - 1
);
assert!(
allowance < FLOOD,
"the flood must be larger than the allowance or nothing is refused"
);
let (addr, hold) = serve(Arc::clone(&gate)).await;
let (codes, sockets) = flood(addr, FLOOD).await;
let admitted = codes.iter().filter(|&&c| c == 200).count();
let refused = codes.iter().filter(|&&c| c == 429).count();
assert_eq!(
admitted, allowance,
"exactly the anonymous allowance may be admitted; got {codes:?}"
);
assert_eq!(
refused,
FLOOD - allowance,
"the excess over the allowance must be refused with 429; got {codes:?}"
);
// The contract the console reads (`gunnar-ui-server`'s `grpc.rs`): a full
// admission gate is a 429 and never a 413. Asserted per request, so a
// refusal that came back as anything else cannot hide inside a count.
assert!(
codes.iter().all(|&c| c == 200 || c == 429),
"every refusal from the admission gate is a 429 — never a 413, which on this listener \
means an oversized object; got {codes:?}"
);
// The emitted counts, the other half of the applied output.
let stats = gate.stats();
assert_eq!(
stats.in_flight, allowance as u64,
"the in-flight gauge must stand at the allowance while the transfers park"
);
assert_eq!(
stats.peak_in_flight, allowance as u64,
"the peak the gate ever held is the allowance, not one above it"
);
assert_eq!(
stats.refused(),
(FLOOD - allowance) as u64,
"the gate's refusal counter must have moved once per refused request"
);
assert_eq!(
stats.refused_keyed_reserve,
(FLOOD - allowance) as u64,
"the refusals must name the keyed reserve (D23b), which is the rule that fired — not a \
per-principal share and not a full server"
);
assert_eq!(
stats.refused_at_capacity, 0,
"the server-wide gate never filled: the reserve refused before it could"
);
assert_eq!(
stats.admitted, allowance as u64,
"exactly the allowance was admitted"
);
drop(sockets);
hold.release();
}
/// **The server-wide gate itself, full, is also a `429`.**
///
/// The test above can no longer reach `Refusal::AtCapacity`, because the keyed
/// reserve refuses an anonymous caller before the last slot is ever taken. That
/// refusal is nonetheless the one an operator meets and the one the gRPC control
/// plane forwards to the console as `ResourceExhausted`, so it keeps its own
/// arm — the original assertions of this file, on the gate they were true of.
///
/// The anonymous door is switched off with
/// [`gunnar_ssh::Admission::with_anonymous_ceiling`]`(None)`, which is what makes
/// the server-wide limit the only thing that can bind: with no anonymous ceiling
/// there is no keyed reserve either (both halves are one decision), so an
/// unauthenticated reader takes an ordinary principal's share and the flood runs
/// straight into the ceiling of three.
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
async fn a_full_server_wide_gate_refuses_with_429_too() {
const CEILING: usize = 3;
let gate = Arc::new(
Admission::with_memory_budget(
Some(CEILING),
None,
gunnar_ssh::governor::DEFAULT_MEMORY_BUDGET_BYTES,
)
.with_anonymous_ceiling(None),
);
// `keyed_reserve()` still *reports* the derived number — it is a property of
// the governed limit, not of this switch — but `try_admit` only applies it
// when there is an anonymous ceiling to be the other half of.
assert_eq!(
gate.max_anonymous(),
None,
"the anonymous door is off, so nothing is reserved against it"
);
let (addr, hold) = serve(Arc::clone(&gate)).await;
let (codes, sockets) = flood(addr, FLOOD).await;
let admitted = codes.iter().filter(|&&c| c == 200).count();
let refused = codes.iter().filter(|&&c| c == 429).count();
assert_eq!(
admitted, CEILING,
"exactly the ceiling may be admitted; got {codes:?}"
);
assert_eq!(
refused,
FLOOD - CEILING,
"the excess over the ceiling must be refused with 429; got {codes:?}"
);
assert!(
codes.iter().all(|&c| c == 200 || c == 429),
"a full admission gate is a 429, not a 413; got {codes:?}"
);
let stats = gate.stats();
assert_eq!(
stats.in_flight, CEILING as u64,
"the in-flight gauge must stand at the ceiling while the transfers park"
);
assert_eq!(
stats.peak_in_flight, CEILING as u64,
"the peak the gate ever held is the ceiling, not one above it"
);
assert_eq!(
stats.refused(),
(FLOOD - CEILING) as u64,
"the gate's refusal counter must have moved once per refused request"
);
assert_eq!(
stats.refused_at_capacity,
(FLOOD - CEILING) as u64,
"the refusals must be the server-wide 'at capacity' kind, not a per-principal share"
);
assert_eq!(
stats.admitted, CEILING as u64,
"exactly the ceiling was admitted"
);
drop(sockets);
hold.release();
}