crates/gunnar-http/src/lfs.rs
//! **Git LFS over HTTP: the batch API, and the basic transfer adapter.**
//!
//! gunnar's fourth surface on the ACL key table, after SSH, gRPC and smart HTTP.
//! Until it existed a `git clone` over `http://` pulled its git objects and
//! then died in `git lfs smudge`, because the LFS endpoint git-lfs derives from
//! the remote — `<remote>/info/lfs` — was not served.
//!
//! # What is implemented, read off the specification and not from memory
//!
//! Verified 2026-08-05 against `git-lfs/docs/api/batch.md` and
//! `git-lfs/docs/api/basic-transfers.md`, and driven end to end by the real
//! `git-lfs` 3.7.1 binary in `gunnar-cli/tests/lfs_http.rs`.
//!
//! ```text
//! POST …/info/lfs/objects/batch application/vnd.git-lfs+json in and out
//! GET …/info/lfs/objects/<oid> the object, application/octet-stream
//! PUT …/info/lfs/objects/<oid> the object, verified before it is committed
//! POST …/info/lfs/objects/verify "you said you stored it; had you"
//! ```
//!
//! The `basic` transfer adapter, and only that one. `multipart-basic` is a
//! GitHub extension and the pure-SSH adapter is [`gunnar_lfs::session`]'s;
//! a client offering neither `basic` nor nothing at all is refused **422 naming
//! what gunnar speaks**, rather than silently served an adapter it did not ask
//! for.
//!
//! # The per-object error, which is the whole reason this is not a copy of SSH
//!
//! The batch response is `200` and the failures are *inside* it, one per
//! object:
//!
//! ```json
//! {"objects":[{"oid":"…","size":12,"error":{"code":404,"message":"…"}}]}
//! ```
//!
//! That shape has no equivalent in the pure-SSH grammar, whose batch line is
//! `<oid> SP <size> SP <action>` with `action ∈ upload | download | noop` and
//! nowhere to put a code. So SSH folds *"the server already has it"* and *"the
//! server does not have it"* into one word, and **git-lfs reads an object with
//! neither an action nor an error as one it already holds**.
//!
//! MEASURED against git-lfs 3.7.1 on 2026-08-05, by rendering `Plan::Missing`
//! as an object with no `actions` and no `error` and asking for something the
//! server does not hold:
//!
//! | | with the per-object 404 | without it |
//! |---|---|---|
//! | `git lfs fetch --all` | fails, naming the object | **`1 object found, done.` and exit 0** — nothing transferred |
//! | `git clone` | fails: `404` / `not on this server` | fails: `Smudge error: error opening media file: … no such file or directory` |
//!
//! So the clone fails either way — its smudge filter cannot open a file that is
//! not there — but it fails blaming *the client's own cache* for a fault that
//! is entirely the server's. `git lfs fetch` has no filter to fail, and that is
//! the row where the bug is silent. Both are asserted in
//! `gunnar-cli/tests/lfs_http.rs`. The mirror image — a push reported
//! successful having uploaded nothing — is pinned in
//! `gunnar-lfs/tests/interop.rs`.
//!
//! This surface does not reproduce it. [`gunnar_lfs::batch::plan_object`] is the
//! one decision both wires make, and it answers with `AlreadyPresent` and
//! `Missing` as **separate** outcomes; SSH renders both `noop` because it must,
//! and here `Missing` becomes a per-object `404`. The divergence is deliberate
//! and it is one function wide.
//!
//! # Authentication is the ACL table, and there is no fourth credential
//!
//! [`crate::auth`] unchanged: `Authorization: Basic <anything>:<token>` or
//! `Bearer <token>`, resolved by `AuthTable::principal_by_token`, and then
//! `Dispatch::authorise_service` with `GitService::LfsTransfer(operation)` —
//! the **same** enum value, carrying the same directional grant, that the SSH
//! exec line produces. Download needs Read, upload needs Write, and neither
//! decision is made here.
//!
//! ## What the LFS specification says about auth, and why gunnar does not do
//! the other half
//!
//! `docs/api/authentication.md` describes two paths. Over HTTP the client sends
//! an `Authorization` header, taken from git's credential helper, and a server
//! that wants one answers `401` with `LFS-Authenticate`. That is this file.
//!
//! The other path is for **ssh remotes**: the client runs `git-lfs-authenticate
//! <path> <operation>` over SSH and the server hands back a JSON blob naming an
//! HTTPS `href` and an `Authorization` header to use against it — an
//! SSH-authenticated key minting a bearer token for a different transport.
//! gunnar does not implement it and should not: its ssh remotes speak git-lfs
//! 3.0's pure-SSH transfer, which needs no token because the connection is
//! already authenticated, and a `git-lfs-authenticate` verb would be a
//! token-minting endpoint — a **fourth credential type**, issued by one surface
//! and honoured by another, with its own expiry and its own revocation story.
//! The ACL table has one kind of secret and this surface uses it.
//!
//! # Anonymous read is the same switch, and it opens no door SSH keeps shut
//!
//! There is no separate anonymous rule here. A request with no `Authorization`
//! header becomes `Principal::anonymous()` exactly as on the git URLs, subject
//! to the same `allow_anonymous` flag the SSH listener is given, and its entire
//! authority is rule 0 of `AuthTable::may`: `Read`, on a `Public` repository.
//! So an anonymous `download` batch on a public repository works; an anonymous
//! `upload` batch is refused, and every batch against a private repository is
//! refused, by the same call that refuses them over SSH.
//!
//! # Nothing here is sized from an object
//!
//! `gunnar-lfs` promises that every LFS byte moves through one
//! [`gunnar_lfs::CHUNK_BYTES`] buffer, and this surface keeps the promise: a
//! download streams from the store into the response body through
//! [`crate::body::Streamed`]'s window, and an upload streams from the request
//! body into [`gunnar_lfs::VerifyingSink`]. The only thing read whole is the
//! batch JSON, which is bounded by [`MAX_BATCH_REQUEST_BYTES`].
use std::sync::Arc;
use bytes::Bytes;
use http::{header, HeaderMap, HeaderValue, Method, Response, StatusCode};
use http_body_util::{BodyExt, Full};
use hyper::body::Incoming;
use gunnar_core::observe::{RequestId, TARGET_GIT};
use gunnar_lfs::{Limits, Oid, Operation};
use gunnar_ssh::seam::{LfsAccess, LfsTransport, ServiceRequest};
use gunnar_ssh::GitService;
use crate::body::Streamed;
use crate::service::Body;
/// The media type every JSON document on this surface carries.
///
/// git-lfs sends it on `Accept` and `Content-Type` and checks it on the way
/// back; a server answering `application/json` here is one git-lfs refuses to
/// talk to.
pub const LFS_CONTENT_TYPE: &str = "application/vnd.git-lfs+json";
/// The one transfer adapter this surface implements.
pub const BASIC: &str = "basic";
/// How much batch JSON is read before the request is refused.
///
/// **4 MiB**, against a default [`Limits::max_batch_objects`] of 10 000. One
/// object costs about 90 bytes on the wire (`{"oid":"<64 hex>","size":<n>},`),
/// so a full batch is ~0.9 MB and this is that with room for a `ref` and
/// whitespace. It is a *byte* ceiling and not an object ceiling because the
/// object count is only knowable after parsing, and "parse whatever arrives,
/// then check how big it was" is the shape of every JSON denial-of-service
/// there has ever been. The object count is checked too, against the operator's
/// own limit, immediately after.
pub const MAX_BATCH_REQUEST_BYTES: usize = 4 * 1024 * 1024;
// ── the wire documents ──────────────────────────────────────────────────────
/// A `POST …/objects/batch` request body.
#[derive(Debug, serde::Deserialize)]
pub struct BatchRequest {
/// `"download"` or `"upload"`. **This is what decides the grant**, so it is
/// required: `#[serde(default)]` here would make a body with no operation
/// authorise as whichever direction the default happened to be.
pub operation: String,
/// Adapters the client can speak, best first. Absent means `["basic"]`.
#[serde(default)]
pub transfers: Vec<String>,
/// The ref the objects belong to. Accepted and unused: gunnar's grants are
/// per repository, not per ref, so a server that read this would be
/// reading a client's claim into an authorisation decision.
#[serde(default)]
pub r#ref: Option<BatchRef>,
/// The objects.
#[serde(default)]
pub objects: Vec<ObjectSpec>,
/// `"sha256"`, or absent. Anything else is refused; see [`batch`].
#[serde(default)]
pub hash_algo: Option<String>,
}
/// The `ref` member of a batch request.
#[derive(Debug, serde::Deserialize)]
pub struct BatchRef {
/// The fully-qualified refspec.
#[serde(default)]
pub name: String,
}
/// One object named by a client, in a request or echoed in a response.
#[derive(Debug, Clone, serde::Deserialize, serde::Serialize)]
pub struct ObjectSpec {
/// The LFS oid: SHA-256 of the raw content, 64 lowercase hex.
pub oid: String,
/// Its size in bytes.
pub size: u64,
}
/// A `200` batch response.
#[derive(Debug, serde::Serialize)]
pub struct BatchResponse {
/// The adapter chosen from the client's list.
pub transfer: &'static str,
/// One entry per requested object, in the order they were asked for.
pub objects: Vec<ObjectResponse>,
/// Echoed so a client cannot be in doubt.
pub hash_algo: &'static str,
}
/// One object's answer: **either** `actions` **or** `error`, never both, and
/// neither when there is nothing to do.
#[derive(Debug, serde::Serialize)]
pub struct ObjectResponse {
/// The oid, as the client spelled it.
pub oid: String,
/// The size.
pub size: u64,
/// `true` means *the `href` is already authenticated; do not attach a
/// credential of your own*.
///
/// gunnar copies the request's own `Authorization` header into
/// [`Action::header`], so it is true by construction — including for an
/// anonymous read, where the truthful statement is that no credential is
/// needed. Left absent (`false`) git-lfs would run the credential helper
/// against the object URL, which for an anonymous public clone means
/// prompting for a password that does not exist.
#[serde(skip_serializing_if = "std::ops::Not::not")]
pub authenticated: bool,
/// What the client must do. Absent when the object needs no transfer.
#[serde(skip_serializing_if = "Option::is_none")]
pub actions: Option<Actions>,
/// Why this object cannot be served. Absent on success.
#[serde(skip_serializing_if = "Option::is_none")]
pub error: Option<ObjectError>,
}
/// The actions a client may take for one object.
#[derive(Debug, Default, serde::Serialize)]
pub struct Actions {
/// Where to `GET` the object.
#[serde(skip_serializing_if = "Option::is_none")]
pub download: Option<Action>,
/// Where to `PUT` the object.
#[serde(skip_serializing_if = "Option::is_none")]
pub upload: Option<Action>,
/// Where to confirm the upload landed.
#[serde(skip_serializing_if = "Option::is_none")]
pub verify: Option<Action>,
}
/// One action: a URL and the headers to send with it.
#[derive(Debug, serde::Serialize)]
pub struct Action {
/// An **absolute** URL. git-lfs feeds it straight to `http.NewRequest`,
/// which will not resolve a relative one.
pub href: String,
/// Headers the client must send. Skipped when empty rather than serialised
/// as `{}`, which is legal but noisy.
#[serde(skip_serializing_if = "std::collections::BTreeMap::is_empty")]
pub header: std::collections::BTreeMap<String, String>,
}
/// A per-object failure inside a `200` batch response.
#[derive(Debug, serde::Serialize)]
pub struct ObjectError {
/// The status code for **this object**, from
/// [`gunnar_lfs::Error::status`] — the same table the SSH surface answers
/// its status lines from, which is why that module exists (one implementation).
pub code: u16,
/// What went wrong, in the words the client is allowed to see.
pub message: String,
}
/// A batch-level failure: a non-`200` response body.
#[derive(Debug, serde::Serialize)]
pub struct ErrorBody {
/// The sentence.
pub message: String,
/// gunnar's own request id, the same value the `x-gunnar-request-id`
/// header carries.
///
/// The specification asks for this field precisely so a user can quote it
/// at an operator, and gunnar already mints one per request for its span.
/// Putting anything else here would be minting a second id for the same
/// request.
pub request_id: String,
}
/// A `POST …/objects/verify` request body.
#[derive(Debug, serde::Deserialize)]
pub struct VerifyRequest {
/// The object.
pub oid: String,
/// What the client believes it uploaded.
pub size: u64,
}
// ── the handlers ────────────────────────────────────────────────────────────
/// Everything one LFS request needs beyond the HTTP layer.
pub struct Context<'a> {
/// The seam to the object store.
pub transport: &'a Arc<dyn LfsTransport>,
/// The request already through the ACL.
pub request: ServiceRequest,
/// The absolute URL prefix `href`s are built from, with no trailing slash:
/// `http://host/team/project.git/info/lfs`.
pub endpoint: String,
/// The `Authorization` header value to echo into the actions, if any.
pub authorization: Option<String>,
/// The in-flight window for a streamed object body.
pub stream_window: usize,
/// This request's id, for the error body.
pub id: RequestId,
/// The request's headers. [`upload`] reads `Content-Length` off them to
/// apply the quota *before* the body arrives, which is the only moment at
/// which applying it costs nothing.
pub headers: HeaderMap,
}
/// `POST …/info/lfs/objects/batch`.
///
/// The body has already been read and parsed by the caller, because the
/// `operation` in it is what selected the [`GitService`] the ACL was asked
/// about — the parse has to happen before the authorisation, and the
/// authorisation before anything touches a store.
pub async fn batch(context: Context<'_>, body: BatchRequest) -> Response<Body> {
// The client sends `hash-algo=sha256` and refuses any other value in the
// response. An LFS oid is SHA-256 by definition; there is nothing else to
// agree about, and a server that quietly accepted `sha1` here would be
// filing objects under ids no client computes. The same sentence the SSH
// session refuses with.
match body.hash_algo.as_deref() {
None | Some("") | Some("sha256") => {}
Some(other) => {
return error(
&context,
StatusCode::UNPROCESSABLE_ENTITY,
format!(
"unsupported hash algorithm `{}`; lfs oids are sha256",
clip(other)
),
)
}
}
// An empty list means "the client did not say", which is `basic` — that is
// what the specification's default is. A non-empty list that does not
// contain `basic` is refused rather than served `basic` anyway: a client
// that asked only for `multipart-basic` and got single-part upload URLs
// would PUT a whole object at a URL expecting one part.
if !body.transfers.is_empty() && !body.transfers.iter().any(|name| name == BASIC) {
return error(
&context,
StatusCode::UNPROCESSABLE_ENTITY,
format!(
"this server implements the `{BASIC}` transfer adapter over HTTP and git-lfs \
3.0's pure-SSH adapter over ssh://; the request offered only [{}]",
body.transfers
.iter()
.map(|name| clip(name))
.collect::<Vec<_>>()
.join(", ")
),
);
}
let operation = match Operation::parse(&body.operation) {
Some(operation) => operation,
// Unreachable from `serve`, which parses the operation first in order
// to choose the grant. Kept because this function is `pub` and a caller
// that skipped that step must not be served as whichever direction is
// cheaper to assume.
None => {
return error(
&context,
StatusCode::UNPROCESSABLE_ENTITY,
format!(
"`{}` is not an lfs operation; it is `download` or `upload`",
clip(&body.operation)
),
)
}
};
debug_assert_eq!(
context.request.service,
GitService::LfsTransfer(operation),
"the batch was authorised for a different direction than it is being served in"
);
let access = match open(&context).await {
Ok(access) => access,
Err(response) => return *response,
};
if body.objects.len() > access.limits.max_batch_objects {
return error(
&context,
StatusCode::UNPROCESSABLE_ENTITY,
format!(
"a batch may name at most {} objects; this one names {}",
access.limits.max_batch_objects,
body.objects.len()
),
);
}
let endpoint = context.endpoint.clone();
let authorization = context.authorization.clone();
let store = Arc::clone(&access.store);
let limits = access.limits;
let allowance = access.allowance.clone();
let specs = body.objects;
// `spawn_blocking`: a batch is one `stat` per object and may name ten
// thousand of them, which is not work to do on a runtime worker.
let planned = tokio::task::spawn_blocking(move || {
specs
.into_iter()
.map(|spec| {
answer_for(
&*store,
&limits,
// The measured number, not the charging door. `batch` is
// ADVISORY — git-lfs's basic transfer `PUT`s whatever the
// batch said and a client may skip the batch entirely — so
// it must not charge the namespace's ledger for objects
// that may never be uploaded. The `PUT` handler below is
// the one that admits, and it is the one that charges.
allowance
.as_ref()
.map(gunnar_lfs::NamespaceAllowance::allowance),
operation,
spec,
&endpoint,
&authorization,
)
})
.collect::<Vec<_>>()
})
.await;
let objects = match planned {
Ok(objects) => objects,
Err(err) => {
tracing::error!(
target: TARGET_GIT,
event = "http.lfs.batch.panicked",
%err,
"the lfs batch planner did not finish"
);
return error(
&context,
StatusCode::INTERNAL_SERVER_ERROR,
"the lfs batch could not be planned".to_owned(),
);
}
};
let missing = objects
.iter()
.filter(|object| object.error.is_some())
.count();
tracing::info!(
target: TARGET_GIT,
event = "http.lfs.batch.served",
operation = %operation,
objects = objects.len(),
// Applied output, not "a batch happened": how many the client will
// actually move, and how many were refused per object.
transfers = objects.iter().filter(|o| o.actions.is_some()).count(),
refused = missing,
"served an lfs batch"
);
json(
StatusCode::OK,
&BatchResponse {
transfer: BASIC,
objects,
hash_algo: "sha256",
},
)
}
/// Decide one object and render it. Blocking; runs on a blocking thread.
fn answer_for(
store: &dyn gunnar_lfs::LfsStore,
limits: &Limits,
allowance: Option<&gunnar_lfs::Allowance>,
operation: Operation,
spec: ObjectSpec,
endpoint: &str,
authorization: &Option<String>,
) -> ObjectResponse {
let mut answer = ObjectResponse {
oid: spec.oid.clone(),
size: spec.size,
authenticated: true,
actions: None,
error: None,
};
let oid = match Oid::parse(&spec.oid) {
Ok(oid) => oid,
Err(err) => {
answer.error = Some(ObjectError {
// 422: the request was well-formed JSON and this *value* was
// not an oid. The batch itself is still 200.
code: 422,
message: err.to_string(),
});
return answer;
}
};
// `taken` is 0 and always will be on this wire: an HTTP batch is one
// request, the allowance was measured for it when `open` ran, and the next
// request measures again off the disk. The SSH session is the one that has
// to carry a running total, because one channel may `put-object` until
// `quit`.
match gunnar_lfs::plan_object(store, limits, allowance, 0, operation, &oid, spec.size) {
Err(err) => {
answer.error = Some(ObjectError {
code: err.status(),
message: err.client_message(),
});
}
// Nothing to do. On an upload batch this is the whole point — the
// server already holds it — and an object with neither actions nor an
// error is exactly how the specification says so.
Ok(gunnar_lfs::Plan::AlreadyPresent) => {}
// **The bug this surface exists not to reproduce.** An object with no
// actions and no error means "already there" to git-lfs, so a download
// of something the server does not hold MUST carry an error or the
// clone succeeds with a pointer file where the content should be.
Ok(gunnar_lfs::Plan::Missing) => {
answer.error = Some(ObjectError {
code: 404,
message: format!("object {oid} is not on this server"),
});
}
Ok(gunnar_lfs::Plan::Transfer) => {
let href = format!("{endpoint}/objects/{oid}");
let header = match authorization {
// Canonical spelling, not `header::AUTHORIZATION.as_str()`,
// which is lower-case: a header *name* is case-insensitive on
// the wire, but this one travels as a **JSON map key** first
// and only becomes a header in the client. Go's
// `http.Header.Set` canonicalises it either way; a client that
// looked the key up in the map would not.
Some(value) => {
std::collections::BTreeMap::from([("Authorization".to_owned(), value.clone())])
}
None => std::collections::BTreeMap::new(),
};
answer.actions = Some(match operation {
Operation::Download => Actions {
download: Some(Action { href, header }),
..Actions::default()
},
Operation::Upload => Actions {
upload: Some(Action {
href,
header: header.clone(),
}),
// The client POSTs here after the PUT succeeds, and gunnar
// answers from the store rather than from the fact that a
// PUT returned 200 — the same check `verify-object` makes
// over SSH.
verify: Some(Action {
href: format!("{endpoint}/objects/verify"),
header,
}),
..Actions::default()
},
});
}
}
answer
}
/// `GET …/info/lfs/objects/<oid>`: the object, streamed.
pub async fn download(context: Context<'_>, oid: &str) -> Response<Body> {
let oid = match Oid::parse(oid) {
Ok(oid) => oid,
Err(err) => {
return error(&context, StatusCode::UNPROCESSABLE_ENTITY, err.to_string());
}
};
let access = match open(&context).await {
Ok(access) => access,
Err(response) => return *response,
};
let store = Arc::clone(&access.store);
let opened = tokio::task::spawn_blocking(move || store.open(&oid)).await;
let object = match opened {
Ok(Ok(Some(object))) => object,
Ok(Ok(None)) => {
return error(
&context,
StatusCode::NOT_FOUND,
format!("object {oid} is not on this server"),
)
}
Ok(Err(err)) => {
tracing::error!(
target: TARGET_GIT,
event = "http.lfs.open.failed", %oid, %err,
"the lfs object store could not open an object"
);
return error(
&context,
StatusCode::INTERNAL_SERVER_ERROR,
err.client_message(),
);
}
Err(err) => {
tracing::error!(target: TARGET_GIT, event = "http.lfs.open.panicked", %oid, %err, "");
return error(
&context,
StatusCode::INTERNAL_SERVER_ERROR,
"the lfs object store could not be read".to_owned(),
);
}
};
let size = object.size;
let (body, writer) = Streamed::pair(context.stream_window);
// **The span does not cross `spawn_blocking`.** It is a thread hand-off,
// not a task spawn, and the current span is thread-local — so without this
// the only line that says how many bytes an LFS object transfer actually
// moved would fall outside `http.request` and carry no `req`, no
// `principal` and no `repo`. MEASURED on a live server 2026-08-05: every
// other `http.lfs.*` line was inside the span and
// `http.lfs.download.served` was the one bare line in the log.
// `.instrument(…)` does not apply here — that is for futures — so the span
// is captured and entered inside the closure. Same trap `gunnar-cli`'s
// `WireTransport::serve` documents.
let span = tracing::Span::current();
// The response head goes out before a byte of the object is read, which is
// what makes this a stream rather than a buffer — and what makes the
// `Content-Length` below a promise the writer then has to keep.
tokio::task::spawn_blocking(move || {
let _entered = span.enter();
use std::io::Write as _;
let mut sink = tokio_util::io::SyncIoBridge::new(writer);
let mut reader = object.reader;
// The same slab `gunnar_lfs::Session::get_object` uses, and stated here
// for the same reason: this is the memory bound, and inheriting
// whatever `std::io::copy` picks would leave it unstated.
let mut slab = vec![0u8; gunnar_lfs::CHUNK_BYTES];
let mut sent = 0u64;
loop {
let n = match reader.read(&mut slab) {
Ok(0) => break,
Ok(n) => n,
Err(err) => {
tracing::warn!(
target: TARGET_GIT,
event = "http.lfs.download.failed", %oid, %err, sent,
"an lfs object ended early on the way out"
);
return;
}
};
if let Err(err) = sink.write_all(&slab[..n]) {
tracing::debug!(
target: TARGET_GIT,
event = "http.lfs.download.disconnected", %oid, %err, sent,
"the client went away mid-object"
);
return;
}
sent += n as u64;
}
let _ = sink.flush();
if sent != size {
// The head already promised `Content-Length: size`, so the only
// honest thing left is to say so in the log. The wire is committed.
tracing::error!(
target: TARGET_GIT,
event = "http.lfs.download.short", %oid, declared = size, sent,
"the object store returned a different number of bytes than it declared"
);
} else {
tracing::info!(
target: TARGET_GIT,
event = "http.lfs.download.served", %oid, bytes = sent,
"served an lfs object"
);
}
});
let mut response = Response::new(BodyExt::boxed(body));
let headers = response.headers_mut();
headers.insert(
header::CONTENT_TYPE,
HeaderValue::from_static("application/octet-stream"),
);
headers.insert(header::CONTENT_LENGTH, header_value(&size.to_string()));
response
}
/// `PUT …/info/lfs/objects/<oid>`: the object, verified before it is committed.
pub async fn upload(context: Context<'_>, oid: &str, incoming: Incoming) -> Response<Body> {
let oid = match Oid::parse(oid) {
Ok(oid) => oid,
Err(err) => return error(&context, StatusCode::UNPROCESSABLE_ENTITY, err.to_string()),
};
let access = match open(&context).await {
Ok(access) => access,
Err(response) => return *response,
};
// **Refused before the bytes arrive**, from the `Content-Length` the basic
// adapter is required to send. A body with no length at all is refused too:
// the quota cannot be applied to an unknown size, and `VerifyingSink` needs
// a declared size to check the object against. The specification's basic
// upload always carries one.
let Some(declared) = content_length(&context.headers) else {
return error(
&context,
StatusCode::LENGTH_REQUIRED,
"an lfs object upload must carry a Content-Length; without one the server \
cannot apply its size limit before the bytes arrive"
.to_owned(),
);
};
if declared > access.limits.max_object_bytes {
return error(
&context,
StatusCode::PAYLOAD_TOO_LARGE,
format!(
"object {oid} is {declared} bytes; this server accepts at most {}",
access.limits.max_object_bytes
),
);
}
// **The tenant's own room**, after the operator's limit and for the reason
// `gunnar_lfs::batch::plan_object` spells out: 413 stays true after the
// customer pays and 507 does not, so the durable refusal goes first.
//
// This is the load-bearing copy on this wire. `batch` above answers the
// same question, but the batch endpoint is advisory — git-lfs's basic
// transfer will `PUT` an object whatever the batch said, and a client is
// free to skip the batch entirely.
//
// **This call used to pass `taken = 0`**, and the comment that justified it
// — *"one HTTP upload is one request against an allowance `open` measured
// for it"* — was true about ONE request and was the bug about two. Ten
// simultaneous `PUT`s each ran their own `open`, each measured the same
// empty namespace, and each was admitted for the whole plan. There is no
// `taken` any more: [`gunnar_lfs::NamespaceAllowance::admit`] charges the
// namespace's ledger, so this request is judged against every other
// transfer admitted since its own `open` measured the disk.
if let Some(allowance) = &access.allowance {
if let Err(err) = allowance.admit(declared) {
return error(
&context,
StatusCode::INSUFFICIENT_STORAGE,
err.client_message(),
);
}
}
let store = Arc::clone(&access.store);
let reader = crate::body::RequestBody::streaming(incoming);
let committed = tokio::task::spawn_blocking(move || -> gunnar_lfs::Result<u64> {
use std::io::Read as _;
let mut source = tokio_util::io::SyncIoBridge::new(reader);
let pending = store.begin_write(&oid, declared)?;
// **The same verifier the SSH `put-object` uses.** Size, overrun and
// SHA-256, and nothing is renamed into place until all three hold.
let mut sink = gunnar_lfs::VerifyingSink::new(pending, oid, declared);
let mut slab = vec![0u8; gunnar_lfs::CHUNK_BYTES];
loop {
let n = source.read(&mut slab)?;
if n == 0 {
break;
}
sink.accept(&slab[..n])?;
}
let received = sink.received();
sink.commit()?;
Ok(received)
})
.await;
match committed {
Ok(Ok(received)) => {
tracing::info!(
target: TARGET_GIT,
event = "http.lfs.upload.stored", %oid, bytes = received,
"stored an lfs object"
);
empty(StatusCode::OK)
}
Ok(Err(err)) => {
tracing::warn!(
target: TARGET_GIT,
event = "http.lfs.upload.refused", %oid, status = err.status(), %err,
"an lfs upload was not committed"
);
error(
&context,
StatusCode::from_u16(err.status()).unwrap_or(StatusCode::UNPROCESSABLE_ENTITY),
err.client_message(),
)
}
Err(err) => {
tracing::error!(target: TARGET_GIT, event = "http.lfs.upload.panicked", %oid, %err, "");
error(
&context,
StatusCode::INTERNAL_SERVER_ERROR,
"the lfs object could not be stored".to_owned(),
)
}
}
}
/// `POST …/info/lfs/objects/verify`.
///
/// Answered **from the store**, not from the fact that a `PUT` returned 200.
/// That is the whole value of the endpoint: it is the client asking whether the
/// object survived, and a server that answered yes because it remembered saying
/// yes would be answering its own question.
pub async fn verify(context: Context<'_>, body: VerifyRequest) -> Response<Body> {
let oid = match Oid::parse(&body.oid) {
Ok(oid) => oid,
Err(err) => return error(&context, StatusCode::UNPROCESSABLE_ENTITY, err.to_string()),
};
let access = match open(&context).await {
Ok(access) => access,
Err(response) => return *response,
};
let store = Arc::clone(&access.store);
let found = tokio::task::spawn_blocking(move || store.size_of(&oid)).await;
match found {
Ok(Ok(None)) => error(
&context,
StatusCode::NOT_FOUND,
format!("object {oid} was not stored by this server"),
),
Ok(Ok(Some(stored))) if stored != body.size => error(
&context,
StatusCode::UNPROCESSABLE_ENTITY,
format!(
"object {oid} is {stored} bytes on this server, not {}",
body.size
),
),
Ok(Ok(Some(_))) => empty(StatusCode::OK),
Ok(Err(err)) => error(
&context,
StatusCode::INTERNAL_SERVER_ERROR,
err.client_message(),
),
Err(_) => error(
&context,
StatusCode::INTERNAL_SERVER_ERROR,
"the lfs object store could not be read".to_owned(),
),
}
}
// ── plumbing ────────────────────────────────────────────────────────────────
/// Open the store, turning a seam refusal into a response.
/// `Err` is the finished response, boxed: it is several times the size of the access.
async fn open(context: &Context<'_>) -> Result<LfsAccess, Box<Response<Body>>> {
let transport = Arc::clone(context.transport);
let request = context.request.clone();
match tokio::task::spawn_blocking(move || transport.open_lfs(&request)).await {
Ok(Ok(access)) => Ok(access),
Ok(Err(err)) => {
tracing::warn!(
target: TARGET_GIT,
event = "http.lfs.store.unavailable",
repo = %context.request.repo.key(),
%err,
"the lfs object store could not be opened for an authorised request"
);
Err(Box::new(error(
context,
// 503 and not 500: the commonest cause is a `gunnar gc` holding
// the repository's store lock, which clears on its own, and the
// message says to retry. A 500 tells a client to give up.
StatusCode::SERVICE_UNAVAILABLE,
err.client_message()
.unwrap_or_else(|| "the repository is unavailable; try again".to_owned()),
)))
}
Err(err) => {
tracing::error!(target: TARGET_GIT, event = "http.lfs.store.panicked", %err, "");
Err(Box::new(error(
context,
StatusCode::INTERNAL_SERVER_ERROR,
"the lfs object store could not be opened".to_owned(),
)))
}
}
}
/// `Content-Length`, if it is present and is a number.
fn content_length(headers: &HeaderMap) -> Option<u64> {
headers
.get(header::CONTENT_LENGTH)?
.to_str()
.ok()?
.trim()
.parse()
.ok()
}
/// Does this request's `Accept` admit an LFS JSON answer?
///
/// The specification says a server **should** answer `406` when it does not.
/// `*/*` and an absent header count as yes: `curl` sends the first and a
/// hand-written client often sends neither, and refusing those would be
/// refusing on a header nobody meant anything by. git-lfs itself always sends
/// the exact type.
pub fn accepts_lfs_json(headers: &HeaderMap) -> bool {
let Some(accept) = headers.get(header::ACCEPT).and_then(|v| v.to_str().ok()) else {
return true;
};
accept.split(',').any(|item| {
let item = item.split(';').next().unwrap_or("").trim();
item.is_empty() || item == "*/*" || item.eq_ignore_ascii_case(LFS_CONTENT_TYPE)
})
}
/// A JSON response with the LFS media type.
fn json<T: serde::Serialize>(code: StatusCode, value: &T) -> Response<Body> {
let encoded = match serde_json::to_vec(value) {
Ok(encoded) => encoded,
Err(err) => {
// Unreachable for these types — every field is a String, a number
// or a map with String keys — and it is still not an `unwrap`,
// because the alternative to this line is a panic inside a
// connection task that hyper reports as a dropped socket.
tracing::error!(target: TARGET_GIT, event = "http.lfs.encode.failed", %err, "");
br#"{"message":"the response could not be encoded"}"#.to_vec()
}
};
let mut response = Response::new(
Full::new(Bytes::from(encoded))
.map_err(|never| match never {})
.boxed(),
);
*response.status_mut() = code;
response
.headers_mut()
.insert(header::CONTENT_TYPE, header_value(LFS_CONTENT_TYPE));
response
}
/// A batch-level error, in the shape the specification defines.
pub fn error(context: &Context<'_>, code: StatusCode, message: String) -> Response<Body> {
error_with_id(code, message, context.id)
}
/// The same, for a refusal raised before there is a [`Context`] to raise it
/// from — a bad `Accept`, an unparsable body, an LFS request to a server that
/// serves no LFS.
pub fn error_with_id(code: StatusCode, message: String, id: RequestId) -> Response<Body> {
json(
code,
&ErrorBody {
message,
request_id: id.to_string(),
},
)
}
/// A response with a status and nothing else. `PUT` and `verify` both answer
/// this on success, which is what the specification says they answer.
fn empty(code: StatusCode) -> Response<Body> {
let mut response = Response::new(
Full::new(Bytes::new())
.map_err(|never| match never {})
.boxed(),
);
*response.status_mut() = code;
response
}
fn header_value(text: &str) -> HeaderValue {
HeaderValue::from_str(text).unwrap_or_else(|_| HeaderValue::from_static("invalid"))
}
/// Truncate a client-supplied string before it goes into a message.
fn clip(text: &str) -> String {
text.chars().take(64).collect()
}
/// Read a bounded request body.
///
/// **Bounded on the way in, not after.** `Incoming::collect()` reads whatever
/// the client sends; this stops at `limit` and refuses, which is the difference
/// between a ceiling and a report of how far past it the server already went.
///
/// Four callers now, not two: the LFS batch JSON
/// ([`MAX_BATCH_REQUEST_BYTES`]), the registry blob
/// ([`crate::service::MAX_REGISTRY_BLOB_BYTES`]), and — since GUN-005 — the
/// gzipped smart-HTTP request body, which used `Incoming::collect()` and had no
/// ceiling on the step it was on at all. The message therefore says "a request
/// body" rather than "an lfs request body"; it was never only LFS's.
pub async fn read_bounded(incoming: Incoming, limit: usize) -> Result<Vec<u8>, String> {
use http_body_util::BodyExt as _;
let mut incoming = incoming;
let mut out = Vec::new();
while let Some(frame) = incoming.frame().await {
let frame = frame.map_err(|err| format!("the request body did not arrive: {err}"))?;
if let Ok(data) = frame.into_data() {
if out.len() + data.len() > limit {
return Err(format!(
"a request body over {limit} bytes was refused without being read"
));
}
out.extend_from_slice(&data);
}
}
Ok(out)
}
/// The absolute URL prefix a client must come back to, as **this** request
/// reached the server.
///
/// ```text
/// https://gunnar.example:8080/team/project.git/info/lfs
/// ```
///
/// The operator's `HttpConfig::external_base` wins. Otherwise the scheme is
/// `X-Forwarded-Proto` when a proxy set it to `https`, else **this listener's
/// own**: `https` when it terminates TLS itself (`--http-tls-cert`,
/// `--http-tls-self`), `http` otherwise. The host is `X-Forwarded-Host`, else
/// `Host`, else the request target's authority, which is where HTTP/2 carries
/// it (`:authority`; an h2 request has no `Host` header, and git-lfs speaks h2
/// to a TLS listener that offers it). A request that names no host at all gets
/// `localhost`, which fails visibly rather than pointing at some other host.
pub fn endpoint_for(
external_base: Option<&str>,
listener_tls: bool,
authority: Option<&str>,
headers: &HeaderMap,
repo_path: &str,
) -> String {
let repo_path = repo_path.trim_end_matches('/');
if let Some(base) = external_base {
return format!("{}{repo_path}/info/lfs", base.trim_end_matches('/'));
}
let header = |name: &str| headers.get(name).and_then(|value| value.to_str().ok());
let scheme = match header(crate::auth::FORWARDED_PROTO) {
Some(proto) if proto.eq_ignore_ascii_case("https") => "https",
_ if listener_tls => "https",
_ => "http",
};
let host = header("x-forwarded-host")
.or_else(|| header(header::HOST.as_str()))
.or(authority)
.unwrap_or("localhost");
format!("{scheme}://{host}{repo_path}/info/lfs")
}
/// Which [`GitService`] — and therefore which grant — an LFS request needs.
///
/// One function, so the direction of a request is decided in one place. `GET`
/// reads and `PUT` writes; `verify` is part of an upload and needs Write, the
/// same as SSH's `verify-object`, which is `upload`-only.
pub fn service_for(
endpoint: &crate::route::Lfs,
method: &Method,
operation: Option<Operation>,
) -> GitService {
match endpoint {
crate::route::Lfs::Batch => {
GitService::LfsTransfer(operation.unwrap_or(Operation::Download))
}
crate::route::Lfs::Verify => GitService::LfsTransfer(Operation::Upload),
crate::route::Lfs::Object(_) => GitService::LfsTransfer(if method == Method::PUT {
Operation::Upload
} else {
Operation::Download
}),
}
}
/// Parse a batch request body.
pub fn parse_batch(bytes: &[u8]) -> Result<BatchRequest, String> {
serde_json::from_slice(bytes).map_err(|err| format!("the batch request is not valid: {err}"))
}
/// Parse a verify request body.
pub fn parse_verify(bytes: &[u8]) -> Result<VerifyRequest, String> {
serde_json::from_slice(bytes).map_err(|err| format!("the verify request is not valid: {err}"))
}
/// The request this batch names, before the store is consulted, so the caller
/// can choose a grant from it.
pub fn operation_of(body: &BatchRequest) -> Option<Operation> {
Operation::parse(&body.operation)
}
#[cfg(test)]
mod tests {
use super::*;
fn headers(pairs: &[(&str, &str)]) -> HeaderMap {
let mut map = HeaderMap::new();
for (name, value) in pairs {
map.insert(
header::HeaderName::from_bytes(name.as_bytes()).expect("a literal header name"),
HeaderValue::from_str(value).expect("a literal header value"),
);
}
map
}
/// **The href a client is sent back to must be the host it arrived on**, or
/// a clone through a proxy fetches its pointers from one server and its
/// content from another that may not exist.
#[test]
fn the_endpoint_follows_the_request_and_the_proxys_word_for_the_scheme() {
assert_eq!(
endpoint_for(
None,
false,
None,
&headers(&[("host", "gunnar:8080")]),
"/team/p.git"
),
"http://gunnar:8080/team/p.git/info/lfs"
);
assert_eq!(
endpoint_for(
None,
false,
None,
&headers(&[("host", "gunnar:8080"), ("x-forwarded-proto", "https")]),
"/team/p.git"
),
"https://gunnar:8080/team/p.git/info/lfs",
"a proxy that terminated TLS must not have gunnar send clients back to http://"
);
assert_eq!(
endpoint_for(
None,
false,
None,
&headers(&[("host", "inside:8080"), ("x-forwarded-host", "git.example")]),
"/team/p.git"
),
"http://git.example/team/p.git/info/lfs"
);
// The operator's own answer wins over both, and does not grow a second
// slash where one already ends the base.
assert_eq!(
endpoint_for(
Some("https://git.example/"),
true,
Some("inside:8080"),
&headers(&[("host", "inside:8080")]),
"/team/p.git"
),
"https://git.example/team/p.git/info/lfs"
);
// The red half: with no Host and no configuration the answer is a URL
// that fails loudly rather than one pointing at some other host.
assert_eq!(
endpoint_for(None, false, None, &HeaderMap::new(), "/team/p.git"),
"http://localhost/team/p.git/info/lfs"
);
}
/// **A TLS listener sends its LFS clients back over https, to the host
/// they reached**, also over HTTP/2 (GUN-FIELD-TEST, 2026-10-31). git-lfs
/// 3.6 speaks h2 to a `--http-tls-cert` listener (ALPN offers h2 first), an
/// h2 request carries its host as `:authority` and has no `Host` header,
/// and the batch answered `http://localhost/...`: every `git lfs push` over
/// https to the appliance failed with "dial tcp 127.0.0.1:80: connection
/// refused".
#[test]
fn a_tls_listener_answers_https_at_the_requests_authority_also_over_h2() {
assert_eq!(
endpoint_for(None, true, Some("127.0.0.1:24101"), &HeaderMap::new(), "/field/lfs.git"),
"https://127.0.0.1:24101/field/lfs.git/info/lfs",
"h2 over TLS: no Host header, the authority names the host, the listener's TLS the scheme"
);
assert_eq!(
endpoint_for(
None,
true,
Some("ignored:1"),
&headers(&[("host", "git.example:8443")]),
"/team/p.git"
),
"https://git.example:8443/team/p.git/info/lfs",
"h1 over TLS: the Host header, and https because this listener is TLS"
);
assert_eq!(
endpoint_for(
None,
false,
Some("gunnar:8080"),
&HeaderMap::new(),
"/team/p.git"
),
"http://gunnar:8080/team/p.git/info/lfs",
"a plain listener stays http (a proxy says otherwise with X-Forwarded-Proto)"
);
}
/// **`PUT` must never be authorised as a read.** The direction of an LFS
/// object transfer is in the method and nowhere else.
#[test]
fn the_grant_follows_the_direction_of_the_request() {
let object = crate::route::Lfs::Object("a".repeat(64));
assert_eq!(
service_for(&object, &Method::PUT, None),
GitService::LfsTransfer(Operation::Upload)
);
assert_eq!(
service_for(&object, &Method::GET, None),
GitService::LfsTransfer(Operation::Download)
);
assert_eq!(
service_for(&crate::route::Lfs::Verify, &Method::POST, None),
GitService::LfsTransfer(Operation::Upload),
"verify is the second half of an upload and needs Write, exactly as \
`verify-object` does over ssh"
);
assert_eq!(
service_for(
&crate::route::Lfs::Batch,
&Method::POST,
Some(Operation::Upload)
),
GitService::LfsTransfer(Operation::Upload)
);
assert_eq!(
service_for(
&crate::route::Lfs::Batch,
&Method::POST,
Some(Operation::Download)
),
GitService::LfsTransfer(Operation::Download)
);
// A batch whose operation could not be parsed falls to the WEAKER
// grant, never the stronger one. It is refused a moment later either
// way; what matters is that the failure direction is Read.
assert_eq!(
service_for(&crate::route::Lfs::Batch, &Method::POST, None),
GitService::LfsTransfer(Operation::Download)
);
// And the two really are different grants, or every assertion above
// would pass against a function returning a constant.
assert_ne!(
GitService::LfsTransfer(Operation::Upload).required_access(),
GitService::LfsTransfer(Operation::Download).required_access()
);
}
#[test]
fn accept_is_honoured_but_not_pedantically() {
assert!(accepts_lfs_json(&HeaderMap::new()));
assert!(accepts_lfs_json(&headers(&[("accept", "*/*")])));
assert!(accepts_lfs_json(&headers(&[(
"accept",
"application/vnd.git-lfs+json"
)])));
assert!(accepts_lfs_json(&headers(&[(
"accept",
"application/vnd.git-lfs+json; charset=utf-8"
)])));
assert!(accepts_lfs_json(&headers(&[(
"accept",
"text/html, application/vnd.git-lfs+json"
)])));
// The red half.
assert!(!accepts_lfs_json(&headers(&[("accept", "text/html")])));
assert!(!accepts_lfs_json(&headers(&[(
"accept",
"application/json"
)])));
}
/// The batch response is JSON git-lfs recognises: `actions` and `error` are
/// mutually exclusive, and an object with **neither** is the "already
/// there" signal, so a `Missing` object must never serialise as one.
#[test]
fn a_missing_download_serialises_as_an_error_and_not_as_an_empty_object() {
let store = gunnar_lfs::MemLfsStore::new();
let here = store.insert(b"content");
let gone = "a".repeat(64);
let limits = Limits::default();
let endpoint = "http://h/r.git/info/lfs".to_owned();
let answer = answer_for(
&store,
&limits,
None,
Operation::Download,
ObjectSpec {
oid: gone.clone(),
size: 7,
},
&endpoint,
&None,
);
let text = serde_json::to_string(&answer).expect("encode");
assert!(
text.contains(r#""error":{"code":404"#),
"a download of an absent object must carry a 404 error object, or git-lfs reads \
it as content it already has and the clone leaves a pointer file: {text}"
);
assert!(!text.contains("actions"), "{text}");
// The present half, so the assertion above is not passing against a
// planner that errors on everything.
let answer = answer_for(
&store,
&limits,
None,
Operation::Download,
ObjectSpec {
oid: here.to_hex(),
size: 7,
},
&endpoint,
&Some("Bearer t".to_owned()),
);
let text = serde_json::to_string(&answer).expect("encode");
assert!(!text.contains("error"), "{text}");
assert!(
text.contains(&format!(r#""href":"{endpoint}/objects/{here}""#)),
"{text}"
);
assert!(
text.contains(r#""Authorization":"Bearer t""#),
"the object href must carry the credential the batch arrived with, or git-lfs \
runs the credential helper again against it: {text}"
);
assert!(text.contains(r#""authenticated":true"#), "{text}");
// An UPLOAD of something already held: no actions AND no error, which
// is how the specification says "nothing to do".
let answer = answer_for(
&store,
&limits,
None,
Operation::Upload,
ObjectSpec {
oid: here.to_hex(),
size: 7,
},
&endpoint,
&None,
);
let text = serde_json::to_string(&answer).expect("encode");
assert!(!text.contains("error"), "{text}");
assert!(
!text.contains("actions"),
"an upload the server need not receive carries neither: {text}"
);
// An UPLOAD it does need: an upload action and a verify action.
let answer = answer_for(
&store,
&limits,
None,
Operation::Upload,
ObjectSpec { oid: gone, size: 7 },
&endpoint,
&None,
);
let text = serde_json::to_string(&answer).expect("encode");
assert!(text.contains(r#""upload""#), "{text}");
assert!(
text.contains(&format!(r#""href":"{endpoint}/objects/verify""#)),
"{text}"
);
}
/// A quota refusal is a per-object error carrying the **quota's** code,
/// which is `gunnar_lfs::Error::status`'s and not a number invented here.
#[test]
fn a_refusal_carries_the_status_the_lfs_error_table_gives_it() {
let store = gunnar_lfs::MemLfsStore::new();
let answer = answer_for(
&store,
&Limits::default().max_object_bytes(4),
None,
Operation::Upload,
ObjectSpec {
oid: "c".repeat(64),
size: 5,
},
"http://h/r.git/info/lfs",
&None,
);
let error = answer
.error
.expect("5 bytes must not fit under a 4-byte quota");
assert_eq!(error.code, 413);
assert_eq!(
error.code,
gunnar_lfs::Error::TooLarge(String::new()).status(),
"the code must come from the one error table, not from a literal here"
);
// A malformed oid is 422 and not 413, or the mapping is a constant.
let answer = answer_for(
&store,
&Limits::default(),
None,
Operation::Download,
ObjectSpec {
oid: "not-an-oid".to_owned(),
size: 1,
},
"http://h/r.git/info/lfs",
&None,
);
assert_eq!(answer.error.expect("a malformed oid is refused").code, 422);
}
}