crates/gunnar-lfs/src/batch.rs
//! **What the server decides about one object named in a batch — once, for both
//! surfaces.**
//!
//! gunnar answers a `batch` on two wires: git-lfs 3.0's pure-SSH
//! `git-lfs-transfer` ([`crate::session`]) and the HTTP batch API
//! (`gunnar_http::lfs`). The *decision* is the same on both — is it here, may it
//! be that big, must the client move it — and only the rendering differs. This
//! module is the decision. Both wires route through [`plan_object`]; neither has
//! a second copy of it (one implementation).
//!
//! # The three outcomes, and why `noop` is not one of them
//!
//! The SSH grammar is `<oid> SP <size> SP <action>` with
//! `action ∈ upload | download | noop`, and it has no fourth token and no error
//! line. It **used to** fold two different facts into `noop`:
//!
//! | | upload batch | download batch |
//! |---|---|---|
//! | the server holds it | `noop` — *you need not send it* | `download` |
//! | the server does not | `upload` | `download` — *was `noop`, see below* |
//!
//! The bottom-right cell was the bug. A `download` batch naming an object the
//! server does not hold answered `noop`, git-lfs takes an object with no action
//! as one it already has, and the fetch reported success having transferred
//! nothing — the mirror image of the push pinned in `tests/interop.rs`.
//!
//! This module therefore keeps the two facts apart as [`Plan::AlreadyPresent`]
//! and [`Plan::Missing`], and lets each surface say what its grammar can say:
//!
//! * HTTP renders [`Plan::Missing`] as the batch API's **per-object `error`
//! object with code 404, inside a 200 response**, which is exactly the case
//! `docs/api/batch.md` defines that shape for, and which makes git-lfs fail
//! the object visibly.
//! * SSH renders it **`download`** ([`Plan::ssh_action`]), and answers the
//! `get-object` that follows with a **404**.
//!
//! # The SSH half, settled 2026-08-05
//!
//! The note that used to stand here said SSH renders both as `noop` because
//! "the grammar cannot carry more". Half right, and the wrong half was load-
//! bearing. The grammar really cannot carry a per-object error — see
//! [`Plan::ssh_action`] for the three citations, including the rule that any
//! `error=404` a server invented would be **specified into being ignored**. But
//! `noop` is not the residue left when there is nothing to say. It is a positive
//! claim, *"there is no action valid for this object"*, and git-lfs acts on it by
//! skipping the object with no error and no output at all.
//!
//! The per-object error exists one command later: `get-object` has a
//! `status-error-response`. It is only ever reached if the batch gives the client
//! an action to ask with. So the bottom-right cell renders `download`, the client
//! asks, and it gets a real 404 for that object while every other object in the
//! batch still transfers — which is what `t/t-fetch.sh` demands over HTTP and
//! what a whole-batch refusal could not have delivered.
//!
//! The four recorded transcripts change accordingly, and one of them —
//! `tests/transcript.rs`, replaying the download against an empty store — is the
//! guard that catches a regression here first.
//!
//! Everything else — the quota, the store lookup, the ordering between them — is
//! identical, which is the point of putting it here.
use crate::allowance::Allowance;
use crate::error::{Error, Result};
use crate::limits::Limits;
use crate::oid::Oid;
use crate::operation::Operation;
use crate::store::LfsStore;
/// What one object named in a `batch` request resolves to.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Plan {
/// The client must move it: **upload** it on an upload batch, **download**
/// it on a download batch.
Transfer,
/// An **upload** batch named an object the server already holds. Nothing to
/// send.
AlreadyPresent,
/// A **download** batch named an object the server does not hold.
///
/// Not the same fact as [`Plan::AlreadyPresent`] and never conflated with
/// it here; see the module docs.
Missing,
}
impl Plan {
/// The action word the pure-SSH grammar carries for this plan.
///
/// # `Missing` renders as `download`, and that is the whole answer to the
/// question this function was left open on
///
/// RESEARCHED 2026-08-05 against the specification, the client's source and
/// the reference server's. **The grammar genuinely cannot carry a per-object
/// error** — that half of the note in the module docs is confirmed, and it is
/// worse than "there is no field for it":
///
/// * `batch-oid-line = PKT-LINE(oid size action *(key=[value]) LF)`
/// (`git-lfs/docs/proposals/ssh_adapter.md`, which is where the pure-SSH
/// protocol is specified — `docs/api/batch.md` governs HTTP only). There is
/// no `error` production and no `code`.
/// * The same document says *"unknown key-value pairs in the `oid-line`
/// production"* **should be ignored**, so a server that invented
/// `error=404` would be specified into being ignored. No extension works.
/// * `tq/ssh.go` confirms it: only `id`, `token`, `expires-in` and
/// `expires-at` are read; every other key is dropped on the floor.
///
/// So no code goes on this line — but it does **not** follow that `noop` is
/// the answer, and that inference is what made the fetch lie. `noop` is not
/// silence, it is a **claim**, and git-lfs acts on it: `tq/ssh.go` skips the
/// action map entirely for `noop`, and `tq/transfer_queue.go` then skips the
/// object with no error and no line printed:
///
/// ```text
/// } else if a == nil && manifest.standaloneTransferAgent == "" {
/// q.Skip(o.Size)
/// q.wait.Done()
/// }
/// ```
///
/// **`get-object` is where the protocol can still tell the truth.** It has a
/// `status-error-response`, so the 404 exists — one command later, per
/// object, exactly where it is needed. It is simply unreachable unless the
/// batch first hands the client an action to ask with. So `Missing` renders
/// `download`: the client asks, gets a real 404 for that object alone, and
/// every other object in the batch still transfers. That is precisely the
/// outcome git-lfs's own suite requires over HTTP (`t/t-fetch.sh`, "fetch
/// with missing object": *"should return non-zero, but should also download
/// all the other valid files too"*), reached over SSH.
///
/// MEASURED with git-lfs 3.7.1: under `noop`, a two-object fetch of which
/// gunnar held one printed "2 objects found, done." and exited **0**. Under
/// `download`, it exits **2** and names the oid.
///
/// The cost is stated rather than hidden: `tq/ssh.go` wraps *every* non-2xx
/// from `get-object` in `errors.NewRetriableError` with no exception, so an
/// absent object walks the backoff ladder — 0.25, 0.5, 1, 2, 4, 8, 10, 10
/// seconds, **35.75 s** — before the failure is reported. Retries re-batch,
/// so that is once for the whole batch, not once per object. Filed upstream
/// as **git-lfs/git-lfs#6319**, with both fixes: a per-object error in the
/// grammar, or simply making a 404 from `get-object` non-retriable.
///
/// [`Plan::AlreadyPresent`] keeps `noop`, where it is not a lie: on an upload
/// batch the server really does hold the bytes and there really is no action.
pub fn ssh_action(self, operation: Operation) -> &'static str {
match (self, operation) {
(Plan::Transfer, Operation::Upload) => "upload",
(Plan::Transfer, Operation::Download) => "download",
(Plan::AlreadyPresent, _) => "noop",
// Only ever produced by a download batch, and answered as one.
(Plan::Missing, _) => "download",
}
}
/// Does the client have to move bytes for this object?
pub fn needs_transfer(self) -> bool {
matches!(self, Plan::Transfer)
}
}
/// Decide one object.
///
/// `Err` is a refusal that carries a status through [`Error::status`]: the
/// quota, or a store that could not be read. The SSH session fails the whole
/// batch on it (as it always has); the HTTP surface may render it per object,
/// which is what the batch API's `error` member is for.
///
/// # The quota is checked before the store
///
/// Deliberately. The store lookup is a `stat` per object and a batch may name
/// [`Limits::max_batch_objects`] of them; an object already over the quota is
/// refused without paying for one.
///
/// # Two quotas, and they are about different things
///
/// `limits.max_object_bytes` is the OPERATOR's — one number for the whole box,
/// a wall around a single object, 413. `allowance` is the TENANT's remaining
/// plan — see [`Allowance`] — a wall around what one namespace may accumulate,
/// 507. Both are checked here so that neither wire has its own copy (one
/// implementation), and the operator's is checked first because it is the one
/// that is the same for everybody: an object no tenant on this box could ever
/// store should be told so in the words that will still be true after they
/// pay.
///
/// `taken` is what this session has already committed against `allowance`; the
/// HTTP wire, where every upload is its own request and the allowance is
/// re-measured for each, always passes `0`.
pub fn plan_object(
store: &dyn LfsStore,
limits: &Limits,
allowance: Option<&Allowance>,
taken: u64,
operation: Operation,
oid: &Oid,
size: u64,
) -> Result<Plan> {
// **The pre-arrival refusal.** An upload batch that names an object over
// the quota is refused here, before the client has sent one byte of any of
// them, not after the disk has taken it.
if operation.may_upload() && size > limits.max_object_bytes {
return Err(Error::TooLarge(format!(
"object {oid} is {size} bytes; this server accepts at most {}",
limits.max_object_bytes
)));
}
// …and the tenant's own room, on an upload only. A download must never see
// it: the allowance bounds what may be ACCUMULATED, an object already
// stored is already accumulated, and a tenant at their ceiling who could
// not read their own data back is a tenant whose data is held hostage —
// which `gunnar_ssh::plan` forbids by name even for the plan states that
// are harsher than this one.
if operation.may_upload() {
if let Some(allowance) = allowance {
allowance.admit(taken, size)?;
}
}
let present = store.size_of(oid)?.is_some();
// Read off the reference server (`Processor.UploadBatch` / `DownloadBatch`):
// upload wants the ones that are MISSING, download wants the ones that are
// PRESENT. The other two cells are the ones SSH cannot distinguish.
Ok(match (operation, present) {
(Operation::Upload, false) | (Operation::Download, true) => Plan::Transfer,
(Operation::Upload, true) => Plan::AlreadyPresent,
(Operation::Download, false) => Plan::Missing,
})
}
#[cfg(test)]
mod tests {
use super::*;
use crate::store::MemLfsStore;
fn limits() -> Limits {
Limits::default()
}
/// **All four cells, and they are four and not three.** A planner that
/// answered `Transfer`/`noop` — which is what the wire used to say — could
/// not tell the two folded cells apart, and the download half of the pair is
/// the bug this module exists for. Since 2026-08-05 the SSH wire does not
/// fold them either; see [`Plan::ssh_action`].
#[test]
fn the_four_cells_are_distinct_and_only_the_wire_folds_two_of_them() {
let store = MemLfsStore::new();
let here = store.insert(b"here");
let gone = Oid::parse(&"a".repeat(64)).expect("literal oid");
let cell = |operation, oid: &Oid| {
plan_object(&store, &limits(), None, 0, operation, oid, 4).expect("no refusal")
};
assert_eq!(cell(Operation::Upload, &gone), Plan::Transfer);
assert_eq!(cell(Operation::Upload, &here), Plan::AlreadyPresent);
assert_eq!(cell(Operation::Download, &here), Plan::Transfer);
assert_eq!(
cell(Operation::Download, &gone),
Plan::Missing,
"a download of an object the server does not hold is MISSING, not `noop`: git-lfs \
reads an object with no action and no error as one it already has, and the fetch \
then reports success having transferred nothing"
);
// **The two the SSH grammar must NOT fold.** This assertion was
// `assert_eq!` — "the line has one word for both facts" — and the word
// it settled on was `noop`, which git-lfs reads as "already there".
// The grammar has no per-object error, but it does have `get-object`'s
// 404, and that is unreachable unless the batch hands back an action.
// See `Plan::ssh_action`.
assert_ne!(
Plan::AlreadyPresent.ssh_action(Operation::Upload),
Plan::Missing.ssh_action(Operation::Download),
"a download batch answered `noop` for an object the server does not hold; \
git-lfs skips a no-action object silently and the fetch exits 0"
);
assert_eq!(Plan::AlreadyPresent.ssh_action(Operation::Upload), "noop");
assert_eq!(
Plan::Missing.ssh_action(Operation::Download),
"download",
"the client must be given an action, or it never asks and never \
reaches the 404 that is the only per-object error this protocol has"
);
assert_eq!(Plan::Transfer.ssh_action(Operation::Upload), "upload");
assert_eq!(Plan::Transfer.ssh_action(Operation::Download), "download");
// The red half: if `ssh_action` returned a constant, the line above and
// the fold above would both still pass.
assert_ne!(
Plan::Transfer.ssh_action(Operation::Upload),
Plan::Transfer.ssh_action(Operation::Download)
);
}
#[test]
fn the_quota_is_refused_before_the_store_is_touched_and_only_on_upload() {
let store = MemLfsStore::new();
let oid = Oid::parse(&"b".repeat(64)).expect("literal oid");
let tight = limits().max_object_bytes(16);
let err = plan_object(&store, &tight, None, 0, Operation::Upload, &oid, 17)
.expect_err("17 bytes must not fit under a 16-byte quota");
assert_eq!(err.status(), 413);
// Exactly at the quota is accepted. Without this the test would pass
// against a planner that refused every upload.
assert_eq!(
plan_object(&store, &tight, None, 0, Operation::Upload, &oid, 16).expect("16 fits"),
Plan::Transfer
);
// A DOWNLOAD is not refused by the upload quota: the bytes are already
// on the server, and refusing to serve what was accepted would strand
// it. The size on a download line is the client's claim, not a request
// for storage.
assert_eq!(
plan_object(&store, &tight, None, 0, Operation::Download, &oid, 1 << 40)
.expect("no refusal"),
Plan::Missing
);
}
}