Cost model
Every user-visible latency is a sum of sequential bucket requests, so round trips are the performance design.
The latency primitives
The bucket is the only durable primitive. Single-object CAS is its only transaction.
| Primitive | Cost | Notes |
|---|---|---|
| GET or PUT of a small object | 60–80 ms p50/p99 | One request is one round trip |
Conditional GET (If-None-Match) answering 304 | 15–18 ms | The cheapest "is anything new?" |
| HEAD | ≈ GET | Never added on a happy path |
| 404 | Free | Probe, don’t list |
| LIST | Slow, paged, eventually-ish | Never on a hot path |
| CAS overwrite of one object | Serialized, ~1 write/s | A throughput cap; 412 is the normal contention signal |
| Range read of a big object | ~100 MB/s per connection | Stripe for more; bulk bytes on their own pool |
Depth before count
Two PUTs in parallel cost one round trip. A healthy push is 6 requests in 4 rounds.
The pending marker follows the CAS so an uncommitted push cannot wake maintenance. Its failure never changes the committed push.
Budgets per operation
Happy path: sequential depth, then total store requests.
| Operation | Depth | Requests |
|---|---|---|
Any read (info/refs, ls-refs, API refs and resolve) | 1 conditional GET (or 0 within wal.freshness_ttl) | 1 |
| Cold refs sync | 1 manifest GET, then 1 round of checkpoint refs ∥ log tail segments | No checkpoint: 1 + tail (2 with one segment); checkpoint: 2 + tail |
| Push | Freshness GET → pack PUT ∥ idx PUT ∥ log PUT → manifest CAS → best-effort pending-marker PUT | Request: 6; already-synced publish: 5 |
| Ref write API | Freshness GET → optional tag pack PUT ∥ idx PUT ∥ log PUT → manifest CAS → pending-marker PUT | Ref-only: 4; annotated tag: 6 |
| Commit or merge write API | Freshness GET → new-object pack PUT ∥ idx PUT ∥ log PUT → manifest CAS → pending-marker PUT | Commit or merge commit: 6; fast-forward: 4; already merged: 1 |
| Checkpoint | Conditional GET → refs PUT ∥ checkpoint PUT → manifest CAS | 3 rounds, 4 requests |
| Lease acquire | 1 GET → 1 CAS put (or 1 Create when absent) | 2 |
| Maintainer pending pass | 1 bounded LIST, then each repository keeps its existing unit depth | Existing requests per repository; up to maintenance.workers repositories overlap |
| Authentication (introspect mode) | 0 on a cache hit; 1 HTTP request to the introspection URL per miss | 0 store requests |
Cost follows pushes
A push writes a pending/ marker and the maintainer lists only those. Nothing lists repos/, so idle cost does not grow with repository count.
| Repositories | Store requests over 5 idle minutes |
|---|---|
| 100 | 118 |
| 1,000 | 118 |
| 50,000 | 120 |
Architectureshows the marker's path through the maintainer.
Rules of thumb
- Let the conditional write be the read
A 412 on Create means it exists; a 412 on Update means someone moved it. Don't GET first.
- Verify on the failure path
Probes run only after a failed Create or CAS. The happy path never pays for rare cases.
- Batch at the CAS
Group commit (
wal.batch_window) turns N concurrent pushes into one log PUT and one CAS.- Carry state in the manifest
Every request fetches it anyway, so a reader never needs a second request to know what to fetch.
- Jitter every retry
A synchronized retry storm serializes itself on a 1 write/s object.
Acceptance bars
| Claim | Bar |
|---|---|
| Cold instance is useful in seconds | ls-remoteof any repository < 1 s on a fresh instance |
| Cache is a cache | Stop the server, delete cache.dir, start it: every repository clones again from the bucket |
| Push | Acknowledged only after the bucket ACKs; one CAS per batch |
| Consistency | Push then fetch anywhere sees it; concurrent pushers: exactly one winner |
| Cost model | A maintainer pass touches only repositories with a pending marker; no periodic LIST of repos/ anywhere |
| Transient store errors | 5xx or throttling on any store operation is retried with backoff and never surfaces as a failed push on its own |