benchmarks#

bench/ contains local benchmarks for ZDS storage paths. The goal is to track scaling curves and compare protocol work against reference implementations, not to produce a single universal score.

The benchmark style follows atproto-bench: state the exact work being measured, keep correctness work visible, compare equivalent operations, and treat run-to-run variance as real.

run#

just bench all
just bench write
just bench read
just bench repo
just bench blob
just bench metastore 10 1000
just bench get-cid 10 1000
just bench get-block 10 1000
just bench decode-record 10 1000
just bench render-record 10 1000
just bench get-record 10 1000
just bench list-records 10 1000
just bench http
just bench official-pds
just bench write-profile 10 500
just bench run --scenario write --records 10000

just bench ... is a Just module declared from the repository root with mod bench.

current scenarios#

• write: applyWrites creating app.bsky.feed.post records.
• read: repeated listRecords queries against one seeded repo.
• repo: full repo CAR materialization through writeRepoCar.
• blob: blob write/read against disk blobstore plus SQLite metadata.
• metastore: Tranquil-shaped apply/get/list benchmark with caller counts and latency percentiles.
• get-cid: CID-only record lookup, matching Tranquil's get_record_cid metastore benchmark.
• get-block: record index lookup plus repo_blocks byte fetch.
• decode-record: DAG-CBOR decode of one seeded record block, without SQLite.
• render-record: DAG-CBOR decode plus JSON rendering, without SQLite.
• get-record: full com.atproto.repo.getRecord storage materialization.
• list-records: full com.atproto.repo.listRecords storage materialization.
• http: starts a local ReleaseFast ZDS server and measures a seeded HTTP route matrix with ApacheBench. This is the route-level check for the httpz server boundary and should be tracked separately from storage microbenchmarks.
• official-pds: copies the official-PDS read probe into a local bluesky-social/atproto checkout and runs it with Vitest.
• write-profile: applyWrites stage timing under write concurrency.

Each run uses temporary SQLite and blobstore state.

methodology#

• Build ZDS with -Doptimize=ReleaseFast.
• Use temporary SQLite and blobstore state for every run.
• Seed records before read benchmarks so lookup/list work is separated from write setup.
• Report throughput plus p50/p95/p99/max/mean latency for concurrent paths.
• Keep HTTP route benchmarks separate from storage benchmarks. Route numbers include request parsing, routing, response serialization, auth/session work, and httpz connection handling.
• Compare ratios from the same machine and same run window. Absolute numbers move with system load, thermal state, compiler version, and storage backend.
• Do not compare paths that perform different correctness work. For example, CID-only metadata lookup is separate from full getRecord materialization.

traffic model#

For a small hosted PDS, the hot paths are not exotic. A hundred active accounts mainly means more session checks, appview-proxied feed/profile reads, record writes from clients and apps, blob uploads, sync subscribers, crawl requests, and occasional repo export/import. The storage benchmarks should therefore keep these domains separate:

• record index: current CID lookups, existence checks, and index-only lists.
• record materialization: index lookup, block fetch, DAG-CBOR decode, and JSON response rendering.
• write path: validation, repo mutation, commit/block persistence, event publication, and crawl notification.
• blob path: upload body ingest, disk write, metadata persistence, and readback.
• sync path: event delivery and repo CAR materialization.

current local baseline#

On this machine, with --records 1000 unless noted:

Current local HTTP route baseline after adopting httpz, measured with just bench http using 5000 requests at concurrency 50 for normal routes and 500 requests at concurrency 10 for blob/CAR routes. The harness covers public metadata, OAuth metadata, session/account reads, preference reads, repo reads, sync reads, blob reads, and repo CAR reads. Stateful write endpoints and WebSocket subscribeRepos need separate harnesses.

The pre-httpz column below is the original narrow route set run against commit 4841aec; it should not be treated as coverage for the expanded route matrix.

reference matrix#

Run Tranquil's metastore benchmark:

cd /tmp/tranquil-pds
SQLX_OFFLINE=true cargo bench -p tranquil-store --bench metastore --features tranquil-store/test-harness

Run the matching ZDS scenarios:

just bench metastore 10 1000
just bench metastore 100 200
just bench get-cid 10 1000
just bench get-cid 100 200

This matrix includes only operation/count pairs measured with the same unit of work. Tranquil's get_record_cid is a record-index operation. The official PDS probe measures its actor-store read paths with Vitest; its single-operation rows are included for matching internal read/write boundaries.

Summary from the May 31, 2026 local run: ZDS is fastest in the single-caller CID lookup and full-materialization read probes. Tranquil is fastest in the 10/100 caller CID lookup probes and its index-only list_records path is faster than ZDS's full listRecords materialization. For writes, Tranquil and ZDS are close around the single-caller 1000-op shape, Tranquil is faster with much tighter tails at 10 callers, and ZDS has higher 100-caller throughput with much worse tail latency.

The direct caller-count concurrency matrix is ZDS and Tranquil only. Official PDS concurrency probes are kept separate until they use the same caller model.

Official PDS batched CID lookup:

Tranquil's metastore list_records benchmark is index-only, while ZDS listRecords joins repo_blocks and materializes record data. Keep these rows as adjacent probes until ZDS grows an explicit index-only list benchmark.

The current Tranquil metastore bench panics at its built-in 1000-caller get_record_cid row with metastore handler backpressure; treat that as a backpressure behavior to investigate, not as a throughput result.

official PDS probe#

The official-PDS probe lives at bench/official-pds-records.bench.ts and is run with:

ATPROTO_REPO=/tmp/bluesky-atproto just bench official-pds

It seeds 1000 app.bsky.feed.post records through the official test network, then measures actor-store reads directly:

The matching ZDS single-caller rows are:

local probes#

These rows are useful for ZDS tuning but are not direct Tranquil comparisons.

The read-path split says SQLite serialization and row materialization dominate single-record reads. Decode/render is effectively invisible for tiny benchmark records; it needs a larger-record probe before we can extrapolate image-heavy or rich-record traffic.

write profile#

write-profile tracks where concurrent writes spend time:

ZDS separates repo ordering from database serialization: writes take a per-DID lane before loading and mutating the repo, then use the DB mutex for the shared SQLite connection. Tranquil remains the reference shape for bounded write queues, explicit backpressure, less full-repo state loading, and a narrower SQLite persistence phase.

comparison work#

Tranquil benchmark source:

• crates/tranquil-store/benches/metastore.rs
• crates/tranquil-store/benches/metastore_scale.rs
• crates/tranquil-store/benches/blockstore.rs
• crates/tranquil-store/benches/eventlog.rs

Official PDS source for matching probes:

• packages/pds/src/actor-store/record/reader.ts
• packages/pds/src/api/com/atproto/repo/applyWrites.ts
• packages/pds/src/actor-store/repo/sql-repo-reader.ts

Open apples-to-apples comparison gaps:

• single-repo write curve as record count grows
• multi-repo concurrent writes
• record-index list latency without record body materialization, across ZDS and Tranquil
• official-PDS direct concurrency harness for the same read/write boundaries
• full repo export cost
• blob/block store throughput

ZDS and Tranquil both report latency percentiles for the metastore-shaped write and CID lookup rows. The remaining comparison gap is broader coverage of blockstore, eventlog, full repo export behavior, an official-PDS benchmark harness, and an explicit ZDS index-only list probe that matches Tranquil's list_records.