benchmarks#

bench/ contains local benchmarks for ZDS storage paths. The goal is to track scaling curves and compare protocol work against reference implementations, not to produce a single universal score.

The benchmark style follows atproto-bench: state the exact work being measured, keep correctness work visible, compare equivalent operations, and treat run-to-run variance as real.

run#

just bench all
just bench write
just bench read
just bench repo
just bench blob
just bench metastore 10 1000
just bench get-cid 10 1000
just bench get-block 10 1000
just bench decode-record 10 1000
just bench render-record 10 1000
just bench get-record 10 1000
just bench list-records 10 1000
just bench repo-size 5000
just bench get-repo 5000 4 5
just bench repo-large 100000
just bench space 1000
just bench http
just bench official-pds
just bench write-profile 10 500
just bench run --scenario write --records 10000

just bench ... is a Just module declared from the repository root with mod bench.

current scenarios#

• write: applyWrites creating app.bsky.feed.post records.
• read: repeated listRecords queries against one seeded repo.
• repo: full repo CAR materialization through writeRepoCar.
• blob: blob write/read against disk blobstore plus SQLite metadata.
• metastore: Tranquil-shaped apply/get/list benchmark with caller counts and latency percentiles.
• get-cid: CID-only record lookup, matching Tranquil's get_record_cid metastore benchmark.
• get-block: record index lookup plus repo_blocks byte fetch.
• decode-record: DAG-CBOR decode of one seeded record block, without SQLite.
• render-record: DAG-CBOR decode plus JSON rendering, without SQLite.
• get-record: full com.atproto.repo.getRecord storage materialization.
• list-records: full com.atproto.repo.listRecords storage materialization.
• repo-size: synthetic small/medium/large repo probes for listRecords(limit=100) and full repo CAR materialization. The default Just target keeps this moderate; use repo-large for intentional 100k+ record runs.
• get-repo: focused com.atproto.sync.getRepo storage-pressure probe. It measures full repo CAR construction, since diff CAR construction after one additional commit, and concurrent full exports against one seeded repo.
• space: experimental permissioned-data storage probes for space discovery, private record writes, private record reads/lists, repo oplog catch-up, and blob readback.
• http: starts a local ReleaseFast ZDS server and measures a seeded HTTP route matrix with ApacheBench. This is the route-level check for the httpz server boundary and should be tracked separately from storage microbenchmarks.
• official-pds: copies the official-PDS read probe into a local bluesky-social/atproto checkout and runs it with Vitest.
• write-profile: applyWrites stage timing under write concurrency.

Each run uses temporary SQLite and blobstore state.

permissioned data#

Permissioned-data benchmarks are separate from the public repo/appview/sync benchmarks. The goal is not broad coverage yet; it is to catch obviously bad access patterns introduced by the experimental com.atproto.space.* storage model.

just bench space seeds one self-owned space, one writer repo, and one small blob, then measures these paths as distinct units of work:

• listSpaces
• createRecord into a space writer repo
• getRecord by (space, repo, collection, rkey)
• listRecords, limit 50, with values included by default
• getBlob storage readback
• listRepoOps catch-up reads

Comparison against Daniel's permissioned-data branch should live in this section once we have a repeatable way to run that PDS locally. Until then, keep ZDS numbers as local trend data and compare only equivalent operations.

repo sizes#

The normal write/read benchmarks use small synthetic repos so they stay fast enough to run while developing. just bench repo-size adds explicit small/medium/large tiers for repo-size-sensitive paths:

• small: 100 records
• medium: 1,000 records, unless the selected maximum is lower
• large: the max_records argument

Use just bench repo-large 100000 for a deliberate larger run. Large public accounts are useful reference shapes because they stress repo export, record indexes, and cursor/list behavior. As of June 15, 2026, the public appview reports about 38k posts for pfrazee.com and about 87k posts for jcsalterego.bsky.social, so a 100k synthetic tier is the right order of magnitude for large-repo stress. Live network corpora should be captured into fixtures before they become stable benchmarks. Do not make routine CI or pre-commit checks depend on live PDS availability.

June 29, 2026 local reachable-export check after switching full com.atproto.sync.getRepo from whole-DID repo_blocks scans to current-root reachability via zat.mst.collectReachableBlocks:

These are synthetic local trend numbers, not cross-PDS comparisons. The old whole-DID range-scan export path is intentionally no longer the production full export behavior because it could include retained stale blocks.

just bench get-repo is the narrower operator-facing lane for the same sync surface. It belongs here first because ZDS owns the storage and route behavior for serving repo exports. Cross-PDS HTTP comparisons, compression behavior, and live backfill traffic should live in atproto-bench once this local seam is stable enough to compare against other implementations.

methodology#

• Build ZDS with -Doptimize=ReleaseFast.
• Use temporary SQLite and blobstore state for every run.
• Seed records before read benchmarks so lookup/list work is separated from write setup.
• Report throughput plus p50/p95/p99/max/mean latency for concurrent paths.
• Keep HTTP route benchmarks separate from storage benchmarks. Route numbers include request parsing, routing, response serialization, auth/session work, and httpz connection handling.
• Compare ratios from the same machine and same run window. Absolute numbers move with system load, thermal state, compiler version, and storage backend.
• Do not compare paths that perform different correctness work. For example, CID-only metadata lookup is separate from full getRecord materialization.

traffic model#

For a small hosted PDS, the hot paths are not exotic. A hundred active accounts mainly means more session checks, appview-proxied feed/profile reads, record writes from clients and apps, blob uploads, sync subscribers, crawl requests, and occasional repo export/import. The storage benchmarks should therefore keep these domains separate:

• record index: current CID lookups, existence checks, and index-only lists.
• record materialization: index lookup, block fetch, DAG-CBOR decode, and JSON response rendering.
• write path: validation, repo mutation, commit/block persistence, event publication, and crawl notification.
• blob path: upload body ingest, disk write, metadata persistence, and readback.
• sync path: event delivery and repo CAR materialization.

current local baseline#

On this machine, with --records 1000 unless noted:

Current local HTTP route baseline after adopting httpz, measured with just bench http using 5000 requests at concurrency 50 for normal routes and 500 requests at concurrency 10 for blob/CAR routes. The harness covers public metadata, OAuth metadata, session/account reads, preference reads, repo reads, sync reads, blob reads, and repo CAR reads. Stateful write endpoints and WebSocket subscribeRepos need separate harnesses.

The pre-httpz column below is the original narrow route set run against commit 4841aec; it should not be treated as coverage for the expanded route matrix.

coverage map#

High-level ZDS surface area represented in bench/:

Still intentionally outside the routine benchmark gate:

• passkey ceremony cryptography and browser UX
• mail delivery providers
• live appview proxy latency
• WebSocket subscriber fanout under network backpressure
• live public-repo corpora until captured as stable fixtures

implementation comparison#

The main PDS implementations do not expose identical benchmark seams, so keep comparisons grouped by equivalent work:

stage checklist#

For each protocol-parity or storage-behavior stage, do this before calling the stage complete:

• Re-read the relevant atproto spec or lexicon, then cross-check the official PDS implementation and Tranquil.
• Decide whether any new parser, codec, verifier, resolver, repository primitive, OAuth helper, or JSON-path helper belongs in zat; keep it local unless the reusable boundary is clear.
• Add or update a focused ZDS benchmark for the changed surface.
• Run the comparable Tranquil benchmark when Tranquil has the same seam.
• Run the comparable official PDS probe when the official PDS has the same seam.
• Record any mismatch as either fixed, intentionally local with rationale, or a concrete follow-up issue. Do not leave “close enough” behavior unnamed.

Run Tranquil's metastore benchmark:

cd /tmp/tranquil-pds
SQLX_OFFLINE=true cargo bench -p tranquil-store --bench metastore --features tranquil-store/test-harness

Run the matching ZDS scenarios:

just bench metastore 10 1000
just bench metastore 100 200
just bench get-cid 10 1000
just bench get-cid 100 200

This matrix includes only operation/count pairs measured with the same unit of work. Tranquil's get_record_cid is a record-index operation. The official PDS probe measures its actor-store read paths with Vitest; its single-operation rows are included for matching internal read/write boundaries.

Summary from the May 31, 2026 local run: ZDS is fastest in the single-caller CID lookup and full-materialization read probes. Tranquil is fastest in the 10/100 caller CID lookup probes and its index-only list_records path is faster than ZDS's full listRecords materialization. For writes, Tranquil and ZDS are close around the single-caller 1000-op shape, Tranquil is faster with much tighter tails at 10 callers, and ZDS has higher 100-caller throughput with much worse tail latency.

The direct caller-count concurrency matrix is ZDS and Tranquil only. Official PDS concurrency probes are kept separate until they use the same caller model.

Official PDS batched CID lookup:

Tranquil's metastore list_records benchmark is index-only, while ZDS listRecords joins repo_blocks and materializes record data. Keep these rows as adjacent probes until ZDS grows an explicit index-only list benchmark.

The current Tranquil metastore bench panics at its built-in 1000-caller get_record_cid row with metastore handler backpressure; treat that as a backpressure behavior to investigate, not as a throughput result.

official PDS probe#

The official-PDS probe lives at bench/official-pds-records.bench.ts and is run with:

ATPROTO_REPO=/tmp/bluesky-atproto just bench official-pds

It seeds 1000 app.bsky.feed.post records through the official test network, then measures actor-store reads directly:

The matching ZDS single-caller rows are:

local probes#

These rows are useful for ZDS tuning but are not direct Tranquil comparisons.

The read-path split says SQLite serialization and row materialization dominate single-record reads. Decode/render is effectively invisible for tiny benchmark records; it needs a larger-record probe before we can extrapolate image-heavy or rich-record traffic.

write profile#

write-profile tracks where concurrent writes spend time:

ZDS separates repo ordering from database serialization: writes take a per-DID lane before loading and mutating the repo, then use the DB mutex for the shared SQLite connection. Tranquil remains the reference shape for bounded write queues, explicit backpressure, less full-repo state loading, and a narrower SQLite persistence phase.

comparison work#

Tranquil benchmark source:

• crates/tranquil-store/benches/metastore.rs
• crates/tranquil-store/benches/metastore_scale.rs
• crates/tranquil-store/benches/blockstore.rs
• crates/tranquil-store/benches/eventlog.rs

Official PDS source for matching probes:

• packages/pds/src/actor-store/record/reader.ts
• packages/pds/src/api/com/atproto/repo/applyWrites.ts
• packages/pds/src/actor-store/repo/sql-repo-reader.ts

Open apples-to-apples comparison gaps:

• single-repo write curve as record count grows
• multi-repo concurrent writes
• record-index list latency without record body materialization, across ZDS and Tranquil
• official-PDS direct concurrency harness for the same read/write boundaries
• full repo export cost
• blob/block store throughput

ZDS and Tranquil both report latency percentiles for the metastore-shaped write and CID lookup rows. The remaining comparison gap is broader coverage of blockstore, eventlog, full repo export behavior, an official-PDS benchmark harness, and an explicit ZDS index-only list probe that matches Tranquil's list_records.

June 18, 2026 write-path check#

ZDS now uses zat.mst.Mst.loadLazy for repo authoring, backed by SQLite repo_blocks, instead of reading every repo block into an in-memory CAR before each write. This completes the earlier zat lazy-MST adoption for the hot write path.

Fresh local 10-caller metastore-shaped rows. The ZDS write row was repeated three times after the change and stayed in the same band: 2312.8-2525.2 ops/s, p95 11.2-12.2 ms.

Two just bench write-profile 10 1000 runs after the lazy-MST change reported 2527.1-2698.4 ops/s, p95 9.3-9.8 ms. Average write time was split roughly across lock wait 23-24%, lazy repo loading 9-10%, staging 10%, commit build 7%, and SQLite/event persistence 50%.