benchmarks#

bench/ contains local benchmarks for ZDS storage paths. The goal is to track scaling curves and compare protocol work against reference implementations, not to produce a single universal score.

The benchmark style follows atproto-bench: state the exact work being measured, keep correctness work visible, compare equivalent operations, and treat run-to-run variance as real.

run#

just bench all
just bench write
just bench read
just bench repo
just bench blob
just bench metastore 10 1000
just bench get-cid 10 1000
just bench get-block 10 1000
just bench decode-record 10 1000
just bench render-record 10 1000
just bench get-record 10 1000
just bench list-records 10 1000
just bench official-pds
just bench write-profile 10 500
just bench run --scenario write --records 10000

just bench ... is a Just module declared from the repository root with mod bench.

current scenarios#

• write: applyWrites creating app.bsky.feed.post records.
• read: repeated listRecords queries against one seeded repo.
• repo: full repo CAR materialization through writeRepoCar.
• blob: blob write/read against disk blobstore plus SQLite metadata.
• metastore: Tranquil-shaped apply/get/list benchmark with caller counts and latency percentiles.
• get-cid: CID-only record lookup, matching Tranquil's get_record_cid metastore benchmark.
• get-block: record index lookup plus repo_blocks byte fetch.
• decode-record: DAG-CBOR decode of one seeded record block, without SQLite.
• render-record: DAG-CBOR decode plus JSON rendering, without SQLite.
• get-record: full com.atproto.repo.getRecord storage materialization.
• list-records: full com.atproto.repo.listRecords storage materialization.
• official-pds: copies the official-PDS read probe into a local bluesky-social/atproto checkout and runs it with Vitest.
• write-profile: applyWrites stage timing under write concurrency.

Each run uses temporary SQLite and blobstore state.

methodology#

• Build ZDS with -Doptimize=ReleaseFast.
• Use temporary SQLite and blobstore state for every run.
• Seed records before read benchmarks so lookup/list work is separated from write setup.
• Report throughput plus p50/p95/p99/max/mean latency for concurrent paths.
• Compare ratios from the same machine and same run window. Absolute numbers move with system load, thermal state, compiler version, and storage backend.
• Do not compare paths that perform different correctness work. For example, CID-only metadata lookup is separate from full getRecord materialization.

traffic model#

For a small hosted PDS, the hot paths are not exotic. A hundred active accounts mainly means more session checks, appview-proxied feed/profile reads, record writes from clients and apps, blob uploads, sync subscribers, crawl requests, and occasional repo export/import. The storage benchmarks should therefore keep these domains separate:

• record index: current CID lookups, existence checks, and index-only lists.
• record materialization: index lookup, block fetch, DAG-CBOR decode, and JSON response rendering.
• write path: validation, repo mutation, commit/block persistence, event publication, and crawl notification.
• blob path: upload body ingest, disk write, metadata persistence, and readback.
• sync path: event delivery and repo CAR materialization.

current local baseline#

On this machine, with --records 1000 unless noted:

reference matrix#

Run Tranquil's metastore benchmark:

cd /tmp/tranquil-pds
SQLX_OFFLINE=true cargo bench -p tranquil-store --bench metastore --features tranquil-store/test-harness

Run the matching ZDS scenarios:

just bench metastore 10 1000
just bench metastore 100 200
just bench get-cid 10 1000
just bench get-cid 100 200

This matrix includes only operation/count pairs measured with the same unit of work. Tranquil's get_record_cid is a record-index operation. The official PDS probe measures its actor-store read paths with Vitest; its single-operation rows are included for matching internal read/write boundaries.

Summary: ZDS is fastest in the single-caller read probes. Tranquil is fastest in the direct 10/100 caller CID lookup probes. The official PDS is slower in these microbenchmarks because the probe goes through its actor-store/test-network stack rather than a bare SQLite reader. For writes, ZDS is fastest in the single-caller row, Tranquil is faster than ZDS at 10 callers, and ZDS is faster than Tranquil at 100 callers but with a worse p99/max tail.

The direct caller-count concurrency matrix is ZDS and Tranquil only. Official PDS concurrency probes are kept separate until they use the same caller model.

Official PDS batched CID lookup:

Tranquil's metastore list_records benchmark is not in this table because the ZDS row reports the official-PDS-shaped listRecords: record index lookup, repo_blocks fetch, DAG-CBOR decode, and JSON materialization. Add an index-only ZDS list probe before comparing that row.

official PDS probe#

The official-PDS probe lives at bench/official-pds-records.bench.ts and is run with:

ATPROTO_REPO=/tmp/bluesky-atproto just bench official-pds

It seeds 1000 app.bsky.feed.post records through the official test network, then measures actor-store reads directly:

The matching ZDS single-caller rows are:

local probes#

These rows are useful for ZDS tuning but are not direct Tranquil comparisons.

The read-path split says SQLite serialization and row materialization dominate single-record reads. Decode/render is effectively invisible for tiny benchmark records; it needs a larger-record probe before we can extrapolate image-heavy or rich-record traffic.

write profile#

write-profile tracks where concurrent writes spend time:

ZDS separates repo ordering from database serialization: writes take a per-DID lane before loading and mutating the repo, then use the DB mutex for the shared SQLite connection. Tranquil remains the reference shape for bounded write queues, explicit backpressure, less full-repo state loading, and a narrower SQLite persistence phase.

comparison work#

Tranquil benchmark source:

• crates/tranquil-store/benches/metastore.rs
• crates/tranquil-store/benches/metastore_scale.rs
• crates/tranquil-store/benches/blockstore.rs
• crates/tranquil-store/benches/eventlog.rs

Official PDS source for matching probes:

• packages/pds/src/actor-store/record/reader.ts
• packages/pds/src/api/com/atproto/repo/applyWrites.ts
• packages/pds/src/actor-store/repo/sql-repo-reader.ts

Open apples-to-apples comparison gaps:

• single-repo write curve as record count grows
• multi-repo concurrent writes
• record-index list latency without record body materialization, across ZDS and Tranquil
• official-PDS direct concurrency harness for the same read/write boundaries
• full repo export cost
• blob/block store throughput

ZDS and Tranquil both report latency percentiles for the metastore-shaped write and CID lookup rows. The remaining comparison gap is broader coverage of blockstore, eventlog, full repo export behavior, an official-PDS benchmark harness, and an explicit ZDS index-only list probe that matches Tranquil's list_records.