atproto pds in zig
pds.zat.dev
pds
atproto
Clarify benchmark comparison matrix
+86
-34
+27
-19
bench/README.md
+27
-19
bench/README.md
···
113
113
This matrix only includes operation/count pairs measured with the same unit of
114
114
work. Tranquil's `get_record_cid` and `list_records` are record-index
115
115
operations: `list_records` returns `rkey + record_cid`, not record body JSON.
116
-
The official PDS probe measures its actor-store read paths with Vitest. That is
117
-
apples-to-apples for the internal read operations, but not for concurrency
118
-
curves: Vitest reports a single async benchmark loop, while ZDS and Tranquil
119
-
also have explicit caller-count runs.
116
+
The official PDS probe measures its actor-store read paths with Vitest. Its
117
+
single-operation rows are apples-to-apples for the internal read boundaries. Its
118
+
batched CID rows are useful throughput shape checks, but their p99 values are
119
+
batch latencies, not per-operation latencies from the ZDS/Tranquil harness.
120
120
121
-
Summary: ZDS is fastest in the single-caller read probes, Tranquil is fastest in
122
-
the 10/100 caller CID lookup probes, and the official PDS is slower in these
121
+
Summary: ZDS is fastest in the single-caller read probes. Tranquil is fastest in
122
+
the direct 10/100 caller CID lookup probes. The official PDS is slower in these
123
123
microbenchmarks because the probe goes through its actor-store/test-network
124
-
stack rather than a bare SQLite reader. For writes, official PDS is faster than
125
-
ZDS in the single-caller row, Tranquil is faster than ZDS at 10 callers, and ZDS
126
-
is faster than Tranquil at 100 callers but with a worse p99/max tail.
124
+
stack rather than a bare SQLite reader. For writes, ZDS is fastest in the
125
+
single-caller row, Tranquil is faster than ZDS at 10 callers, and ZDS is faster
126
+
than Tranquil at 100 callers but with a worse p99/max tail.
127
127
128
128
| operation | callers | zds | tranquil | official PDS |
129
129
|---|---:|---:|---:|---:|
130
-
| apply one record commit | 1 | 642 ops/s, p95 2.9 ms | 287 ops/s, p95 4.1 ms | 486 ops/s, p99 3.9 ms |
131
-
| current record CID lookup | 1 | 414k ops/s, p95 4 us | 96.8k ops/s, p95 15 us | 3.2k ops/s, p99 477 us |
130
+
| apply one record commit | 1 | 642 ops/s, p95 2.9 ms | 287 ops/s, p95 4.1 ms | 448 ops/s, p99 4.0 ms |
131
+
| current record CID lookup | 1 | 414k ops/s, p95 4 us | 96.8k ops/s, p95 15 us | 3.1k ops/s, p99 511 us |
132
132
133
-
The current caller-count concurrency matrix is only ZDS and Tranquil until the
134
-
official-PDS probe grows explicit concurrent workers.
133
+
The direct caller-count concurrency matrix is only ZDS and Tranquil. The
134
+
official-PDS rows below it use a batched Vitest probe and are kept separate.
135
135
136
136
| operation | callers | ops | zds | tranquil |
137
137
|---|---:|---:|---:|---:|
···
139
139
| apply one record commit | 100 | 20000 | 1223 ops/s, p95 67.5 ms | 1007 ops/s, p95 109 ms |
140
140
| current record CID lookup | 10 | 10000 | 222k ops/s, p95 225 us | 455k ops/s, p95 43 us |
141
141
| current record CID lookup | 100 | 20000 | 211k ops/s, p95 887 us | 537k ops/s, p95 233 us |
142
+
143
+
Official PDS batched CID lookup:
144
+
145
+
| operation | batch shape | converted throughput | batch p99 |
146
+
|---|---|---:|---:|
147
+
| current record CID lookup | 10 callers x 100 lookups | 3.3k lookups/s | 317 ms |
148
+
| current record CID lookup | 100 callers x 20 lookups | 3.2k lookups/s | 630 ms |
142
149
143
150
Tranquil's metastore `list_records` benchmark is not in this table because ZDS
144
151
does not currently expose an equivalent "list rkeys and CIDs only" benchmark.
···
159
166
160
167
| operation | official PDS |
161
168
|---|---:|
162
-
| apply one record commit | 486 ops/s, p99 3.9 ms |
163
-
| current record CID lookup | 3.2k ops/s, p99 477 us |
164
-
| full getRecord materialization | 3.1k ops/s, p99 469 us |
165
-
| full listRecords, limit 50 | 639 ops/s, p99 2.3 ms |
169
+
| apply one record commit | 448 ops/s, p99 4.0 ms |
170
+
| current record CID lookup | 3.1k ops/s, p99 511 us |
171
+
| full getRecord materialization | 3.0k ops/s, p99 525 us |
172
+
| full listRecords, limit 50 | 635 ops/s, p99 2.4 ms |
166
173
167
174
These are useful for the official-PDS column, but they should not be mixed with
168
-
ZDS's caller-count rows. The matching ZDS single-caller rows are:
175
+
ZDS/Tranquil direct caller-count latency rows. The matching ZDS single-caller
176
+
rows are:
169
177
170
178
| operation | zds |
171
179
|---|---:|
···
232
240
- multi-repo concurrent writes
233
241
- record-index list latency without record body materialization, across ZDS and
234
242
Tranquil
235
-
- official-PDS concurrency probe for the same read/write boundaries
243
+
- official-PDS direct concurrency harness for the same read/write boundaries
236
244
- full repo export cost
237
245
- blob/block store throughput
238
246
+1
-1
bench/justfile
+1
-1
bench/justfile
+43
-7
bench/official-pds-records.bench.ts
+43
-7
bench/official-pds-records.bench.ts
···
3
3
import { TestNetworkNoAppView } from '@atproto/dev-env'
4
4
import { prepareCreate } from './src/repo/index.js'
5
5
6
+
type Op = () => Promise<void>
7
+
6
8
const records: { uri: string; cid: string; rkey: string }[] = []
7
9
let network: TestNetworkNoAppView
8
10
let did: string
···
43
45
})
44
46
45
47
bench('current record CID lookup', async () => {
48
+
await currentCidLookup()
49
+
})
50
+
51
+
bench('apply one record commit', async () => {
52
+
await applyOneRecordCommit()
53
+
})
54
+
55
+
bench('full getRecord materialization', async () => {
56
+
await fullGetRecord()
57
+
})
58
+
59
+
bench('full listRecords materialization, limit 50', async () => {
60
+
await fullListRecords()
61
+
})
62
+
63
+
bench('current record CID lookup batch, 10 callers x 100 ops', async () => {
64
+
await runConcurrent(10, 100, currentCidLookup)
65
+
})
66
+
67
+
bench('current record CID lookup batch, 100 callers x 20 ops', async () => {
68
+
await runConcurrent(100, 20, currentCidLookup)
69
+
})
70
+
71
+
async function currentCidLookup() {
46
72
const rec = records[Math.floor(Math.random() * records.length)]
47
73
const cid = await network.pds.ctx.actorStore.read(did, (store) =>
48
74
store.record.getCurrentRecordCid(new AtUri(rec.uri)),
49
75
)
50
76
if (!cid) throw new Error('missing cid')
51
-
})
77
+
}
52
78
53
-
bench('apply one record commit', async () => {
79
+
async function applyOneRecordCommit() {
54
80
const i = writeIdx++
55
81
const write = await prepareCreate({
56
82
did,
···
68
94
return next
69
95
})
70
96
await network.pds.ctx.accountManager.updateRepoRoot(did, commit.cid, commit.rev)
71
-
})
97
+
}
72
98
73
-
bench('full getRecord materialization', async () => {
99
+
async function fullGetRecord() {
74
100
const rec = records[Math.floor(Math.random() * records.length)]
75
101
const record = await network.pds.ctx.actorStore.read(did, (store) =>
76
102
store.record.getRecord(new AtUri(rec.uri), rec.cid),
77
103
)
78
104
if (!record) throw new Error('missing record')
79
-
})
105
+
}
80
106
81
-
bench('full listRecords materialization, limit 50', async () => {
107
+
async function fullListRecords() {
82
108
const rows = await network.pds.ctx.actorStore.read(did, (store) =>
83
109
store.record.listRecordsForCollection({
84
110
collection: 'app.bsky.feed.post',
···
87
113
}),
88
114
)
89
115
if (rows.length === 0) throw new Error('missing records')
90
-
})
116
+
}
117
+
118
+
async function runConcurrent(callers: number, opsPerCaller: number, op: Op) {
119
+
await Promise.all(
120
+
Array.from({ length: callers }, async () => {
121
+
for (let i = 0; i < opsPerCaller; i++) {
122
+
await op()
123
+
}
124
+
}),
125
+
)
126
+
}
+15
-7
docs/benchmarking.md
+15
-7
docs/benchmarking.md
···
24
24
- sync event delivery
25
25
26
26
ZDS, Tranquil, and the official PDS now have matching single-caller numbers for
27
-
apply-one-record and current-CID lookup. ZDS and Tranquil also have explicit
28
-
10/100 caller concurrency curves; the official probe should grow those workers
29
-
before it is included in the concurrency matrix.
27
+
apply-one-record and current-CID lookup. ZDS and Tranquil also have direct
28
+
10/100 caller concurrency curves. The official probe has a batched CID lookup
29
+
shape check, but its latency values are batch latencies from Vitest, not per-op
30
+
latencies from the same harness as the ZDS and Tranquil rows.
30
31
31
32
## read path
32
33
···
53
54
54
55
| operation | result |
55
56
|---|---:|
56
-
| apply one record commit | 486 ops/s, p99 3.9 ms |
57
-
| current CID lookup | 3.2k ops/s, p99 477 us |
58
-
| full getRecord | 3.1k ops/s, p99 469 us |
59
-
| full listRecords, limit 50 | 639 ops/s, p99 2.3 ms |
57
+
| apply one record commit | 448 ops/s, p99 4.0 ms |
58
+
| current CID lookup | 3.1k ops/s, p99 511 us |
59
+
| full getRecord | 3.0k ops/s, p99 525 us |
60
+
| full listRecords, limit 50 | 635 ops/s, p99 2.4 ms |
61
+
62
+
The official PDS batched CID lookup probe reports:
63
+
64
+
| batch shape | converted throughput | batch p99 |
65
+
|---|---:|---:|
66
+
| 10 callers x 100 lookups | 3.3k lookups/s | 317 ms |
67
+
| 100 callers x 20 lookups | 3.2k lookups/s | 630 ms |
60
68
61
69
For tiny records, DAG-CBOR decode and JSON rendering are below useful timer
62
70
resolution. The important next read benchmark is larger records plus an