+69 -8

bench/README.md

Reviewed

··· 23 23 just bench render-record 10 1000 24 24 just bench get-record 10 1000 25 25 just bench list-records 10 1000 26 26 + just bench repo-size 5000 27 27 + just bench repo-large 100000 28 28 + just bench space 1000 26 29 just bench http 27 30 just bench official-pds 28 31 just bench write-profile 10 500 ··· 47 50 - `render-record`: DAG-CBOR decode plus JSON rendering, without SQLite. 48 51 - `get-record`: full `com.atproto.repo.getRecord` storage materialization. 49 52 - `list-records`: full `com.atproto.repo.listRecords` storage materialization. 53 53 + - `repo-size`: synthetic small/medium/large repo probes for 54 54 + `listRecords(limit=100)` and full repo CAR materialization. The default Just 55 55 + target keeps this moderate; use `repo-large` for intentional 100k+ record 56 56 + runs. 57 57 + - `space`: experimental permissioned-data storage probes for space discovery, 58 58 + private record writes, private record reads/lists, repo oplog catch-up, and 59 59 + blob readback. 50 60 - `http`: starts a local ReleaseFast ZDS server and measures a seeded HTTP 51 61 route matrix with ApacheBench. This is the route-level check for the `httpz` 52 62 server boundary and should be tracked separately from storage ··· 64 74 access patterns introduced by the experimental `com.atproto.space.*` storage 65 75 model. 66 76 67 67 - When adding the first permissioned-data probe, keep it out of `all` until the 68 68 - protocol shape settles. A useful first pass should seed one self-owned space, 69 69 - one writer repo, and one small blob, then measure these paths as distinct units 70 70 - of work: 77 77 + `just bench space` seeds one self-owned space, one writer repo, and one small 78 78 + blob, then measures these paths as distinct units of work: 71 79 72 72 - - `createSpace` 73 73 - - `createRecord` or `applyWrites` into a space writer repo 80 80 + - `listSpaces` 81 81 + - `createRecord` into a space writer repo 74 82 - `getRecord` by `(space, repo, collection, rkey)` 75 83 - `listRecords`, limit 50, index-only response shape 76 76 - - ranged `getBlob` 84 84 + - `getBlob` storage readback 77 85 - `getRepoOplog` catch-up reads 78 86 79 87 Comparison against Daniel's permissioned-data branch should live in this 80 88 section once we have a repeatable way to run that PDS locally. Until then, keep 81 89 ZDS numbers as local trend data and compare only equivalent operations. 82 90 91 91 + ## repo sizes 92 92 + 93 93 + The normal write/read benchmarks use small synthetic repos so they stay fast 94 94 + enough to run while developing. `just bench repo-size` adds explicit 95 95 + small/medium/large tiers for repo-size-sensitive paths: 96 96 + 97 97 + - small: 100 records 98 98 + - medium: 1,000 records, unless the selected maximum is lower 99 99 + - large: the `max_records` argument 100 100 + 101 101 + Use `just bench repo-large 100000` for a deliberate larger run. Large public 102 102 + accounts are useful reference shapes because they stress repo export, record 103 103 + indexes, and cursor/list behavior. As of June 15, 2026, the public appview 104 104 + reports about 38k posts for `pfrazee.com` and about 87k posts for 105 105 + `jcsalterego.bsky.social`, so a 100k synthetic tier is the right order of 106 106 + magnitude for large-repo stress. Live network corpora should be captured into 107 107 + fixtures before they become stable benchmarks. Do not make routine CI or 108 108 + pre-commit checks depend on live PDS availability. 109 109 + 83 110 ## methodology 84 111 85 112 - Build ZDS with `-Doptimize=ReleaseFast`. ··· 152 179 | `getSession` | 19.9k req/s, p95 8 ms | 20.8k req/s, p95 6 ms | 153 180 | `listRecords`, limit 50 | 19.8k req/s, p95 11 ms | 22.4k req/s, p95 7 ms | 154 181 155 155 - ## reference matrix 182 182 + ## coverage map 183 183 + 184 184 + High-level ZDS surface area represented in `bench/`: 185 185 + 186 186 + | PDS surface | representative benchmark | 187 187 + |---|---| 188 188 + | repo writes | `write`, `metastore`, `write-profile` | 189 189 + | repo reads | `get-cid`, `get-block`, `get-record`, `list-records` | 190 190 + | repo export/sync snapshots | `repo`, `repo-size` | 191 191 + | sync event storage | `metastore` write path plus smoke tests | 192 192 + | blob storage | `blob`, `space` blob readback | 193 193 + | HTTP routing/serialization/auth overhead | `http` | 194 194 + | OAuth/session/account routes | `http` route matrix | 195 195 + | permissioned spaces | `space` | 196 196 + | official PDS read-path comparison | `official-pds` | 197 197 + 198 198 + Still intentionally outside the routine benchmark gate: 199 199 + 200 200 + - passkey ceremony cryptography and browser UX 201 201 + - mail delivery providers 202 202 + - live appview proxy latency 203 203 + - WebSocket subscriber fanout under network backpressure 204 204 + - live public-repo corpora until captured as stable fixtures 205 205 + 206 206 + ## implementation comparison 207 207 + 208 208 + The main PDS implementations do not expose identical benchmark seams, so keep 209 209 + comparisons grouped by equivalent work: 210 210 + 211 211 + | implementation | comparable bench surface | notes | 212 212 + |---|---|---| 213 213 + | ZDS | `bench/main.zig`, `tools/http_bench.sh` | local SQLite/blobstore, storage microbenchmarks, HTTP route matrix, permissioned-space storage | 214 214 + | Tranquil | `crates/tranquil-store/benches/metastore*.rs`, `blockstore.rs`, `eventlog.rs` | best match for metastore-shaped record/index/event/blockstore comparisons | 215 215 + | official Bluesky PDS | `bench/official-pds-records.bench.ts` copied into `packages/pds` | best match for actor-store record CID/get/list/write paths | 216 216 + | Pegasus | `pegasus/bench/bench_repository.ml`, `mist/bench/bench_mst.ml` | best match for repository block/MST internals; not a full HTTP/PDS route comparison yet | 156 217 157 218 Run Tranquil's metastore benchmark: 158 219

+12

bench/justfile

Reviewed

··· 57 57 list-records callers="10" ops_per_caller="1000": 58 58 {{zig}} build bench -Doptimize=ReleaseFast -- --scenario list-records --callers {{callers}} --ops-per-caller {{ops_per_caller}} 59 59 60 60 + # benchmark listRecords and repo CAR materialization across synthetic repo sizes 61 61 + repo-size max_records="5000": 62 62 + {{zig}} build bench -Doptimize=ReleaseFast -- --scenario repo-size --records {{max_records}} 63 63 + 64 64 + # deliberately slow large-repo lane for Paul/Jerry-shaped synthetic repositories 65 65 + repo-large records="100000": 66 66 + {{zig}} build bench -Doptimize=ReleaseFast -- --scenario repo-size --records {{records}} 67 67 + 68 68 + # benchmark experimental permissioned-space storage paths 69 69 + space records="1000": 70 70 + {{zig}} build bench -Doptimize=ReleaseFast -- --scenario space --records {{records}} 71 71 + 60 72 # run the official Bluesky PDS read probe from a local atproto checkout 61 73 official-pds: 62 74 ./run-official-pds.sh

+230 -2

bench/main.zig

Reviewed

··· 14 14 render_record, 15 15 get_record, 16 16 list_records, 17 17 + repo_size, 18 18 + space, 17 19 write_profile, 18 20 }; 19 21 20 22 const bench_collection = "dev.zds.bench.record"; 23 23 + const space_collection = "fm.plyr.stg.track"; 24 24 + const space_type = "fm.plyr.stg.privateMedia"; 25 25 + const valid_bench_did = "did:plc:zdsbenchzdsbenchzdsbenchzd"; 21 26 22 27 const Options = struct { 23 28 scenario: Scenario = .all, ··· 117 122 .render_record => try benchRenderRecord(allocator, options), 118 123 .get_record => try benchGetRecord(allocator, options), 119 124 .list_records => try benchListRecords(allocator, options), 125 125 + .repo_size => try benchRepoSizes(allocator, options.records), 126 126 + .space => try benchSpace(allocator, options), 120 127 .write_profile => try benchWriteProfile(allocator, options), 121 128 } 122 129 } ··· 138 145 "bench.test", 139 146 "bench@test.com", 140 147 "password", 141 141 - "did:plc:zdsbench", 148 148 + valid_bench_did, 142 149 true, 143 150 ); 144 151 ··· 467 474 result.print(); 468 475 } 469 476 477 477 + fn benchRepoSizes(allocator: std.mem.Allocator, max_records: usize) !void { 478 478 + const tiers = [_]struct { 479 479 + name: []const u8, 480 480 + records: usize, 481 481 + }{ 482 482 + .{ .name = "repo small", .records = 100 }, 483 483 + .{ .name = "repo medium", .records = @min(1000, max_records) }, 484 484 + .{ .name = "repo large", .records = max_records }, 485 485 + }; 486 486 + 487 487 + std.debug.print("\n=== zds repo-size benchmarks ===\n", .{}); 488 488 + var last_records: usize = 0; 489 489 + for (tiers) |tier| { 490 490 + if (tier.records == 0 or tier.records == last_records) continue; 491 491 + last_records = tier.records; 492 492 + std.debug.print("seeding {s} with {d} records\n", .{ tier.name, tier.records }); 493 493 + var state = try initBench(allocator); 494 494 + errdefer state.deinit(); 495 495 + try seedIndexedRecords(allocator, state.account, tier.records); 496 496 + (try benchListRecordsForRepo(allocator, state.account, tier.records)).print(); 497 497 + (try benchRepoCarForRepo(allocator, state.account, tier.records)).print(); 498 498 + state.deinit(); 499 499 + } 500 500 + } 501 501 + 502 502 + fn benchListRecordsForRepo( 503 503 + allocator: std.mem.Allocator, 504 504 + account: zds.auth.tokens.Account, 505 505 + records: usize, 506 506 + ) !BenchResult { 507 507 + var arena = std.heap.ArenaAllocator.init(allocator); 508 508 + defer arena.deinit(); 509 509 + const iterations: usize = if (records >= 100_000) 50 else 200; 510 510 + const start = nowNs(); 511 511 + for (0..iterations) |_| { 512 512 + _ = arena.reset(.retain_capacity); 513 513 + const found = try zds.storage.store.listRecords(arena.allocator(), account.did, bench_collection, 100); 514 514 + if (found.len == 0) return error.MissingRecords; 515 515 + } 516 516 + return .{ 517 517 + .name = "repo tier list", 518 518 + .ops = iterations, 519 519 + .elapsed_ns = nowNs() - start, 520 520 + }; 521 521 + } 522 522 + 523 523 + fn benchRepoCarForRepo( 524 524 + allocator: std.mem.Allocator, 525 525 + account: zds.auth.tokens.Account, 526 526 + records: usize, 527 527 + ) !BenchResult { 528 528 + var arena = std.heap.ArenaAllocator.init(allocator); 529 529 + defer arena.deinit(); 530 530 + const iterations: usize = if (records >= 100_000) 2 else 5; 531 531 + var total_bytes: usize = 0; 532 532 + const start = nowNs(); 533 533 + for (0..iterations) |_| { 534 534 + _ = arena.reset(.retain_capacity); 535 535 + const car = try zds.storage.store.writeRepoCar(arena.allocator(), account.did); 536 536 + if (car.len == 0) return error.EmptyRepoCar; 537 537 + total_bytes += car.len; 538 538 + } 539 539 + return .{ 540 540 + .name = "repo tier CAR", 541 541 + .ops = iterations, 542 542 + .bytes = total_bytes, 543 543 + .elapsed_ns = nowNs() - start, 544 544 + }; 545 545 + } 546 546 + 547 547 + fn benchSpace(allocator: std.mem.Allocator, options: Options) !void { 548 548 + var state = try initBench(allocator); 549 549 + defer state.deinit(); 550 550 + 551 551 + var arena = std.heap.ArenaAllocator.init(allocator); 552 552 + defer arena.deinit(); 553 553 + const a = arena.allocator(); 554 554 + const records = options.records; 555 555 + const space = try seedSpaceRecords(a, state.account, records); 556 556 + const cid = try zds.storage.store.putBlob(a, std.Options.debug_io, state.account, "hello permissioned audio", "audio/mpeg"); 557 557 + 558 558 + std.debug.print("\n=== zds permissioned-space benchmarks ===\n", .{}); 559 559 + (try benchSpaceListSpaces(allocator, state.account, space.uri)).print(); 560 560 + (try benchSpaceWrite(allocator, state.account, space.uri, records)).print(); 561 561 + (try benchSpaceGetRecord(allocator, state.account, space.uri, records)).print(); 562 562 + (try benchSpaceListRecords(allocator, state.account, space.uri, records)).print(); 563 563 + (try benchSpaceOplog(allocator, state.account, space.uri)).print(); 564 564 + (try benchSpaceBlob(allocator, state.account, cid)).print(); 565 565 + } 566 566 + 567 567 + fn seedSpaceRecords( 568 568 + allocator: std.mem.Allocator, 569 569 + account: zds.auth.tokens.Account, 570 570 + records: usize, 571 571 + ) !zds.storage.store.SpaceConfig { 572 572 + const space = try zds.storage.store.createSpace(allocator, .{ 573 573 + .actor_did = account.did, 574 574 + .owner_did = account.did, 575 575 + .space_type = space_type, 576 576 + .skey = "self", 577 577 + .is_owner = true, 578 578 + .managing_app = "https://api-stg.plyr.fm", 579 579 + .is_public = false, 580 580 + .app_access_mode = "allow", 581 581 + .app_exceptions_json = "[]", 582 582 + }); 583 583 + for (0..records) |i| { 584 584 + const value = try spaceRecordValue(allocator, i); 585 585 + const prepared = try zds.storage.store.prepareRecordValue( 586 586 + allocator, 587 587 + space_collection, 588 588 + try std.fmt.allocPrint(allocator, "track{d:0>8}", .{i}), 589 589 + value, 590 590 + ); 591 591 + _ = try zds.storage.store.createSpaceRecord( 592 592 + allocator, 593 593 + space.uri, 594 594 + account.did, 595 595 + space_collection, 596 596 + try std.fmt.allocPrint(allocator, "track{d:0>8}", .{i}), 597 597 + prepared, 598 598 + ); 599 599 + } 600 600 + return space; 601 601 + } 602 602 + 603 603 + fn benchSpaceListSpaces(allocator: std.mem.Allocator, account: zds.auth.tokens.Account, space: []const u8) !BenchResult { 604 604 + var arena = std.heap.ArenaAllocator.init(allocator); 605 605 + defer arena.deinit(); 606 606 + const iterations: usize = 500; 607 607 + const start = nowNs(); 608 608 + for (0..iterations) |_| { 609 609 + _ = arena.reset(.retain_capacity); 610 610 + const spaces = try zds.storage.store.listSpaces(arena.allocator(), account.did, account.did, space_type, null, 50); 611 611 + if (spaces.len == 0 or !std.mem.eql(u8, spaces[0].uri, space)) return error.MissingSpace; 612 612 + } 613 613 + return .{ .name = "space listSpaces", .ops = iterations, .elapsed_ns = nowNs() - start }; 614 614 + } 615 615 + 616 616 + fn benchSpaceWrite(allocator: std.mem.Allocator, account: zds.auth.tokens.Account, space: []const u8, offset: usize) !BenchResult { 617 617 + var arena = std.heap.ArenaAllocator.init(allocator); 618 618 + defer arena.deinit(); 619 619 + const iterations: usize = 100; 620 620 + const start = nowNs(); 621 621 + for (0..iterations) |i| { 622 622 + _ = arena.reset(.retain_capacity); 623 623 + const idx = offset + i; 624 624 + const rkey = try std.fmt.allocPrint(arena.allocator(), "track{d:0>8}", .{idx}); 625 625 + const value = try spaceRecordValue(arena.allocator(), idx); 626 626 + const prepared = try zds.storage.store.prepareRecordValue(arena.allocator(), space_collection, rkey, value); 627 627 + const result = try zds.storage.store.createSpaceRecord(arena.allocator(), space, account.did, space_collection, rkey, prepared); 628 628 + if (result.cid.len == 0) return error.UnexpectedWriteResult; 629 629 + } 630 630 + return .{ .name = "space createRecord", .ops = iterations, .elapsed_ns = nowNs() - start }; 631 631 + } 632 632 + 633 633 + fn benchSpaceGetRecord(allocator: std.mem.Allocator, account: zds.auth.tokens.Account, space: []const u8, records: usize) !BenchResult { 634 634 + var arena = std.heap.ArenaAllocator.init(allocator); 635 635 + defer arena.deinit(); 636 636 + const iterations: usize = 1000; 637 637 + const start = nowNs(); 638 638 + for (0..iterations) |i| { 639 639 + _ = arena.reset(.retain_capacity); 640 640 + const idx = if (records == 0) 0 else (i * 13) % records; 641 641 + const rkey = try std.fmt.allocPrint(arena.allocator(), "track{d:0>8}", .{idx}); 642 642 + const record = (try zds.storage.store.getSpaceRecord(arena.allocator(), space, account.did, space_collection, rkey)) orelse return error.MissingRecord; 643 643 + if (record.value_json.len == 0) return error.MissingRecord; 644 644 + } 645 645 + return .{ .name = "space getRecord", .ops = iterations, .elapsed_ns = nowNs() - start }; 646 646 + } 647 647 + 648 648 + fn benchSpaceListRecords(allocator: std.mem.Allocator, account: zds.auth.tokens.Account, space: []const u8, records: usize) !BenchResult { 649 649 + var arena = std.heap.ArenaAllocator.init(allocator); 650 650 + defer arena.deinit(); 651 651 + const iterations: usize = 500; 652 652 + const start = nowNs(); 653 653 + for (0..iterations) |_| { 654 654 + _ = arena.reset(.retain_capacity); 655 655 + const found = try zds.storage.store.listSpaceRecords(arena.allocator(), space, account.did, space_collection, null, false, 50); 656 656 + if (records > 0 and found.len == 0) return error.MissingRecords; 657 657 + } 658 658 + return .{ .name = "space listRecords", .ops = iterations, .elapsed_ns = nowNs() - start }; 659 659 + } 660 660 + 661 661 + fn benchSpaceOplog(allocator: std.mem.Allocator, account: zds.auth.tokens.Account, space: []const u8) !BenchResult { 662 662 + var arena = std.heap.ArenaAllocator.init(allocator); 663 663 + defer arena.deinit(); 664 664 + const iterations: usize = 500; 665 665 + const start = nowNs(); 666 666 + for (0..iterations) |_| { 667 667 + _ = arena.reset(.retain_capacity); 668 668 + const ops = try zds.storage.store.listSpaceRecordOplog(arena.allocator(), space, account.did, null, 100); 669 669 + if (ops.len == 0) return error.MissingRecords; 670 670 + } 671 671 + return .{ .name = "space getRepoOplog", .ops = iterations, .elapsed_ns = nowNs() - start }; 672 672 + } 673 673 + 674 674 + fn benchSpaceBlob(allocator: std.mem.Allocator, account: zds.auth.tokens.Account, cid: []const u8) !BenchResult { 675 675 + var arena = std.heap.ArenaAllocator.init(allocator); 676 676 + defer arena.deinit(); 677 677 + const iterations: usize = 1000; 678 678 + const start = nowNs(); 679 679 + for (0..iterations) |_| { 680 680 + _ = arena.reset(.retain_capacity); 681 681 + const blob = zds.storage.store.getBlob(arena.allocator(), account.did, cid) orelse return error.MissingBlob; 682 682 + if (blob.data.len == 0) return error.MissingBlob; 683 683 + } 684 684 + return .{ .name = "space getBlob", .ops = iterations, .bytes = iterations * "hello permissioned audio".len, .elapsed_ns = nowNs() - start }; 685 685 + } 686 686 + 470 687 fn benchDecodeRecord(allocator: std.mem.Allocator, options: Options) !void { 471 688 try benchRecordBlockCpu(allocator, options, .decode); 472 689 } ··· 748 965 return try std.json.parseFromSliceLeaky(std.json.Value, allocator, json, .{}); 749 966 } 750 967 968 968 + fn spaceRecordValue(allocator: std.mem.Allocator, index: usize) !std.json.Value { 969 969 + const json = try std.fmt.allocPrint( 970 970 + allocator, 971 971 + "{{\"$type\":\"{s}\",\"title\":\"private track {d}\",\"createdAt\":\"2026-06-15T00:{d:0>2}:00.000Z\",\"audioRef\":\"bench-audio-{d}\",\"size\":{d}}}", 972 972 + .{ space_collection, index, index % 60, index, 1024 + index }, 973 973 + ); 974 974 + return try std.json.parseFromSliceLeaky(std.json.Value, allocator, json, .{}); 975 975 + } 976 976 + 751 977 fn parseOptions(init: std.process.Init) !Options { 752 978 var options: Options = .{}; 753 979 var args = std.process.Args.Iterator.init(init.minimal.args); ··· 800 1026 if (std.mem.eql(u8, value, "render-record")) return .render_record; 801 1027 if (std.mem.eql(u8, value, "get-record")) return .get_record; 802 1028 if (std.mem.eql(u8, value, "list-records")) return .list_records; 1029 1029 + if (std.mem.eql(u8, value, "repo-size")) return .repo_size; 1030 1030 + if (std.mem.eql(u8, value, "space")) return .space; 803 1031 if (std.mem.eql(u8, value, "write-profile")) return .write_profile; 804 1032 return error.UnknownScenario; 805 1033 } 806 1034 807 1035 fn usage() void { 808 1036 std.debug.print( 809 809 - \\usage: zds-bench [--scenario all|write|read|blob|repo|metastore|get-cid|get-block|decode-record|render-record|get-record|list-records|write-profile] [--records N] [--blobs N] [--blob-size BYTES] [--callers N --ops-per-caller N] 1037 1037 + \\usage: zds-bench [--scenario all|write|read|blob|repo|metastore|get-cid|get-block|decode-record|render-record|get-record|list-records|repo-size|space|write-profile] [--records N] [--blobs N] [--blob-size BYTES] [--callers N --ops-per-caller N] 810 1038 \\ 811 1039 , .{}); 812 1040 }