+3

AGENTS.md

Reviewed

··· 72 72 zig zen 73 73 ``` 74 74 75 75 + Also run `just smoke-permissioned` when touching `com.atproto.space.*`, 76 76 + permissioned-data storage, or the `/spaces` resident view. 77 77 + 75 78 Use `just invite` for admin-minted invite codes. Assume `ZDS_ADMIN_TOKEN` is 76 79 set in the repo `.env`; source it before running the target, for example 77 80 `set -a; . ./.env; set +a; just invite`. Use `just bench ...` for local

-68

PLYR_OAUTH_PAR_HANDOFF.md

Reviewed

··· 1 1 - # ZDS `/oauth/par` rejecting plyr.fm's permission-set includes → login broken on pds.zat.dev 2 2 - 3 3 - **From:** plyr.fm side investigation (2026-06-09 ~01:20 UTC) 4 4 - **Symptom:** signing into plyr.fm staging as a `pds.zat.dev` account (e.g. `bufo.uk`, 5 5 - `did:plc:b64lsctzqnzpv6vd4ry3qktw`) fails. Accounts on *other* PDSes (e.g. 6 6 - `zzstoatzz.io` on `pds.zzstoatzz.io`) sign in fine. So this is specific to ZDS, 7 7 - and it is **new** — `bufo.uk` signed in successfully at 23:34 UTC, then broke. 8 8 - Something changed ZDS-side since then (a redeploy?). 9 9 - 10 10 - ## Where it fails 11 11 - 12 12 - plyr's OAuth client (`atproto_oauth`) dies at the **PAR step** of 13 13 - `start_authorization`, *after* successfully fetching ZDS's `.well-known` 14 14 - metadata (both 200) and *before* a usable PAR response. On plyr's side the 15 15 - exception arrived with an empty `str()` (we just shipped a fix to log the real 16 16 - type/traceback — next failed login will show it precisely in our logfire). 17 17 - 18 18 - ## The ZDS-side lead (your logs) 19 19 - 20 20 - `fly logs -a zds-pds` shows, for plyr's client: 21 21 - 22 22 - ``` 23 23 - error oauth validate include failed scope=include:fm.plyr.stg.privateMedia err=InvalidPermissionSet 24 24 - error oauth authorize scope_html failed client=https://api-stg.plyr.fm/oauth-client-metadata.json \ 25 25 - scope=atproto blob:*/* include:fm.plyr.stg.authFullApp include:fm.plyr.stg.privateMedia \ 26 26 - err=InvalidPermissionSet 27 27 - ``` 28 28 - 29 29 - So ZDS is rejecting plyr's permission-set `include:` scopes with 30 30 - **`InvalidPermissionSet`**. plyr's *current* (post-revert) login requests only: 31 31 - 32 32 - ``` 33 33 - atproto blob:*/* include:fm.plyr.stg.authFullApp 34 34 - ``` 35 35 - 36 36 - If ZDS now fails to resolve **`include:fm.plyr.stg.authFullApp`** the same way it 37 37 - failed `privateMedia`, PAR aborts and login dies. 38 38 - 39 39 - ## What to check 40 40 - 41 41 - 1. Resolve `include:fm.plyr.stg.authFullApp` (and `include:fm.plyr.stg.privateMedia`) 42 42 - from ZDS right now — why `InvalidPermissionSet`? The permission-set records are 43 43 - published as `com.atproto.lexicon.schema` on the **plyr.fm** account 44 44 - `did:plc:vs3hnzq2daqbszxlysywzy54` (PDS `chalciporus.us-west.host.bsky.network`), 45 45 - resolved via DNS `_lexicon.stg.plyr.fm` TXT → that DID. Confirm ZDS can still 46 46 - fetch + parse those records (DNS, DID resolution, getRecord, `buildExpandedScope`). 47 47 - This worked earlier today, so suspect a regression in the last ZDS deploy. 48 48 - 49 49 - 2. `/oauth/par` returns **HTTP 500** on bad input rather than a clean 4xx OAuth 50 50 - error. Example (my probe, missing client assertion): 51 51 - `error oauth client_auth rejected missing_assertion_type` → `failed to serve 52 52 - /oauth/par: MissingFormField` → 500. A 500 (especially with an empty/aborted 53 53 - body) surfaces on the client as an opaque `httpx.RemoteProtocolError`. Returning 54 54 - a proper `{"error": "...", "error_description": "..."}` 4xx would make failures 55 55 - debuggable on both sides. 56 56 - 57 57 - ## Please ignore in the logs 58 58 - 59 59 - My manual `/oauth/par` probes around **01:19–01:20 UTC** (`client=test`, and 60 60 - `client=...plyr...` *without* a client assertion) are mine, not plyr's — they 61 61 - lack the confidential-client assertion and are not representative. plyr always 62 62 - sends `client_assertion` (private_key_jwt). 63 63 - 64 64 - ## To reproduce faithfully 65 65 - 66 66 - Have a `pds.zat.dev` account sign into `https://stg.plyr.fm` and watch the 67 67 - `/oauth/par` handler + permission-set resolution for 68 68 - `client=https://api-stg.plyr.fm/oauth-client-metadata.json`.

+4

README.md

Reviewed

··· 37 37 - [operations](docs/operations.md) 38 38 - [invite codes](docs/invite-codes.md) 39 39 - [passkeys](docs/passkeys.md) 40 40 + - [permissioned data](docs/permissioned-data.md) 40 41 - [references](docs/references.md) 41 42 - [benchmarks](bench/README.md) 42 43 ··· 128 129 credential IDs, public keys, counters, names, and last-use timestamps. 129 130 - `/security` is the local account-security page for passkey and app-password 130 131 management. 132 132 + - `com.atproto.space.*` permissioned-data routes are experimental and operator 133 133 + gated with `ZDS_PERMISSIONED_DATA`. ZDS implements the storage and credential 134 134 + substrate, while reader/group semantics are left to applications. 131 135 132 136 ## references 133 137

+6 -6

docs/benchmarking.md

Reviewed

··· 99 99 100 100 Permissioned-data measurements should stay separate from the existing repo, 101 101 blob, sync, and HTTP route matrices. The experimental `com.atproto.space.*` 102 102 - surface has its own storage model, actor-local space state, LtHash-backed 103 103 - member/record state, oplogs, credentials, and ranged blob reads, so mixing it 104 104 - into the public repo tables would hide the actual access pattern. 102 102 + surface has its own storage model, actor-local space state, writer repo state, 103 103 + record oplogs, credentials, and ranged blob reads, so mixing it into the public 104 104 + repo tables would hide the actual access pattern. 105 105 106 106 The first benchmark slice should be small and diagnostic rather than exhaustive: 107 107 108 108 - - seed one self-owned private space and one member repo. 109 109 - - measure `createSpace`, member add/remove, record write, record read, 110 110 - `listRecords`, ranged `getBlob`, and oplog catch-up separately. 108 108 + - seed one self-owned private space and one writer repo. 109 109 + - measure `createSpace`, record write, record read, `listRecords`, ranged 110 110 + `getBlob`, and writer repo oplog catch-up separately. 111 111 - report p50/p95/p99/max for the concurrent paths, but keep write, read, blob, 112 112 and oplog rows separate. 113 113 - compare to Daniel's permissioned-data branch only after we can run that PDS

+5 -3

docs/operations.md

Reviewed

··· 81 81 - `ZDS_PERMISSIONED_DATA`: set to `true` to opt into experimental 82 82 `com.atproto.space.*` permissioned-data routes. Default: disabled. The 83 83 current implementation is intended to track the upstream experimental 84 84 - permissioned-data branch, including space lifecycle, membership, private 85 85 - records, grants, credentials, oplogs, ranged blob reads, and notify routes. 86 86 - Use `just smoke-permissioned` for the experimental smoke lane. 84 84 + permissioned-data work while following the newer direction that removes 85 85 + protocol member lists. ZDS provides space lifecycle, private records, grants, 86 86 + credentials, writer repo state/oplogs, ranged blob reads, and notify routes. 87 87 + Reader/group semantics are application policy. Use `just smoke-permissioned` 88 88 + for the experimental smoke lane. 87 89 - Passkeys do not require a separate deployment secret. WebAuthn RP ID is 88 90 derived from the public URL host. 89 91

+81 -92

docs/permissioned-data.md

Reviewed

··· 1 1 - # permissioned data plan 1 1 + # permissioned data 2 2 3 3 - Permissioned data support is experimental. ZDS should ship it behind an 4 4 - operator opt-in flag, off by default, before any `com.atproto.space.*` route is 5 5 - publicly reachable. 6 6 - 7 7 - Planned flag: 3 3 + Permissioned data support is experimental and operator gated. ZDS only exposes 4 4 + `com.atproto.space.*` when explicitly enabled: 8 5 9 6 ```sh 10 7 ZDS_PERMISSIONED_DATA=true 11 8 ``` 12 9 13 13 - Until that flag is enabled, ZDS should reject permissioned-data routes as 14 14 - unimplemented and should not create or mutate permissioned-space rows. 15 15 - 16 16 - When enabled, ZDS should expose the full experimental 17 17 - `com.atproto.space.*` surface described by the current permissioned-data 18 18 - lexicons. Downstream apps should treat the namespace as experimental protocol 19 19 - support, not as a plyr.fm-specific private-media slice. 10 10 + When the flag is off, permissioned-data routes stay dark and should not create 11 11 + or mutate permissioned-space rows. When the flag is on, the API reference marks 12 12 + the space routes with `x-zds-experimental: true`. 20 13 21 14 ## references 22 15 23 16 - Discourse: <https://discourse.atprotocol.community/t/permissioned-data-pds-lexicons/879> 17 17 + - Member-list direction: 18 18 + <https://discourse.atprotocol.community/t/removing-the-member-list/895> 24 19 - Branch: <https://github.com/bluesky-social/atproto/tree/permissioned-data> 25 20 - Lexicons: 26 21 <https://github.com/bluesky-social/atproto/tree/permissioned-data/lexicons/com/atproto/space> 27 22 - Reference implementation path: 28 23 `packages/pds/src/actor-store/space` on the permissioned-data branch 29 24 30 30 - The Discourse post rates the lexicon shape higher than the implementation. Use 31 31 - the lexicons as the main contract, and use the branch implementation to 32 32 - understand storage and flow rather than copying it blindly. 25 25 + The upstream design is still moving. The May branch included protocol-level 26 26 + member lists. The later discussion is moving toward making space credentials 27 27 + the protocol substrate and leaving reader/group semantics to applications or 28 28 + space-host policy. ZDS follows that newer direction because this deployment has 29 29 + no external production users, so subtractive experimental changes are cheaper 30 30 + than carrying compatibility for a shape we no longer want. 33 31 34 34 - Feedback should respect the research already behind the sketch and avoid 35 35 - overfitting to ZDS's idiosyncratic needs. The protocol is trying to cover likely 36 36 - use cases from large shared spaces down to small contextual ones, while leaving 37 37 - substantial application-specific authorization and UX in user land. ZDS feedback 38 38 - should name concrete implementation pressure, but frame it as design input for 39 39 - that broader scope. 32 32 + Feedback should respect the research behind the sketch and avoid overfitting to 33 33 + ZDS or plyr.fm. Name concrete implementation pressure, but frame it as input for 34 34 + the broader protocol shape. 40 35 41 41 - ## surface 36 36 + ## current surface 42 37 43 43 - The branch adds these `com.atproto.space.*` methods: 38 38 + ZDS currently keeps the permissioned-data substrate: 44 39 45 40 - space lifecycle: `createSpace`, `getSpace`, `listSpaces`, 46 41 `updateSpaceConfig`, `deleteSpace` 47 47 - - membership: `addMember`, `removeMember`, `getMembers`, `getMemberState`, 48 48 - `getMemberOplog`, `getMemberGrant`, `notifyMembership` 49 49 - - records: `createRecord`, `putRecord`, `deleteRecord`, `applyWrites`, 50 50 - `getRecord`, `listRecords`, `getBlob`, `getRepoState`, `getRepoOplog`, 51 51 - `notifyWrite` 52 52 - - credentials and deletion fanout: `getSpaceCredential`, `notifySpaceDeleted` 42 42 + - records and blobs: `createRecord`, `putRecord`, `deleteRecord`, 43 43 + `applyWrites`, `getRecord`, `listRecords`, `getBlob` 44 44 + - writer state: `getRepoState`, `getRepoOplog`, `notifyWrite` 45 45 + - credentials and deletion fanout: `getMemberGrant`, `getSpaceCredential`, 46 46 + `notifySpaceDeleted` 47 47 + 48 48 + ZDS deliberately does not expose protocol member-list routes: 49 49 + 50 50 + - `addMember` 51 51 + - `removeMember` 52 52 + - `getMembers` 53 53 + - `getMemberState` 54 54 + - `getMemberOplog` 55 55 + - `notifyMembership` 56 56 + 57 57 + `getSpace` and `listSpaces` do not return `members` or `isMember`. 58 58 + 59 59 + ## access model 60 60 + 61 61 + ZDS treats permissioned data as a space credential and writer-repo substrate, 62 62 + not as a universal group-membership system. 63 63 + 64 64 + Applications own reader/group semantics: label rosters, supporter access, 65 65 + personal private libraries, community roles, follower-only access, and similar 66 66 + policy all belong above the PDS substrate. If an application needs portable 67 67 + access state, it should model that state as application records in a 68 68 + permissioned space rather than relying on a PDS-wide member list. 69 69 + 70 70 + For private spaces, ZDS currently mints space credentials only to the space 71 71 + owner DID unless the space is public. That is the conservative default until 72 72 + the protocol settles on an application or space-host policy hook for broader 73 73 + credential issuance. 53 74 54 54 - This is not a thin wrapper around public repo writes. It is a separate 55 55 - space-scoped data model with its own read perimeter, write notifications, 56 56 - member state, record state, oplogs, and signed set commitments. 75 75 + Writer notifications are keyed by writer repo. `notifyWrite` verifies service 76 76 + auth from the writer repo to the space DID, then fans out to registered 77 77 + credential recipients. 57 78 58 79 ## storage shape 59 80 60 60 - The reference branch stores these tables inside each actor store: 81 81 + ZDS stores many actor repos in one SQLite database, so permissioned data uses 82 82 + explicit space-scoped tables instead of the public repo tables: 61 83 62 62 - - `space`: URI, owner/member flags, app access policy, creation/deletion 63 63 - timestamps 64 64 - - `space_member`: member DIDs and member revision 65 65 - - `space_record`: current records by `(space, collection, rkey)` with CID, 66 66 - DAG-CBOR value, repo revision, and indexed timestamp 67 67 - - `space_repo`: record set hash and current repo revision 68 68 - - `space_member_state`: member set hash and current member revision 69 69 - - `space_record_oplog`: incremental record changes by `(space, rev, idx)` 70 70 - - `space_member_oplog`: incremental membership changes by `(space, rev, idx)` 71 71 - - `space_credential_recipient`: registered services to notify for writes 84 84 + - `permissioned_spaces`: canonical owner-space config keyed by space URI 85 85 + - `permissioned_space_actor_state`: actor-local owner/deleted state keyed by 86 86 + `(space, actor_did)` 87 87 + - `permissioned_space_records`: current records keyed by 88 88 + `(space, repo_did, collection, rkey)` with CID, DAG-CBOR value, repo revision, 89 89 + and indexed timestamp 90 90 + - `permissioned_space_repos`: writer repo state and current record-set hash 91 91 + - `permissioned_space_record_oplog`: incremental record changes by 92 92 + `(space, repo_did, rev, idx)` 93 93 + - `permissioned_space_credentials`: short-lived grants and credentials 94 94 + - `permissioned_space_credential_recipients`: services to notify for writes and 95 95 + space deletion 72 96 73 73 - ZDS has one public repo namespace per DID: `records`, `repo_blocks`, 74 74 - `commits`, `seq_events`, `blobs`, plus account/auth state. Permissioned data 75 75 - must not be squeezed into those public repo tables. It uses space-scoped 76 76 - storage in `src/storage/store.zig` and route behavior in 77 77 - `src/atproto/space.zig`. 97 97 + Permissioned records and blobs are not public repo records. They must not be 98 98 + squeezed into `records`, `repo_blocks`, `commits`, or `seq_events`. 78 99 79 79 - Because ZDS stores many actor repos in one SQLite database instead of one 80 80 - actor-store per DID, it must preserve the same actor-local semantics explicitly: 81 81 - 82 82 - - canonical space config lives in `permissioned_spaces`, keyed by space URI 83 83 - - actor-local owner/member/deleted state lives in 84 84 - `permissioned_space_actor_state`, keyed by `(space, actor_did)` 85 85 - - member-list state remains space-owner state 86 86 - - record state and oplogs carry the member `repo_did` explicitly 100 100 + Existing databases from the earlier experiment drop the dedicated 101 101 + permissioned-space member tables during migration: 87 102 88 88 - Do not collapse actor-local space state back into a single global row. The same 89 89 - space URI can legitimately have a different local view for different actors on 90 90 - the same ZDS process. 103 103 + - `permissioned_space_members` 104 104 + - `permissioned_space_member_state` 105 105 + - `permissioned_space_member_oplog` 91 106 92 107 ## Zat first 93 108 94 94 - Before implementing local primitives, check Zat's current public API. Recent 95 95 - Zat work is relevant: 96 96 - 97 97 - - `zat.mst.Mst` has the performance-oriented middle layer ZDS already uses: 98 98 - lazy CID stubs, cached clean-node CIDs, direct DAG-CBOR node serialization, 99 99 - borrowed-key insertion, `collectBlocks`, and ordered `walk`. 100 100 - - `zat.car.streamBlocks` supports zero-allocation CAR iteration for large repo 101 101 - workflows. 102 102 - - `zat.cbor` owns DAG-CBOR values, CIDs, encoding, and parsing helpers. 103 103 - - `zat.Tid`, `zat.Did`, `zat.Nsid`, `zat.Rkey`, and `zat.AtUri` own syntax 104 104 - parsing and formatting for standard AT Protocol identifiers. 105 105 - - `zat.Keypair`, `zat.jwt`, `verifyJose`, and strict repo verification helpers 106 106 - keep the JOSE-vs-repo-signature distinction explicit. 107 107 - - `zat.oauth` owns OAuth client/primitive helpers, while ZDS owns PDS policy, 108 108 - stored grants, and route behavior. 109 109 + Before implementing local primitives, check Zat's current public API. ZDS uses 110 110 + `zat` for syntax, TIDs, DID/handle resolution, JWT helpers, DAG-CBOR, CAR, MST, 111 111 + repo verification, OAuth helpers, JSON helpers, and streaming clients. 109 112 110 113 Likely future Zat candidates: 111 114 ··· 116 119 Do not edit or patch the sibling `zat` repo from ZDS without stopping and 117 120 making the proposed Zat change explicit. 118 121 119 119 - ## implementation milestones 120 120 - 121 121 - Current implementation notes: 122 122 - 123 123 - 1. Permissioned data remains behind `ZDS_PERMISSIONED_DATA`. 124 124 - 2. Spaces, members, member/repo LtHash state, record oplogs, member oplogs, 125 125 - grants, owner-signed space credentials, credential-gated reads, ranged blob 126 126 - reads, and best-effort notify fanout live in ZDS. 127 127 - 3. The current branch is still experimental because the upstream proposal is 128 128 - experimental. Do not narrow future work to one downstream app's needs unless 129 129 - the protocol shape itself narrows. 130 130 - 131 131 - Each step should keep permissioned data dark unless `ZDS_PERMISSIONED_DATA` is 132 132 - enabled. 133 133 - 134 122 ## code boundaries 135 123 136 124 Permissioned data is experimental code and should stay segmented: 137 125 138 126 - HTTP dispatch and XRPC semantics live in `src/atproto/space.zig`. 139 139 - - Experimental protocol primitives live in 140 140 - `src/internal/permissioned_data.zig`. 127 127 + - Experimental protocol helpers live in `src/internal/permissioned_data.zig`. 128 128 + - The resident private-record browser lives in 129 129 + `src/internal/private_spaces.zig`. 141 130 - OAuth scope parsing for `space:` scopes lives in `src/internal/scopes.zig`. 142 131 - Persistent state lives in permissioned-data tables in 143 132 `src/storage/store.zig`.