atproto pds in zig
pds.zat.dev
pds
atproto
Simplify project docs
+155
-135
+60
-36
README.md
+60
-36
README.md
···
1
1
# zds
2
2
3
-
A Zig experiment toward an AT Protocol Personal Data Server.
4
-
5
-
> Name credit: Jim (`calabro.io`) suggested `zds`.
3
+
a zig implementation of an at protocol personal data server.
6
4
7
-
`zds` is a Zig implementation of an AT Protocol Personal Data Server. The goal
8
-
is a rigorous, production-correct PDS that follows the protocol and official PDS
9
-
behavior while keeping Bluesky-specific compatibility isolated from the core
10
-
server:
5
+
> name credit: jim (`calabro.io`) suggested `zds`.
11
6
12
-
- real account, identity, repo, sync, and blob hosting
13
-
- durable storage with explicit migration and recovery paths
14
-
- production account migration support
15
-
- compatibility with existing PDS implementations and clients
16
-
- isolated Bluesky app-view compatibility where clients require it
7
+
`zds` stores atproto accounts, repos, records, blobs, sessions, and identity
8
+
state. it uses [`zat`](../zat.dev/zat) for atproto primitives such as syntax,
9
+
did resolution, jwt helpers, dag-cbor, car, mst, repo verification, and key
10
+
encoding.
17
11
18
-
## Reference Projects
12
+
## references
19
13
20
-
The initial design pass is informed by:
14
+
the implementation is informed by:
21
15
22
16
- `tranquil.farm/tranquil-pds`
23
17
- `futur.blue/pegasus`
···
26
20
- `chadtmiller.com/pds.js`
27
21
- `zzstoatzz.io/pds-message-poc`
28
22
- `zzstoatzz.io/pollz`
23
+
- `haileyok/cocoon`
29
24
30
-
See [docs/reference-study.md](docs/reference-study.md).
25
+
see [docs/reference-study.md](docs/reference-study.md).
31
26
32
-
## Development
27
+
## run locally
33
28
34
29
```sh
35
30
zig build test
···
37
32
caddy run --config dev/Caddyfile
38
33
```
39
34
40
-
Before committing, run the relevant probes plus:
35
+
before committing:
41
36
42
37
```sh
43
38
zig build test
44
39
zig zen
45
40
```
46
41
47
-
Then reflect on whether the change reduces what future readers must remember.
42
+
## endpoints
48
43
49
-
The local server currently exposes:
44
+
current server surface includes:
50
45
51
46
- `GET /`
52
47
- `GET /.well-known/did.json`
···
58
53
- `POST /xrpc/com.atproto.server.refreshSession`
59
54
- `GET /xrpc/com.atproto.server.getSession`
60
55
- `GET /xrpc/com.atproto.server.getServiceAuth`
61
-
- migration-oriented repo, blob, account-status, email, and PLC submission endpoints
56
+
- repo read/write/import/blob endpoints
57
+
- sync repo/blob/status endpoints
58
+
- identity plc credential/sign/submit endpoints
59
+
- a small set of `app.bsky.*` compatibility endpoints
62
60
63
-
Blob upload requests default to a 100,000,000-byte ceiling for Bluesky video
64
-
compatibility and can be changed with `ZDS_BLOB_UPLOAD_LIMIT` or
65
-
`--blob-upload-limit`.
61
+
blob uploads default to `100000000` bytes for bluesky video compatibility. set
62
+
`ZDS_BLOB_UPLOAD_LIMIT` or `--blob-upload-limit` to change it.
66
63
67
-
See [docs/smoke.md](docs/smoke.md) for the first `atproto-smoke` run.
64
+
see [docs/smoke.md](docs/smoke.md) for the first `atproto-smoke` run.
68
65
69
-
## Production Migration
66
+
## deploy
70
67
71
-
For a throwaway migration target, run with a public HTTPS URL and mail config:
68
+
run behind a public https origin:
72
69
73
70
```sh
74
71
ZDS_RESEND_API_KEY=... \
···
83
80
--server-did did:web:pds.example.com
84
81
```
85
82
86
-
The migration path now includes service-auth-gated account creation for existing
87
-
DIDs, real repo writes with per-account signing keys, verified repo CAR import,
88
-
repo export, blob import checks, account activation state, recommended DID
89
-
credentials, PLC operation signing/submission, and email delivery via Resend
90
-
with local log fallback. Deployment also serves the PDS `did:web` document and
91
-
handle `/.well-known/atproto-did` resolution.
83
+
configuration:
92
84
93
-
The next production risks are operational rather than fake core shortcuts:
94
-
sequenced subscription events, durable session/recovery flows, and a filesystem
95
-
or object-store blob backend with backup/restore tooling. See
96
-
[docs/production-migration.md](docs/production-migration.md).
85
+
- `--public-url`: public pds origin used in did credentials and blob urls
86
+
- `--server-did`: pds service did, usually `did:web:<host>`
87
+
- `ZDS_HANDLE_DOMAINS`: comma-separated handle domains advertised by
88
+
`describeServer`
89
+
- `ZDS_JWT_SECRET`: stable secret for access and refresh jwt signing
90
+
- `ZDS_RESEND_API_KEY` and `ZDS_EMAIL_FROM`: email delivery for account and plc
91
+
tokens
92
+
- `ZDS_BLOB_UPLOAD_LIMIT`: generic `com.atproto.repo.uploadBlob` body limit
93
+
94
+
## account migration
95
+
96
+
zds implements the pds endpoints used by browser-based migration tools such as
97
+
[pds moover](https://pdsmoover.com/):
98
+
99
+
- service-auth-gated `createAccount` for an existing did
100
+
- repo car import/export
101
+
- missing blob reporting and blob upload
102
+
- preferences import/export
103
+
- recommended did credentials
104
+
- plc operation email token, signing, and submission
105
+
- account activation/deactivation and status checks
106
+
107
+
see [docs/migration.md](docs/migration.md).
108
+
109
+
## storage
110
+
111
+
sqlite stores accounts, records, repo blocks, commits, expected blobs, imported
112
+
blobs, preferences, bookmarks, and mute state. uploaded blobs are currently
113
+
stored in sqlite as blob rows.
114
+
115
+
## next
116
+
117
+
- `com.atproto.sync.subscribeRepos`
118
+
- password reset and account recovery flows
119
+
- filesystem or object-store blob storage
120
+
- backup/restore docs for the sqlite database and blob store
+95
docs/migration.md
+95
docs/migration.md
···
1
+
# migration
2
+
3
+
`zds` implements the pds endpoints used by browser-based account migration
4
+
tools, including pds moover.
5
+
6
+
## pds moover flow
7
+
8
+
pds moover asks for:
9
+
10
+
- current handle and password
11
+
- new pds url
12
+
- new email
13
+
- new handle
14
+
- invite code
15
+
16
+
the migration is not complete until the user enters a plc token delivered by
17
+
email.
18
+
19
+
## implemented endpoints
20
+
21
+
implemented:
22
+
23
+
- `com.atproto.server.createAccount`
24
+
- `com.atproto.server.checkAccountStatus`
25
+
- `com.atproto.server.activateAccount`
26
+
- `com.atproto.server.deactivateAccount`
27
+
- `com.atproto.server.refreshSession`
28
+
- `com.atproto.repo.importRepo`
29
+
- `com.atproto.repo.listMissingBlobs`
30
+
- `com.atproto.sync.getRepo`
31
+
- `com.atproto.sync.listBlobs`
32
+
- `com.atproto.sync.getRepoStatus`
33
+
- `com.atproto.identity.getRecommendedDidCredentials`
34
+
- `com.atproto.identity.requestPlcOperationSignature`
35
+
- `com.atproto.identity.signPlcOperation`
36
+
- `com.atproto.identity.submitPlcOperation`
37
+
- `app.bsky.actor.getPreferences`
38
+
- `app.bsky.actor.putPreferences`
39
+
40
+
implementation details:
41
+
42
+
- `createAccount` verifies migration service jwts against the migrating did
43
+
- session tokens are signed with the configured `ZDS_JWT_SECRET`
44
+
- stored passwords use salted pbkdf2-sha256 hashes
45
+
- repo writes produce signed commit blocks and mst updates
46
+
- imported repo cars are verified against the account did and signing key
47
+
- each account has its own signing key for repo commits and plc credentials
48
+
- `uploadBlob` defaults to 100,000,000 bytes and is configurable with
49
+
`ZDS_BLOB_UPLOAD_LIMIT` or `--blob-upload-limit`
50
+
- the server serves `/.well-known/did.json` for the pds service did and
51
+
`/.well-known/atproto-did` for hosted handles
52
+
53
+
## not yet implemented
54
+
55
+
- `com.atproto.sync.subscribeRepos` and real sequenced account/identity events
56
+
- password reset and account recovery paths
57
+
- filesystem or object-store blob backend with backup/restore tooling
58
+
59
+
## deployment
60
+
61
+
run behind a public https origin:
62
+
63
+
```sh
64
+
ZDS_RESEND_API_KEY=... \
65
+
ZDS_EMAIL_FROM='ZDS <pds@example.com>' \
66
+
ZDS_BLOB_UPLOAD_LIMIT=100000000 \
67
+
ZDS_HANDLE_DOMAINS='.example.com,example.com' \
68
+
ZDS_JWT_SECRET='at-least-32-random-bytes-here' \
69
+
zig build run -- \
70
+
--port 2583 \
71
+
--db /var/lib/zds/zds.sqlite3 \
72
+
--public-url https://pds.example.com \
73
+
--server-did did:web:pds.example.com
74
+
```
75
+
76
+
the public url is used for did credentials and blob views. `ZDS_RESEND_API_KEY`
77
+
and `ZDS_EMAIL_FROM` enable real email delivery; without them, token emails are
78
+
logged to stdout for local development. `ZDS_BLOB_UPLOAD_LIMIT` controls the
79
+
generic `com.atproto.repo.uploadBlob` request body ceiling.
80
+
`ZDS_HANDLE_DOMAINS` is what `describeServer` advertises to clients.
81
+
`ZDS_JWT_SECRET` signs app-session access and refresh jwts; keep it stable
82
+
across deploys or existing sessions will be invalidated.
83
+
84
+
the at protocol uploadBlob lexicon does not define one universal byte limit; it
85
+
says blob restrictions are enforced when a record references the uploaded blob.
86
+
the official pds exposes a configurable blob upload limit, while the bluesky
87
+
video embed lexicon allows 100,000,000-byte mp4 blobs. `zds` uses that as the
88
+
default generic upload ceiling so migration tests do not fail below bluesky's
89
+
current video shape.
90
+
91
+
deployment also needs sqlite data-directory backups and operational access to
92
+
inspect account status, repo import, missing blobs, and plc submission failures.
93
+
94
+
core atproto account, repo, sync, and identity code should stay separate from
95
+
bluesky app-view compatibility. bluesky compatibility lives in `bsky_*` modules.
-99
docs/production-migration.md
-99
docs/production-migration.md
···
1
-
# Production Migration
2
-
3
-
`zds` now has the migration surface needed to try a low-stakes throwaway account
4
-
migration. Passing the local smoke suite was only the baseline; the important
5
-
surface here is account creation under service auth, repo import/export, blob
6
-
transfer, PLC handoff, and email-token delivery.
7
-
8
-
## PDS MOOver Shape
9
-
10
-
PDS MOOver asks for:
11
-
12
-
- current handle and password
13
-
- new PDS URL
14
-
- new email
15
-
- new handle
16
-
- invite code
17
-
18
-
The migration is not complete until the user enters a PLC token delivered by
19
-
email. That means `zds` needs both migration endpoints and outbound email before
20
-
it can be used safely with that flow.
21
-
22
-
## Implemented Migration Surface
23
-
24
-
Implemented:
25
-
26
-
- `com.atproto.server.createAccount`
27
-
- `com.atproto.server.checkAccountStatus`
28
-
- `com.atproto.server.activateAccount`
29
-
- `com.atproto.server.deactivateAccount`
30
-
- `com.atproto.server.refreshSession`
31
-
- `com.atproto.repo.importRepo`
32
-
- `com.atproto.repo.listMissingBlobs`
33
-
- `com.atproto.sync.getRepo`
34
-
- `com.atproto.sync.listBlobs`
35
-
- `com.atproto.sync.getRepoStatus`
36
-
- `com.atproto.identity.getRecommendedDidCredentials`
37
-
- `com.atproto.identity.requestPlcOperationSignature`
38
-
- `com.atproto.identity.signPlcOperation`
39
-
- `com.atproto.identity.submitPlcOperation`
40
-
- `app.bsky.actor.getPreferences`
41
-
- `app.bsky.actor.putPreferences`
42
-
43
-
The migration-critical implementation details are also in place:
44
-
45
-
- `createAccount` verifies migration service JWTs against the migrating DID
46
-
- session tokens are signed with the configured `ZDS_JWT_SECRET`
47
-
- stored passwords use salted PBKDF2-SHA256 hashes
48
-
- repo writes produce signed commit blocks and MST updates, not synthetic refs
49
-
- imported repo CARs are verified against the account DID and signing key
50
-
- each account has its own signing key for repo commits and PLC credentials
51
-
- `uploadBlob` defaults to 100,000,000 bytes and is configurable with
52
-
`ZDS_BLOB_UPLOAD_LIMIT` or `--blob-upload-limit`
53
-
- the server serves `/.well-known/did.json` for the PDS service DID and
54
-
`/.well-known/atproto-did` for hosted handles
55
-
56
-
Still to harden after a throwaway migration:
57
-
58
-
- `com.atproto.sync.subscribeRepos` and real sequenced account/identity events
59
-
- password reset and account recovery paths
60
-
- filesystem or object-store blob backend with backup/restore tooling
61
-
62
-
## Deployment Requirements
63
-
64
-
Run behind a public HTTPS origin:
65
-
66
-
```sh
67
-
ZDS_RESEND_API_KEY=... \
68
-
ZDS_EMAIL_FROM='ZDS <pds@example.com>' \
69
-
ZDS_BLOB_UPLOAD_LIMIT=100000000 \
70
-
ZDS_HANDLE_DOMAINS='.example.com,example.com' \
71
-
ZDS_JWT_SECRET='at-least-32-random-bytes-here' \
72
-
zig build run -- \
73
-
--port 2583 \
74
-
--db /var/lib/zds/zds.sqlite3 \
75
-
--public-url https://pds.example.com \
76
-
--server-did did:web:pds.example.com
77
-
```
78
-
79
-
The public URL is used for DID credentials and blob views. `ZDS_RESEND_API_KEY`
80
-
and `ZDS_EMAIL_FROM` enable real email delivery; without them, token emails are
81
-
logged to stdout for local development. `ZDS_BLOB_UPLOAD_LIMIT` controls the
82
-
generic `com.atproto.repo.uploadBlob` request body ceiling.
83
-
`ZDS_HANDLE_DOMAINS` is what `describeServer` advertises to clients.
84
-
`ZDS_JWT_SECRET` signs app-session access and refresh JWTs; keep it stable
85
-
across deploys or existing sessions will be invalidated.
86
-
87
-
The AT Protocol uploadBlob lexicon does not define one universal byte limit; it
88
-
says blob restrictions are enforced when a record references the uploaded blob.
89
-
The official PDS exposes a configurable blob upload limit, while the Bluesky
90
-
video embed lexicon allows 100,000,000-byte MP4 blobs. `zds` uses that as the
91
-
default generic upload ceiling so migration tests do not fail below Bluesky's
92
-
current video shape.
93
-
94
-
Deployment also needs SQLite data-directory backups and operational access to
95
-
inspect account status, repo import, missing blobs, and PLC submission failures.
96
-
97
-
The implementation should keep following the current split: core atproto
98
-
account, repo, sync, and identity code should not gain Bluesky app-view
99
-
behavior; Bluesky compatibility stays in `bsky_*` modules.