···11# Contrail
2233-> **Work in progress!** Pre-alpha, expect breaking changes.
33+> **Pre-alpha.** Expect breaking changes.
4455-Contrail is a library (and a small set of sibling packages) for building AT Protocol appviews. Define collections, get automatic Jetstream ingestion, PDS backfill, typed XRPC endpoints, permissioned spaces for private records, group-controlled communities, and a client-side reactive sync layer.
55+A library for building AT Protocol appviews. Declare your collections, get automatic Jetstream ingestion, typed XRPC endpoints, optional permissioned spaces for private records, group-controlled communities, and a client-side reactive sync layer. Runs on Cloudflare Workers + D1, Node.js + PostgreSQL, or SvelteKit.
66+77+## Install
88+99+```bash
1010+pnpm add @atmo-dev/contrail
1111+```
1212+1313+## Minimal example
1414+1515+```ts
1616+import { Contrail } from "@atmo-dev/contrail";
1717+1818+const contrail = new Contrail({
1919+ namespace: "com.example",
2020+ db, // D1, node:sqlite, or @atmo-dev/contrail/postgres
2121+ collections: {
2222+ event: {
2323+ collection: "community.lexicon.calendar.event",
2424+ queryable: { startsAt: { type: "range" } },
2525+ searchable: ["name", "description"],
2626+ },
2727+ },
2828+});
2929+3030+await contrail.init();
3131+await contrail.ingest(); // pulls from Jetstream
3232+```
3333+3434+Mount the XRPC handler in any fetch-style framework:
3535+3636+```ts
3737+import { createHandler } from "@atmo-dev/contrail/server";
3838+export default { fetch: createHandler(contrail) };
3939+```
4040+4141+## Docs
4242+4343+- [Indexing](./docs/indexing.md) — the core: collections, queries, ingestion, adapters
4444+- [Spaces](./docs/spaces.md) — permissioned records stored by the appview
4545+- [Communities](./docs/communities.md) — group-controlled atproto DIDs
4646+- [Sync](./docs/sync.md) — reactive client-side store over `watchRecords`
4747+- [Lexicons](./docs/lexicons.md) — `contrail-lex` CLI and codegen
648749## Packages
85099-| Package | Description |
5151+| Package | |
1052|---|---|
1111-| [`@atmo-dev/contrail`](./packages/contrail) | Core library — indexing, XRPC server, spaces, communities, realtime publishing. |
1212-| [`@atmo-dev/contrail-sync`](./packages/sync) | Client-side reactive watch-store over `watchRecords`. SSE + WebSocket, IndexedDB cache. |
1313-| [`@atmo-dev/contrail-lexicons`](./packages/lexicons) | Lexicon codegen from a Contrail config + CLI (`contrail-lex`) wrapping `@atcute/lex-cli`. |
5353+| `@atmo-dev/contrail` | Core library — indexing, XRPC server, spaces, communities, realtime |
5454+| `@atmo-dev/contrail-sync` | Client-side reactive watch-store with optional IndexedDB cache |
5555+| `@atmo-dev/contrail-lexicons` | Codegen + `contrail-lex` CLI |
14561515-## Apps (reference deployments — `workspace:*`-linked)
1616-1717-| App | Description |
1818-|---|---|
1919-| [`rsvp-atmo`](./apps/rsvp-atmo) | Cloudflare Workers + D1 indexer for `community.lexicon.calendar.*`. |
2020-| [`group-chat`](./apps/group-chat) | Full-featured SvelteKit + Workers group chat using spaces, communities, and realtime. |
2121-| [`postgres`](./apps/postgres) | Node + PostgreSQL minimal indexer. |
2222-| [`cloudflare-workers`](./apps/cloudflare-workers) | Minimal Worker example. |
2323-| [`sveltekit-cloudflare-workers`](./apps/sveltekit-cloudflare-workers) | SvelteKit Statusphere-style example. |
2424-2525-## Setup
2626-2727-```bash
2828-pnpm install
2929-pnpm build
3030-pnpm test
3131-```
3232-3333-Per-package commands run through Turbo:
3434-3535-```bash
3636-pnpm build # turbo run build
3737-pnpm typecheck # turbo run typecheck
3838-pnpm test # turbo run test
3939-pnpm --filter @atmo-dev/contrail build
4040-pnpm --filter rsvp-atmo dev
4141-```
4242-4343-## Releasing
4444-4545-Changesets drive versioning. `@atmo-dev/contrail` and `@atmo-dev/contrail-sync` are `linked` so their versions stay aligned (they share the realtime wire protocol).
4646-4747-```bash
4848-pnpm changeset # add a changeset
4949-pnpm changeset version # bump versions
5050-pnpm release # build + publish
5151-```
5757+Working in this repo? See [development.md](https://github.com/flo-bit/contrail/blob/main/development.md) for the monorepo layout and commands.
+51
development.md
···11+# Contrail
22+33+> **Work in progress!** Pre-alpha, expect breaking changes.
44+55+Contrail is a library (and a small set of sibling packages) for building AT Protocol appviews. Define collections, get automatic Jetstream ingestion, PDS backfill, typed XRPC endpoints, permissioned spaces for private records, group-controlled communities, and a client-side reactive sync layer.
66+77+## Packages
88+99+| Package | Description |
1010+|---|---|
1111+| [`@atmo-dev/contrail`](./packages/contrail) | Core library — indexing, XRPC server, spaces, communities, realtime publishing. |
1212+| [`@atmo-dev/contrail-sync`](./packages/sync) | Client-side reactive watch-store over `watchRecords`. SSE + WebSocket, IndexedDB cache. |
1313+| [`@atmo-dev/contrail-lexicons`](./packages/lexicons) | Lexicon codegen from a Contrail config + CLI (`contrail-lex`) wrapping `@atcute/lex-cli`. |
1414+1515+## Apps (reference deployments — `workspace:*`-linked)
1616+1717+| App | Description |
1818+|---|---|
1919+| [`rsvp-atmo`](./apps/rsvp-atmo) | Cloudflare Workers + D1 indexer for `community.lexicon.calendar.*`. |
2020+| [`group-chat`](./apps/group-chat) | Full-featured SvelteKit + Workers group chat using spaces, communities, and realtime. |
2121+| [`postgres`](./apps/postgres) | Node + PostgreSQL minimal indexer. |
2222+| [`cloudflare-workers`](./apps/cloudflare-workers) | Minimal Worker example. |
2323+| [`sveltekit-cloudflare-workers`](./apps/sveltekit-cloudflare-workers) | SvelteKit Statusphere-style example. |
2424+2525+## Setup
2626+2727+```bash
2828+pnpm install
2929+pnpm build
3030+pnpm test
3131+```
3232+3333+Per-package commands run through Turbo:
3434+3535+```bash
3636+pnpm build # turbo run build
3737+pnpm typecheck # turbo run typecheck
3838+pnpm test # turbo run test
3939+pnpm --filter @atmo-dev/contrail build
4040+pnpm --filter rsvp-atmo dev
4141+```
4242+4343+## Releasing
4444+4545+Changesets drive versioning. `@atmo-dev/contrail` and `@atmo-dev/contrail-sync` are `linked` so their versions stay aligned (they share the realtime wire protocol).
4646+4747+```bash
4848+pnpm changeset # add a changeset
4949+pnpm changeset version # bump versions
5050+pnpm release # build + publish
5151+```
+54
docs/communities.md
···11+# Communities
22+33+Group-controlled atproto DIDs. A community is a DID whose signing/rotation keys are held by the appview on behalf of multiple members, with tiered access levels. Built on top of [spaces](./spaces.md).
44+55+## When to use this
66+77+When you want atproto records published under a *shared* identity — a team, a project, a channel — not a single user. Think: a group's published calendar events, a community's published posts.
88+99+## Two modes
1010+1111+- **Minted** — contrail creates a fresh `did:plc` for the community, holds the signing + rotation keys, publishes from it.
1212+- **Adopted** — contrail takes over an existing DID whose rotation keys were handed over by the owner.
1313+1414+Either way, the result is the same: a DID that multiple members can act through, gated by access levels.
1515+1616+## Access levels
1717+1818+Each member has a level (ranked). Levels map to write permissions. Owners can grant/revoke levels. Two reserved levels exist: `owner` and `member`. Your deployment defines the rest:
1919+2020+```ts
2121+community: {
2222+ masterKey: ENV.COMMUNITY_MASTER_KEY, // 32-byte encryption key for stored credentials
2323+ serviceDid: "did:web:example.com",
2424+ levels: ["admin", "moderator"], // ranked, highest-first
2525+}
2626+```
2727+2828+Stored credentials (app passwords for adopted communities, signing keys for minted) are envelope-encrypted with `masterKey`. Never ship the placeholder.
2929+3030+## How it composes with spaces
3131+3232+A community *owns* spaces. Members of the community get access to community-owned spaces based on their level. Grant access per-space per-level:
3333+3434+```
3535+community.space.grant { spaceUri, level: "admin", perms: "write" }
3636+```
3737+3838+The spaces layer stays ignorant of access levels — it just sees "this DID is a member with these perms." The community layer projects member × level → space perms.
3939+4040+## XRPCs
4141+4242+- `com.example.community.mint | adopt | list | delete`
4343+- `com.example.community.invite.create | redeem | revoke | list`
4444+- `com.example.community.setAccessLevel | revoke | listMembers`
4545+- `com.example.community.space.create | grant | revoke | ...` — community-owned spaces
4646+- `com.example.community.putRecord | deleteRecord` — publish records as the community DID
4747+4848+## What's not here
4949+5050+- No per-record per-level ACLs. Model as spaces.
5151+- No auto-rotation on key compromise yet.
5252+- Adoption is irreversible without manual key surrender back to the owner.
5353+5454+The design follows zicklag's [Arbiter design sketch](https://zicklag.leaflet.pub/3mjrvb5pul224) for group management on atproto. The post is an early design note; our implementation will track it as the spec firms up.
+96
docs/indexing.md
···11+# Indexing
22+33+Contrail's core job: mirror atproto records into your DB and expose them via XRPC. You describe what to index with a config object; everything else is automatic.
44+55+## Collection shape
66+77+```ts
88+collections: {
99+ event: {
1010+ collection: "community.lexicon.calendar.event", // full NSID
1111+ queryable: {
1212+ mode: {}, // ?mode=online
1313+ startsAt: { type: "range" }, // ?startsAtMin=...&startsAtMax=...
1414+ },
1515+ searchable: ["name", "description"], // FTS5 / tsvector
1616+ relations: {
1717+ rsvps: {
1818+ collection: "rsvp",
1919+ groupBy: "status",
2020+ groups: { going: "community.lexicon.calendar.rsvp#going" },
2121+ },
2222+ },
2323+ references: {
2424+ event: { collection: "event", field: "subject.uri" },
2525+ },
2626+ },
2727+}
2828+```
2929+3030+- **queryable** — string equality or range, exposed as query params.
3131+- **searchable** — FTS5 on D1/Postgres. Not available on `node:sqlite`.
3232+- **relations** — materialized many-to-one counts (`rsvpsGoingCount`).
3333+- **references** — forward lookups. Hydrate inline with `?hydrateEvent=true`.
3434+3535+## Ingestion
3636+3737+Three ways records land in the DB:
3838+3939+```ts
4040+await contrail.ingest(); // one Jetstream cycle, then stops
4141+await contrail.runPersistent(); // long-lived connection, auto-reconnect
4242+await contrail.notify(uri); // immediate PDS fetch for one record
4343+```
4444+4545+Call `notify()` after your app writes to a PDS and needs the change reflected now. Jetstream catches the same event later; the duplicate is detected by CID.
4646+4747+## Discovery + backfill
4848+4949+```ts
5050+await contrail.discover(); // find users from relays
5151+await contrail.backfill({ concurrency: 100 }); // pull their history
5252+await contrail.sync(); // both
5353+```
5454+5555+## Querying
5656+5757+```ts
5858+const { records, cursor } = await contrail.query("event", {
5959+ filters: { mode: "online" },
6060+ sort: { field: "startsAt", direction: "asc" },
6161+ limit: 20,
6262+});
6363+```
6464+6565+Or via HTTP once the handler is mounted:
6666+6767+```
6868+/xrpc/com.example.event.listRecords?mode=online&sort=startsAt&limit=20
6969+/xrpc/com.example.event.getRecord?uri=at://did:plc:.../...
7070+```
7171+7272+## Adapters
7373+7474+| Adapter | Use when | FTS |
7575+|---|---|---|
7676+| Cloudflare D1 | Workers | ✅ |
7777+| `@atmo-dev/contrail/sqlite` | Node 22+ local dev | ❌ |
7878+| `@atmo-dev/contrail/postgres` | Node server | ✅ |
7979+8080+```ts
8181+import { createPostgresDatabase } from "@atmo-dev/contrail/postgres";
8282+const db = createPostgresDatabase(pool);
8383+```
8484+8585+## Top-level config
8686+8787+| Key | Default | |
8888+|---|---|---|
8989+| `namespace` | — | Reverse-domain for XRPC paths |
9090+| `profiles` | `["app.bsky.actor.profile"]` | Profile NSIDs, auto-hydrated via `?profiles=true` |
9191+| `jetstreams` | Bluesky | Jetstream URLs |
9292+| `relays` | Bluesky | Relay URLs for discovery |
9393+| `notify` | off | `true` opens `notifyOfUpdate`; a string requires `Bearer` |
9494+| `spaces` | — | See [Spaces](./spaces.md) |
9595+| `community` | — | See [Communities](./communities.md) |
9696+| `realtime` | — | See [Sync](./sync.md) |
+91
docs/lexicons.md
···11+# Lexicons
22+33+Contrail emits lexicon JSON for every XRPC method it exposes. Your app needs those JSON files for two things: publishing to your PDS (so other apps can discover them) and generating TypeScript types for typed clients. The `contrail-lex` CLI handles both.
44+55+```bash
66+pnpm add -D @atmo-dev/contrail-lexicons @atcute/lex-cli
77+```
88+99+`@atcute/lex-cli` is a peer dep — you pin the version.
1010+1111+## CLI
1212+1313+```bash
1414+contrail-lex generate # emit lexicon JSON from your Contrail config
1515+contrail-lex pull # wraps `lex-cli pull` (fetches external lexicons)
1616+contrail-lex types # wraps `lex-cli generate` (JSON → TS types)
1717+contrail-lex all # generate → pull → generate → pull → types
1818+contrail-lex all --no-types # skip the type step
1919+```
2020+2121+The CLI auto-detects your config at `contrail.config.ts`, `app/config.ts`, or `src/lib/contrail/config.ts`. Override with `--config <path>`.
2222+2323+## What lands where
2424+2525+Running `contrail-lex all` against a config with `namespace: "com.example"` writes:
2626+2727+```
2828+lexicons-generated/ # JSON lexicons emitted from your config
2929+lexicons-pulled/ # external NSIDs fetched by lex-cli pull
3030+lex.config.js # regenerated each run (add to .gitignore)
3131+src/lexicon-types/ # TS types emitted by lex-cli generate
3232+```
3333+3434+`lexicons-generated/` and `lexicons-pulled/` should be **committed** — that way CI doesn't need network access and consumers of your repo can pull the JSON as a lexicon source. `lex.config.js` and `src/lexicon-types/` are generated on demand and safe to gitignore.
3535+3636+## Publishing to a PDS
3737+3838+Once committed, publish the lexicons to an atproto account so other apps can resolve them:
3939+4040+```ts
4141+import { publishLexicons } from "@atmo-dev/contrail-lexicons";
4242+4343+await publishLexicons({
4444+ generatedDir: "lexicons-generated",
4545+ identifier: process.env.LEXICON_ACCOUNT_IDENTIFIER,
4646+ password: process.env.LEXICON_ACCOUNT_PASSWORD,
4747+});
4848+```
4949+5050+Writes each lexicon as a `com.atproto.lexicon.schema` record under `at://<did>/com.atproto.lexicon.schema/<nsid>`. You'll also need DNS TXT records — publishing prints them for you.
5151+5252+## Programmatic API
5353+5454+```ts
5555+import { generateLexicons, extractXrpcMethods } from "@atmo-dev/contrail-lexicons";
5656+5757+const generated = generateLexicons({
5858+ config,
5959+ rootDir: process.cwd(),
6060+ outputDir: "lexicons-generated",
6161+});
6262+6363+const methods = extractXrpcMethods(generated); // NSIDs of every query + procedure
6464+```
6565+6666+Handy for emitting the method list for your OAuth permission set.
6767+6868+## Consuming a third-party contrail instance
6969+7070+Someone else runs a contrail deployment; you want typed client access to their XRPCs. Point `lex-cli pull` at their git repo:
7171+7272+```js
7373+// lex.config.js
7474+import { defineLexiconConfig } from "@atcute/lex-cli";
7575+7676+export default defineLexiconConfig({
7777+ outdir: "src/lexicon-types/",
7878+ imports: ["@atcute/atproto"],
7979+ files: ["lexicons/**/*.json"],
8080+ pull: {
8181+ outdir: "lexicons/",
8282+ sources: [{
8383+ type: "git",
8484+ remote: "https://github.com/them/their-contrail.git",
8585+ pattern: ["lexicons-generated/**/*.json"],
8686+ }],
8787+ },
8888+});
8989+```
9090+9191+Then `npx lex-cli pull && npx lex-cli generate`, and their endpoints are typed in your client.
+70
docs/spaces.md
···11+# Spaces
22+33+Auth-gated store for records that can't live on public PDSes — private events, invite-only groups, members-only chat. Opt-in; zero cost if you don't enable it.
44+55+## Mental model
66+77+> A **space** is a bag of records with one lock. The **member list** says who has the key.
88+99+- One owner (DID), one type (NSID), one key. Identified by `at://<owner>/<type>/<key>`.
1010+- Members have `read` or `write`. Owner is implicit `write`.
1111+- Optional **app policy** gates which OAuth clients can act in the space.
1212+1313+Every permission boundary is its own space. No nested ACLs. Richer roles = more spaces or app-layer checks.
1414+1515+## Enable
1616+1717+```ts
1818+import type { ContrailConfig } from "@atmo-dev/contrail";
1919+2020+const config: ContrailConfig = {
2121+ namespace: "com.example",
2222+ collections: { /* ... */ },
2323+ spaces: {
2424+ type: "com.example.event.space",
2525+ serviceDid: "did:web:example.com",
2626+ },
2727+};
2828+```
2929+3030+Each collection gets a parallel `spaces_records_<short>` table. Opt out per-collection:
3131+3232+```ts
3333+public_only: { collection: "com.example.public", allowInSpaces: false }
3434+```
3535+3636+## Auth
3737+3838+atproto service-auth JWTs via `@atcute/xrpc-server`. Middleware validates signature, `aud`, and `lxm` before the handler runs. Apps acting in a space mint a JWT against the user's PDS with `Atproto-Proxy: did:web:example.com#com_example_space`, forward to your service, it verifies and executes.
3939+4040+**Note:** Use the plain DID (no `#fragment`) as `serviceDid` — the fragment form only belongs in your DID doc's service entry.
4141+4242+## Unified `listRecords`
4343+4444+| Call | Returns |
4545+|---|---|
4646+| no auth, no `spaceUri` | public only |
4747+| `?spaceUri=…` + JWT | one space (ACL-gated) |
4848+| JWT, no `spaceUri` | public **unioned** with every space the caller is a member of |
4949+5050+Filters, sorts, hydration, and references work across all three. Records from a space carry a `space: <spaceUri>` field.
5151+5252+## Invites
5353+5454+Stored as hashed tokens. Redemption is a single atomic UPDATE that checks `!revoked && !expired && !exhausted`. Generate, hand out the plaintext once, verify later.
5555+5656+## XRPCs
5757+5858+- `com.example.space.create | get | list | delete`
5959+- `com.example.space.putRecord | deleteRecord | listRecords | getRecord`
6060+- `com.example.space.invite.create | redeem | revoke | list`
6161+- `com.example.space.listMembers | removeMember`
6262+6363+## What's not here
6464+6565+- No E2EE (data is operator-readable).
6666+- No FTS on `?spaceUri=…` yet.
6767+- No per-space sharding — one DB, one operator.
6868+- Not a long-term replacement for real atproto permissioned repos.
6969+7070+The design follows Daniel Holmgren's [permissioned data rough spec](https://dholms.leaflet.pub/3mhj6bcqats2o). The goal is that when real atproto permissioned repos ship, migration is mostly data movement — the API your app speaks doesn't change.
+93
docs/sync.md
···11+# Sync
22+33+Client-side reactive store over contrail's `watchRecords` endpoints. Subscribes once, reconciles forever, ships with optimistic updates and an optional IndexedDB cache.
44+55+Lives in its own package:
66+77+```bash
88+pnpm add @atmo-dev/contrail-sync
99+```
1010+1111+## Basic use
1212+1313+```ts
1414+import { createWatchStore } from "@atmo-dev/contrail-sync";
1515+1616+const store = createWatchStore({
1717+ url: "/xrpc/com.example.message.watchRecords?roomUri=at://...",
1818+ transport: "sse", // or "ws"
1919+});
2020+2121+store.subscribe(({ records, status }) => { /* re-render */ });
2222+store.start();
2323+```
2424+2525+Framework-agnostic — wrap it in Svelte `$state`, React `useSyncExternalStore`, Vue `ref`, whatever.
2626+2727+## Transports
2828+2929+- **SSE** (default) — one HTTP request, simplest. Works everywhere.
3030+- **WS** — a two-step handshake: HTTP GET returns a snapshot + watch-scoped ticket, then you upgrade to WS. On Cloudflare, the WS terminates on a Durable Object that hibernates idle connections. Same event stream either way.
3131+3232+## Authenticated watches
3333+3434+Pass `mintTicket` for any non-public endpoint. One-shot string for SSR-minted tickets, function for fresh tickets per reconnect:
3535+3636+```ts
3737+mintTicket: async () => (await fetch("/api/ticket")).then((r) => r.text()),
3838+```
3939+4040+Tickets are minted server-side via `com.example.realtime.ticket` (or any app-specific route).
4141+4242+## Optimistic updates
4343+4444+```ts
4545+store.addOptimistic({ rkey, did, record: { text: "hi" } });
4646+// later, on mutation failure:
4747+store.markFailed(rkey, err);
4848+// or explicit rollback:
4949+store.removeOptimistic(rkey);
5050+```
5151+5252+When a real record with the same `rkey` arrives via the stream, the optimistic entry is dropped automatically.
5353+5454+## IndexedDB cache
5555+5656+Instant first paint from last session's records:
5757+5858+```ts
5959+import { createIndexedDBCache } from "@atmo-dev/contrail-sync/cache-idb";
6060+6161+createWatchStore({
6262+ url,
6363+ cache: createIndexedDBCache(),
6464+ cacheMaxRecords: 200,
6565+});
6666+```
6767+6868+Cached records show immediately; the live snapshot reconciles when the connection opens.
6969+7070+## Server-side config
7171+7272+Enable `watchRecords` emission in your Contrail config:
7373+7474+```ts
7575+realtime: {
7676+ ticketSecret: ENV.REALTIME_TICKET_SECRET, // 32 bytes
7777+ pubsub: new DurableObjectPubSub(env.PUBSUB), // or in-memory for dev
7878+}
7979+```
8080+8181+See [indexing.md](./indexing.md) for the full config surface.
8282+8383+## Lifecycle
8484+8585+```
8686+idle → connecting → snapshot → live
8787+ ↓ (disconnect)
8888+ reconnecting → snapshot → live
8989+ ↓ (stop)
9090+ closed
9191+```
9292+9393+Stale records stay visible across reconnects until the fresh snapshot arrives, at which point anything the server didn't re-send is evicted. Survives offline periods cleanly.