@atjam/tests#

Integration test harness for atjam. Drives real PDS writes against four test accounts (test-1 … test-4), then asserts via the same Constellation backlinks the web app uses for reads. Catches lexicon-shape regressions and (once invitation validation lands) bilateral correctness.

This is not unit tests — it talks to the PDS and to Constellation. Network-dependent, slower than unit tests by design. The point is to verify the system as it actually works, not as it's modeled.

Setup#

You need four ATProto accounts the harness can sign in to. By default the harness points at https://bsky.social; works against any ATProto-compliant PDS that exposes the standard server APIs.

Option 1 — Automated (bsky.social or self-hosted PDS)#

cd tests
GMAIL_ADDRESS=you@gmail.com pnpm create-accounts > .env

What happens:

1. Calls com.atproto.server.createAccount four times, creating test-1.bsky.social through test-4.bsky.social (or whatever HANDLE_DOMAIN you point it at).
2. Plus-addresses your gmail per signup (you+test-1@gmail.com, …) so verification emails all land in one inbox.
3. Mints an app password per account via com.atproto.server.createAppPassword.
4. Writes a complete .env block to stdout containing handles, app passwords, and the right PDS_URL.

You still need to click the verification email for each account in your inbox. Bsky.social may also push you to SMS verify on first use — handle that in the bsky.app UI if it happens.

Custom count: pnpm create-accounts 6 for six accounts (but only the first four are read by the harness today). Custom PDS: PDS_URL=https://your-pds pnpm create-accounts > .env.

Option 2 — Manual#

Create four accounts however you like (bsky.app web flow, goat, Tranquil UI), generate app passwords in each account's settings, then:

cd tests
cp .env.example .env
# edit .env, fill in TEST1_HANDLE / TEST1_APP_PASSWORD, etc.

.env is gitignored. Don't commit it.

Running#

pnpm --filter @atjam/tests test          # run scenarios once, exit code reflects pass/fail
pnpm --filter @atjam/tests test:watch    # vitest watch mode while developing

pnpm --filter @atjam/tests probe:lag     # measure firehose → Constellation latency

Each scenario:

1. Logs in to all four test accounts
2. Resets each account's at.atjam.* records (deletes whatever's there)
3. Runs the scenario's writes
4. Asserts the resulting state via Constellation backlinks
5. Cleans up again at the end

Reset means scenarios are hermetic — rerunning gives the same result, no accumulation.

Account naming#

The four accounts are referred to as test1, test2, test3, test4 internally (TS identifiers can't contain hyphens). The actual bsky handles use hyphens: test-1.bsky.social. Each scenario file aliases them locally to semantic roles for readability:

const organizer    = agents.test1;
const participant1 = agents.test2;
const participant2 = agents.test3;
// test4 is reserved for chain/forgery scenarios

What's covered today#

When the invitation validator lands (likely in @atjam/lexicons or as a module in web/app/lib/), un-skip the stub files and replace it.todo(...) with real assertions. The setup pattern from open-round.test.ts is the template.

Probes#

probe:lag measures how long it takes a record written to the PDS to become visible via Constellation. Useful for:

• Sanity-checking the PDS → relay → indexer pipeline is alive
• Setting realistic waitForBacklinks timeouts in scenarios
• Spotting if Constellation has gotten slower over time

Probes are plain TS scripts (no vitest), invoked via pnpm probe:*. Easy place to add one-off experiments without ceremony.

Adding a scenario#

1. Drop a new file under src/scenarios/*.test.ts.
2. beforeAll → loginAll() then resetAll(agents).
3. Write whatever sequence of records the scenario needs using helpers from writes.ts.
4. Assert via waitForBacklinks (handles firehose lag) or by directly fetching with listMyRecords.
5. afterAll → resetAll(agents) again so the next run starts clean.

Use types from @atjam/lexicons for record bodies — the compiler catches lexicon-shape mistakes before they reach the PDS.

Why this isn't unit tests#

1. The behavior under test is bilateral — records on multiple PDSes that converge via the firehose. Unit tests can fake half of it; integration can't.
2. Constellation is a real dependency — testing the read path with a fake indexer doesn't tell you whether production reads work.
3. The lexicon is the contract. Real PDS writes validate it end-to-end; in-process mocks of createRecord don't.

The cost is speed and brittleness (network flakes, Constellation lag) — accepted in exchange for the realism.

Caveats with bsky.social#

The default PDS_URL is bsky.social, which works but has constraints:

• Records go to the global firehose. Anything you write during testing is public. Test data has throwaway content; just don't put anything sensitive in scenario inputs.
• Use disposable accounts. reset.ts deletes all at.atjam.* records on each account at the start of every run. Point the harness at burner handles, not your personal one.
• Rate limits apply. Per-account write limits are tighter than a self-hosted PDS. ~15 writes per run is fine; running back-to-back-to-back dozens of times might trip them.
• Phone verification may gate things. Bsky started pushing SMS verify in 2024. The harness can sign in to a verified account fine; an unverified one may have limited capabilities.

For automated CI eventually, point at a self-hosted PDS via PDS_URL — no global firehose pollution, no rate limits, no SMS gates.