Version Packages

fix: avoid race condition while resolving PDS

# Conflicts:
# packages/contrail/src/core/backfill.ts

Version Packages

Following feeds fix unbounded pruning

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* feat(contrail-community): provision schema, types, adapter CRUD

* feat(contrail-community): PDS + PLC + service-auth helpers

* feat(contrail-community): provision orchestrator + XRPC route + session cache

* feat(contrail-community): reap CLI for stuck provisioning rows

* test+docs(contrail): provision e2e, docs, changeset, deps

* fix(contrail-community): harden provision allowlist + reap (PR #31 review)

Security/review follow-ups on the community DID provisioning feature:

- Provisioning fails closed: with allowProvisioning=true, an empty/undefined
allowlist no longer accepts any caller pdsEndpoint (fail-open). Require a
non-empty allowlist, or the explicit, loud allowAnyProvisionPdsEndpoint.
Renamed allowedPdsEndpoints -> allowedProvisionPdsEndpoints to scope it to
provisioning (Contrail still reads/indexes from every PDS).

- reap --all-stuck gains a mandatory age floor (listStuckAttempts(olderThanMs)
+ --older-than <minutes>, default 30) so a bulk run can't tombstone an
in-flight provision. --attempt-id still targets a known row regardless.

- reap is reachable: ship it as a contrail-community bin (under pnpm the core
contrail CLI can't resolve the community package; the dynamic import there
now logs instead of an empty catch). Adds Postgres support via
--db/DATABASE_URL alongside the D1 binding.

- archiveStuckAttempt is now atomic (db.batch) + idempotent (ON CONFLICT DO
NOTHING), so a retry after a partial failure can't PK-conflict.

- Minor: export bytesToB64url from plc and reuse in service-auth; drop the
duplicate reap mutual-exclusion validator.

mint-route allowlist gap (pre-existing on main) tracked separately in om-vyok.

* test(contrail-community): drop tautological builder-shape assertions

Remove the 'buildTombstoneOp produces the expected shape' case and the two
pass-through type/prev assertions in the update-op test: both assert literals
the builder just set or args just passed in, exercising object-literal
semantics rather than runtime behavior. The real signing behavior
(signTombstoneOp / signUpdateOp base64url sig) and the live-PLC CID round-trip
remain covered.

* docs(contrail-community): document SSRF exposure of allowAnyProvisionPdsEndpoint

The accept-any escape hatch bypasses endpoint validation: the caller-supplied
pdsEndpoint reaches describeServer + createAccount fetches with no
private-IP/metadata guard. Note inline that it must be paired with trusted auth
+ egress network policy + IMDSv2; the default allowlist path contains this by
construction. No behavior change.

* feat(contrail-base): networkOverrides config for private-network deployments

Add optional ContrailConfig.networkOverrides with three subfields:
plcUrl, slingshotUrl, additionalAllowedHosts. Plumb config through
resolvePDS, getPDSViaDidDoc (inlined; module-level didResolver removed),
getPDS, resolvePDSCached, getClient, and all five identity.ts functions
(fetchAndSave, resolveIdentity, resolveIdentities, resolveActor,
refreshStaleIdentities). Pass config through backfill.ts call sites
(getClient + getPDS) and the persistent.ts:160 refreshStaleIdentities
call. additionalAllowedHosts is exact + case-insensitive + port-agnostic;
the default SSRF validator still runs for non-listed hosts.

All changes are additive — omitting networkOverrides preserves current
public-internet defaults. validatePdsUrl remains internal (no new
exports beyond the networkOverrides field itself).

* test(contrail-base): network overrides + SSRF regression coverage

Add three test files exercising the networkOverrides plumbing:
- client.test.ts: ContrailConfig.networkOverrides type, validatePdsUrl
regression baseline (public HTTPS accepted; non-HTTPS + private CIDRs
+ localhost + link-local rejected), additionalAllowedHosts allowlist
(allowed host accepted; non-listed hosts still rejected; port-agnostic
match), getClient + getPDS + resolvePDS plumb-through with mocked fetch.
- identity-config.test.ts: resolveIdentity, resolveIdentities, resolveActor,
refreshStaleIdentities all route to override slingshot URL.
- network-overrides.test.ts: integration test against node:http stub PLC
and Slingshot servers. Asserts resolvePDS, getClient, and
refreshStaleIdentities actually hit the configured override URLs end to
end, exercising the full chain.

The SSRF regression cases give the existing validator implicit baseline
explicit coverage in this PR.

* feat(contrail-base): apply networkOverrides.resolver to spaces credential verifier

* feat(contrail-appview): apply networkOverrides to label-endpoint resolution

* fix(contrail-appview): use IF NOT EXISTS on schema migrations to remove init race

Concurrent `initSchema` calls previously raced on `ALTER TABLE ADD COLUMN`
because the SQL had no `IF NOT EXISTS` clause, and the two consumer-site
`try { ... } catch { /* Column already exists — ignore */ }` blocks silently
swallowed *every* DDL error — masking missing tables, syntax errors, and
type mismatches alongside the intended duplicate-column case.

L3 replaces both swallows with a dialect-aware `addColumnIfNotExists` helper:
- Postgres: emits `ALTER TABLE ... ADD COLUMN IF NOT EXISTS` (atomic in PG).
- SQLite: pre-checks `PRAGMA table_info`, then narrowly absorbs only the
literal "duplicate column name" error from the still-possible PRAGMA->ALTER
race. Every other error surfaces.

The `MIGRATIONS` array is now structured `{ table, column, columnDef, target }`
data routed through the same helper. `target: "spaces"` routes to `spacesDb`
when one is configured (split-DB deployments), `target: "feeds"` is gated on
feeds being configured so the absent `feed_backfills` table no longer
silently fails.

Tests at `packages/contrail/tests/schema-idempotency.test.ts` cover:
- sequential and concurrent (3-way) `initSchema` calls
- repeated init does not leave duplicate count columns
- helper is a no-op when column already exists
- helper surfaces "no such table" errors (the L3-removed swallow case)
- `initSchema` propagates errors thrown from extension schema modules

OpenMeet's consumer-side Postgres advisory lock around `contrail.init()`
exists only to work around this race; with L3 in place that lock can be
deleted in a follow-up.

* test: L3 concurrent CREATE race on Postgres

* chore: add changeset for private-network deployment support

* fix(contrail-appview): thread networkOverrides config through all resolver/identity call sites

The networkOverrides config param was threaded into the resolver/identity
functions but dropped at ~8 appview call sites (live-ingest refresh cycle,
on-demand refresh, and router getProfile/getFeed/collection/profiles/notify),
so private-network deploys silently fell back to the public resolver and the
un-widened SSRF guard on those paths.

Pass the in-scope config at each site. Add a regression test driving an
override config through the live-ingest refresh path (runIngestCycle) and a
router actor-resolution path (getProfile), asserting the override slingshot
is hit instead of the default.

* refactor(contrail-base): dedup SSRF validator into shared validateExternalUrl

labels/resolve.ts validateEndpointUrl duplicated contrail-base validatePdsUrl,
and this PR had edited the additionalAllowedHosts allowlist into both copies —
a future SSRF-rule fix applied to one copy would leave the other exploitable.

Export a single validateExternalUrl(url, allowedHosts?) from contrail-base and
consume it from both the PDS client and labeler-endpoint resolution.
validateEndpointUrl stays exported as a thin alias for backward compat.
Behavior is identical (covered by existing SSRF tests).

Also document the PDS resolve cache's process-wide-config assumption (keyed by
DID only; safe now that all callers thread the same config).

* docs(contrail-base): document validateExternalUrl threat-model scope

Note inline that the SSRF guard is best-effort literal-match only: no DNS
resolution, partial IPv6 / encoded-IP coverage, egress network policy
expected for fully untrusted resolver inputs. No behavior change.

another big refactor

I noticed about 1/4 of identities in my app did not have a handle after
backfill. Looks like getPDS over dids run twice over the same set of
dids creates race condition. One is from this loop, and the other getPDS
is invoked from getClient function.

I tested with my set of data and edited variant always gives 100%
of handles in identities table.

fix(spaces): owner-signed enroll, enrollment-only default binding, declaration validation

merge: resolve conflicts with main

Two trust assumptions in the binding layer cannot be closed at the
Contrail layer alone:

- DID-doc service-entry rebind: deployments using
createDidDocBindingResolver inherit PLC's rotation-key authorization
model. Any rotation key on the owner's account can rewrite the
#atproto_space_authority service entry.

- PDS app password held for managed communities: community.provision
and community.adopt hold unscoped app passwords that can write any
record on the user-owned PDS, including the binding declaration
read by createPdsBindingResolver.

Both are constraints of their respective binding sources and
onboarding flows, not bugs. Document each so operators picking a
binding strategy can size them against their own threat model, with
pointers to the canonical issues for the upstream/protocol-level
work that would close them: #38 (DID-doc), #39 (PDS app password).

The PDS binding resolver trusted body.value.authority on any record
returned for the space's URI, with only a startsWith('did:') sniff. A
malformed or attacker-shaped record at the same NSID could rebind
authority — record content alone, no signature verification.

Validate before trusting:
- $type must equal the URI's type (the space-type lexicon embeds the
declaration fields per tools.atmo.space.declaration's contract)
- createdAt must be a string
- authority must match a strict did:plc | did:web regex

Fail closed on any deviation. Adds three negative tests covering each
deviation path.

The default in-process credential verifier composed
[Enrollment, Local], so a never-enrolled space still resolved its
authority via LocalBindingResolver — collapsing the host-side consent
layer the enrollment table is meant to provide.

Default to enrollment-only. createLocalBindingResolver remains exported
for explicit single-tenant wiring via options.credentialVerifier.

The recordHost.enroll endpoint accepted either an owner-signed call OR
a self-attested authority call. The latter let any DID claim to be the
authority for any space's URI and rebind that space's authority.

Drop the self-attesting-authority branch. Require the caller to be the
space owner (sa.issuer === parts.ownerDid). The owner is still free to
designate any authority via body.authority — only the legitimacy of the
caller changes.

Updates the existing test that codified the old behavior to assert the
new rejection, and adds a positive test for the canonical owner-driven
split-deployment enrollment flow.

Main added record-filter, identity-events, follow-feed, and lex-gen-fix
features in `packages/contrail/src/core/*` that PR #30 had reorganized
into the new package split. Each conflict resolved by:

1. Keeping PR #30's re-export shims in `packages/contrail/src/core/*`
(these files now `export * from "@atmo-dev/contrail-appview"`)
2. Porting main's additions to the corresponding files in
`packages/contrail-appview/src/core/*`, which is the new home of the
actual implementation.

Files re-shimmed:
- backfill.ts, db/records.ts, db/schema.ts, identity.ts, jetstream.ts,
persistent.ts, router/feed.ts, types.ts

Ports landed in contrail-appview/src/core (or contrail-base where types
and identity helpers actually live):
- backfill.ts, db/records.ts, db/schema.ts, jetstream.ts, persistent.ts,
router/feed.ts (+1.0K lines from main)
- constellation.ts (new 181-line module)
- contrail-base/src/types.ts: FeedTargetConfig, ConstellationConfig,
buildFeedTargetCaps, normalizeFeedTarget, feedTargetMaxItems,
CollectionConfig.{timeField, subjectField, recordFilter}, and
resolveConfig defaulting `discover: false` for `app.bsky.*`
- contrail-base/src/identity.ts: applyIdentityEvent (handle change handler)

Test mock for `persistent.test.ts` updated to mock
`@atmo-dev/contrail-base` (post-split home of identity helpers) and
include both `refreshStaleIdentities` and `applyIdentityEvent`.

Test status post-merge: 314 passed, 7 failed in
`packages/contrail/tests/search.test.ts` — these 7 failures pre-exist on
plain `origin/main` and are unrelated to the merge.

No behavior changes intended beyond what main and PR #30 already deliver.

Version Packages

test(contrail-e2e): adopt post-PR30 spaces split + community integration

Bring the e2e suite back to green under the package split:

- makeSpacesConfig() wraps the old flat { type, serviceDid, resolver }
shape under authority / recordHost and generates a fresh signing key
per call. The config validator now requires spaces.authority whenever
community is set, so this is the minimum viable shape for any test
that uses spaces.
- setupCommunityContrail() wires the community module via
createCommunityIntegration({ db, config: resolveConfig(...) }) and
passes the result as communityIntegration to the Contrail constructor.
The community config field is now opaque to contrail core; routes are
registered through the integration option.
- 7 test files updated to use the helpers; 4 community tests pick up a
workspace dep on @atmo-dev/contrail-community.

35/35 e2e tests pass against devnet. No source-package changes.

Version Packages

Version Packages

Version Packages

Version Packages