contrail/docs/ at changeset-release/main · flo-bit.dev/contrail · Tangled

flo-bit.dev / contrail

0

[READ-ONLY] Mirror of https://github.com/flo-bit/contrail. atproto backend in a bottle flo-bit.dev/contrail/

0

contrail / docs /

at changeset-release/main 1 folder 10 files

Tom Scanlan perf(appview): skip feed-prune sweep on idle ingest ticks 5w ago

small fixes, update starter

2 months ago

perf(appview): skip feed-prune sweep on idle ingest ticks The feed-items prune ran on every ingest tick (cron) and every sweep interval (persistent), walking every actor to issue a cutoff DELETE that almost always deleted nothing. On an operated deployment this no-op sweep was the #1 D1 query by count and runtime (~825k/day, reading ~6.3M rows to delete ~0). It is not an indexing problem; the cost is running the sweep too often. A feed can only exceed its cap right after a feed-mutating record is applied (an event fanning out to followers, or a follow backfilling a feed). Gate the sweep on that: - Cron (runIngestCycle): sweep only when the tick ingested a feed-mutating collection, or a 6h recovery interval has elapsed. The recovery clock is persisted in _contrail_meta since the isolate recycles each tick. - Persistent (streamAndFlush): a state.feedDirty flag is set when a flushed batch contains a feed-mutating record; the sweep is gated on feedDirty or the recovery interval. - getFeedMutatingNsids(config) derives the feed-mutating NSID set from the configured feed targets and follow collections, so the gate cannot drift from what actually writes feed_items. - Recovery completes a full pass per interval (tracking the last completed pass, not a single slice) and runs on no-op notify calls, so over-cap rows from a lowered cap or bulk import still drain with no live activity. Idle deployments drop from a sweep every tick (~1,440/day on a 1-min cron) to ~4/day; correctness is unchanged (feeds are caches, and the recovery interval bounds how long a feed can stay briefly over cap).

1 month ago

update docs

3 months ago

add lexicon endpoint

3 months ago

perf(appview): skip feed-prune sweep on idle ingest ticks The feed-items prune ran on every ingest tick (cron) and every sweep interval (persistent), walking every actor to issue a cutoff DELETE that almost always deleted nothing. On an operated deployment this no-op sweep was the #1 D1 query by count and runtime (~825k/day, reading ~6.3M rows to delete ~0). It is not an indexing problem; the cost is running the sweep too often. A feed can only exceed its cap right after a feed-mutating record is applied (an event fanning out to followers, or a follow backfilling a feed). Gate the sweep on that: - Cron (runIngestCycle): sweep only when the tick ingested a feed-mutating collection, or a 6h recovery interval has elapsed. The recovery clock is persisted in _contrail_meta since the isolate recycles each tick. - Persistent (streamAndFlush): a state.feedDirty flag is set when a flushed batch contains a feed-mutating record; the sweep is gated on feedDirty or the recovery interval. - getFeedMutatingNsids(config) derives the feed-mutating NSID set from the configured feed targets and follow collections, so the gate cannot drift from what actually writes feed_items. - Recovery completes a full pass per interval (tracking the last completed pass, not a single slice) and runs on no-op notify calls, so over-cap rows from a lowered cap or bulk import still drain with no live activity. Idle deployments drop from a sweep every tick (~1,440/day on a 1-min cron) to ~4/day; correctness is unchanged (feeds are caches, and the recovery interval bounds how long a feed can stay briefly over cap).

1 month ago

docs(auth): document binding-layer trust assumptions Two trust assumptions in the binding layer cannot be closed at the Contrail layer alone: - DID-doc service-entry rebind: deployments using createDidDocBindingResolver inherit PLC's rotation-key authorization model. Any rotation key on the owner's account can rewrite the #atproto_space_authority service entry. - PDS app password held for managed communities: community.provision and community.adopt hold unscoped app passwords that can write any record on the user-owned PDS, including the binding declaration read by createPdsBindingResolver. Both are constraints of their respective binding sources and onboarding flows, not bugs. Document each so operators picking a binding strategy can size them against their own threat model, with pointers to the canonical issues for the upstream/protocol-level work that would close them: #38 (DID-doc), #39 (PDS app password).

2 months ago

fix deploy, update docs, update ci

2 months ago

07-communities.md

feat(contrail): community DID provisioning on stock PDS (#31) * feat(contrail-community): provision schema, types, adapter CRUD * feat(contrail-community): PDS + PLC + service-auth helpers * feat(contrail-community): provision orchestrator + XRPC route + session cache * feat(contrail-community): reap CLI for stuck provisioning rows * test+docs(contrail): provision e2e, docs, changeset, deps * fix(contrail-community): harden provision allowlist + reap (PR #31 review) Security/review follow-ups on the community DID provisioning feature: - Provisioning fails closed: with allowProvisioning=true, an empty/undefined allowlist no longer accepts any caller pdsEndpoint (fail-open). Require a non-empty allowlist, or the explicit, loud allowAnyProvisionPdsEndpoint. Renamed allowedPdsEndpoints -> allowedProvisionPdsEndpoints to scope it to provisioning (Contrail still reads/indexes from every PDS). - reap --all-stuck gains a mandatory age floor (listStuckAttempts(olderThanMs) + --older-than <minutes>, default 30) so a bulk run can't tombstone an in-flight provision. --attempt-id still targets a known row regardless. - reap is reachable: ship it as a contrail-community bin (under pnpm the core contrail CLI can't resolve the community package; the dynamic import there now logs instead of an empty catch). Adds Postgres support via --db/DATABASE_URL alongside the D1 binding. - archiveStuckAttempt is now atomic (db.batch) + idempotent (ON CONFLICT DO NOTHING), so a retry after a partial failure can't PK-conflict. - Minor: export bytesToB64url from plc and reuse in service-auth; drop the duplicate reap mutual-exclusion validator. mint-route allowlist gap (pre-existing on main) tracked separately in om-vyok. * test(contrail-community): drop tautological builder-shape assertions Remove the 'buildTombstoneOp produces the expected shape' case and the two pass-through type/prev assertions in the update-op test: both assert literals the builder just set or args just passed in, exercising object-literal semantics rather than runtime behavior. The real signing behavior (signTombstoneOp / signUpdateOp base64url sig) and the live-PLC CID round-trip remain covered. * docs(contrail-community): document SSRF exposure of allowAnyProvisionPdsEndpoint The accept-any escape hatch bypasses endpoint validation: the caller-supplied pdsEndpoint reaches describeServer + createAccount fetches with no private-IP/metadata guard. Note inline that it must be paired with trusted auth + egress network policy + IMDSv2; the default allowlist path contains this by construction. No behavior change.

1 month ago

small fixes, update starter

2 months ago

small fixes, update starter

2 months ago

10-deployment-shapes.md

docs(auth): document binding-layer trust assumptions Two trust assumptions in the binding layer cannot be closed at the Contrail layer alone: - DID-doc service-entry rebind: deployments using createDidDocBindingResolver inherit PLC's rotation-key authorization model. Any rotation key on the owner's account can rewrite the #atproto_space_authority service entry. - PDS app password held for managed communities: community.provision and community.adopt hold unscoped app passwords that can write any record on the user-owned PDS, including the binding declaration read by createPdsBindingResolver. Both are constraints of their respective binding sources and onboarding flows, not bugs. Document each so operators picking a binding strategy can size them against their own threat model, with pointers to the canonical issues for the upstream/protocol-level work that would close them: #38 (DID-doc), #39 (PDS app password).

2 months ago