Substack → atproto migration tool#

Move a Substack archive into the open web. Drop your Substack export ZIP, sign in to your atproto account, walk away. Your published posts land in your PDS as portable long-form records — visible across reader apps that consume them.

Primary target: offprint.app. Secondary compatibility with pckt.blog. Leaflet is metadata-level only.

Runs entirely in your browser. Your Substack export and atproto credentials never touch a backend; the only server-side piece is a stateless image proxy that bypasses CORS for Substack's CDN.

Status#

Working in development. All 15 posts of the test fixture migrate cleanly: HTML cleaned, markdown converted to offprint blocks, images resized + uploaded to PDS, records written with the right shapes and scopes. Open question: offprint.app rendering. We write four record types offprint requires (site.standard.publication, site.standard.document, app.offprint.document.article, app.offprint.actor.profile) with the correct shapes and paths — but as of this writing, articles don't yet surface in offprint's UI. Investigation ongoing with the offprint maintainer.

What it does#

Parses your Substack export ZIP (browser-side, via @zip.js/zip.js + papaparse)
Filters to published newsletter posts; refuses subscribers.csv and analytics files for privacy
Converts each post's HTML to GFM markdown via Turndown with Substack-specific cleanup (drops subscribe widgets, share buttons, SVGs, lightbox wrappers; preserves headings, lists, blockquotes, code, footnotes, embeds-as-typed-links)
Resolves the user's atproto OAuth session (PAR + PKCE + DPoP via @atproto/oauth-client-browser)
Rehosts each Substack image through a stateless proxy → resize (1600px max, JPEG Q82, cover image iteratively shrunk to ≤1MB) → uploadBlob to the user's PDS
Transforms the markdown into offprint's content block schema — headings, paragraphs with byte-indexed facets (bold, italic, code, strikethrough, link), bullet/ordered/task lists, blockquotes, horizontal rules, code blocks with language, standalone image blocks
Writes the records to the user's PDS:
- site.standard.publication (one-time per migration — or user picks an existing one created by offprint signup)
- site.standard.document per post (envelope with content: { $type: 'app.offprint.content', items: [...] })
- app.offprint.document.article per post (strongRef pointer offprint's indexer reads)
- app.offprint.actor.profile (presence marker the indexer requires)
Reports per-post status, warnings, downloadable JSON + Markdown summary

Re-runs are idempotent via deterministic rkeys (subst-<post_id>) + listRecords pre-flight. Picking a different publication on re-run retargets existing records.

Design + research#

The architecture rationale, primary-source research, and the journey of discovering offprint's four record-type requirements live here:

Spec / design doc — feature requirements, architecture decisions, open items
Lexicon compatibility research — how we figured out what to write, with citations

Tech stack#

SvelteKit (Svelte 5 runes) on Vercel — single-page app with one server endpoint for the image proxy
Tailwind v4 for styles, editorial / quiet palette
@atproto/api + @atproto/oauth-client-browser for atproto OAuth and record writes
marked for markdown AST → offprint blocks
turndown for HTML → markdown
Browser-native createImageBitmap + OffscreenCanvas + convertToBlob for image resize (no wasm/codec dependency)
vitest + happy-dom for tests (180 unit + snapshot tests as of this writing)

OAuth scopes requested#

Granular per the atproto-oauth profile:

atproto
repo:site.standard.publication?action=create
repo:site.standard.document?action=create&action=update
repo:app.offprint.document.article?action=create&action=update
repo:app.offprint.actor.profile?action=create&action=update
blob:image/*

No transition:generic. Each scope is justified by a specific record-write inventory.

Run locally#

pnpm install
pnpm dev
# opens at http://127.0.0.1:5173/

Note: dev server binds to 127.0.0.1 (not localhost) because the atproto OAuth loopback flow requires the literal IP in redirect_uri. A SvelteKit hook redirects any localhost:5173 request to 127.0.0.1:5173 automatically.

pnpm check    # typecheck (svelte-check)
pnpm test     # vitest run, all suites
pnpm build    # production build (Vercel adapter)

Project structure#

src/
├── lib/
│   ├── auth/                      atproto OAuth client (BrowserOAuthClient)
│   ├── substack/                  ZIP parser
│   ├── transform/                 HTML→Markdown + Markdown→offprint blocks
│   ├── images/                    proxy fetch + decode/resize/uploadBlob
│   ├── atproto/                   record builders + writers
│   └── runner/                    per-post pipeline + checkpoint + circuit breaker
├── routes/
│   ├── +page.svelte               landing (handle input, target picker)
│   ├── migrate/                   ZIP drop, publication picker, progress, report
│   ├── oauth/callback/            OAuth callback handler
│   ├── api/img/                   stateless image proxy (allowlist: substackcdn, substack-post-media, substack.com)
│   ├── oauth-client-metadata.json hosted client metadata for OAuth discovery
│   └── dev/smoke/                 single-post end-to-end test page
└── hooks.server.ts                localhost → 127.0.0.1 redirect in dev

License#

MIT — see LICENSE.

Contributing#

Issues and pull requests welcome. The codebase is mirrored from GitHub to tangled.org automatically; PRs go to GitHub.

Configure Feed