A diet Tap option for handling atproto sync
Readme update
Bailey Townsend
(Jun 30, 2026, 12:26 PM -0500)
8d1658f2
b32e30c1
+74
-49
+64
-47
README.md
+64
-47
README.md
···
1
-
`tab`: atproto sync utility
1
+
`tab`: a diet atproto sync utility
2
2
========================================
3
+
4
+
Tab is a fork of [Tap](https://github.com/bluesky-social/indigo/tree/main/cmd/tap) that brings a few extra features that Tap does not offer. These changes have been PR upstream, but I thought I'd go ahead and fork to make then available to the community and myself.
5
+
6
+
New features:
7
+
- Can use [lightrail](https://tangled.org/microcosm.blue/lightrail) for finding collections. This lets you set multiple signal collections via `TAB_LIGHT_RAIL_SIGNAL_COLLECTIONS`, vs the single collection on tap
8
+
- You can use Jetstream over the firehose for reduce bandwidth by setting `TAB_JETSTREAM_URL`. **NOTE**. The jestream is not fully verifiable and you do lose that verification for the benefit for reduced bandwidth. Can read more on that [here](https://atproto.com/blog/jetstream#tradeoffs-and-use-cases)
9
+
- Note that this is called Ta**b**, not Ta**p**. You will also need to change any environment variables from `TAP_` to `TAB_` when using this fork
10
+
11
+
12
+
13
+
**Docker setup for Tab:**
14
+
15
+
A pre-built Docker image is also available:
16
+
```bash
17
+
docker pull fatfingers23/tab:latest
18
+
docker run -p 2480:2480 fatfingers23/tab:latest
19
+
20
+
# or using atcr.io
21
+
docker pull atcr.io/pds.dad/tab:latest
22
+
docker run -p 2480:2480 atcr.io/pds.dad/tab:latest
23
+
24
+
```
25
+
26
+
To persist data, mount a volume at `/data`:
27
+
```bash
28
+
docker run -p 2480:2480 -v ./data:/data fatfingers23/tab:latest
29
+
```
30
+
31
+
32
+
### Original read me with docs for tap/tab as a whole
3
33
4
34
tab simplifies AT sync by handling the firehose/jetstream connection, verification, backfill, and filtering. Your application connects to a tab and receives simple JSON events for only the repos and collections you care about, no need to worry about binary formats for validating cryptographic signatures.
5
35
···
73
103
74
104
Environment variables or CLI flags:
75
105
76
-
- `tab_DATABASE_URL`: database connection string, SQLite or PostgreSQL (default: `sqlite://./tab.db`)
77
-
- `tab_MAX_DB_CONNS`: maximum number of database connections (default: `32`)
78
-
- `tab_BIND`: HTTP server address (default: `:2480`)
79
-
- `tab_PLC_URL`: PLC directory HTTP/HTTPS URL (default: `https://plc.directory`)
80
-
- `tab_RELAY_URL`: AT Protocol relay HTTP/HTTPS URL (default: `https://relay1.us-east.bsky.network`)
81
-
- `tab_LIGHT_RAIL_URL`: Light Rail HTTP/HTTPS URL (default: ``, but `https://lightrail.microcosm.blue` is a instance you can use)
82
-
- `tab_JETSTREAM_URL`: If set uses the Jetstream over the firehose/relay. Other firehose settings apply to the jetstream connection (default: ``, but `https://jetstream1.us-east.bsky.network` is a public node hosted by Bluesky that you can use)
83
-
- `tab_FIREHOSE_PARALLELISM`: concurrent firehose event processors (default: `10`)
84
-
- `tab_RESYNC_PARALLELISM`: concurrent resync workers (default: `5`)
85
-
- `tab_OUTBOX_PARALLELISM`: concurrent outbox workers (default: `1`)
86
-
- `tab_CURSOR_SAVE_INTERVAL`: how often to persist upstream firehose cursor (default: `1s`)
87
-
- `tab_NO_REPLAY`: skip saved cursor and connect to firehose head on startup; incompatible with `tab_FULL_NETWORK`, not recommended for production (default: `false`)
88
-
- `tab_REPO_FETCH_TIMEOUT`: timeout for fetching repo CARs from PDS (default: `300s`)
89
-
- `tab_IDENT_CACHE_SIZE`: size of in-process identity cache (default: `2000000`)
90
-
- `tab_OUTBOX_CAPACITY`: rough size of outbox before back pressure is applied (default: `100000`)
91
-
- `tab_FULL_NETWORK`: track all repos on the network (default: `false`)
92
-
- `tab_SIGNAL_COLLECTION`: track all repos with at least one record in this collection (e.g. `app.bsky.actor.profile`)
93
-
- `tab_LIGHT_RAIL_SIGNAL_COLLECTIONS`: track all repos with at least one record in these collections. Can not be used with `tab_SIGNAL_COLLECTION` (e.g. `community.lexicon.calendar.rsvp,community.lexicon.calendar.event`)
94
-
- `tab_COLLECTION_FILTERS`: comma-separated collection filters, wildcards accepted (e.g., `app.bsky.feed.post,app.bsky.graph.*`)
95
-
- `tab_DISABLE_ACKS`: fire-and-forget mode, no client acks (default: `false`)
96
-
- `tab_WEBHOOK_URL`: webhook URL for event delivery (disables WebSocket mode)
97
-
- `tab_OUTBOX_ONLY`: run in outbox-only mode (no firehose, resync, or enumeration) (default: `false`)
98
-
- `tab_ADMIN_PASSWORD`: Basic auth admin password required for all requests (if set)
99
-
- `tab_RETRY_TIMEOUT`: timeout before retrying unacked events (default: `60s`)
100
-
- `tab_LOG_LEVEL`: log verbosity (`debug`, `info`, `warn`, `error`, default: `info`)
101
-
- `tab_METRICS_LISTEN`: address for metrics/pprof server (disabled if empty)
106
+
- `TAB_DATABASE_URL`: database connection string, SQLite or PostgreSQL (default: `sqlite://./tab.db`)
107
+
- `TAB_MAX_DB_CONNS`: maximum number of database connections (default: `32`)
108
+
- `TAB_BIND`: HTTP server address (default: `:2480`)
109
+
- `TAB_PLC_URL`: PLC directory HTTP/HTTPS URL (default: `https://plc.directory`)
110
+
- `TAB_RELAY_URL`: AT Protocol relay HTTP/HTTPS URL (default: `https://relay1.us-east.bsky.network`)
111
+
- `TAB_LIGHT_RAIL_URL`: Light Rail HTTP/HTTPS URL (default: ``, but `https://lightrail.microcosm.blue` is a instance you can use)
112
+
- `TAB_JETSTREAM_URL`: If set uses the Jetstream over the firehose/relay. Other firehose settings apply to the jetstream connection (default: ``, but `https://jetstream1.us-east.bsky.network` is a public node hosted by Bluesky that you can use)
113
+
- `TAB_FIREHOSE_PARALLELISM`: concurrent firehose event processors (default: `10`)
114
+
- `TAB_RESYNC_PARALLELISM`: concurrent resync workers (default: `5`)
115
+
- `TAB_OUTBOX_PARALLELISM`: concurrent outbox workers (default: `1`)
116
+
- `TAB_CURSOR_SAVE_INTERVAL`: how often to persist upstream firehose cursor (default: `1s`)
117
+
- `TAB_NO_REPLAY`: skip saved cursor and connect to firehose head on startup; incompatible with `TAB_FULL_NETWORK`, not recommended for production (default: `false`)
118
+
- `TAB_REPO_FETCH_TIMEOUT`: timeout for fetching repo CARs from PDS (default: `300s`)
119
+
- `TAB_IDENT_CACHE_SIZE`: size of in-process identity cache (default: `2000000`)
120
+
- `TAB_OUTBOX_CAPACITY`: rough size of outbox before back pressure is applied (default: `100000`)
121
+
- `TAB_FULL_NETWORK`: track all repos on the network (default: `false`)
122
+
- `TAB_SIGNAL_COLLECTION`: track all repos with at least one record in this collection (e.g. `app.bsky.actor.profile`)
123
+
- `TAB_LIGHT_RAIL_SIGNAL_COLLECTIONS`: track all repos with at least one record in these collections. Can not be used with `TAB_SIGNAL_COLLECTION` (e.g. `community.lexicon.calendar.rsvp,community.lexicon.calendar.event`)
124
+
- `TAB_COLLECTION_FILTERS`: comma-separated collection filters, wildcards accepted (e.g., `app.bsky.feed.post,app.bsky.graph.*`)
125
+
- `TAB_DISABLE_ACKS`: fire-and-forget mode, no client acks (default: `false`)
126
+
- `TAB_WEBHOOK_URL`: webhook URL for event delivery (disables WebSocket mode)
127
+
- `TAB_OUTBOX_ONLY`: run in outbox-only mode (no firehose, resync, or enumeration) (default: `false`)
128
+
- `TAB_ADMIN_PASSWORD`: Basic auth admin password required for all requests (if set)
129
+
- `TAB_RETRY_TIMEOUT`: timeout before retrying unacked events (default: `60s`)
130
+
- `TAB_LOG_LEVEL`: log verbosity (`debug`, `info`, `warn`, `error`, default: `info`)
131
+
- `TAB_METRICS_LISTEN`: address for metrics/pprof server (disabled if empty)
102
132
103
133
## Delivery Modes
104
134
···
106
136
107
137
**WebSocket with acks** (default): Client sends acks each event once it has been processed/persisted. Ensures that no data is lost and client does not need to handle cursors. It's recommended to use a client library such as [@atproto/tab](https://github.com/bluesky-social/atproto/tree/main/packages/tab/README.md) when using this mode.
108
138
109
-
**Fire-and-forget**: Set `tab_DISABLE_ACKS=true`. Events are sent and considered "acked" once the client receives them. Simpler but may result in data loss. Recommended for testing purposes or when data integrity is not critical.
139
+
**Fire-and-forget**: Set `TAB_DISABLE_ACKS=true`. Events are sent and considered "acked" once the client receives them. Simpler but may result in data loss. Recommended for testing purposes or when data integrity is not critical.
110
140
111
-
**Webhook**: Set `tab_WEBHOOK_URL=http://...`. Events are POSTed as JSON. Events considered "acked" once the webhook responds with a 200. Recommended for lower throughput serverless environments.
141
+
**Webhook**: Set `TAB_WEBHOOK_URL=http://...`. Events are POSTed as JSON. Events considered "acked" once the webhook responds with a 200. Recommended for lower throughput serverless environments.
112
142
113
143
114
144
## Network Boundary Modes
···
117
147
118
148
**Dynamically Configured** (default): tab starts out tracking no repos. Specific repos can then by added via `/repos/add` and removed via `/repos/remove`.
119
149
120
-
**Collection Signal**: Set `tab_SIGNAL_COLLECTION=com.example.nsid`. Track all repos that have at least one record in the specified collection. Many applications create a "declaration" or "profile" in a repo when that repo uses that application
150
+
**Collection Signal**: Set `TAB_SIGNAL_COLLECTION=com.example.nsid`. Track all repos that have at least one record in the specified collection. Many applications create a "declaration" or "profile" in a repo when that repo uses that application
121
151
122
-
**Full Network**: Set `tab_FULL_NETWORK=true`. Enumerates and tracks all findable repos on the entire network. Resource-intensive and takes days/weeks to complete backfill.
152
+
**Full Network**: Set `TAB_FULL_NETWORK=true`. Enumerates and tracks all findable repos on the entire network. Resource-intensive and takes days/weeks to complete backfill.
123
153
124
154
## Collection Filtering
125
155
126
156
After narrowing down the network to a subset of repos, tab can further filter records down to a specified set of collections. Filters apply to record events only. Identity events are always delivered for tracked repos.
127
157
128
-
If you are interested syncing all of a single record type, it is important to specify that collection as both the signal collection and the filter collection. For example: `tab_SIGNAL_COLLECTION=com.example.nsid tab_COLLECTION_FILTERS=com.example.nsid`
158
+
If you are interested syncing all of a single record type, it is important to specify that collection as both the signal collection and the filter collection. For example: `TAB_SIGNAL_COLLECTION=com.example.nsid TAB_COLLECTION_FILTERS=com.example.nsid`
129
159
130
160
Collection filters use wildcards but only at the period breaks in NSIDs. For example:
131
161
132
-
`tab_COLLECTION_FILTERS=app.bsky.feed.post,app.bsky.graph.*`
162
+
`TAB_COLLECTION_FILTERS=app.bsky.feed.post,app.bsky.graph.*`
133
163
134
164
135
165
## Event Format
···
208
238
209
239
## Authentication
210
240
211
-
If exposing tab to the internet, you should set `tab_ADMIN_PASSWORD` to require authentication for all API requests.
241
+
If exposing tab to the internet, you should set `TAB_ADMIN_PASSWORD` to require authentication for all API requests.
212
242
213
243
tab uses HTTP Basic authentication with the username `admin`. Basic auth works by concatenating the username and password with a colon (`admin:yourpassword`), then base64-encoding the result. This is sent in the `Authorization` header as `Basic <encoded-value>`.
214
244
···
239
269
```bash
240
270
go build -o tab ./cmd/tab
241
271
./tab run
242
-
```
243
-
244
-
**Docker:**
245
-
246
-
A pre-built Docker image is also available:
247
-
```bash
248
-
docker pull ghcr.io/bluesky-social/indigo/tab:latest
249
-
docker run -p 2480:2480 ghcr.io/bluesky-social/indigo/tab:latest
250
-
```
251
-
252
-
To persist data, mount a volume at `/data`:
253
-
```bash
254
-
docker run -p 2480:2480 -v ./data:/data ghcr.io/bluesky-social/indigo/tab:latest
255
272
```
256
273
257
274
The Dockerfile is included in the repo at `cmd/tab/Dockerfile` if you need to build your own image.
+2
-2
cmd/tab/Dockerfile
+2
-2
cmd/tab/Dockerfile
···
31
31
32
32
CMD ["/tab", "run"]
33
33
34
-
LABEL org.opencontainers.image.source=https://github.com/bluesky-social/indigo
35
-
LABEL org.opencontainers.image.description="AT Protocol Sync Utility (tab)"
34
+
LABEL org.opencontainers.image.source=https://tangled.org/pds.dad/tab
35
+
LABEL org.opencontainers.image.description="A Diet AT Protocol Sync Utility (tab)"
36
36
LABEL org.opencontainers.image.licenses=MIT