Commits
Picked up by the new pre-commit hook at ~/.config/git/hooks/pre-commit
which runs `zig fmt --check` on staged .zig files. Pure whitespace /
trailing-comma drift, no behavior changes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Captures the design + lineage in a single doc:
- the three atomic Lua scripts as the entire state machine
- comptime-erased task registration mechanism
- io.async + Future.cancel shutdown semantics (including mid-task
cancellation behavior under the "next cancelation point" rule)
- dual-backend story: same scripts, redis://... vs memory://...
- what's done, what's deferred
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the atomic-stop-flag pattern with proper cancellation:
- `Worker.runForever` is now a normal function that loops calling
`tick()`. when a `tick()` returns `error.Canceled`, the loop exits
cleanly. all other errors are logged-and-continued.
- `tick()` propagates `error.Canceled` through every backend call site
(promoteDueTasks / consumeStream / handleMessage) by switching on
the catch and re-throwing Canceled vs. swallowing other errors.
- `handleMessage` handles mid-task cancellation: a task body whose Io
op returned Canceled propagates up; the worker still calls
`ackMessage(..., "failed")` (the "next cancelation point" rule says
ack runs normally), then re-throws Canceled so `runForever` exits.
- Removed `Worker.stop()` and the `stopping: std.atomic.Value(bool)`
field. Callers use `io.async(Worker.runForever, .{&w})` and
`future.cancel(io)` to drive lifecycle.
Memory backend's `consumeStream` previously ignored `block_ms` and
busy-looped when empty. It now sleeps the configured interval via
`std.Io.sleep`, which is both a cancellation point AND CPU-friendly.
Real redis already blocked via XREADGROUP, so this brings the two
backends into shape parity.
Example updated: spawns `Worker.runForever` via `io.async`, sleeps
10s, calls `future.cancel`. End-to-end runs at ~10.2s wall clock on
both backends with five perpetual ticks observed and clean exit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pulls in the per-command response arena fix from the redis client.
docket's existing integration tests (memory + redis) now run leak-clean
under DOCKET_TEST_REDIS_URL — previously 77 leaks were reported every
redis test pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The only fields on TaskOptions are runtime data (perpetual interval,
future TTLs / retry policies); forcing the whole record to be
comptime meant prefect-server's late_runs couldn't pass an
env-derived interval into Perpetual.every_micros without baking the
value into a comptime literal.
Drops the `comptime` qualifier on the options arg in `register` /
`makeTask`. Erased.invoke is still comptime-generated (it has to be —
it closes over ParamsType + the user fn) — only the data alongside
it relaxes.
5/6 tests pass (memory + redis-gated integration tests unaffected).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Consumers ( prefect-server's late_runs port among them) want to type a
field as `?docket.ExecutionHandle` and there was no re-export at the
docket root — they had to dig into the internal execution_mod.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pulls in the burner-redis commits that add SET/GET/MGET/KEYS/TTL/
EXPIRE/INCR/DECR, SADD/SMEMBERS/SISMEMBER/SREM/SCARD, full list ops,
ZRANGE/ZCOUNT, XREAD/XRANGE/XTRIM/streamLastId/xgroupDestroy,
sweepExpired, plus XAUTOCLAIM for the docket worker orphan-recovery
path filed under task #19.
Memory + redis perpetual smoke tests pass; integration tests 5/6
(redis test still gated on DOCKET_TEST_REDIS_URL).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wires `tangled.org/zzstoatzz.io/burner-redis` (zig 0.16 port of
PrefectHQ/burner-redis, in-process Redis-compatible engine) as
docket's second backend.
src/backend/memory.zig
- MemoryBackend owns a burner.Store + burner.scripting.LuaEngine.
- scheduleTask / promoteDueTasks / ackMessage EVAL the SAME three
Lua scripts the redis backend uses — single source of truth at
src/scripts/*.lua, ports cleanly across backends.
- ensureConsumerGroup + consumeStream skip Lua and call straight
into the store (matches the redis backend's xgroupCreate /
xreadgroup path, which also doesn't go through EVAL).
- read_arena holds the bytes of the most recent consumeStream
result; reset on each call so previous slices stop being valid,
matching the redis client's read-buffer behaviour.
src/backend.zig
- Backend union learns the .memory variant.
- connect() now resolves memory:// URLs to MemoryBackend instances.
tests/integration.zig
- drives Docket end-to-end against both backends through the
perpetual schedule → promote → consume → ack → reschedule cycle.
- memory backend tests run unconditionally (no external services).
- redis backend test gated on DOCKET_TEST_REDIS_URL so CI / local
runs can opt in.
- new `zig build test-integration` step.
Smoke test (manual):
DOCKET_URL=memory://docket-example ./zig-out/bin/perpetual
→ 4 perpetual ticks observed in 10s (50ms interval), clean exit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fills out the skeleton with a working single-worker tick loop:
- composite ops in src/backend/redis.zig: scheduleTask, promoteDueTasks,
ackMessage, consumeStream, ensureConsumerGroup — each backed by an
embedded Lua script (schedule.lua / promote_due.lua / ack.lua) so the
state transitions are atomic.
- Worker.tick: promote due ZSET entries -> XREADGROUP one -> dispatch
via the comptime-erased task shim -> reschedule perpetual BEFORE ack
so a perpetual task never silently dies.
- examples/perpetual: registers a 2s perpetual task, runs 10s, observes
~5 ticks. Used as the v0 smoke gate.
- compose.yml: redis:7-alpine on :6379 for local dev.
The bug that gated this: EVAL returns Lua values as RESP bulk strings,
not simple strings. The first cut only matched .string in the reply
switch, fell through to an error, and the run looked like a hang rather
than a clear failure. stringValue() in backend/redis.zig now accepts
either variant; scheduleTask/ackMessage route through it.
3/3 tests pass; perpetual example produces 5 ticks in 10s as expected.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
scaffolding for the distributed background task system. compiles + 3/3 tests
pass. composite ops (Lua scripts) not yet wired; backend.connect and
docket.add are placeholders.
structure:
- src/docket.zig — Docket struct, register + add public API
- src/task.zig — comptime-erased Task registration (validates signature,
generates JSON-dispatch shim per task)
- src/execution.zig — TaskContext, scheduling primitives
- src/worker.zig — Worker shell with runForever loop placeholder
- src/backend.zig — Backend tagged union, currently redis-only
- src/backend/redis.zig — RedisBackend with redis:// URL parser
- src/scripts/*.lua — schedule + promote_due (stubs for v0 wiring)
deps: zzstoatzz.io/redis (commit 42ef79c), codeberg uuid-zig 0.5.0.
zig 0.16, no cron, no poolio.
Captures the design + lineage in a single doc:
- the three atomic Lua scripts as the entire state machine
- comptime-erased task registration mechanism
- io.async + Future.cancel shutdown semantics (including mid-task
cancellation behavior under the "next cancelation point" rule)
- dual-backend story: same scripts, redis://... vs memory://...
- what's done, what's deferred
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the atomic-stop-flag pattern with proper cancellation:
- `Worker.runForever` is now a normal function that loops calling
`tick()`. when a `tick()` returns `error.Canceled`, the loop exits
cleanly. all other errors are logged-and-continued.
- `tick()` propagates `error.Canceled` through every backend call site
(promoteDueTasks / consumeStream / handleMessage) by switching on
the catch and re-throwing Canceled vs. swallowing other errors.
- `handleMessage` handles mid-task cancellation: a task body whose Io
op returned Canceled propagates up; the worker still calls
`ackMessage(..., "failed")` (the "next cancelation point" rule says
ack runs normally), then re-throws Canceled so `runForever` exits.
- Removed `Worker.stop()` and the `stopping: std.atomic.Value(bool)`
field. Callers use `io.async(Worker.runForever, .{&w})` and
`future.cancel(io)` to drive lifecycle.
Memory backend's `consumeStream` previously ignored `block_ms` and
busy-looped when empty. It now sleeps the configured interval via
`std.Io.sleep`, which is both a cancellation point AND CPU-friendly.
Real redis already blocked via XREADGROUP, so this brings the two
backends into shape parity.
Example updated: spawns `Worker.runForever` via `io.async`, sleeps
10s, calls `future.cancel`. End-to-end runs at ~10.2s wall clock on
both backends with five perpetual ticks observed and clean exit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The only fields on TaskOptions are runtime data (perpetual interval,
future TTLs / retry policies); forcing the whole record to be
comptime meant prefect-server's late_runs couldn't pass an
env-derived interval into Perpetual.every_micros without baking the
value into a comptime literal.
Drops the `comptime` qualifier on the options arg in `register` /
`makeTask`. Erased.invoke is still comptime-generated (it has to be —
it closes over ParamsType + the user fn) — only the data alongside
it relaxes.
5/6 tests pass (memory + redis-gated integration tests unaffected).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pulls in the burner-redis commits that add SET/GET/MGET/KEYS/TTL/
EXPIRE/INCR/DECR, SADD/SMEMBERS/SISMEMBER/SREM/SCARD, full list ops,
ZRANGE/ZCOUNT, XREAD/XRANGE/XTRIM/streamLastId/xgroupDestroy,
sweepExpired, plus XAUTOCLAIM for the docket worker orphan-recovery
path filed under task #19.
Memory + redis perpetual smoke tests pass; integration tests 5/6
(redis test still gated on DOCKET_TEST_REDIS_URL).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wires `tangled.org/zzstoatzz.io/burner-redis` (zig 0.16 port of
PrefectHQ/burner-redis, in-process Redis-compatible engine) as
docket's second backend.
src/backend/memory.zig
- MemoryBackend owns a burner.Store + burner.scripting.LuaEngine.
- scheduleTask / promoteDueTasks / ackMessage EVAL the SAME three
Lua scripts the redis backend uses — single source of truth at
src/scripts/*.lua, ports cleanly across backends.
- ensureConsumerGroup + consumeStream skip Lua and call straight
into the store (matches the redis backend's xgroupCreate /
xreadgroup path, which also doesn't go through EVAL).
- read_arena holds the bytes of the most recent consumeStream
result; reset on each call so previous slices stop being valid,
matching the redis client's read-buffer behaviour.
src/backend.zig
- Backend union learns the .memory variant.
- connect() now resolves memory:// URLs to MemoryBackend instances.
tests/integration.zig
- drives Docket end-to-end against both backends through the
perpetual schedule → promote → consume → ack → reschedule cycle.
- memory backend tests run unconditionally (no external services).
- redis backend test gated on DOCKET_TEST_REDIS_URL so CI / local
runs can opt in.
- new `zig build test-integration` step.
Smoke test (manual):
DOCKET_URL=memory://docket-example ./zig-out/bin/perpetual
→ 4 perpetual ticks observed in 10s (50ms interval), clean exit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fills out the skeleton with a working single-worker tick loop:
- composite ops in src/backend/redis.zig: scheduleTask, promoteDueTasks,
ackMessage, consumeStream, ensureConsumerGroup — each backed by an
embedded Lua script (schedule.lua / promote_due.lua / ack.lua) so the
state transitions are atomic.
- Worker.tick: promote due ZSET entries -> XREADGROUP one -> dispatch
via the comptime-erased task shim -> reschedule perpetual BEFORE ack
so a perpetual task never silently dies.
- examples/perpetual: registers a 2s perpetual task, runs 10s, observes
~5 ticks. Used as the v0 smoke gate.
- compose.yml: redis:7-alpine on :6379 for local dev.
The bug that gated this: EVAL returns Lua values as RESP bulk strings,
not simple strings. The first cut only matched .string in the reply
switch, fell through to an error, and the run looked like a hang rather
than a clear failure. stringValue() in backend/redis.zig now accepts
either variant; scheduleTask/ackMessage route through it.
3/3 tests pass; perpetual example produces 5 ticks in 10s as expected.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
scaffolding for the distributed background task system. compiles + 3/3 tests
pass. composite ops (Lua scripts) not yet wired; backend.connect and
docket.add are placeholders.
structure:
- src/docket.zig — Docket struct, register + add public API
- src/task.zig — comptime-erased Task registration (validates signature,
generates JSON-dispatch shim per task)
- src/execution.zig — TaskContext, scheduling primitives
- src/worker.zig — Worker shell with runForever loop placeholder
- src/backend.zig — Backend tagged union, currently redis-only
- src/backend/redis.zig — RedisBackend with redis:// URL parser
- src/scripts/*.lua — schedule + promote_due (stubs for v0 wiring)
deps: zzstoatzz.io/redis (commit 42ef79c), codeberg uuid-zig 0.5.0.
zig 0.16, no cron, no poolio.