at main
8 folders
22 files
knotserver: fix SigV4 canonical-path encoding in clone-perf diagnostic S3 probe
The R2 egress probe (r2Get/r2RangeGet) signed the canonical request with Go's
url.EscapedPath(), which leaves ':' literal in non-first path segments. R2, like
S3, percent-encodes the path for the SigV4 canonical request, so did:plc:<id>
repo keys must sign as did%3Aplc%3A<id> — the literal-colon mismatch produced a
SignatureDoesNotMatch 403, leaving container->R2 egress latency unmeasured.
- canonicalPath now AWS-URI-encodes the path (unreserved + '/' pass through,
everything else %XX uppercase), matching what R2 echoes in its 403 body.
- probeS3Get captures a bounded prefix of non-2xx response bodies into the error
field, so future signing/scope failures are self-diagnosing instead of a bare
'http status 403'.
- unit tests lock in the did:plc colon-encoding and the empty-path case.
Verified live against knot.solpbc.org: r2Get 200 (5.18MB, ~22-30 MB/s),
r2RangeGet 206 (~78ms round-trip floor, stable warm/cold).
Patch category: aerie-distribution
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
feat(knotserver): threshold-gate and local-stage post-receive maintenance
Post-receive maintenance previously ran `git repack -a -d
--write-bitmap-index` plus `commit-graph write --reachable` in place over
the network-backed FUSE mount on every successful push, blocking the push
handler for minutes even on trivial pushes to large repos.
Gate the work behind a cheap check run on every push: a single
objects/pack listing yields pack count and bitmap presence, plus a
bounded, early-exiting loose-object scan. Repack only when pack count or
loose count crosses a threshold, or when a non-empty repo has no bitmap
(the bitmap floor keeps clones fast and is not defeatable by high
thresholds). A sub-threshold push does nothing beyond the gate.
When the gate fires, stage the repack on ephemeral disk: copy the bare
repo to a per-run dir under the configured stage dir, repack and write the
commit-graph on the copy, then write back additively. New packs install
companion-first with the .idx last so a reader never sees a pack without
its index; the commit-graph installs via temp-file rename. Only
snapshotted old packs are pruned, .idx first, and only after every new
file installs, so an interruption or concurrent push never leaves the live
repo invalid and loose objects are never removed. Pack files are
content-addressed, so same-named live files are skipped rather than
rewritten.
Add a KNOT_MAINTENANCE_ config section (loose threshold, pack threshold,
stage dir) threaded into the maintainer, create the stage dir in the
image, and keep the call synchronous with the per-repo lock.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Patch category: aerie-distribution