Commits
Signed-off-by: Xe Iaso <me@xeiaso.net>
Instrument the three things an operator needs to see and serve them on a
dedicated listener (-metrics-bind, default :9090; empty disables):
- s3fs: count + latency per S3 API operation (GetObject, PutObject, …) via a
process-level observer set with s3fs.SetMetricsObserver, keeping s3fs free of
any prometheus import.
- git ops: count + latency + in-flight gauge per protocol+service+status,
wrapped once per transport handler (http/git/ssh).
- auth: count + latency per transport+operation+decision, routed through a new
(*daemon).authorize chokepoint that all three transports now call.
Extras: push-hook runs (ok/error/timeout) with latency, repos auto-created
counter, and the default registry's Go-runtime + process collectors.
Per the operator's call there is no repo label (unbounded cardinality); git
ops are keyed by protocol+service+status only.
All vectors live in internal/metrics via promauto on the default registry,
exposed by promhttp.Handler under the existing errgroup Serve/Shutdown idiom.
Hooks now run synchronously and stream stdout/stderr to the pushing client
as "remote: ..." lines over the git sideband progress channel (band 2).
Previously they ran asynchronously after the push response, and output was
captured to slog only — the client never saw it.
To support streaming, fork go-git's transport.ReceivePack (v6) into
cmd/objgitd/receivepack.go as receivePackStreaming(). go-git keeps the
sideband Muxer internal and sends the closing flush-pkt before returning,
so there's no public seam to inject progress. The fork adds one callback
onUpdated(progress io.Writer), invoked after report-status but before the
final flush. When sideband is negotiated, progress writes to band 2
(ProgressMessage); otherwise it's nil and hooks fall back to logging.
HTTP adds a flush-on-write wrapper (flushWriter) to the receive-pack
response so net/http buffering doesn't hold the "remote:" lines back.
git:// and SSH write through live sockets and need no such wrapper.
Remove the now-dead async machinery: hookWG field and its shutdown drain.
Hooks complete before receivePackStreaming returns, so the connection
already holds them until completion.
Update tests to assert hook output appears in the push output (as "remote:"
lines) rather than in logs. Add TestSmartHTTPHookStreams to cover the HTTP
flush path. All three transports (git://, HTTP, SSH) verified.
Note: Clients now wait for hooks to finish (bounded by -hook-timeout,
default 60s) before the push completes, rather than push completing while
hooks run in the background. This is inherent to streaming output live.
Assisted-by: Claude Opus 4.8 via Claude Code
Signed-off-by: Xe Iaso <me@xeiaso.net>
Address code-review feedback:
- Stat the absolute "/test.git/config" path (memfs can resolve
relative vs absolute lookups differently); matches the rest of
the package.
- Close the listener in startSSHServer cleanup as well as the
server, avoiding a listener leak if Cleanup fires before the
concurrent Serve registers the listener.
- Quote the key path in GIT_SSH_COMMAND so paths with spaces are
not mis-split when git execs ssh.
- Rename the TestGitServiceFor loop variable tc -> tt for
package-wide consistency.
Assisted-by: Claude Opus 4.8 via Claude Code
Add comprehensive SSH integration tests covering:
- push creates repo on demand and clone round-trips match HEAD
- push rejected when disabled, repo not created
- clone of missing repository fails
- receive-pack hooks fire asynchronously after push response
Tests drive a real git client over ssh:// against an in-process
SSH server backed by memfs, generating ephemeral ed25519 client
keys and bypassing host key checking for CI environments. Helpers:
gitSSHEnv() configures GIT_SSH_COMMAND and isolation env vars,
gitWithEnv() runs git with custom env (callers can inspect errors),
startSSHServer() creates an in-memory daemon on an ephemeral port.
Assisted-by: Claude Sonnet 4.6 via Claude Code
Signed-off-by: Xe Iaso <me@xeiaso.net>
gliderlabs/ssh already stashes the connecting key for Session.PublicKey(),
so the pubKeyContextKey type and PublicKeyHandler SetValue call were
redundant. Read the key directly in handleSSH. Also relax the command
arity guard to len < 2 and document the intentional protocol-v2 omission.
Adds newSSHServer, handleSSH, and gitServiceFor to ssh.go, wiring
gliderlabs/ssh into the daemon so git clone/push over SSH mirrors
the git:// handler: persistent-stream no-op closers, streamingStorer
for receive-pack, and auth.Authorizer gating via pubKeyContextKey.
Adds TestGitServiceFor covering all valid service names and invalid
inputs.
Signed-off-by: Xe Iaso <me@xeiaso.net>
Replace the allowPush bool gate in resolve with a call to d.authz.Authorize,
extracting HTTP Basic credentials via credFromRequest. Remove the now-redundant
allowPush field from the daemon struct and all daemon literals in tests. Add
TestSmartHTTPAnonymousReadWhilePushDisabled to verify reads succeed when writes
are denied.
Signed-off-by: Xe Iaso <me@xeiaso.net>
Signed-off-by: Xe Iaso <me@xeiaso.net>
Implement automatic hook execution after a successful push: when started with
-allow-hooks, objgitd runs .objgit/hooks/receive-pack from the pushed commit
in a restricted shell (kefka virtual bash). The hook sees a read-only view of
the commit tree at /src and writable scratch at /tmp. Execution is async
post-response so hooks cannot reject a push. Only coreutils commands available
(cat, grep, ls, head, tail, sort, sha256sum, etc.) — no arbitrary binaries,
network, or git command.
Refs before/after push are snapshotted to detect branch changes since
transport.ReceivePack does not report them. One hook runs per created/updated
branch; deletions are skipped. Output and exit status logged to slog only.
New packages:
- internal/treefs: lazy read-only billy.Filesystem view of a git tree
(blobs fetched on open, no checkout to disk)
- internal/mountfs: path-prefix composite FS routing /src and /tmp to
separate mounted filesystems
- internal/kefkash: vendored copy of kefka's billysh handler wiring
(adapted to allow writes so /tmp redirections work)
Includes: daemon integration, flags (-allow-hooks, -hook-timeout), tests
(treefs unit tests, diffRefs, e2e push tests), example hook with regression
test, and CLAUDE.md architecture documentation.
Assisted-by: Claude Opus 4.8 via Claude Code
Signed-off-by: Xe Iaso <me@xeiaso.net>
Store POSIX file attributes (uid, gid, mode, mtime) as x-amz-meta-*
headers on S3 objects so POSIX filesystems layered on Tigris can
round-trip ownership and permissions instead of always reporting
mode 0666 with no owner.
The feature is opt-in and off by default: NewS3FS takes functional
options, and without WithUnixMetadata the filesystem behaves exactly
as before. Callers opt in by passing WithUnixMetadata(uid, gid, umask)
and resolve any name strings to numeric IDs themselves.
New unixmeta package implements Encode/Decode and PosixMode/GoFileMode
conversion; the wire format is documented in docs/reference.
Assisted-by: Claude Opus 4.7 via Claude Code
Signed-off-by: Xe Iaso <me@xeiaso.net>
Captures the non-obvious bits: the shared daemon backing both
transports, why git:// receive-pack hides PackfileWriter to avoid
deadlocking on EOF, why s3fs needs a tempBuffer for go-git's
read-while-write PackWriter, and the suffix-based smart-HTTP dispatch.
Assisted-by: Claude Opus 4.7 via Claude Code
Signed-off-by: Xe Iaso <me@xeiaso.net>
Captures the design decisions and rollout plan for adding HTTP smart
protocol support to objgitd alongside the existing git:// transport.
Signed-off-by: Xe Iaso <me@xeiaso.net>
Move daemon.go and daemon_test.go to git_protocol.go and git_protocol_test.go
to better reflect their purpose. No logic changes.
Signed-off-by: Xe Iaso <me@xeiaso.net>
Implement native git protocol support in objgitd daemon. Enables clients
to use standard git commands (git clone, git fetch) over the git protocol
instead of requiring HTTP. Includes proper request handling and response
formatting for upload-pack operations.
Assisted-by: Claude Haiku 4.5 via Claude Code
Signed-off-by: Xe Iaso <me@xeiaso.net>
Instrument the three things an operator needs to see and serve them on a
dedicated listener (-metrics-bind, default :9090; empty disables):
- s3fs: count + latency per S3 API operation (GetObject, PutObject, …) via a
process-level observer set with s3fs.SetMetricsObserver, keeping s3fs free of
any prometheus import.
- git ops: count + latency + in-flight gauge per protocol+service+status,
wrapped once per transport handler (http/git/ssh).
- auth: count + latency per transport+operation+decision, routed through a new
(*daemon).authorize chokepoint that all three transports now call.
Extras: push-hook runs (ok/error/timeout) with latency, repos auto-created
counter, and the default registry's Go-runtime + process collectors.
Per the operator's call there is no repo label (unbounded cardinality); git
ops are keyed by protocol+service+status only.
All vectors live in internal/metrics via promauto on the default registry,
exposed by promhttp.Handler under the existing errgroup Serve/Shutdown idiom.
Hooks now run synchronously and stream stdout/stderr to the pushing client
as "remote: ..." lines over the git sideband progress channel (band 2).
Previously they ran asynchronously after the push response, and output was
captured to slog only — the client never saw it.
To support streaming, fork go-git's transport.ReceivePack (v6) into
cmd/objgitd/receivepack.go as receivePackStreaming(). go-git keeps the
sideband Muxer internal and sends the closing flush-pkt before returning,
so there's no public seam to inject progress. The fork adds one callback
onUpdated(progress io.Writer), invoked after report-status but before the
final flush. When sideband is negotiated, progress writes to band 2
(ProgressMessage); otherwise it's nil and hooks fall back to logging.
HTTP adds a flush-on-write wrapper (flushWriter) to the receive-pack
response so net/http buffering doesn't hold the "remote:" lines back.
git:// and SSH write through live sockets and need no such wrapper.
Remove the now-dead async machinery: hookWG field and its shutdown drain.
Hooks complete before receivePackStreaming returns, so the connection
already holds them until completion.
Update tests to assert hook output appears in the push output (as "remote:"
lines) rather than in logs. Add TestSmartHTTPHookStreams to cover the HTTP
flush path. All three transports (git://, HTTP, SSH) verified.
Note: Clients now wait for hooks to finish (bounded by -hook-timeout,
default 60s) before the push completes, rather than push completing while
hooks run in the background. This is inherent to streaming output live.
Assisted-by: Claude Opus 4.8 via Claude Code
Signed-off-by: Xe Iaso <me@xeiaso.net>
Address code-review feedback:
- Stat the absolute "/test.git/config" path (memfs can resolve
relative vs absolute lookups differently); matches the rest of
the package.
- Close the listener in startSSHServer cleanup as well as the
server, avoiding a listener leak if Cleanup fires before the
concurrent Serve registers the listener.
- Quote the key path in GIT_SSH_COMMAND so paths with spaces are
not mis-split when git execs ssh.
- Rename the TestGitServiceFor loop variable tc -> tt for
package-wide consistency.
Assisted-by: Claude Opus 4.8 via Claude Code
Add comprehensive SSH integration tests covering:
- push creates repo on demand and clone round-trips match HEAD
- push rejected when disabled, repo not created
- clone of missing repository fails
- receive-pack hooks fire asynchronously after push response
Tests drive a real git client over ssh:// against an in-process
SSH server backed by memfs, generating ephemeral ed25519 client
keys and bypassing host key checking for CI environments. Helpers:
gitSSHEnv() configures GIT_SSH_COMMAND and isolation env vars,
gitWithEnv() runs git with custom env (callers can inspect errors),
startSSHServer() creates an in-memory daemon on an ephemeral port.
Assisted-by: Claude Sonnet 4.6 via Claude Code
Adds newSSHServer, handleSSH, and gitServiceFor to ssh.go, wiring
gliderlabs/ssh into the daemon so git clone/push over SSH mirrors
the git:// handler: persistent-stream no-op closers, streamingStorer
for receive-pack, and auth.Authorizer gating via pubKeyContextKey.
Adds TestGitServiceFor covering all valid service names and invalid
inputs.
Replace the allowPush bool gate in resolve with a call to d.authz.Authorize,
extracting HTTP Basic credentials via credFromRequest. Remove the now-redundant
allowPush field from the daemon struct and all daemon literals in tests. Add
TestSmartHTTPAnonymousReadWhilePushDisabled to verify reads succeed when writes
are denied.
Implement automatic hook execution after a successful push: when started with
-allow-hooks, objgitd runs .objgit/hooks/receive-pack from the pushed commit
in a restricted shell (kefka virtual bash). The hook sees a read-only view of
the commit tree at /src and writable scratch at /tmp. Execution is async
post-response so hooks cannot reject a push. Only coreutils commands available
(cat, grep, ls, head, tail, sort, sha256sum, etc.) — no arbitrary binaries,
network, or git command.
Refs before/after push are snapshotted to detect branch changes since
transport.ReceivePack does not report them. One hook runs per created/updated
branch; deletions are skipped. Output and exit status logged to slog only.
New packages:
- internal/treefs: lazy read-only billy.Filesystem view of a git tree
(blobs fetched on open, no checkout to disk)
- internal/mountfs: path-prefix composite FS routing /src and /tmp to
separate mounted filesystems
- internal/kefkash: vendored copy of kefka's billysh handler wiring
(adapted to allow writes so /tmp redirections work)
Includes: daemon integration, flags (-allow-hooks, -hook-timeout), tests
(treefs unit tests, diffRefs, e2e push tests), example hook with regression
test, and CLAUDE.md architecture documentation.
Assisted-by: Claude Opus 4.8 via Claude Code
Signed-off-by: Xe Iaso <me@xeiaso.net>
Store POSIX file attributes (uid, gid, mode, mtime) as x-amz-meta-*
headers on S3 objects so POSIX filesystems layered on Tigris can
round-trip ownership and permissions instead of always reporting
mode 0666 with no owner.
The feature is opt-in and off by default: NewS3FS takes functional
options, and without WithUnixMetadata the filesystem behaves exactly
as before. Callers opt in by passing WithUnixMetadata(uid, gid, umask)
and resolve any name strings to numeric IDs themselves.
New unixmeta package implements Encode/Decode and PosixMode/GoFileMode
conversion; the wire format is documented in docs/reference.
Assisted-by: Claude Opus 4.7 via Claude Code
Signed-off-by: Xe Iaso <me@xeiaso.net>
Captures the non-obvious bits: the shared daemon backing both
transports, why git:// receive-pack hides PackfileWriter to avoid
deadlocking on EOF, why s3fs needs a tempBuffer for go-git's
read-while-write PackWriter, and the suffix-based smart-HTTP dispatch.
Assisted-by: Claude Opus 4.7 via Claude Code
Signed-off-by: Xe Iaso <me@xeiaso.net>
Implement native git protocol support in objgitd daemon. Enables clients
to use standard git commands (git clone, git fetch) over the git protocol
instead of requiring HTTP. Includes proper request handling and response
formatting for upload-pack operations.
Assisted-by: Claude Haiku 4.5 via Claude Code
Signed-off-by: Xe Iaso <me@xeiaso.net>