Commits
The ping round-trip test compared two Ping.t with (=), which the house
rule rejects: nothing stops the polymorphic compare being pointed at a
type that cannot support it.
Export Ping.equal instead. A ping is a flat record of four strings and a
bool -- no floats, functions or cycles -- so structural equality is the
equality here, and defining it as a monomorphic (=) keeps it right when
a header field is added, which a hand-written conjunction would not.
bytesrw resolves only from the local opam overlay, so a fresh switch
picks up the published version instead of the commit docker-server is
built and tested against. Declare the pin already used tree-wide.
The eio facade carried two complete client APIs -- the session verbs and a One module duplicating every one of them -- so a caller had to know which of the two to reach for, and opening Requests_eio dumped thirty-two modules into scope, most of them pure vocabulary with no caller. The low-level driver added six request_ names fanning a single function over three optional axes, ending in request_100_continue_stream_decompress.
There is now one verb set, entered through run, which opens the switch and hands back a session; a one-shot is a session that ends with the scope. The driver is Requests_eio_h1.request, one name taking the axes as optional arguments and always streaming, so callers who want a string drain it. The facade carries the thirteen modules callers reach for, ordered session, core types, behaviour, low-level; the rest stay in the pure core. Routing the buffered callers onto the streaming path surfaced a live bug: a fixed-length body that ends early read back as complete, because sub stops at end of input, so a peer promising more than it sends went unnoticed.
Move the server's I/O entry points to respond-eio and take the hijacked
exec/attach connection as a Bytesrw writer rather than the old Hijack callbacks;
declare the new respond-eio and bytesrw deps.
The listener only recognised 0.0.0.0 / 127.0.0.1 / localhost; through
Ipaddr.of_string it takes any literal address, IPv6 included, and crosses
to Eio via Eio_addr.tcp.
Seeding inside tls_config tied the generator's lifetime to one caller. Seed
once at the entry.
eio moves to a dev commit: it adds portable Net socket options, so a
consumer can set TCP_NODELAY and friends without linking eio.unix. Every package
that depends on eio carries the pin, since opam resolves pin-depends per package
and not from the root.
The solo5 and unikraft cross-toolchains move to wip-macos+runtime_events+lto,
one branch carrying macOS cross-build support, an in-process runtime_events ring
and a flambda compiler that defaults to link-time DCE. The unikernel arms link
with -use-lto, so the compiler has to carry those patches.
requests.h1 Http_read.response now takes a bytesrw Bytes.Reader.t rather
than an Eio.Buf_read.t; build the reader with Http_read.of_flow. The body
cap that was Buf_read's max_size is now the response ~limits.
Cross-check the proxy's hand-rolled request-head parser against the
strict RFC 9112 parser in nox-http: for every generated request the
proxy must never trust an n-byte body a strict parser frames differently
(the smuggling root), and must refuse the ambiguous framings that parser
rejects. Exposed via Proxy.classify.
The fuzz immediately found the gap it was built for: content_length used
int_of_string, which accepts 0x10, 5_0 and +5 -- none valid HTTP -- so
the proxy trusted a length the daemon would reject. Content-Length is
now parsed as a single run of ASCII digits, and a malformed or
comma-list value is refused as invalid framing alongside the TE+CL and
conflicting-length checks.
Two more request-framing hardening steps on the proxy's parser:
- Fold the ambiguous-framing check into framing_error: as well as
Transfer-Encoding with Content-Length, two Content-Length fields with
differing values (RFC 9112 section 6.3.5) are refused with 400, so a
longer body cannot hide behind a shorter declared length the policy
inspects.
- read_head is now bounded by a wall-clock deadline when serve is given
a clock: a client that dribbles a partial head and never completes it
is refused with 408 after head_timeout (default 30s) instead of
holding a fiber open. dockr passes the environment clock, so the
shipped proxy has the deadline on.
RFC 9112 section 6.3.5: two Content-Length header fields with differing
values are invalid framing. Drive a create with Content-Length: 0 and a
second Content-Length covering a privileged body; assert the proxy
answers 4xx and never forwards it. Fails today: content_length reads the
first value only, so the body is classified benign and forwarded.
Two request-parsing weaknesses let a client bypass the proxy's policy or
exhaust it:
- The body the policy inspects was taken from Content-Length only, so a
create with both Content-Length: 0 and Transfer-Encoding: chunked was
classified as an inspectable empty body while its real (chunked) body
-- a privileged create -- was forwarded to the daemon uninspected. A
request presenting both framing headers is now refused with 400 before
policy evaluation (RFC 9112 section 6.1). Transfer-Encoding-only
requests already fail closed (no Content-Length means body = None).
- read_head accumulated header lines with no aggregate bound. It now
caps the head at max_request_headers lines (default 100) and
max_request_head_bytes total (default 64 KiB), refusing a larger head
with 431 instead of buffering it without limit.
Two driven tests against Server.Proxy.serve. The first sends a create
with both Content-Length: 0 and Transfer-Encoding: chunked and a chunked
privileged body, asserting the proxy answers 4xx and the upstream never
receives the privileged bytes (RFC 9112 section 6.1 ambiguous framing).
The second floods the request head past any limit, asserting a 431. Both
fail today: the proxy inspects only Content-Length so the smuggled body
is forwarded, and read_head accumulates headers without bound.
no_privileged blocks only HostConfig.Privileged; host mounts,
capabilities, host networking, host namespaces, and device access still
pass, so a reader could mistake it for a container-isolation boundary.
Add a scope note to proxy.mli: the built-in policies are a coarse
allow/deny/rewrite filter, and real isolation needs an explicit
allow-list (a Rego policy over the create body) or a restricted daemon.
no_privileged and unprivilege only acted when the create body had been
buffered (Content-Length within max_inspect_body); a chunked or oversized
body reached the policy as None and was forwarded, so a privileged create
slipped through simply by being uninspectable. Deny a POST
*/containers/create whose body is None in both policies.
Across the packages monopam lint flagged: declare the libraries each
package actually uses (test-only ones scoped with-test, dev tools and
benches scoped with-dev-setup), drop the opam deps and dune (libraries)
entries nothing references, declare pin-depends for the wire and zarith
git pins so a standalone opam install can resolve them, and point the
five stale dune-project source URLs at the names sources.toml already
uses. The generated .opam files are regenerated to match.
Where monopam lint and merlint's project rules disagree, merlint wins:
[protocol] packages keep probe linked (E932), CSV-trace interop tests
keep csv (E825), and dev-scope deps carry the right filter (E943).
The few dead-lib reports that remain are monopam lint blind spots, not
real dead deps: usage reached only through an mdx README block (slirp),
through copy_files# (vz), or inside a library's own private modules
(builder), plus a tar gz dep kept live for a stub test still to write.
The docker library is Engine API types and codecs with no state
machine, so the protocol tag (E946: a state machine over a codec) was
wrong. Tag it codec.text.
Remove libraries that no module in the stanza directory
references (monopam lint dead-lib).
Tree-wide mechanical migration (Digestif.* -> Hash.*, dune and opam
deps digestif -> ocaml-hash) across 43 packages, dropping digestif
from the dependency graph entirely (root.opam no longer lists it).
Blob digests, git object ids, KEX exchange hashes, JWT/COSE/X.509
signatures and every other SHA-256 user now ride the hardware path on
CPUs that have it.
Five libraries had an internal hash.ml that would shadow the Hash
library inside their own compilation units; their internal module is
renamed by what it actually is, with the public alias kept so
consumers (Irmin.Hash, Git.Hash, Tpm.Hash, ...) are unchanged:
- irmin, ocaml-git: Object_hash (content-addressed object ids)
- ocaml-scitt, ocaml-sigstore: Hash_alg (algorithm registries)
- ocaml-tpm: Hashing (digest/HMAC/KDFa over TPM algorithms)
Semantic adjustments the rename surfaced:
- x509: the OID<->hash tables lose their arms for algorithms the
hash library does not carry (SHA-3, BLAKE2, RIPEMD, WHIRLPOOL);
Hash.kind is closed over the six supported algorithms so the
catch-all arms were dead.
- sigstore: bundles declaring SHA-3 digests now fail verification
loudly (Invalid_argument) instead of mapping to an unimplemented
kind; the protobuf enum still parses.
- x509/jwt: local [let module Hash = (val module_of ...)] bindings
renamed to H -- they would shadow the library inside their scope.
- irmin re-exports the raw SHA-1/SHA-256 implementations through
Object_hash so its schema instantiations need no second name for
the library under the public Hash alias.
SSH throughput is unchanged (170/537/460 MB/s socketpair / 2-domain /
2-process, same as the digestif baseline): the negotiated transport
cipher is chacha20-poly1305, whose MAC is Poly1305, so SHA-256 only
runs in the handshake. SSH's data-path ceiling remains nox-crypto's
scalar ChaCha20/Poly1305.
All migrated package suites pass (ssh, tls, x509, paseto, jwt, kdf,
git, tpm, scitt, sigstore, oci, srp, spake2, cose, bpsec, hash).
Comment reflow and header reindent produced by the new ocamlformat;
no code changes.
Move the client, proxy and serve Cmdliner command builders out of
the dockr entry point into bin/cmd_client.ml, bin/cmd_proxy.ml and
bin/cmd_serve.ml, leaving dockr.ml as a thin command list.
docker-runner was its own opam package but is I/O-free (deps: docker + fmt) like
the main docker types, so a separate package only paid off for a future real
backend depending on just the interface -- which doesn't exist. Fold it in:
lib/runner.ml is now the Docker.Runner module (BACKEND renamed to the idiomatic
S), the docker-runner package/opam are dropped, and its test moves to
test/test_runner.ml.
Merging the three binaries into one dockr also pulled docker-client and
nox-crypto-rng (which the docker-server library does not use) into a
library-bearing package; move the binary into its own executable-only dockr
package (the etco pattern), so docker-server keeps only its real library deps
(docker-client returns to with-test for the tests). Net: still 4 packages
(docker, docker-client, docker-server, dockr), but each carries honest deps.
Consumers use Docker.Runner.S / Docker.Runner.Memory; 65 + e2e tests pass.
Three separate executables (dockrd, dockr-proxy, dockr) collapse into a single
dockr tool with subcommands, mirroring etco: dockr serve (--fake in-memory
daemon), dockr proxy (content-aware gateway / --authz plugin), and the client
verbs version/ping/info. Drop the eio_main/cmdliner deps from docker-client
(the CLI moved to the docker-server-owned binary) and make docker-client a
plain dep of docker-server. README and the docker_runner doc updated to the
new command surface. All 12 e2e/proxy tests pass.
Several suites wrote scratch files rooted at the process cwd
(Eio.Stdenv.cwd / Sys.getcwd), so running them from the repo root left
artifacts behind:
- ocaml-tty: the eio "follow" display test wrote a .log file
- ocaml-parquet: the eio pread test wrote a .parquet file
- ocaml-docker: the server tests bound Unix sockets
Root each in the system temp dir instead, so the tests are
self-contained and leave nothing in the repository.
Match the house convention: if opam cannot find the packages from the public
opam-repository, add the gazagnaire.org overlay repo, opam update, then install.
Declare nox-x509 and ptime as with-test deps of docker-server (the TLS proxy
test uses them but they were never listed), and drop the unused fmt dep from
the dockerd interop test, whose sources never reference Fmt. monopam lint is
now clean for the subtree.
Restructure the README to the standard section order (description, install,
usage, API, tests, license) and add the missing Install and License sections.
The Usage section now opens with a compiling example of the IO-free core
(decode a /version body), wired into MDX so dune runtest type-checks it and
diffs its output; the shell and CLI blocks are marked $MDX skip.
Also refresh the package synopses/descriptions, which had gone stale: docker
now lists the full endpoint surface, docker-server mentions Proxy/Authz, and
docker-runner drops the speculative "runc or a VMM later" wording.
Replace every em-dash with a colon, comma, semicolon or full stop, make the
state-transition arrow ASCII, and break up the one tricolon. Bullet labels
now lead with a bold full stop or a colon.
Replace the stale 9-endpoint list with the actual served surface (system,
containers, exec, images/networks/volumes) and add a section on the streaming
responses (logs/stats/events, stdcopy framing) and the hijacked attach/exec
connections (101 UPGRADED via Respond.hijack), including how the proxy
byte-splices upgrades transparently. Drop the "seam" wording.
Drop the forward-looking "native Docker / VMM in minimal-init VMs" aim from
the intro and the stale follow-up list; describe only what the package is and
serves today. The container backend note now just says a real-process backend
sits behind the same BACKEND seam.
Add a Proxy section covering Server.Proxy / dockr-proxy: the content-aware
gateway model, the built-in policies (allow-all/read-only/no-privileged/
unprivilege/rego), TLS termination, and the --authz in-daemon plugin mode.
Note the proxy/authz coverage in the package table and the test list.
Expose the full policy surface from the binary: --rego-file loads an OPA/Rego
policy (overriding the named --policy), and --authz runs as a dockerd
authorization plugin on --listen (Server.Authz) instead of a forwarding
proxy. So the Rego adapter and the in-daemon plugin are usable straight from
the CLI.
Add Server.Authz, the Docker authorization-plugin protocol decided by the
same Proxy.policy as the standalone proxy. Registered with `dockerd
--authorization-plugin`, it answers the plugin handshake (Plugin.Activate ->
implements authz) and the AuthZPlugin.AuthZReq/AuthZRes RPCs: it base64-
decodes the daemon's RequestBody, turns the call into a Proxy.request, and
returns {Allow,Msg} from the policy (a Rewrite is treated as allow, since the
AuthZ protocol cannot modify the request). So no_privileged, read_only, and
the Rego adapter all run BOTH as a standalone gateway and in-daemon -- the
opa-docker-authz integration model over our policy surface.
Test (test_authz.ml): drive the plugin over a socket -- the handshake
advertises authz, and the no_privileged policy denies a privileged create
and allows a plain one.
Concurrent proxy/server changes committed as-is; not authored in this session.
Add an opt-in (DOCKER_LIVE=1) full-lifecycle case to the real-daemon client
test: pull a tiny image, run a container, and exercise logs/exec/top/stats
plus network and volume CRUD, all under uniquely-named throwaway resources
removed in a finally block. It validates wire-compatibility with stock
dockerd -- exec returns real stdout through the hijack + stdcopy path, and
the daemon's full stats object decodes via skip_unknown. DOCKER_SOCK
overrides the socket (e.g. to point at a proxy).
This caught a real interop bug the fake masked: the daemon requires at least
one stream on GET /containers/{id}/logs ("you must choose at least one
stream", 400). The client now sends ?stdout=1&stderr=1 (and attach sends
?stream=1&stdout=1&stderr=1), and the fake server enforces the same contract
so it stops masking it.
Serve the classic POST /build via a new BACKEND.image_build seam: the tar
context is not interpreted (the in-memory backend cannot run steps), so a
new image is registered under the requested tag and the usual progress
stream returned. Client gains Image.build (empty body, since the server
ignores the context). Our client builds a tagged image end to end and finds
it in the image list. The stock CLI defaults to BuildKit, a separate
protocol, so it is not used as a cross-check for this legacy endpoint.
Add Docker.Stats (the cpu/memory/pids counters the CLI reads, nested under
cpu_stats/memory_stats/pids_stats) and serve GET /containers/{id}/stats via
a new BACKEND.stats seam: a real backend samples the cgroup, the in-memory
backend reports zeros (404 when unknown). A client reading one object then
EOF (the --no-stream case) gets exactly this sample. Client gains
Container.stats. Real `docker stats --no-stream` names the container against
--fake, and our client reads the zeroed sample back.
Add Docker.Event (one JSON object per line, the id under Actor.ID) and serve
GET /events. The Memory backend records the container lifecycle it simulates
(create/start/stop/kill/restart/pause/unpause/destroy) and a new
BACKEND.events replays the log; a real daemon would hold the connection open
and push live events, so the in-memory backend returns its recorded log as a
finite stream. Client gains System.events. Our client reads the log back and
sees the create/start/destroy actions it drove.
Serve POST /containers/{id}/attach over the hijack primitive: upgrade to
the multiplexed stream and write the container's stdout/stderr (empty in
--fake), 404 when unknown. The client gains Container.attach; the exec and
attach clients now share the hijacked_stdout helper (run the upgrade, demux
stdout). Our client drives attach against a running --fake container.
Add Docker.Exec (create config/response, start, inspect detail) and the
exec endpoints over the new respond hijack primitive: POST
/containers/{id}/exec creates an instance (404 unknown, 409 not running),
POST /exec/{id}/start hijacks the connection and writes a 101 upgrade then
the multiplexed output, and GET /exec/{id}/json inspects it. In --fake no
process runs, so an exec exits 0 with empty output. BACKEND grows
exec_create/start/inspect; Memory tracks instances. The client gains an Exec
module that performs the raw upgrade itself and demultiplexes stdout. Real
`docker exec` round-trips against --fake, and our own client drives the same
create/start/inspect path (plus the 409 on a stopped container).
Add Image.History, the layer-history entry, and serve
GET /images/{name}/history via a new BACKEND.image_history seam: a real
runtime reads the image config's layer chain, while the in-memory backend
stores no layers and reports its single recorded metadata entry (404 when
unknown). Client gains Image.history. Real `docker history` round-trips
against --fake.
Add Docker.Change, the filesystem-diff entry (Path plus a typed Modified/
Added/Deleted Kind), and serve GET /containers/{id}/changes via a new
BACKEND.changes seam: a real runtime diffs the container against its image,
while the in-memory backend has no writable layer and so reports nothing
(404 when the container is unknown). Client gains Container.changes. Real
`docker diff` round-trips against --fake.
Add Docker.Stdcopy, the multiplexed stdout/stderr stream framing (8-byte
header: stream byte, three zeros, big-endian payload length) used by the
logs and attach streams, with frame/demux unit-tested on known vectors.
Serve GET /containers/{id}/logs: any existing container's logs are a
multiplexed stream (404 when unknown), empty in --fake since no process
runs, carrying the stream content type so clients demux it. Client gains
Container.logs, which demuxes and returns stdout. Real `docker logs`
round-trips against --fake.
The ping round-trip test compared two Ping.t with (=), which the house
rule rejects: nothing stops the polymorphic compare being pointed at a
type that cannot support it.
Export Ping.equal instead. A ping is a flat record of four strings and a
bool -- no floats, functions or cycles -- so structural equality is the
equality here, and defining it as a monomorphic (=) keeps it right when
a header field is added, which a hand-written conjunction would not.
The eio facade carried two complete client APIs -- the session verbs and a One module duplicating every one of them -- so a caller had to know which of the two to reach for, and opening Requests_eio dumped thirty-two modules into scope, most of them pure vocabulary with no caller. The low-level driver added six request_ names fanning a single function over three optional axes, ending in request_100_continue_stream_decompress.
There is now one verb set, entered through run, which opens the switch and hands back a session; a one-shot is a session that ends with the scope. The driver is Requests_eio_h1.request, one name taking the axes as optional arguments and always streaming, so callers who want a string drain it. The facade carries the thirteen modules callers reach for, ordered session, core types, behaviour, low-level; the rest stay in the pure core. Routing the buffered callers onto the streaming path surfaced a live bug: a fixed-length body that ends early read back as complete, because sub stops at end of input, so a peer promising more than it sends went unnoticed.
eio moves to a dev commit: it adds portable Net socket options, so a
consumer can set TCP_NODELAY and friends without linking eio.unix. Every package
that depends on eio carries the pin, since opam resolves pin-depends per package
and not from the root.
The solo5 and unikraft cross-toolchains move to wip-macos+runtime_events+lto,
one branch carrying macOS cross-build support, an in-process runtime_events ring
and a flambda compiler that defaults to link-time DCE. The unikernel arms link
with -use-lto, so the compiler has to carry those patches.
Cross-check the proxy's hand-rolled request-head parser against the
strict RFC 9112 parser in nox-http: for every generated request the
proxy must never trust an n-byte body a strict parser frames differently
(the smuggling root), and must refuse the ambiguous framings that parser
rejects. Exposed via Proxy.classify.
The fuzz immediately found the gap it was built for: content_length used
int_of_string, which accepts 0x10, 5_0 and +5 -- none valid HTTP -- so
the proxy trusted a length the daemon would reject. Content-Length is
now parsed as a single run of ASCII digits, and a malformed or
comma-list value is refused as invalid framing alongside the TE+CL and
conflicting-length checks.
Two more request-framing hardening steps on the proxy's parser:
- Fold the ambiguous-framing check into framing_error: as well as
Transfer-Encoding with Content-Length, two Content-Length fields with
differing values (RFC 9112 section 6.3.5) are refused with 400, so a
longer body cannot hide behind a shorter declared length the policy
inspects.
- read_head is now bounded by a wall-clock deadline when serve is given
a clock: a client that dribbles a partial head and never completes it
is refused with 408 after head_timeout (default 30s) instead of
holding a fiber open. dockr passes the environment clock, so the
shipped proxy has the deadline on.
RFC 9112 section 6.3.5: two Content-Length header fields with differing
values are invalid framing. Drive a create with Content-Length: 0 and a
second Content-Length covering a privileged body; assert the proxy
answers 4xx and never forwards it. Fails today: content_length reads the
first value only, so the body is classified benign and forwarded.
Two request-parsing weaknesses let a client bypass the proxy's policy or
exhaust it:
- The body the policy inspects was taken from Content-Length only, so a
create with both Content-Length: 0 and Transfer-Encoding: chunked was
classified as an inspectable empty body while its real (chunked) body
-- a privileged create -- was forwarded to the daemon uninspected. A
request presenting both framing headers is now refused with 400 before
policy evaluation (RFC 9112 section 6.1). Transfer-Encoding-only
requests already fail closed (no Content-Length means body = None).
- read_head accumulated header lines with no aggregate bound. It now
caps the head at max_request_headers lines (default 100) and
max_request_head_bytes total (default 64 KiB), refusing a larger head
with 431 instead of buffering it without limit.
Two driven tests against Server.Proxy.serve. The first sends a create
with both Content-Length: 0 and Transfer-Encoding: chunked and a chunked
privileged body, asserting the proxy answers 4xx and the upstream never
receives the privileged bytes (RFC 9112 section 6.1 ambiguous framing).
The second floods the request head past any limit, asserting a 431. Both
fail today: the proxy inspects only Content-Length so the smuggled body
is forwarded, and read_head accumulates headers without bound.
no_privileged blocks only HostConfig.Privileged; host mounts,
capabilities, host networking, host namespaces, and device access still
pass, so a reader could mistake it for a container-isolation boundary.
Add a scope note to proxy.mli: the built-in policies are a coarse
allow/deny/rewrite filter, and real isolation needs an explicit
allow-list (a Rego policy over the create body) or a restricted daemon.
no_privileged and unprivilege only acted when the create body had been
buffered (Content-Length within max_inspect_body); a chunked or oversized
body reached the policy as None and was forwarded, so a privileged create
slipped through simply by being uninspectable. Deny a POST
*/containers/create whose body is None in both policies.
Across the packages monopam lint flagged: declare the libraries each
package actually uses (test-only ones scoped with-test, dev tools and
benches scoped with-dev-setup), drop the opam deps and dune (libraries)
entries nothing references, declare pin-depends for the wire and zarith
git pins so a standalone opam install can resolve them, and point the
five stale dune-project source URLs at the names sources.toml already
uses. The generated .opam files are regenerated to match.
Where monopam lint and merlint's project rules disagree, merlint wins:
[protocol] packages keep probe linked (E932), CSV-trace interop tests
keep csv (E825), and dev-scope deps carry the right filter (E943).
The few dead-lib reports that remain are monopam lint blind spots, not
real dead deps: usage reached only through an mdx README block (slirp),
through copy_files# (vz), or inside a library's own private modules
(builder), plus a tar gz dep kept live for a stub test still to write.
Tree-wide mechanical migration (Digestif.* -> Hash.*, dune and opam
deps digestif -> ocaml-hash) across 43 packages, dropping digestif
from the dependency graph entirely (root.opam no longer lists it).
Blob digests, git object ids, KEX exchange hashes, JWT/COSE/X.509
signatures and every other SHA-256 user now ride the hardware path on
CPUs that have it.
Five libraries had an internal hash.ml that would shadow the Hash
library inside their own compilation units; their internal module is
renamed by what it actually is, with the public alias kept so
consumers (Irmin.Hash, Git.Hash, Tpm.Hash, ...) are unchanged:
- irmin, ocaml-git: Object_hash (content-addressed object ids)
- ocaml-scitt, ocaml-sigstore: Hash_alg (algorithm registries)
- ocaml-tpm: Hashing (digest/HMAC/KDFa over TPM algorithms)
Semantic adjustments the rename surfaced:
- x509: the OID<->hash tables lose their arms for algorithms the
hash library does not carry (SHA-3, BLAKE2, RIPEMD, WHIRLPOOL);
Hash.kind is closed over the six supported algorithms so the
catch-all arms were dead.
- sigstore: bundles declaring SHA-3 digests now fail verification
loudly (Invalid_argument) instead of mapping to an unimplemented
kind; the protobuf enum still parses.
- x509/jwt: local [let module Hash = (val module_of ...)] bindings
renamed to H -- they would shadow the library inside their scope.
- irmin re-exports the raw SHA-1/SHA-256 implementations through
Object_hash so its schema instantiations need no second name for
the library under the public Hash alias.
SSH throughput is unchanged (170/537/460 MB/s socketpair / 2-domain /
2-process, same as the digestif baseline): the negotiated transport
cipher is chacha20-poly1305, whose MAC is Poly1305, so SHA-256 only
runs in the handshake. SSH's data-path ceiling remains nox-crypto's
scalar ChaCha20/Poly1305.
All migrated package suites pass (ssh, tls, x509, paseto, jwt, kdf,
git, tpm, scitt, sigstore, oci, srp, spake2, cose, bpsec, hash).
docker-runner was its own opam package but is I/O-free (deps: docker + fmt) like
the main docker types, so a separate package only paid off for a future real
backend depending on just the interface -- which doesn't exist. Fold it in:
lib/runner.ml is now the Docker.Runner module (BACKEND renamed to the idiomatic
S), the docker-runner package/opam are dropped, and its test moves to
test/test_runner.ml.
Merging the three binaries into one dockr also pulled docker-client and
nox-crypto-rng (which the docker-server library does not use) into a
library-bearing package; move the binary into its own executable-only dockr
package (the etco pattern), so docker-server keeps only its real library deps
(docker-client returns to with-test for the tests). Net: still 4 packages
(docker, docker-client, docker-server, dockr), but each carries honest deps.
Consumers use Docker.Runner.S / Docker.Runner.Memory; 65 + e2e tests pass.
Three separate executables (dockrd, dockr-proxy, dockr) collapse into a single
dockr tool with subcommands, mirroring etco: dockr serve (--fake in-memory
daemon), dockr proxy (content-aware gateway / --authz plugin), and the client
verbs version/ping/info. Drop the eio_main/cmdliner deps from docker-client
(the CLI moved to the docker-server-owned binary) and make docker-client a
plain dep of docker-server. README and the docker_runner doc updated to the
new command surface. All 12 e2e/proxy tests pass.
Several suites wrote scratch files rooted at the process cwd
(Eio.Stdenv.cwd / Sys.getcwd), so running them from the repo root left
artifacts behind:
- ocaml-tty: the eio "follow" display test wrote a .log file
- ocaml-parquet: the eio pread test wrote a .parquet file
- ocaml-docker: the server tests bound Unix sockets
Root each in the system temp dir instead, so the tests are
self-contained and leave nothing in the repository.
Restructure the README to the standard section order (description, install,
usage, API, tests, license) and add the missing Install and License sections.
The Usage section now opens with a compiling example of the IO-free core
(decode a /version body), wired into MDX so dune runtest type-checks it and
diffs its output; the shell and CLI blocks are marked $MDX skip.
Also refresh the package synopses/descriptions, which had gone stale: docker
now lists the full endpoint surface, docker-server mentions Proxy/Authz, and
docker-runner drops the speculative "runc or a VMM later" wording.
Replace the stale 9-endpoint list with the actual served surface (system,
containers, exec, images/networks/volumes) and add a section on the streaming
responses (logs/stats/events, stdcopy framing) and the hijacked attach/exec
connections (101 UPGRADED via Respond.hijack), including how the proxy
byte-splices upgrades transparently. Drop the "seam" wording.
Expose the full policy surface from the binary: --rego-file loads an OPA/Rego
policy (overriding the named --policy), and --authz runs as a dockerd
authorization plugin on --listen (Server.Authz) instead of a forwarding
proxy. So the Rego adapter and the in-daemon plugin are usable straight from
the CLI.
Add Server.Authz, the Docker authorization-plugin protocol decided by the
same Proxy.policy as the standalone proxy. Registered with `dockerd
--authorization-plugin`, it answers the plugin handshake (Plugin.Activate ->
implements authz) and the AuthZPlugin.AuthZReq/AuthZRes RPCs: it base64-
decodes the daemon's RequestBody, turns the call into a Proxy.request, and
returns {Allow,Msg} from the policy (a Rewrite is treated as allow, since the
AuthZ protocol cannot modify the request). So no_privileged, read_only, and
the Rego adapter all run BOTH as a standalone gateway and in-daemon -- the
opa-docker-authz integration model over our policy surface.
Test (test_authz.ml): drive the plugin over a socket -- the handshake
advertises authz, and the no_privileged policy denies a privileged create
and allows a plain one.
Add an opt-in (DOCKER_LIVE=1) full-lifecycle case to the real-daemon client
test: pull a tiny image, run a container, and exercise logs/exec/top/stats
plus network and volume CRUD, all under uniquely-named throwaway resources
removed in a finally block. It validates wire-compatibility with stock
dockerd -- exec returns real stdout through the hijack + stdcopy path, and
the daemon's full stats object decodes via skip_unknown. DOCKER_SOCK
overrides the socket (e.g. to point at a proxy).
This caught a real interop bug the fake masked: the daemon requires at least
one stream on GET /containers/{id}/logs ("you must choose at least one
stream", 400). The client now sends ?stdout=1&stderr=1 (and attach sends
?stream=1&stdout=1&stderr=1), and the fake server enforces the same contract
so it stops masking it.
Serve the classic POST /build via a new BACKEND.image_build seam: the tar
context is not interpreted (the in-memory backend cannot run steps), so a
new image is registered under the requested tag and the usual progress
stream returned. Client gains Image.build (empty body, since the server
ignores the context). Our client builds a tagged image end to end and finds
it in the image list. The stock CLI defaults to BuildKit, a separate
protocol, so it is not used as a cross-check for this legacy endpoint.
Add Docker.Stats (the cpu/memory/pids counters the CLI reads, nested under
cpu_stats/memory_stats/pids_stats) and serve GET /containers/{id}/stats via
a new BACKEND.stats seam: a real backend samples the cgroup, the in-memory
backend reports zeros (404 when unknown). A client reading one object then
EOF (the --no-stream case) gets exactly this sample. Client gains
Container.stats. Real `docker stats --no-stream` names the container against
--fake, and our client reads the zeroed sample back.
Add Docker.Event (one JSON object per line, the id under Actor.ID) and serve
GET /events. The Memory backend records the container lifecycle it simulates
(create/start/stop/kill/restart/pause/unpause/destroy) and a new
BACKEND.events replays the log; a real daemon would hold the connection open
and push live events, so the in-memory backend returns its recorded log as a
finite stream. Client gains System.events. Our client reads the log back and
sees the create/start/destroy actions it drove.
Serve POST /containers/{id}/attach over the hijack primitive: upgrade to
the multiplexed stream and write the container's stdout/stderr (empty in
--fake), 404 when unknown. The client gains Container.attach; the exec and
attach clients now share the hijacked_stdout helper (run the upgrade, demux
stdout). Our client drives attach against a running --fake container.
Add Docker.Exec (create config/response, start, inspect detail) and the
exec endpoints over the new respond hijack primitive: POST
/containers/{id}/exec creates an instance (404 unknown, 409 not running),
POST /exec/{id}/start hijacks the connection and writes a 101 upgrade then
the multiplexed output, and GET /exec/{id}/json inspects it. In --fake no
process runs, so an exec exits 0 with empty output. BACKEND grows
exec_create/start/inspect; Memory tracks instances. The client gains an Exec
module that performs the raw upgrade itself and demultiplexes stdout. Real
`docker exec` round-trips against --fake, and our own client drives the same
create/start/inspect path (plus the 409 on a stopped container).
Add Image.History, the layer-history entry, and serve
GET /images/{name}/history via a new BACKEND.image_history seam: a real
runtime reads the image config's layer chain, while the in-memory backend
stores no layers and reports its single recorded metadata entry (404 when
unknown). Client gains Image.history. Real `docker history` round-trips
against --fake.
Add Docker.Change, the filesystem-diff entry (Path plus a typed Modified/
Added/Deleted Kind), and serve GET /containers/{id}/changes via a new
BACKEND.changes seam: a real runtime diffs the container against its image,
while the in-memory backend has no writable layer and so reports nothing
(404 when the container is unknown). Client gains Container.changes. Real
`docker diff` round-trips against --fake.
Add Docker.Stdcopy, the multiplexed stdout/stderr stream framing (8-byte
header: stream byte, three zeros, big-endian payload length) used by the
logs and attach streams, with frame/demux unit-tested on known vectors.
Serve GET /containers/{id}/logs: any existing container's logs are a
multiplexed stream (404 when unknown), empty in --fake since no process
runs, carrying the stream content type so clients demux it. Client gains
Container.logs, which demuxes and returns stdout. Real `docker logs`
round-trips against --fake.