core: Add 5-landmark affine face alignment
The pipeline stage between the detector and the embedder. Given detected
landmarks and the canonical ArcFace template, fits a similarity transform
(uniform scale + rotation + translation), inverts it, and warps the source
frame into a 112×112 chip.
# 2D-specific math
ArcFace alignment is a 2D-to-2D similarity transform, so we get a closed-
form least-squares fit without invoking SVD. The 2x2 rotation+scale can
be parameterised by a single (u, v) pair (u = s·cos θ, v = s·sin θ) and
solved as four sums over the centred points. This keeps the alignment
code free of any linear-algebra dependency — no nalgebra, no manual SVD.
# Behaviour at the edges
- Non-RGB8 frames return InvalidInputFormat rather than producing garbage.
- All-coincident or otherwise degenerate landmarks return a Backend
error (the linear system has no solution). The daemon should fall
through to password in this case rather than panic.
- Inverse-mapped samples that fall outside the source frame return
black pixels. This produces an aligned chip with some black border
when the face is near the source edge — a useful downstream signal
rather than a fatal error.
# Tests
19 unit tests covering the SimilarityTransform type (identity, scale,
rotation, inverse round-trip, singularity guard), the closed-form fit
(pure translation, pure scale, pure rotation, mismatched / empty / coincident
inputs), align_face (format validation, output shape, identity-warp
solid-colour preservation, metadata propagation, degenerate-landmark
error), and the bilinear sampler (out-of-bounds, NaN, exact-integer hits).
2 property tests guard the headline invariants — the fit-then-apply
reproduces any well-conditioned target within f32 tolerance, and the
inverse round-trips any non-singular transform within f32 tolerance —
so future micro-optimisations of the math can't silently drift.
core: Add SCRFD detector implementation
First real Detector impl. Wires SCRFD-10G (loaded by OnnxSession from the
nix-pinned ONNX file) into the pipeline's Detector trait. Covers the full
flow: letterbox preprocess, normalised NCHW tensor, single inference call,
9-output anchor decoding across 3 stride levels, NMS, coordinate
rescaling.
Trait refinement: Detector::detect and Embedder::embed now take `&mut self`.
ONNX Runtime's Session::run is `&mut self` and the daemon owns one
detector per worker (never sharing concurrently), so this matches the
usage pattern without forcing interior mutability. As a bonus the
ScriptedDetector mock can now pop its queue properly instead of the
peek-and-pretend-to-mutate hack the previous `&self` signature forced.
Implementation notes:
- Letterbox uses the standard SCRFD layout: resized image at top-left,
zero-padded right/bottom, single scale factor. The OpenCV half-pixel
offset convention in the bilinear resampler matches reference Python
implementations, so detection coordinates compare correctly against
insightface goldens later.
- Bilinear resize is hand-rolled (~30 LOC) rather than pulling in the
`image` crate. We'll add `image` in M3.5 when fixture loading needs
it; not paying for it now keeps the inference feature's transitive dep
surface small.
- Decoder bails (returns no detections) rather than panicking when output
tensor shapes don't match the expected stride layout. A model variant
with different strides shouldn't take down the daemon, just report no
faces and let PAM fall through to password.
- Default thresholds (score 0.5, NMS IoU 0.4) match InsightFace
recommendations.
Tests:
- 18 unit tests on the pure helpers (NMS, anchor decoding, letterbox,
bilinear resize, coordinate rescaling). The decode test exercises the
anchor centre → bbox + landmarks math against hand-computed expected
outputs; the NMS tests cover score-order invariance, IoU-threshold
edge cases, and disjoint-box preservation.
- 1 hardware-gated end-to-end test loading the real SCRFD model and
running it on a synthetic black frame. Asserts no detections (no face
exists) and no panic; the runtime path is what matters here. Currently
~210ms wall-clock on CPU on the dev machine.
core,nix: Add end-to-end pipeline orchestration with golden test
`pipeline::recognise` ties the three model-touching stages — Detector,
align_face, Embedder — into a single call. The output is a Vec of
(Detection, Embedding) pairs, one per face the detector found,
preserving the detector's ordering. This is what the daemon's PAM path
will eventually drive once enrollment + matching land in M4.
Pure orchestration code: no feature gate, no new runtime deps. It
operates on the existing trait + type surface, so it compiles in every
workspace configuration including --no-default-features.
# Errors abort the pipeline
Per-face partial successes (e.g. 'detected three faces, aligned two,
embedded one') would be hard to reason about in the matching layer, so
any stage error aborts the whole call with no results. The unit tests
pin this behaviour: detector errors, degenerate landmarks (align fails),
and embedder errors all surface as the original error variant with no
side effects.
# Golden test against real models
A new hardware-gated test runs the full pipeline against the InsightFace
sample face JPEG (t1.jpg, ~128 KB, MIT-licensed via their repo). It
asserts:
- ≥1 detection (the fixture has a clearly visible face; finding zero
would mean the detector regressed).
- Every detection's confidence clears the default 0.5 threshold.
- Every embedding is 512-d with L2 magnitude ≈ 1.0.
- Running the pipeline twice on the same frame yields bit-identical
embeddings (CPU EP determinism guarantee).
Stops short of pinning exact embedding values — that's the matching
layer's concern in M4, where similarity-against-stored-baseline becomes
the natural assertion.
# Fixture infrastructure
The fixture image is pinned in nix/models.nix with sha256, fetched at
devShell build time, and exposed via PAREIDOLIA_TEST_FACE so the
hardware test runs without manual setup inside `nix develop`. Outside
Nix, set the env var manually (the InsightFace URL is in
nix/models.nix).
`image` (jpeg + png decoders only, default-features off) joins the
workspace dev-dependencies for fixture loading; production code never
needs to decode images because the camera produces RGB8 directly.
core: Add property tests for SCRFD's pure stages
The SCRFD module's pure stages (NMS, letterbox preprocess, bilinear
resize, anchor decode) had hand-picked goldens but no property coverage.
Those are exactly the code paths where structural bugs silently degrade
detection quality without crashing — the kind property tests catch best.
# Coverage added
NMS (the largest gap):
- Output count is never greater than input count.
- Every pair of surviving detections has IoU ≤ threshold — the
postcondition NMS exists to enforce.
- The globally-best detection (highest confidence) always survives.
- Idempotency: running NMS on its own output produces the same set.
- Input-order independence: reversing the input order produces the same
surviving set.
letterbox_preprocess:
- Output tensor is always [1, 3, target, target] regardless of source
dimensions.
- det_scale is finite, positive, and produces source dimensions that
fit inside the target (within sub-pixel rounding).
bilinear_resize_rgb8:
- Output buffer length is always exactly dst_w · dst_h · 3.
- Solid colour preservation across arbitrary resize ratios (stronger
than the unit test which fixed one ratio).
decode_stride:
- Never panics on arbitrary tensor lengths or grid sizes. A malformed
model output should produce zero detections, never an index-out-of-
bounds panic that takes down the daemon.
# Performance
Letterbox proptests are bounded to 400×400 source dimensions because
each iteration runs a full bilinear resize. The 'output shape' and
'det_scale fit' properties are structural and don't gain coverage from
larger inputs; capping the bound keeps the test in single-digit seconds.
# Cost to CI
Zero: the proptests live inside the inference-feature-gated scrfd
module, so CI's --no-default-features build doesn't compile or run them.
The full-feature local suite picks them up via `just test-features`.