core: Add per-device IR calibration model and store
Calibration records how a given IR camera delivers illuminated frames so
capture can be optimized per device. It is purely an optimization and
capability layer: `capture_ir_face` already works without it (capture a
burst, keep the brightest), so a missing/unmatched/corrupt record just
falls back to the safe mode-agnostic burst — never an error. That's why
this store, unlike the enrollment store, needs no HMAC.
# Device identity (card + bus)
Records are keyed by `CameraIdentity { card, bus }`. Neither field alone
is a usable key: `/dev/videoN` renumbers across reboots, the card name
isn't unique across identical models, and the bus (port topology)
changes when a device is replugged elsewhere. `resolve_calibration`
matches leniently, returning a typed `CalibrationMatch`:
- `Exact` — card + bus both match.
- `MovedPort` — card matches, bus differs, and exactly one connected
camera has that card name, so the device just moved ports: safe to
adopt and refresh the stored bus.
- `Ambiguous` — card matches but several connected cameras share that
name, so we can't guess which: the caller should ask the user.
- `None` — no record; fall back to mode-agnostic capture.
This mirrors the decisions-in-core / rendering-in-CLI shape of
`camera::select::IrCameraResolution`.
# Store
`CalibrationStore` persists all records in one small CBOR file (the set
is tiny and the matching API wants it whole). Writes are atomic
(temp-file + fsync + rename), matching the enrollment store's crash
discipline. `upsert` replaces by exact identity; `replace_by_card` drops
a stale-bus record when a device moved ports.
# CameraInfo.bus
`CameraInfo` gains a `bus: Option<String>`, filled by `enumerate()` from
the V4L2 bus info and left `None` for mock/synthetic sources and the
by-path `open()` path. This is the input `CameraIdentity::from_info`
reads. Calibration shares the `v4l2` feature gate since it depends on the
backend's `IrCaptureMode`.
core,cli: Add dual-modality authentication
Single-modality face auth is identity-matching, not liveness detection.
ArcFace is trained to cluster the embedding space by identity; a sharp
photo of a face produces an embedding very close to the face's own,
because identity is what ArcFace was optimised to preserve. We
confirmed the practical impact: `pareidolia test` (RGB) passes when
shown a phone-screen photo of a relative, with similarity solidly
above the 0.6 threshold.
The same query against the IR camera, however, didn't even reach the
matcher — SCRFD failed to detect a face in the IR view of the phone
screen at all, because phone/laptop displays emit visible light but
almost no near-infrared. Printed photos behave similarly in most
cases. Requiring *both* modalities to pass independently therefore
closes the common presentation-attack vector without adding any
anti-spoof ML.
# Core: `authenticate_dual` + `DualMatchOutcome` + `SingleVerdict`
`Enrollment::authenticate_dual(rgb_query, ir_query, threshold)` takes
two `Option<&Embedding>` queries (one per modality) and returns a
`DualMatchOutcome`:
- `Pass { rgb, ir }` — both modalities cleared the threshold against
their respective enrolled pools. Returns the winning `MatchResult`
from each pool for audit / logging.
- `Reject { rgb, ir }` — at least one modality failed. Both
per-modality `SingleVerdict`s are preserved so the failure mode is
fully diagnosable (which side failed, and how).
- `NoEnrollment` — fires only when the enrollment has zero samples
total; per-modality emptiness collapses into `SingleVerdict::NoSamples`
on that side instead.
`SingleVerdict` carries the four states a per-modality decision can
land in: `Pass`, `BelowThreshold` (face found, similarity below
threshold), `NoFace` (caller didn't supply an embedding — capture
error, no face detected, modality disabled), `NoSamples` (enrollment
has no samples for this kind). The `Option` parameter shape lets the
caller express 'couldn't produce an embedding for this side' directly,
guaranteeing the matcher rejects rather than silently degrading to
single-modality.
# Tests
Eight new unit tests pin the full truth table (empty enrollment, both
queries present and matching, RGB pass + IR below threshold, missing
RGB query, both queries missing, RGB-only enrollment with IR queried,
both below threshold, is_pass helper). Two new property tests pin the
soundness invariants:
- `authenticate_dual_pass_iff_both_single_modality_pass` — the
primary contract: dual.is_pass() ⟺ both single-modality authenticate
calls would have passed. Callers can reason about
`authenticate_dual` as 'AND over single-modality outcomes'.
- `authenticate_dual_with_either_query_missing_never_passes_on_nonempty`
— a missing query on either side always forces Reject for any
non-empty enrollment, regardless of threshold.
One integration smoke test
(`public_dual_modality_authenticate_requires_both_pools_to_pass`)
covers the end-to-end public API including the Pass shape and the
None-query Reject shape.
# CLI: `pareidolia test` learns dual mode
The `conflicts_with = rgb_camera` mutex on `--ir-camera` is removed.
Mode selection becomes:
- `--ir-camera` unset → single RGB mode against the RGB pool
(unchanged from M4.6).
- `--ir-camera` set → dual mode: capture one frame from each sensor
serially (sibling-node cooldown between, like enroll), run
`authenticate_dual`. 'No face detected' on one side becomes
`SingleVerdict::NoFace` rather than a hard error, so the verdict
surfaces fully.
The dual output path prints both per-modality verdicts on reject
('RGB: pass (similarity 0.823 ≥ 0.600, sample 1) / IR: no face
detected in captured frame'), making it obvious which side blocked
auth. The previous IR-single-camera testing flow gives way to dual:
running `pareidolia test --ir-camera /dev/videoX` against the typical
RGB+IR enrollment is now a strictly stronger check than before.
# Documentation
A new `## Security` section in CONTRIBUTING.md explains the threat
model: what ArcFace matches (identity, not liveness), what dual-mode
defeats (phone-screen photos, most paper photos) and what it doesn't
(NIR-tuned prints, 3D masks, NIR-illuminated photographs), and the
separate concern of family-resemblance false-accepts. A defense
matrix summarises the cells.
core,nix: Add end-to-end pipeline orchestration with golden test
`pipeline::recognise` ties the three model-touching stages — Detector,
align_face, Embedder — into a single call. The output is a Vec of
(Detection, Embedding) pairs, one per face the detector found,
preserving the detector's ordering. This is what the daemon's PAM path
will eventually drive once enrollment + matching land in M4.
Pure orchestration code: no feature gate, no new runtime deps. It
operates on the existing trait + type surface, so it compiles in every
workspace configuration including --no-default-features.
# Errors abort the pipeline
Per-face partial successes (e.g. 'detected three faces, aligned two,
embedded one') would be hard to reason about in the matching layer, so
any stage error aborts the whole call with no results. The unit tests
pin this behaviour: detector errors, degenerate landmarks (align fails),
and embedder errors all surface as the original error variant with no
side effects.
# Golden test against real models
A new hardware-gated test runs the full pipeline against the InsightFace
sample face JPEG (t1.jpg, ~128 KB, MIT-licensed via their repo). It
asserts:
- ≥1 detection (the fixture has a clearly visible face; finding zero
would mean the detector regressed).
- Every detection's confidence clears the default 0.5 threshold.
- Every embedding is 512-d with L2 magnitude ≈ 1.0.
- Running the pipeline twice on the same frame yields bit-identical
embeddings (CPU EP determinism guarantee).
Stops short of pinning exact embedding values — that's the matching
layer's concern in M4, where similarity-against-stored-baseline becomes
the natural assertion.
# Fixture infrastructure
The fixture image is pinned in nix/models.nix with sha256, fetched at
devShell build time, and exposed via PAREIDOLIA_TEST_FACE so the
hardware test runs without manual setup inside `nix develop`. Outside
Nix, set the env var manually (the InsightFace URL is in
nix/models.nix).
`image` (jpeg + png decoders only, default-features off) joins the
workspace dev-dependencies for fixture loading; production code never
needs to decode images because the camera produces RGB8 directly.