core,cli: Add dual-modality authentication · isaaccorbrey.com/pareidolia@1168233

+80

CONTRIBUTING.md

··· 177 177 that genuinely need another internal crate's default features can flip 178 178 `default-features = true` back on in their own dep entry. 179 179 180 + ## Security 181 + 182 + ### What Pareidolia matches 183 + 184 + Pareidolia performs **face-identity matching** via the ArcFace model on 185 + embeddings extracted from each captured face. The model produces a 186 + 512-dimensional vector per face; cosine similarity between vectors 187 + clusters by identity. The matcher picks the highest-similarity enrolled 188 + sample and grants auth when that similarity clears a configurable 189 + threshold (0.6 by default). 190 + 191 + This is **not liveness detection**. ArcFace by itself cannot distinguish 192 + a real face from a sharp photograph of one — that is an architectural 193 + property of every single-modality identity-matching system without an 194 + added anti-spoof model. 195 + 196 + ### Defense mode: dual-modality authentication 197 + 198 + The recommended mode whenever the user has IR samples enrolled is 199 + dual-modality: pass `--rgb-camera` and `--ir-camera` together to both 200 + `pareidolia enroll` and `pareidolia test`. With both pools populated, 201 + [`Enrollment::authenticate_dual`] requires *both* modalities to clear the 202 + threshold independently before granting auth. The daemon (M5+) will 203 + default to dual mode automatically whenever the enrollment supports it. 204 + 205 + Dual-modality auth defeats the common presentation attack — a printed 206 + photo or a face displayed on a phone/laptop screen — without any 207 + additional ML model: 208 + 209 + - Phone and laptop screens emit visible light but almost no 210 + near-infrared. In an IR illuminator's beam such a screen is just a 211 + dim rectangle, and SCRFD typically does not detect a face in it. 212 + - Printed photos vary in NIR reflectivity by pigment, but most prints 213 + either fail face detection in IR or produce IR embeddings too distant 214 + from the enrolled IR samples to clear the threshold. 215 + 216 + This is **a partial defense, not bulletproof**. Specifically, it does 217 + *not* defeat: 218 + 219 + - NIR-tuned print attacks (photos printed with pigments chosen for 220 + skin-like NIR reflectivity). 221 + - 3D masks built with realistic skin-like NIR behaviour. 222 + - A photograph illuminated by an NIR source matching the camera 223 + illuminator's spectrum. 224 + 225 + Real PAD (presentation-attack detection) via a dedicated anti-spoof 226 + model is a future milestone, and would be required to defend against 227 + the high-effort attacks above. 228 + 229 + ### Family resemblance and identity discrimination 230 + 231 + ArcFace clusters by identity, and biological relatives occupy regions 232 + of the embedding space that are *closer together* than unrelated 233 + people. A child, sibling, or parent may produce a similarity above the 234 + default 0.6 threshold against a relative's enrollment — this is identity 235 + confusion, not spoofing, and exists independently of liveness or the 236 + choice of single-vs-dual modality. Remedies: 237 + 238 + - Raise `--threshold` to be stricter at the cost of more false-rejects 239 + on poses or lighting the user didn't enroll for. 240 + - Re-run `pareidolia enroll` to add more sample variety to the user's 241 + own pool. Best-of-N matching is monotonically improving in N, so this 242 + raises the bar against close-relative false-accepts without making 243 + the user's own auth more fragile. 244 + 245 + ### Defense matrix 246 + 247 + | Attack | Single (RGB only) | Dual (RGB + IR) | 248 + | ------------------------------------- | ----------------- | --------------- | 249 + | Phone or laptop screen showing a photo | passes | rejected (SCRFD vetoes the IR side) | 250 + | Printed photo on paper | passes | likely rejected | 251 + | NIR-tuned printed photo | passes | may pass | 252 + | 3D mask with skin-like NIR | passes | may pass | 253 + | Live family member | may pass | may pass | 254 + | Same person, hardware variation | passes | passes | 255 + 256 + `pareidolia test` mirrors the modes available to the daemon, so it's 257 + useful for sanity-checking which attacks your current enrollment would 258 + let through before exposing it to PAM. 259 + 180 260 ## Source control 181 261 182 262 The canonical workflow uses [Jujutsu](https://jj-vcs.github.io/jj/) (`jj`).

+138 -31

crates/cli/src/commands/test.rs

··· 1 1 //! `pareidolia test` — capture a frame, embed the face, match against a 2 2 //! user's existing enrollment, report the outcome. 3 3 //! 4 - //! The non-PAM smoke test: lets you verify "did my enrollment actually 5 - //! work" without involving any of the daemon / PAM / IPC plumbing that 6 - //! arrives in M5. Useful both as a development tool and as a triage step 7 - //! once the full login path exists ("auth is failing; does 8 - //! `pareidolia test` pass?"). 4 + //! Three operating modes selected by which `--*-camera` flags are set: 5 + //! 6 + //! - **Single RGB** (default): only `--rgb-camera` is in play. Matches 7 + //! the captured face against the user's RGB pool with 8 + //! [`Enrollment::authenticate`]. 9 + //! - **Dual**: both `--rgb-camera` and `--ir-camera` are set. Captures 10 + //! one frame from each sensor and runs 11 + //! [`Enrollment::authenticate_dual`], which requires *both* pools to 12 + //! pass independently before granting auth. This is the recommended 13 + //! mode whenever the user has IR samples enrolled, because it 14 + //! defeats the common presentation-attack (printed photo / phone 15 + //! screen) at the cost of needing both sensors live. 9 16 //! 10 17 //! Order matters: enrollment is loaded *first*. If the user isn't 11 - //! enrolled, we report that and exit cleanly without ever touching the 12 - //! camera or loading models. That keeps a fast no-op path for what's 13 - //! often the actual question ("am I enrolled?") and avoids wedging 14 - //! hardware on the way to a "no" answer. 18 + //! enrolled, we report that and exit cleanly without ever touching 19 + //! the camera or loading models. That keeps a fast no-op path for 20 + //! what's often the actual question ("am I enrolled?") and avoids 21 + //! wedging hardware on the way to a "no" answer. 15 22 16 23 use std::path::{Path, PathBuf}; 24 + use std::thread; 25 + use std::time::Duration; 17 26 18 27 use clap::Args as ClapArgs; 19 28 20 29 use pareidolia_core::camera::CameraKind; 21 30 use pareidolia_core::enrollment::store::StoreError; 22 - use pareidolia_core::enrollment::{MatchOutcome, DEFAULT_THRESHOLD}; 31 + use pareidolia_core::enrollment::{ 32 + DualMatchOutcome, MatchOutcome, MatchResult, SingleVerdict, DEFAULT_THRESHOLD, 33 + }; 23 34 use pareidolia_core::pipeline::arcface::{ArcfaceConfig, ArcfaceEmbedder}; 35 + use pareidolia_core::pipeline::recognise::FaceResult; 24 36 use pareidolia_core::pipeline::scrfd::{ScrfdConfig, ScrfdDetector}; 25 37 26 38 use crate::{pipeline_runtime, store_config, CliError}; ··· 36 48 #[arg(long, default_value_t = DEFAULT_THRESHOLD)] 37 49 pub threshold: f32, 38 50 39 - /// RGB V4L2 camera device. Mutually exclusive with `--ir-camera`. 51 + /// RGB V4L2 camera device. Always used in single mode; combined 52 + /// with `--ir-camera` triggers dual-modality mode where *both* 53 + /// modalities must pass independently. 40 54 #[arg(long, default_value = "/dev/video0")] 41 55 pub rgb_camera: PathBuf, 42 56 43 - /// Use this IR camera instead of the RGB one. When set, the match 44 - /// runs against the user's IR-enrolled samples only. 45 - #[arg(long, conflicts_with = "rgb_camera")] 57 + /// IR V4L2 camera device. Setting this opts into dual-modality 58 + /// authentication: a single frame is captured from each sensor and 59 + /// the auth passes only when both modalities clear the threshold 60 + /// against their respective enrolled pools. 61 + #[arg(long)] 46 62 pub ir_camera: Option<PathBuf>, 47 63 48 64 /// SCRFD ONNX model path. Falls back to $PAREIDOLIA_TEST_SCRFD. ··· 85 101 "ArcFace", 86 102 )?; 87 103 88 - // clap's `conflicts_with = "rgb_camera"` on `ir_camera` keeps the 89 - // two from being set together at parse time, so we can branch on 90 - // `ir_camera` alone. 91 - let (camera_path, kind) = match &args.ir_camera { 92 - Some(path) => (path.clone(), CameraKind::Ir), 93 - None => (args.rgb_camera.clone(), CameraKind::Rgb), 94 - }; 95 - 96 - let mut source = pipeline_runtime::open_camera(&camera_path, kind)?; 97 104 let mut detector = 98 105 ScrfdDetector::open(&scrfd_path, ScrfdConfig::default()).map_err(CliError::Pipeline)?; 99 106 let mut embedder = ArcfaceEmbedder::open(&arcface_path, ArcfaceConfig::default()) 100 107 .map_err(CliError::Pipeline)?; 101 108 102 - let face = pipeline_runtime::capture_one_face(&mut source, &mut detector, &mut embedder)?; 103 - let outcome = enrollment.authenticate(&face.embedding, kind, args.threshold); 109 + match &args.ir_camera { 110 + Some(ir_path) => { 111 + // Dual mode: capture from each sensor (serially — never 112 + // both open at once, see `cli/src/commands/enroll.rs` for 113 + // the rationale) and let `authenticate_dual` decide. 114 + let rgb_face = try_capture_face( 115 + &args.rgb_camera, 116 + CameraKind::Rgb, 117 + &mut detector, 118 + &mut embedder, 119 + )?; 120 + // Match the enroll-loop cooldown so consecutive opens of 121 + // sibling video nodes on the same physical UVC device 122 + // don't fight each other. 123 + thread::sleep(Duration::from_millis(500)); 124 + let ir_face = try_capture_face(ir_path, CameraKind::Ir, &mut detector, &mut embedder)?; 125 + 126 + let outcome = enrollment.authenticate_dual( 127 + rgb_face.as_ref().map(|f| &f.embedding), 128 + ir_face.as_ref().map(|f| &f.embedding), 129 + args.threshold, 130 + ); 131 + print_dual_outcome(&outcome, args.threshold, &args.user); 132 + } 133 + None => { 134 + // Single RGB mode (default). Unchanged from M4.6. 135 + let mut source = pipeline_runtime::open_camera(&args.rgb_camera, CameraKind::Rgb)?; 136 + let face = 137 + pipeline_runtime::capture_one_face(&mut source, &mut detector, &mut embedder)?; 138 + let outcome = enrollment.authenticate(&face.embedding, CameraKind::Rgb, args.threshold); 139 + print_single_outcome(&outcome, args.threshold, &args.user); 140 + } 141 + } 142 + 143 + Ok(()) 144 + } 104 145 146 + /// Open `path`, capture one frame, run detect/embed. "No face detected" 147 + /// is a *verdict* in dual mode (one modality failed to produce an 148 + /// embedding), not an error to abort on, so it's folded into `Ok(None)` 149 + /// here. All other errors still bubble. 150 + fn try_capture_face( 151 + path: &Path, 152 + kind: CameraKind, 153 + detector: &mut ScrfdDetector, 154 + embedder: &mut ArcfaceEmbedder, 155 + ) -> Result<Option<FaceResult>, CliError> { 156 + let mut source = pipeline_runtime::open_camera(path, kind)?; 157 + match pipeline_runtime::capture_one_face(&mut source, detector, embedder) { 158 + Ok(face) => Ok(Some(face)), 159 + Err(CliError::NoFaceDetected) => Ok(None), 160 + Err(e) => Err(e), 161 + } 162 + } 163 + 164 + fn print_single_outcome(outcome: &MatchOutcome, threshold: f32, user: &str) { 105 165 match outcome { 106 166 MatchOutcome::Pass(result) => { 107 167 println!( 108 168 "Pass: similarity {:.3} ≥ threshold {:.3} (matched sample {})", 109 - result.similarity, args.threshold, result.sample_id.0, 169 + result.similarity, threshold, result.sample_id.0, 110 170 ); 111 171 } 112 172 MatchOutcome::Reject { best } => { 113 173 println!( 114 174 "Reject: best similarity {:.3} < threshold {:.3} (closest sample {})", 115 - best.similarity, args.threshold, best.sample_id.0, 175 + best.similarity, threshold, best.sample_id.0, 116 176 ); 117 177 } 118 178 MatchOutcome::NoEnrollment => { 119 179 // Shouldn't reach here — the loaded-but-empty case above 120 180 // already short-circuited — but handle it for completeness 121 181 // in case Enrollment's invariants ever change. 122 - println!("Not enrolled: {} has no samples", args.user); 182 + println!("Not enrolled: {user} has no samples"); 123 183 } 124 184 MatchOutcome::NoSamplesForModality { kind } => { 125 185 let (label, flag) = match kind { ··· 127 187 CameraKind::Ir => ("IR", "--ir-camera"), 128 188 }; 129 189 println!( 130 - "Not enrolled for {label}: {} has no {label} samples \ 190 + "Not enrolled for {label}: {user} has no {label} samples \ 131 191 (re-run `pareidolia enroll` with {flag} to add them)", 132 - args.user, 133 192 ); 134 193 } 135 194 } 195 + } 136 196 137 - Ok(()) 197 + fn print_dual_outcome(outcome: &DualMatchOutcome, threshold: f32, user: &str) { 198 + match outcome { 199 + DualMatchOutcome::Pass { rgb, ir } => { 200 + println!( 201 + "Pass (dual): RGB similarity {:.3} ≥ {:.3} (sample {}), \ 202 + IR similarity {:.3} ≥ {:.3} (sample {})", 203 + rgb.similarity, 204 + threshold, 205 + rgb.sample_id.0, 206 + ir.similarity, 207 + threshold, 208 + ir.sample_id.0, 209 + ); 210 + } 211 + DualMatchOutcome::Reject { rgb, ir } => { 212 + println!("Reject (dual): both modalities must pass independently"); 213 + println!(" RGB: {}", verdict_summary(rgb, threshold)); 214 + println!(" IR : {}", verdict_summary(ir, threshold)); 215 + } 216 + DualMatchOutcome::NoEnrollment => { 217 + println!("Not enrolled: {user} has no samples"); 218 + } 219 + } 220 + } 221 + 222 + fn verdict_summary(verdict: &SingleVerdict, threshold: f32) -> String { 223 + match verdict { 224 + SingleVerdict::Pass(MatchResult { 225 + sample_id, 226 + similarity, 227 + }) => format!( 228 + "pass (similarity {:.3} ≥ {:.3}, sample {})", 229 + similarity, threshold, sample_id.0, 230 + ), 231 + SingleVerdict::BelowThreshold(MatchResult { 232 + sample_id, 233 + similarity, 234 + }) => format!( 235 + "reject (similarity {:.3} < {:.3}, closest sample {})", 236 + similarity, threshold, sample_id.0, 237 + ), 238 + SingleVerdict::NoFace => "no face detected in captured frame".to_string(), 239 + SingleVerdict::NoSamples => { 240 + "no samples enrolled for this modality (re-run `pareidolia enroll` \ 241 + with the matching camera flag)" 242 + .to_string() 243 + } 244 + } 138 245 }

+361

crates/core/src/enrollment.rs

··· 257 257 MatchOutcome::Reject { best } 258 258 } 259 259 } 260 + 261 + /// Decide whether the caller authenticates against *both* enrolled 262 + /// modalities. The RGB and IR pools are matched independently and 263 + /// the overall outcome is [`DualMatchOutcome::Pass`] only when each 264 + /// pool clears the threshold on its own. 265 + /// 266 + /// # Why both must pass 267 + /// 268 + /// ArcFace is an identity model, not a liveness model: a sharp 269 + /// photo of a face produces an embedding very close to the face's. 270 + /// Single-modality auth is therefore vulnerable to presentation 271 + /// attacks. IR sensing, while not designed for liveness, defeats 272 + /// the common case in practice — phone screens emit almost no NIR, 273 + /// printed photos reflect NIR very differently from skin, and the 274 + /// SCRFD detector typically fails to even find a face in IR views 275 + /// of either. Requiring both modalities to pass independently 276 + /// closes those attacks without any anti-spoof model. 277 + /// 278 + /// This is a partial defense, not bulletproof: NIR-aware print 279 + /// attacks and 3D masks aren't defeated. Real PAD (presentation- 280 + /// attack detection) is a separate future milestone. 281 + /// 282 + /// # Inputs 283 + /// 284 + /// Each modality's query is [`Option`] so the caller can express 285 + /// "couldn't produce an embedding for this side" (capture error, 286 + /// no face detected, modality disabled) directly. A `None` query 287 + /// translates to [`SingleVerdict::NoFace`] for that modality and 288 + /// guarantees the overall outcome is [`DualMatchOutcome::Reject`]. 289 + /// 290 + /// # Outcomes 291 + /// 292 + /// - [`DualMatchOutcome::NoEnrollment`] when the user has no 293 + /// samples at all. Same remediation as the single-modality 294 + /// variant. 295 + /// - [`DualMatchOutcome::Pass`] when both per-modality verdicts 296 + /// are [`SingleVerdict::Pass`]. 297 + /// - [`DualMatchOutcome::Reject`] otherwise, with both per-modality 298 + /// verdicts surfaced so callers can log which side failed and 299 + /// how. 300 + pub fn authenticate_dual( 301 + &self, 302 + rgb_query: Option<&Embedding>, 303 + ir_query: Option<&Embedding>, 304 + threshold: f32, 305 + ) -> DualMatchOutcome { 306 + if self.samples.is_empty() { 307 + return DualMatchOutcome::NoEnrollment; 308 + } 309 + let rgb = single_verdict(self, rgb_query, CameraKind::Rgb, threshold); 310 + let ir = single_verdict(self, ir_query, CameraKind::Ir, threshold); 311 + match (rgb, ir) { 312 + (SingleVerdict::Pass(rgb), SingleVerdict::Pass(ir)) => { 313 + DualMatchOutcome::Pass { rgb, ir } 314 + } 315 + (rgb, ir) => DualMatchOutcome::Reject { rgb, ir }, 316 + } 317 + } 318 + } 319 + 320 + /// Compute a single-modality verdict, used internally by 321 + /// [`Enrollment::authenticate_dual`] to produce one half of the 322 + /// dual-modality outcome. Free function (not a method) because the 323 + /// public surface is the dual variant; this is a building block. 324 + fn single_verdict( 325 + enrollment: &Enrollment, 326 + query: Option<&Embedding>, 327 + kind: CameraKind, 328 + threshold: f32, 329 + ) -> SingleVerdict { 330 + let Some(query) = query else { 331 + return SingleVerdict::NoFace; 332 + }; 333 + match enrollment.best_match(query, kind) { 334 + Some(m) if m.similarity >= threshold => SingleVerdict::Pass(m), 335 + Some(m) => SingleVerdict::BelowThreshold(m), 336 + None => SingleVerdict::NoSamples, 337 + } 260 338 } 261 339 262 340 /// Result of comparing one query embedding against one enrollment. ··· 306 384 Self::Reject { best } => Some(*best), 307 385 Self::NoEnrollment | Self::NoSamplesForModality { .. } => None, 308 386 } 387 + } 388 + } 389 + 390 + /// Per-modality verdict in a [`DualMatchOutcome::Reject`]. 391 + /// 392 + /// Surfacing both verdicts on a reject — rather than collapsing to a 393 + /// single "failed" — keeps the failure mode debuggable: an operator can 394 + /// tell at a glance whether RGB was below threshold (poor capture 395 + /// quality), IR found no face (presentation-attack defense triggered), 396 + /// neither modality had any enrolled samples to match against 397 + /// (enrollment incomplete), or some combination. 398 + #[derive(Debug, Clone, Copy, PartialEq)] 399 + pub enum SingleVerdict { 400 + /// Best match in this modality's pool met or exceeded the 401 + /// threshold. 402 + Pass(MatchResult), 403 + /// Best match in this modality's pool was below the threshold. 404 + BelowThreshold(MatchResult), 405 + /// No embedding was supplied for this modality (capture failed, no 406 + /// face detected, or this modality wasn't engaged at all). The 407 + /// matcher can't decide one way or the other so the dual outcome 408 + /// is forced to [`DualMatchOutcome::Reject`]. 409 + NoFace, 410 + /// The enrollment exists but contains no samples for this modality. 411 + /// Remediation is to enroll for the missing modality; the 412 + /// equivalent of [`MatchOutcome::NoSamplesForModality`] inside the 413 + /// dual outcome. 414 + NoSamples, 415 + } 416 + 417 + /// Outcome of a dual-modality authentication attempt. 418 + /// 419 + /// Mirrors [`MatchOutcome`] but requires both modalities to pass 420 + /// independently for the overall outcome to be [`Self::Pass`]. See 421 + /// [`Enrollment::authenticate_dual`] for the security rationale. 422 + #[derive(Debug, Clone, Copy, PartialEq)] 423 + pub enum DualMatchOutcome { 424 + /// Both modalities cleared the threshold against the user's 425 + /// enrollment. The per-modality best matches are kept for 426 + /// audit / logging. 427 + Pass { rgb: MatchResult, ir: MatchResult }, 428 + /// At least one modality failed. Both per-modality verdicts are 429 + /// preserved so the failure mode is fully diagnosable. 430 + Reject { 431 + rgb: SingleVerdict, 432 + ir: SingleVerdict, 433 + }, 434 + /// The enrollment has no samples at all — distinguished from 435 + /// [`Self::Reject`] for the same reason 436 + /// [`MatchOutcome::NoEnrollment`] is: the remediation is to enroll, 437 + /// not to retry. The same outcome can also surface inside `Reject` 438 + /// as `SingleVerdict::NoSamples` for *one* modality; this variant 439 + /// fires only when *both* pools are empty. 440 + NoEnrollment, 441 + } 442 + 443 + impl DualMatchOutcome { 444 + /// Convenience for the PAM-side decision: only `Pass` authenticates. 445 + pub fn is_pass(&self) -> bool { 446 + matches!(self, Self::Pass { .. }) 309 447 } 310 448 } 311 449 ··· 550 688 .is_none()); 551 689 } 552 690 691 + // ----- authenticate_dual ------------------------------------------------- 692 + 693 + /// Build an enrollment with N RGB samples and M IR samples, all 694 + /// along axis 0. Test fixture; the actual embedding values don't 695 + /// matter beyond "lets us reason about Pass/Reject by varying the 696 + /// query". 697 + fn dual_enrollment(rgb_count: u64, ir_count: u64) -> Enrollment { 698 + let mut e = Enrollment::new("alice"); 699 + let emb = unit_along(0, 4); 700 + for i in 0..rgb_count { 701 + e.push(sample(i + 1, emb.clone())); 702 + } 703 + for i in 0..ir_count { 704 + let mut s = sample(rgb_count + i + 1, emb.clone()); 705 + s.camera_kind = CameraKind::Ir; 706 + e.push(s); 707 + } 708 + e 709 + } 710 + 711 + #[test] 712 + fn authenticate_dual_empty_enrollment_yields_no_enrollment_regardless_of_queries() { 713 + let e = Enrollment::new("alice"); 714 + let q = unit_along(0, 4); 715 + assert_eq!( 716 + e.authenticate_dual(Some(&q), Some(&q), 0.6), 717 + DualMatchOutcome::NoEnrollment, 718 + ); 719 + // Same outcome even when no queries supplied — empty enrollment 720 + // short-circuits before any per-modality work. 721 + assert_eq!( 722 + e.authenticate_dual(None, None, 0.6), 723 + DualMatchOutcome::NoEnrollment, 724 + ); 725 + } 726 + 727 + #[test] 728 + fn authenticate_dual_both_queries_above_threshold_yields_pass_with_both_matches() { 729 + let e = dual_enrollment(2, 2); 730 + let q = unit_along(0, 4); 731 + match e.authenticate_dual(Some(&q), Some(&q), 0.6) { 732 + DualMatchOutcome::Pass { rgb, ir } => { 733 + assert!(rgb.similarity >= 0.6); 734 + assert!(ir.similarity >= 0.6); 735 + // RGB samples come first by construction (ids 1..=2), IR after. 736 + assert!(matches!(rgb.sample_id, SampleId(1 | 2))); 737 + assert!(matches!(ir.sample_id, SampleId(3 | 4))); 738 + } 739 + other => panic!("expected Pass, got {other:?}"), 740 + } 741 + } 742 + 743 + #[test] 744 + fn authenticate_dual_rgb_passes_ir_below_threshold_yields_reject_with_per_modality_verdicts() { 745 + let e = dual_enrollment(1, 1); 746 + let aligned = unit_along(0, 4); // matches RGB and IR samples (both along axis 0) 747 + let orthogonal = unit_along(1, 4); // similarity 0 against axis-0 samples 748 + match e.authenticate_dual(Some(&aligned), Some(&orthogonal), 0.6) { 749 + DualMatchOutcome::Reject { 750 + rgb: SingleVerdict::Pass(_), 751 + ir: SingleVerdict::BelowThreshold(best), 752 + } => { 753 + assert!(best.similarity.abs() < 1e-6); 754 + } 755 + other => panic!("expected Reject(Pass, BelowThreshold), got {other:?}"), 756 + } 757 + } 758 + 759 + #[test] 760 + fn authenticate_dual_missing_rgb_query_surfaces_no_face_and_rejects() { 761 + let e = dual_enrollment(1, 1); 762 + let q = unit_along(0, 4); 763 + match e.authenticate_dual(None, Some(&q), 0.6) { 764 + DualMatchOutcome::Reject { 765 + rgb: SingleVerdict::NoFace, 766 + ir: SingleVerdict::Pass(_), 767 + } => {} 768 + other => panic!("expected Reject(NoFace, Pass), got {other:?}"), 769 + } 770 + } 771 + 772 + #[test] 773 + fn authenticate_dual_both_queries_missing_rejects_with_two_no_face_verdicts() { 774 + let e = dual_enrollment(1, 1); 775 + match e.authenticate_dual(None, None, 0.6) { 776 + DualMatchOutcome::Reject { 777 + rgb: SingleVerdict::NoFace, 778 + ir: SingleVerdict::NoFace, 779 + } => {} 780 + other => panic!("expected Reject(NoFace, NoFace), got {other:?}"), 781 + } 782 + } 783 + 784 + #[test] 785 + fn authenticate_dual_rgb_only_enrollment_with_ir_query_surfaces_no_samples_for_ir() { 786 + // RGB-only enrollment; the user supplies queries for both 787 + // modalities but the enrollment has no IR pool. RGB passes, IR 788 + // collapses to NoSamples — the overall outcome rejects with 789 + // both verdicts visible. 790 + let e = dual_enrollment(2, 0); 791 + let q = unit_along(0, 4); 792 + match e.authenticate_dual(Some(&q), Some(&q), 0.6) { 793 + DualMatchOutcome::Reject { 794 + rgb: SingleVerdict::Pass(_), 795 + ir: SingleVerdict::NoSamples, 796 + } => {} 797 + other => panic!("expected Reject(Pass, NoSamples), got {other:?}"), 798 + } 799 + } 800 + 801 + #[test] 802 + fn authenticate_dual_both_modalities_below_threshold_preserves_both_below_threshold_verdicts() { 803 + let e = dual_enrollment(1, 1); 804 + let orthogonal = unit_along(1, 4); 805 + match e.authenticate_dual(Some(&orthogonal), Some(&orthogonal), 0.6) { 806 + DualMatchOutcome::Reject { 807 + rgb: SingleVerdict::BelowThreshold(_), 808 + ir: SingleVerdict::BelowThreshold(_), 809 + } => {} 810 + other => panic!("expected Reject(BelowThreshold, BelowThreshold), got {other:?}"), 811 + } 812 + } 813 + 814 + #[test] 815 + fn dual_match_outcome_is_pass_only_for_pass_variant() { 816 + let pass = DualMatchOutcome::Pass { 817 + rgb: MatchResult { 818 + sample_id: SampleId(1), 819 + similarity: 0.9, 820 + }, 821 + ir: MatchResult { 822 + sample_id: SampleId(2), 823 + similarity: 0.8, 824 + }, 825 + }; 826 + assert!(pass.is_pass()); 827 + let reject = DualMatchOutcome::Reject { 828 + rgb: SingleVerdict::NoFace, 829 + ir: SingleVerdict::NoFace, 830 + }; 831 + assert!(!reject.is_pass()); 832 + assert!(!DualMatchOutcome::NoEnrollment.is_pass()); 833 + } 834 + 553 835 // ----- property tests ---------------------------------------------------- 554 836 555 837 fn nonzero_vec(dim: usize) -> impl Strategy<Value = Vec<f32>> { ··· 644 926 } 645 927 let out = e.authenticate(&Embedding::normalized(query), CameraKind::Rgb, 1.001); 646 928 prop_assert!(!out.is_pass(), "threshold above 1.0 should reject everything"); 929 + } 930 + 931 + /// Dual-modality `Pass` is equivalent to both single-modality 932 + /// `authenticate` calls passing independently. The soundness 933 + /// invariant of the dual-modality auth design: callers can 934 + /// reason about `authenticate_dual` as "AND over the two 935 + /// single-modality outcomes" without having to inspect the 936 + /// dual implementation. 937 + #[test] 938 + fn authenticate_dual_pass_iff_both_single_modality_pass( 939 + rgb_samples in proptest::collection::vec(nonzero_vec(8), 1..=4), 940 + ir_samples in proptest::collection::vec(nonzero_vec(8), 1..=4), 941 + rgb_query in nonzero_vec(8), 942 + ir_query in nonzero_vec(8), 943 + threshold in -1.0f32..=1.0, 944 + ) { 945 + let mut e = Enrollment::new("alice"); 946 + let mut next_id = 1u64; 947 + for s in &rgb_samples { 948 + e.push(sample(next_id, Embedding::normalized(s.clone()))); 949 + next_id += 1; 950 + } 951 + for s in &ir_samples { 952 + let mut sm = sample(next_id, Embedding::normalized(s.clone())); 953 + sm.camera_kind = CameraKind::Ir; 954 + e.push(sm); 955 + next_id += 1; 956 + } 957 + let rgb_q = Embedding::normalized(rgb_query); 958 + let ir_q = Embedding::normalized(ir_query); 959 + 960 + let single_rgb = e.authenticate(&rgb_q, CameraKind::Rgb, threshold); 961 + let single_ir = e.authenticate(&ir_q, CameraKind::Ir, threshold); 962 + let dual = e.authenticate_dual(Some(&rgb_q), Some(&ir_q), threshold); 963 + 964 + let both_singles_passed = single_rgb.is_pass() && single_ir.is_pass(); 965 + prop_assert_eq!( 966 + dual.is_pass(), 967 + both_singles_passed, 968 + "dual.is_pass()={} but single_rgb.is_pass()={} && single_ir.is_pass()={}", 969 + dual.is_pass(), 970 + single_rgb.is_pass(), 971 + single_ir.is_pass(), 972 + ); 973 + } 974 + 975 + /// A missing query in either modality forces `Reject` for any 976 + /// non-empty enrollment, regardless of what the other modality 977 + /// would have decided. This is the design contract that lets 978 + /// callers express "couldn't capture this side" as `None` 979 + /// without worrying that it might silently degrade to a 980 + /// single-modality pass. 981 + #[test] 982 + fn authenticate_dual_with_either_query_missing_never_passes_on_nonempty( 983 + rgb_samples in proptest::collection::vec(nonzero_vec(8), 1..=3), 984 + ir_samples in proptest::collection::vec(nonzero_vec(8), 1..=3), 985 + present_query in nonzero_vec(8), 986 + present_on_rgb in proptest::bool::ANY, 987 + threshold in -1.0f32..=1.0, 988 + ) { 989 + let mut e = Enrollment::new("alice"); 990 + let mut next_id = 1u64; 991 + for s in &rgb_samples { 992 + e.push(sample(next_id, Embedding::normalized(s.clone()))); 993 + next_id += 1; 994 + } 995 + for s in &ir_samples { 996 + let mut sm = sample(next_id, Embedding::normalized(s.clone())); 997 + sm.camera_kind = CameraKind::Ir; 998 + e.push(sm); 999 + next_id += 1; 1000 + } 1001 + let q = Embedding::normalized(present_query); 1002 + let dual = if present_on_rgb { 1003 + e.authenticate_dual(Some(&q), None, threshold) 1004 + } else { 1005 + e.authenticate_dual(None, Some(&q), threshold) 1006 + }; 1007 + prop_assert!(!dual.is_pass(), "expected !pass with one side None, got {dual:?}"); 647 1008 } 648 1009 649 1010 /// Removing a sample never *increases* the best-match similarity.

+43 -1

crates/integration-tests/tests/integration/enrollment.rs

··· 3 3 use pareidolia_core::camera::CameraKind; 4 4 use pareidolia_core::enrollment::store::{EnrollmentStore, StoreError, KEY_LEN}; 5 5 use pareidolia_core::enrollment::{ 6 - EnrolledSample, Enrollment, MatchOutcome, ModelFingerprint, SampleId, DEFAULT_THRESHOLD, 6 + DualMatchOutcome, EnrolledSample, Enrollment, MatchOutcome, ModelFingerprint, SampleId, 7 + SingleVerdict, DEFAULT_THRESHOLD, 7 8 }; 8 9 use pareidolia_core::pipeline::Embedding; 9 10 use std::time::SystemTime; ··· 70 71 kind: CameraKind::Ir 71 72 } 72 73 ); 74 + } 75 + 76 + #[test] 77 + fn public_dual_modality_authenticate_requires_both_pools_to_pass() { 78 + // End-to-end through the public dual-modality auth API. Two 79 + // observations pinned at once: 80 + // 1. Pass requires both RGB *and* IR queries to clear the 81 + // threshold against their respective pools. 82 + // 2. A missing query in either modality collapses the outcome to 83 + // Reject with NoFace surfacing on the missing side — the 84 + // contract that lets daemon/CLI callers express "couldn't 85 + // capture this side" as None without risking a silent 86 + // single-modality pass. 87 + let mut enrollment = Enrollment::new("alice"); 88 + let face = unit_along(0, 512); 89 + // One RGB sample. 90 + enrollment.push(sample(1, face.clone())); 91 + // One IR sample (override the default-Rgb camera_kind on `sample`). 92 + let mut ir = sample(2, face.clone()); 93 + ir.camera_kind = CameraKind::Ir; 94 + enrollment.push(ir); 95 + 96 + // Both queries present and matching → Pass with both per-modality 97 + // results carried through. 98 + match enrollment.authenticate_dual(Some(&face), Some(&face), DEFAULT_THRESHOLD) { 99 + DualMatchOutcome::Pass { rgb, ir } => { 100 + assert_eq!(rgb.sample_id, SampleId(1)); 101 + assert_eq!(ir.sample_id, SampleId(2)); 102 + } 103 + other => panic!("expected dual Pass, got {other:?}"), 104 + } 105 + 106 + // Missing IR query → Reject with NoFace on the IR side, regardless 107 + // of how strongly RGB matches. 108 + match enrollment.authenticate_dual(Some(&face), None, DEFAULT_THRESHOLD) { 109 + DualMatchOutcome::Reject { 110 + rgb: SingleVerdict::Pass(_), 111 + ir: SingleVerdict::NoFace, 112 + } => {} 113 + other => panic!("expected Reject(Pass, NoFace), got {other:?}"), 114 + } 73 115 } 74 116 75 117 #[test]

Configure Feed

Configure Feed