fix: deterministic documentation-figure rendering (end the flaky figures job) (#131)
* fix: make documentation-figure rendering deterministic across machines
The "Documentation figures up to date" CI job was intermittently failing on the
heavy compute figures (excitation_signals, multiple_shock, sii_vocal_efforts,
sottek_specific_loudness, tonality_roughness_demo): their multi-threaded
numerical backends reorder floating-point reductions depending on the available
core count, so the CI runner produced byte-different SVG/PNG output than the
committed (single-threaded) baseline, and a plain re-run usually "fixed" it.
Pin the numerical thread pools to a single thread so rendering is reproducible
regardless of the host core count:
- generate_graphs.py / generate_diagrams.py set OMP/MKL/OPENBLAS/NUMEXPR/NUMBA/
VECLIB thread counts to 1 before the numeric backends import (covers direct
invocation).
- The Makefile `graphs` target also exports those plus PYTHONHASHSEED=0 before
the interpreter starts (PYTHONHASHSEED cannot be set from within the process);
CI runs `make graphs`, so it inherits this with no workflow change.
Regenerating every figure under these settings produces a zero-byte diff against
the committed images, confirming the current baseline is already the
single-threaded canonical output — so no figure files change here.
* fix: compare documentation figures within tolerance, not byte-exact
The "Documentation figures up to date" job regenerated .github/images with
`make graphs` and required a byte-identical result. That is unachievable on
GitHub's hardware-heterogeneous runner fleet: the pinned software stack
computes a few plotted path coordinates ~1 ULP apart depending on which CPU
microarchitecture (SIMD kernel) the run lands on, so the job failed
intermittently even though the figures were numerically and visually
identical. Single-thread pinning removes intra-machine reduction races but
cannot remove this cross-CPU last-bit difference.
Replace the byte diff with scripts/check_figures.py, which compares the
freshly regenerated figures against the committed versions within a tolerance:
* SVG -- non-numeric structure (elements, text, colours, ordering) must match
exactly; every numeric token must agree within an absolute/relative
tolerance. A moved element or relabelled axis fails; a last-bit
coordinate wobble passes.
* PNG -- identical dimensions, at most a small number of pixels changed beyond
a level threshold (catches localised changes a global RMS dilutes),
and a bounded RMS (catches broad changes the pixel count misses).
* other -- exact byte compare. Added/removed files always fail.
The thread-pinning determinism work from the previous commit is retained (it
still removes genuine intra-machine flakiness); this makes the check robust to
the cross-machine floating-point non-determinism it cannot eliminate.
* fix: harden figure check against hash-id drift and orphan figures
Two robustness gaps found reviewing the tolerance-aware figure check:
* SVG clip-path / marker ids are SHA256 hashes of the underlying float
geometry (matplotlib backend_svg._make_id). A cross-CPU 1-ULP wobble in that
geometry avalanches the whole hash, so the id text would change and trip the
strict structural gate -- the very drift the check exists to tolerate. Now
canonicalise every id and its url(#...)/href="#..." references to
first-appearance placeholders before comparing, so equivalent documents match
regardless of hash text while a genuine id add/remove/reorder/repoint fails.
* The generators only overwrite files, never delete, so a figure that is no
longer produced would survive on disk byte-identical to HEAD and pass the
"committed figure no longer generated" check. Clear the generated SVG/PNG at
the start of the `graphs` target before regenerating (animations *.gif/*.webm
are from the separate `animations` target and are preserved). A full
clear+regenerate reproduces the committed tree exactly, confirming no
committed figure is orphaned.