Kleidos Linux QA harness#

A reusable Linux QA-mode harness for the Kleidos hardware password manager: BLE-HID + WiFi-admin, normal and attack paths. It runs on a Linux QA server and is driven either from the dev machine (over SSH) or directly on the box.

The QA server is typically a PRODUCTION machine. Read the safety model below before running anything. The harness is built so that it can only ever touch the one Bluetooth controller and the one WiFi NIC named in .env.

Configuration (no secrets in git)#

All coordinates — SSH host/port/user, interface names, SoftAP subnet, optional DUT addresses — live in a gitignored .env. Tracked code carries placeholders only and reads everything from .env (or matching QA_* env vars / CLI flags).

cd scripts/qa/linux-harness
cp .env.example .env
$EDITOR .env          # fill in QA_SSH_*, QA_BT_HCI, QA_WIFI_IFACE, QA_ETH_IFACE, ...

Key variables (see .env.example for the full, commented set):

The harness fails closed: if QA_BT_HCI, QA_WIFI_IFACE, or QA_ETH_IFACE are unset, it aborts (SAFETY_FAIL) rather than guessing an interface.

Harness selection policy#

• Preferred: the Linux QA server (this harness) when .env is configured. It gives a real BlueZ + WiFi stack (numeric-comparison pairing agent, monitor mode, evil-twin, portal fuzzing) on commodity hardware.
• Fallback: when no Linux server is available, use the ESP32 serial harness — build env m5stickc_plus1_host and run the Plus1 serial harness against a DUT.

run.py enforces this: if QA_SSH_HOST is empty and you did not pass --local, it stops and points you at the Plus1 serial fallback.

Safety model (non-negotiable)#

1. Interface pinning. Every script reads the controller / WiFi / Ethernet names from .env. run.py refuses an --iface/--hci that does not match the configured ones; the bash guards (lib/safety.sh) refuse the Ethernet iface and Docker bridge names (br-*/docker0/veth*) outright.
2. The default route is sacred. kqa_assert_route_safe asserts ip route show default is via $QA_ETH_IFACE and not via $QA_WIFI_IFACE — it runs before and after every WiFi op and aborts (SAFETY_FAIL, exit 3) otherwise.
3. No DHCP, no default route via WiFi. $QA_WIFI_IFACE is brought up with a static IP in the SoftAP subnet (subnet route only). No DHCP client is ever started.
4. WiFi iface left DOWN when not actively testing; the controller's BLE knobs (sc, public-addr) are always restored to their original values.
5. The bluetooth service is used only via btmgmt/D-Bus; hostapd is masked so it never auto-starts, and is run by hand for the rogue-AP scenario only.

Layout#

scripts/qa/linux-harness/
├── .env.example        # config template (committed); copy to .env (gitignored)
├── run.py              # QA-mode runner: SSH wrapper + local dispatcher
├── setup.sh            # idempotent provisioning (venv + apt tools + qa-env)
├── selftest.sh         # device-independent self-test orchestrator
├── lib/
│   ├── safety.sh       # route/iface/hci guards (sourced)
│   └── qa_common.sh    # .env loader + OK/FAIL/SKIP emitters (sourced)
├── ble/
│   ├── nc_agent.py     # BlueZ numeric-comparison auto-confirm agent (system py)
│   ├── bluez_util.py   # BlueZ bond-state + uhid/evdev helpers (no bleak dep)
│   ├── ble_normal.py   # scan/connect/pair/HOGP/subscribe/reconnect (bleak)
│   ├── ble_attack.sh   # E1-E6 attack scenarios (btmgmt) + N2 fuzz hook
│   └── ble_fuzz.py     # N2 malformed-ATT-PDU robustness fuzz (raw L2CAP/ATT)
└── wifi/
    ├── wifi_net.sh     # WiFi iface up(static)/connect/down helpers (sourced)
    ├── wifi_normal.sh  # station connect (WPA2/SAE) + portal checks
    ├── portal_checks.py# admin-portal HTTP normal + negative/fuzz (requests)
    └── wifi_attack.sh  # monitor capture, evil-twin, injection probe

Output uses the machine-greppable OK/FAIL/SKIP marker convention: *_OK / *_FAIL / *_SKIP lines plus a SUITE_SUMMARY ... pass= fail= skip=. *_SKIP means intentionally not run (e.g. no DUT in range) and does not fail the suite.

Quick start#

# First time: provision the server (idempotent — safe to re-run).
python3 scripts/qa/linux-harness/run.py --suite setup

# Device-independent self-test (no DUT needed).
python3 scripts/qa/linux-harness/run.py --suite selftest

# Individual suites.
python3 scripts/qa/linux-harness/run.py --suite ble-normal
python3 scripts/qa/linux-harness/run.py --suite ble-attack --scenario blerp
python3 scripts/qa/linux-harness/run.py --suite ble-fuzz
python3 scripts/qa/linux-harness/run.py --suite wifi-normal
python3 scripts/qa/linux-harness/run.py --suite wifi-attack --scenario evil-twin

run.py SSHes to $QA_SSH_HOST, syncs the harness to $QA_DEPLOY_DIR/harness (tar-over-ssh, no rsync needed — the gitignored .env is carried along so the server-side suites read the same config), and runs the suite there. To run on the server directly, add --local.

Suites#

setup#

Creates $QA_DEPLOY_DIR/venv (--system-site-packages so it inherits python3-dbus/gi/requests) and installs bleak; apt-installs only iw wireless-tools wpasupplicant hostapd aircrack-ng tcpdump rfkill; masks the hostapd service; writes the pinned $QA_DEPLOY_DIR/qa-env. Re-runnable.

ble-normal (BLE controller)#

Scans for the $QA_DUT_NAME_PREFIX* device (a re-scan loop that covers the DUT's ~30 s advertising window — a single short scan can race), hands the discovered BLEDevice object straight to BleakClient, connects, pairs via LE Secure Connections + Numeric Comparison (nc_agent.py D-Bus agent auto-confirms the matched value), validates HOGP, decodes typed keystrokes, reconnects the bonded peer, lists bonds. If no DUT is advertising, the scan mechanics still run and the DUT steps report *_SKIP.

Two robustness details:

• Bond is asserted on the persisted record, not Pair()'s return value or a transient live flag. BlueZ Device1.Pair() returns void/None on success, so a real bond was being mis-scored as AUTH_FAIL; and the live Device1.Bonded D-Bus property can momentarily read False right after Pair() while bluetoothd is still committing the keys, mis-scoring a real bond as not-bonded. The suite now polls for the persisted bond record (/var/lib/bluetooth/<adapter>/<dev>/info carrying a long-term-key section), which BlueZ only writes once the bond is settled to disk, and scores AUTH_OK on that — falling back to the live Paired/Bonded/Connected+ServicesResolved flags only if the persisted record never appears. (BlueZ names the bond dir by the peer's identity address — which differs from the address bleak connected on when the DUT advertises an RPA — and writes the key under [PeripheralLongTermKey], so the check accepts any long-term-key section and scans every bond info rather than keying on one address/name; the suite bonds exactly one DUT per run.)
• HOGP is validated through the kernel uhid node, not GATT. The default bluetoothd input (HoG) plugin claims HID 0x1812 and builds a real Linux keyboard, so a GATT central (bleak) can't discover/subscribe the report characteristic. The suite locates the /dev/input/eventN node the plugin created for the DUT (matched by BD address in /proc/bus/input/devices) and decodes the EV_KEY keydowns — exactly what a real host's HID stack does, and non-invasive (no bluetoothd restart on the production box). If the HID service is exposed to GATT (input plugin disabled), it falls back to a GATT subscribe.

Keystroke privacy: by default only the decoded length / keydown count is printed. Pass --show-text (via -- --show-text) only with a known test credential.

Driving the DUT (serial debug commands)#

To stage the DUT for ble-normal/ble-attack, drive it over its serial debug console (a _debug build). The commands that actually exist on the firmware are BLETYPE [id] [ALL|USER|PASS] (begin advertising + stage typing), BLESEND (send the staged payload), BLEPAIR <ACCEPT|REJECT> (resolve a pending numeric-comparison request — requires the argument), BLEAUTOACCEPT ON|OFF, BLESTOP, BLEFORGET ALL, BLE?/BLEDIAG (introspection).

Doc discrepancy flagged for the firmware team (do not "fix" in the harness): docs/testing/serial-debug-commands.md lists "BLEPAIR / BLETYPE | Begin advertising / typing flows", but bare BLEPAIR is "Unknown command" — the firmware registers it as a Match::Prefix on "BLEPAIR " (trailing space), so it only matches BLEPAIR <ACCEPT|REJECT> and never begins advertising. BLETYPE is what begins advertising; BLEPAIR only resolves the NC prompt. The doc conflates the two. The harness uses BLETYPE.

ble-attack (BLE controller) — maps to enhancements E1–E6 + N2#

DUT-side assertions need QA_DUT_ADDR (and QA_SPOOF_ADDR for blerp); without them the controller-side mechanics (knob toggles, agent registration, PDU crafting) are self-tested and the DUT assertions are *_SKIP / BASELINE notes. The rpa scenario asserts E1 for real (anti-tracking + IRK resolution); accept-list is already wired for real DUT assertions so it needs no rewrite when E2 ships — it just starts passing.

accept-list clears the local bond (to become an unbonded identity) by default; re-run ble-normal afterwards to re-bond, or set QA_ACCEPTLIST_CLEAR_BOND=0 to keep the bond and skip the unbonded assertion.

BLERP public-address spoof is opt-in. Overriding the controller's public address is default-SKIP and only runs with QA_ALLOW_ADDR_SPOOF=1. It needs a spoof-capable controller (e.g. a CSR8510 USB dongle): Intel controllers tx-timeout on the vendor command (HCI 0xfc31) and wedge with "No way to reset", recoverable only by a USB re-enumeration of the btusb device. The default run never issues that command.

ble-fuzz (BLE controller) — N2 malformed-PDU / ATT-GATT robustness#

Sends a small, bounded catalogue of malformed / oversized / boundary ATT PDUs at the DUT and asserts it survives — a secrets-typing peripheral must never crash on hostile input. Run standalone (--suite ble-fuzz) or as the fuzz scenario of ble-attack.

• Transport (safety): a kernel-managed AF_BLUETOOTH / BTPROTO_L2CAP socket connected to the ATT fixed CID 0x0004 via a raw sockaddr_l2 (ctypes). This rides the normal BLE stack — bluetoothd stays up, the controller is not seized with an HCI user channel, and nothing touches WiFi / Ethernet / the default route. Bounded: a fixed PDU catalogue, short timeouts, one connection at a time, no flooding.
• What it sends (tagged FUZZ_PDU_SENT <tag>): reserved opcode 0xFF, truncated/under-length requests, invalid/reversed/zero/out-of-range handles, Read-By-Group/Type length mismatches, oversized writes (> MTU), prepared-write with a bogus offset, a truncated signed write, an unsolicited confirmation, and a spoofed server PDU.
• Survival check: after every burst it re-opens an ATT connection + Exchange-MTU. Still answering ⇒ FUZZ_BURST_SURVIVED_OK; stops answering ⇒ FUZZ_DUT_UNRESPONSIVE_FAIL (a firmware finding — the harness reports, it does not fix firmware).
• Address type: set QA_DUT_ADDR_TYPE (random for the Kleidos static-random address, the default; public otherwise).
• No DUT: self-tests the PDU crafting + sockaddr_l2 packing and prints the tagged catalogue (green in selftest); the live sends are *_SKIP.
• Live fuzz needs the ATT CID free. Opening the ATT fixed CID (0x0004) from a userspace L2CAP socket requires bind()ing to the local controller and only succeeds when bluetoothd does not own that CID. On a bluetoothd-managed adapter the connect hangs in EINPROGRESS and the scenario FUZZ_LIVENESS_SKIPs with that explanation; validated live on mastodon by briefly freeing the CID (DUT returned proper ATT Error Responses to every malformed PDU and stayed responsive — FUZZ_DONE_OK).

wifi-normal (WiFi iface)#

Brings the WiFi iface up (static IP, no default route), scans for the $QA_DUT_NAME_PREFIX* SoftAP, associates (WPA2 or QA_AP_MODE=sae for WPA3), runs the portal checks, tears the iface back DOWN. Provide QA_AP_SSID + QA_AP_PSK to run the full station+portal pass; otherwise it self-tests bring-up/scan/teardown with route assertions and *_SKIPs the connect.

portal_checks.py: portal reachable; status endpoint is accepted either as a public 200 with the expected shape or as an auth-required 401/403 (requiring auth on /api/status is the more-secure behavior and must not be scored as a failure); auth-required sensitive endpoints reject unauthenticated access; and a negative/fuzz pass (path traversal, oversize paths, wrong methods, malformed bodies) that must not 5xx. Response bodies are never printed.

wifi-attack (WiFi iface)#

• inject-probe — empirically runs aireplay-ng --test in monitor mode. If the WiFi chip cannot inject, the deauth/injection scenarios auto-disable (INJECT_PROBE_SKIP + DEAUTH_SKIP, clean exit — not a failure). The probe verdict is printed as INJECT_SUPPORTED=yes|no.
• monitor — monitor-mode capture of management frames near the SoftAP.
• evil-twin — rogue AP cloning the Kleidos admin SSID via hostapd.

Monitor + evil-twin work on most chips; injection/deauth generally do not on Intel iwlwifi — they need an mt76/Atheros/Realtek USB dongle. The harness detects this and skips cleanly.

Run-later (DUT-dependent) staging#

When a Kleidos DUT is available, set its coordinates in .env (or pass as env):

# BLE: drive the DUT with BLETYPE (begin advertising + stage typing) on its
# serial debug console, put it in pairing range, then:
python3 .../run.py --suite ble-normal -- --show-text   # known test credential only
QA_DUT_ADDR=<dut-ble-addr> \
  python3 .../run.py --suite ble-attack --scenario sc-reject
QA_DUT_ADDR=<dut-ble-addr> \
  python3 .../run.py --suite ble-attack --scenario accept-list   # E2 baseline
QA_DUT_ADDR=<dut-ble-addr> \
  python3 .../run.py --suite ble-attack --scenario rpa           # E1 anti-tracking + IRK resolution
QA_DUT_ADDR=<dut-ble-addr> QA_DUT_ADDR_TYPE=random \
  python3 .../run.py --suite ble-fuzz                            # N2 robustness
QA_DUT_ADDR=<dut-ble-addr> QA_SPOOF_ADDR=<bonded-central-addr> \
  python3 .../run.py --suite ble-attack --scenario blerp

# WiFi: put the DUT in ADMIN_MODE (SoftAP up), then:
QA_AP_SSID=<softap-ssid> QA_AP_PSK=<softap-pass> QA_AP_MODE=wpa2 \
  python3 .../run.py --suite wifi-normal

(run.py forwards these to the remote suite.)