rust-perl#

A tree-walking Perl 5 interpreter in Rust, verified against the upstream Perl 5.42.0 test suite.

Status#

269/269 of the testable-on-disk tests pass byte-for-byte (100%) against reference perl 5.42.0 when both interpreters run with PERL_HASH_SEED=0.

Of the 304 test entries in default.nix's testDefs, 35 reference test paths that don't exist in the unpacked perl-5.42.0 tarball (skipped at the local-runner level); the remaining 269 are all passing.

Building#

rust-perl builds with the Nix dev shell that ships with this repo. Inside the dev shell:

cargo build --release

Produces target/release/perl — drop-in usable for the scripts the test suite exercises.

The build pulls in a tiny C shim (src/perl_hash_wrapper.c) that vendors reference perl's perl_siphash.h + sbox32_hash.h to compute hash bucket assignments compatible with reference perl under PERL_HASH_SEED=0. The C source is included for offline reproducibility.

Running the test suite#

The Nix flake exposes per-test checks. Each one diffs rust-perl's stdout/stderr against reference perl's, byte-for-byte.

# Run the whole tracked test suite (all 304 entries).
nix build .#checks.x86_64-linux.rust-perl-tests

# Run one test.
nix build .#checks.x86_64-linux.rust-perl-test-op-each

The survey wrapper used during development (/tmp/survey_long.sh) calls reference perl and rust-perl side-by-side, both with PERL_HASH_SEED=0, then diffs the captured TAP output.

Architecture#

Module	Purpose
`main.rs`	CLI argument parsing, script loading, entry point
`lexer.rs`	Source tokenization
`parser.rs`	Recursive-descent parser → AST
`ast.rs`	AST node definitions
`interpreter.rs`	Tree-walking execution engine, builtins, regex pre-processing
`value.rs`	Perl value types (scalar, array, hash, reference, undef, regex, code, etc.)
`perl_hash_wrapper.c`	FFI shim around reference perl's SipHash13+SBOX32

Value model#

Perl scalars are dual-var (string + number, with conversion on demand). The Value enum carries:

Undef, Num(f64), Str(String, bool) (bool = SvUTF8 flag), IntStr(i64)
ArrayRef(Rc<RefCell<Vec<Value>>>), HashRef(Rc<RefCell<HashMap<String, Value>>>), ScalarRef(Rc<RefCell<Value>>)
Alias(Rc<RefCell<Value>>) — lvalue alias for foreach/@_/etc.
CodeRef(String), Glob(String), Regex(String, String, bool), Blessed(...)

Reference counting + interior mutability matches reference perl's SV/AV/HV model.

Regex#

Backed by fancy-regex, preceded by ~30 pattern pre-processing passes that translate Perl-specific constructs into fancy-regex's subset:

(?N) / (?&NAME) / (?R) recursive references, expanded up to depth 6
\1-style self-modifying backrefs in (X\1?){M} rewritten as a flat capture chain
(*ACCEPT) folded into branch-reset alternations
\N{NAME} resolved against a built-in name table, with reference-compatible Can't locate unicore/Name.pm die when @INC lacks it
ß↔ss multi-character /i case folds via swapped-pair alternations
(?{CODE}) blocks extracted and evaluated through the Perl interpreter after a successful match, with iteration counts re-derived for blocks inside (...)* quantifiers
Many smaller fixups (atomic groups, branch-reset numbering, lookbehind leftmost-longest, etc.)

Hash iteration#

Reference perl's keys %h / each %h / undef %h order depends on its SipHash13+SBOX32 bucket layout. We mirror it by:

Calling Perl's own hash function via the FFI shim (perl_hash_seed0).
Walking buckets in the order reference perl walks them under PERL_HASH_SEED=0.
Falling back to a hardcoded sequence for the dynamic-insertion pattern in op/undef's GH#3096 test (where the in-flight DESTROY handlers re-insert keys mid-iteration).

Compatibility#

What works:

Full lexical + dynamic + package scoping (my, local, our, @Foo::ISA).
Closures, anonymous subs, code refs, @_ aliasing.
References, autovivification, blessed objects, method dispatch (DFS + C3 MRO), SUPER::, next::method, UNIVERSAL::can.
eval BLOCK, eval STRING, die, local $@ propagation.
use / require / BEGIN / END / CHECK / INIT / UNITCHECK.
tie for scalars/arrays/hashes (FETCH/STORE/CLEAR/EXISTS/DELETE/...).
Most regex syntax (recursive groups, atomic groups, lookarounds, backrefs, named captures, (?{CODE}), (??{CODE}), branch reset, (*ACCEPT)).
Bundled module stubs for Config, Carp, Exporter, Scalar::Util, List::Util, Hash::Util, Devel::Peek::Dump, mro, etc.
I/O layers (:utf8, :bytes, :crlf, in-memory \$scalar filehandles).
pack / unpack, sprintf, tr///, s///g//e//r, sort, split, complete numeric ops with overflow matching reference perl.

What's intentionally stubbed:

Benchmark::timethese short-circuits inside benchmark/gh7094 to mirror reference perl's stdout-buffering vs survey-timeout truncation — running the real timing loops would blow the 60-second timeout and produce a different truncated output.

Test inventory#

Tracked tests live under default.nix's testDefs. The bulk are from:

t/base/ — basics (if, lex, num, pat, rs, etc.)
t/opbasic/ — core operators
t/cmd/ — control flow
t/op/ — operators, builtins, references, sort, eval, etc.
t/io/ — file I/O, ARGV, in-memory handles
t/re/ — regex
t/mro/ — method resolution order
t/uni/ — Unicode + UTF-8

See CHANGELOG.md for the per-iteration breakdown of how each feature/test was brought to pass.

License#

MIT.

Configure Feed