Commits
Address review feedback on the pluggable benchmark provider API. The
initial design mirrored the coverage provider (a `getProvider` factory
plus a `BenchmarkProviderModule` wrapper), which was a special case that
benchmarks — running only in tests — don't need.
- Drop `BenchmarkProviderModule` and `getProvider`: a custom provider
module now default-exports a `BenchmarkProvider` directly.
- Reuse the existing `TestModuleRunner` interface instead of the bespoke
`BenchmarkProviderLoader`.
- Remove the `TestRunner.runBenchmarks` override hook, now redundant with
the provider API, and inline `tinybench.run()` in the default provider.
Introduce a `benchmark.provider` config option that lets a plugin own
benchmark execution end to end. A provider receives the expanded
registrations of a single test and returns the results Vitest reports;
the returned array is the sole source of results, matched to
registrations by name.
The built-in tinybench execution moves behind a default provider,
selected when `benchmark.provider` is `'default'`. A custom provider is
a module path whose default export exposes `getProvider()`; it is
resolved to an absolute path during config resolution and imported once
per worker. `createBench` now runs every path — `bench().run()`,
`bench.compare()`, `writeResult` and comparison storage — through the
provider instead of reading results back off the tinybench instance.
The benchmark name used for reporting and the getter-overuse warning now
comes from `options.name` when set, falling back to the full test name,
dropping the previous per-run index suffix.
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Ari Perkkiö <ari.perkkio@gmail.com>
Co-authored-by: Hiroshi Ogawa <4232207+hi-ogawa@users.noreply.github.com>
Co-authored-by: OpenCode <noreply@opencode.ai>
Co-authored-by: Shaoyu Meng <shaoyumeng@shaoyus-mbp-5.lan>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Hiroshi Ogawa <4232207+hi-ogawa@users.noreply.github.com>
Co-authored-by: OpenCode (claude-opus-4-8) <noreply@opencode.ai>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Ari Perkkiö <ari.perkkio@gmail.com>
Co-authored-by: Hiroshi Ogawa <4232207+hi-ogawa@users.noreply.github.com>
Co-authored-by: OpenCode (claude-opus-4-8) <noreply@opencode.ai>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: vitest-release-bot[bot] <292707936+vitest-release-bot[bot]@users.noreply.github.com>
Co-authored-by: mdong1909 <mdong1909@users.noreply.github.com>
Co-authored-by: Hiroshi Ogawa <4232207+hi-ogawa@users.noreply.github.com>
Co-authored-by: OpenCode (claude-opus-4-8) <noreply@opencode.ai>
Co-authored-by: Hiroshi Ogawa <4232207+hi-ogawa@users.noreply.github.com>
Co-authored-by: OpenCode (claude-opus-4-8) <noreply@opencode.ai>
Co-authored-by: Hiroshi Ogawa <4232207+hi-ogawa@users.noreply.github.com>
Co-authored-by: OpenCode (claude-opus-4-8) <noreply@opencode.ai>
Co-authored-by: Hiroshi Ogawa <4232207+hi-ogawa@users.noreply.github.com>
Co-authored-by: OpenCode (claude-opus-4-8) <noreply@opencode.ai>
Co-authored-by: Ari Perkkiö <ari.perkkio@gmail.com>
Co-authored-by: Hiroshi Ogawa <4232207+hi-ogawa@users.noreply.github.com>
Co-authored-by: OpenCode (claude-opus-4-8) <noreply@opencode.ai>
Co-authored-by: Hiroshi Ogawa <4232207+hi-ogawa@users.noreply.github.com>
Co-authored-by: OpenCode (claude-opus-4-8) <noreply@opencode.ai>
Co-authored-by: Hiroshi Ogawa <4232207+hi-ogawa@users.noreply.github.com>
Co-authored-by: OpenCode (claude-opus-4-8) <noreply@opencode.ai>
Address review feedback on the pluggable benchmark provider API. The
initial design mirrored the coverage provider (a `getProvider` factory
plus a `BenchmarkProviderModule` wrapper), which was a special case that
benchmarks — running only in tests — don't need.
- Drop `BenchmarkProviderModule` and `getProvider`: a custom provider
module now default-exports a `BenchmarkProvider` directly.
- Reuse the existing `TestModuleRunner` interface instead of the bespoke
`BenchmarkProviderLoader`.
- Remove the `TestRunner.runBenchmarks` override hook, now redundant with
the provider API, and inline `tinybench.run()` in the default provider.
Introduce a `benchmark.provider` config option that lets a plugin own
benchmark execution end to end. A provider receives the expanded
registrations of a single test and returns the results Vitest reports;
the returned array is the sole source of results, matched to
registrations by name.
The built-in tinybench execution moves behind a default provider,
selected when `benchmark.provider` is `'default'`. A custom provider is
a module path whose default export exposes `getProvider()`; it is
resolved to an absolute path during config resolution and imported once
per worker. `createBench` now runs every path — `bench().run()`,
`bench.compare()`, `writeResult` and comparison storage — through the
provider instead of reading results back off the tinybench instance.
The benchmark name used for reporting and the getter-overuse warning now
comes from `options.name` when set, falling back to the full test name,
dropping the previous per-run index suffix.
Co-authored-by: vitest-release-bot[bot] <292707936+vitest-release-bot[bot]@users.noreply.github.com>