[READ-ONLY] Mirror of https://github.com/vitest-dev/vitest. Next generation testing framework powered by Vite.
vitest.dev
test
testing-tools
vite
docs: update Visual Regression Testing guide (#10514)
authored by
Raul Macarie
and committed by
GitHub
(Jun 11, 2026, 9:33 AM +0200)
8f85d20d
cbd862b3
+390
-331
+20
docs/.vitepress/components/MoonPhase.vue
+20
docs/.vitepress/components/MoonPhase.vue
···
1
+
<script setup lang="ts">
2
+
const phases = [
3
+
{ emoji: '🌑', alt: 'new moon' },
4
+
{ emoji: '🌒', alt: 'waxing crescent moon' },
5
+
{ emoji: '🌓', alt: 'first quarter moon' },
6
+
{ emoji: '🌔', alt: 'waxing gibbous moon' },
7
+
{ emoji: '🌕', alt: 'full moon' },
8
+
{ emoji: '🌖', alt: 'waning gibbous moon' },
9
+
{ emoji: '🌗', alt: 'last quarter moon' },
10
+
{ emoji: '🌘', alt: 'waning crescent moon' },
11
+
]
12
+
13
+
const phase = phases[
14
+
Math.floor(Math.random() * phases.length)
15
+
]
16
+
</script>
17
+
18
+
<template>
19
+
<span :aria-label="phase.alt">{{ phase.emoji }}</span>
20
+
</template>
+370
-331
docs/guide/browser/visual-regression-testing.md
+370
-331
docs/guide/browser/visual-regression-testing.md
···
3
3
outline: [2, 3]
4
4
---
5
5
6
+
<script setup>
7
+
import MoonPhase from '../../.vitepress/components/MoonPhase.vue'
8
+
</script>
9
+
6
10
# Visual Regression Testing
7
11
8
-
Vitest can run visual regression tests out of the box. It captures screenshots
9
-
of your UI components and pages, then compares them against reference images to
10
-
detect unintended visual changes.
12
+
Vitest can run visual regression tests out of the box. It captures screenshots of your UI components and pages, then compares them against reference images to detect unintended visual changes.
11
13
12
-
Unlike functional tests that verify behavior, visual tests catch styling issues,
13
-
layout shifts, and rendering problems that might otherwise go unnoticed without
14
-
thorough manual testing.
14
+
Unlike functional tests that verify behavior, visual tests catch styling issues, layout shifts, and rendering problems that might otherwise go unnoticed without thorough manual testing.
15
15
16
-
## Why Visual Regression Testing?
16
+
## Why visual regression testing?
17
17
18
-
Visual bugs don’t throw errors, they just look wrong. That’s where visual
19
-
testing comes in.
18
+
Visual bugs don’t throw errors, they just look wrong. That’s where visual testing comes in.
20
19
21
20
- That button still submits the form... but why is it hot pink now?
22
21
- The text fits perfectly... until someone views it on mobile
23
-
- Everything works great... except those two containers are out of viewport
22
+
- Everything works great... except those two containers are outside the viewport
24
23
- That careful CSS refactor works... but broke the layout on a page no one tests
25
24
26
-
Visual regression testing acts as a safety net for your UI, automatically
27
-
catching these visual changes before they reach production.
25
+
Visual regression testing acts as a safety net for your UI, automatically catching these visual changes before they reach production.
28
26
29
-
## Getting Started
27
+
## Example
30
28
31
-
::: warning Browser Rendering Differences
32
-
Visual regression tests are **inherently unstable across different
33
-
environments**. Screenshots will look different on different machines because
34
-
of:
35
-
36
-
- Font rendering (the big one. Windows, macOS, Linux, they all render text
37
-
differently)
38
-
- GPU drivers and hardware acceleration
39
-
- Whether you're running headless or not
40
-
- Browser settings and versions
41
-
- ...and honestly, sometimes just the phase of the moon
42
-
43
-
That's why Vitest includes the browser and platform in screenshot names (like
44
-
`button-chromium-darwin.png`).
45
-
46
-
For stable tests, use the same environment everywhere. We **strongly recommend**
47
-
cloud services like
48
-
[Azure App Testing](https://azure.microsoft.com/en-us/products/app-testing/)
49
-
or [Docker containers](https://playwright.dev/docs/docker).
50
-
:::
51
-
52
-
Visual regression testing in Vitest can be done through the
53
-
[`toMatchScreenshot` assertion](/api/browser/assertions.html#tomatchscreenshot):
29
+
Visual regression testing in Vitest can be done through the [`toMatchScreenshot` assertion](/api/browser/assertions#tomatchscreenshot):
54
30
55
31
```ts
56
32
import { expect, test } from 'vitest'
57
33
import { page } from 'vitest/browser'
58
34
59
-
test('hero section looks correct', async () => {
60
-
// ...the rest of the test
35
+
test('button renders in default state', async () => {
36
+
// render your component
61
37
62
38
// capture and compare screenshot
63
-
await expect(page.getByTestId('hero')).toMatchScreenshot('hero-section')
39
+
await expect(page.getByRole('button')).toMatchScreenshot()
64
40
})
65
41
```
66
42
67
-
### Creating References
43
+
## Getting started
68
44
69
-
When you run a visual test for the first time, Vitest creates a reference (also
70
-
called baseline) screenshot and fails the test with the following error message:
45
+
### Environmental stability
46
+
47
+
Visual regression tests are **sensitive to environmental differences** because rendering is not perfectly deterministic across environments and depends on multiple factors:
48
+
49
+
- GPU, drivers, and hardware acceleration
50
+
- Operating System
51
+
- Font rendering pipelines
52
+
- Browser, browser versions, and settings
53
+
- Whether the browser is running headless or headed
54
+
- Screen scaling, color profiles, and display settings
55
+
- ...and occasionally what feels like the phase of the moon <MoonPhase />
56
+
57
+
In practice, even seemingly identical environments can occasionally produce subtle rendering differences. For this reason, **visual regression tests are most reliable when run in a standardized and tightly controlled environment**. This is also why [Docker containers](https://playwright.dev/docs/docker), [CI-only visual testing workflows, or cloud services](#visual-testing-for-teams) are strongly recommended.
58
+
59
+
### Not a replacement for behavior testing
60
+
61
+
When a visual test fails alongside behavior tests, it's harder to tell what's actually broken or why. Visual failures are also expected during intentional UI work, but a failing unit test usually is not. Keeping them separate means each suite can fail loudly for the right reasons.
62
+
63
+
It's worth calling out that **`toMatchScreenshot` is not a substitute for proper assertions**.
64
+
65
+
A test that renders a button and just takes a screenshot is just documenting the current state. There's no way to tell from a screenshot whether users can interact with the button. **Visual tests work best as a complementary layer on top of behavior tests, not a replacement for them**.
66
+
67
+
Put another way, **visual testing doesn't tell you why something renders the way it does**. It just tells you that something rendered a certain way, or a different way than it did last time.
68
+
69
+
For example, take a business requirement to sort recent purchases in a table by purchase date. If you're looking only at the visual regression tests, you might notice that the same items from the last test are in a different order. This could be because you just introduced the sorting or because the sorting is broken. Either way, you don't know why the order is different just by looking at the UI. Someone could dismiss the visual diff as noise because the table "looks the same", even though the ordering logic is now broken. Now you have a broken business requirement in production.
70
+
71
+
### Project structure
72
+
73
+
Separating your visual suite from other tests gives you cleaner failure signals and a more deliberate update workflow. The recommended setup uses [projects](/guide/projects) with a `[name].vrt.test.[ext]` naming convention to keep them distinct, and runs them in headless mode for consistency. As the browser instance might have a different default size, it also sets a specific viewport size.
74
+
75
+
```ts [vitest.config.ts]
76
+
import { defaultExclude, defineConfig } from 'vitest/config'
77
+
78
+
const vrtPattern = '**/*.vrt.test.[tj]s?(x)'
79
+
80
+
export default defineConfig({
81
+
test: {
82
+
// ...other configurations
83
+
projects: [
84
+
{
85
+
extends: true,
86
+
test: {
87
+
name: 'unit',
88
+
exclude: [vrtPattern, ...defaultExclude],
89
+
},
90
+
},
91
+
{
92
+
extends: true,
93
+
test: {
94
+
name: 'vrt',
95
+
browser: {
96
+
headless: true,
97
+
instances: [
98
+
{
99
+
browser: '[browser-name]',
100
+
viewport: { width: 1280, height: 720 },
101
+
},
102
+
],
103
+
},
104
+
include: [vrtPattern],
105
+
},
106
+
},
107
+
],
108
+
},
109
+
})
110
+
```
111
+
112
+
With this configuration in place, add scripts to launch each project separately:
113
+
114
+
```json [package.json]
115
+
{
116
+
"scripts": {
117
+
"test:unit": "vitest --project unit",
118
+
"test:visual": "vitest --project vrt"
119
+
}
120
+
}
121
+
```
122
+
123
+
### Creating references
124
+
125
+
When you run a visual test for the first time, Vitest creates a reference (also called baseline) screenshot and fails the test with the following error message:
71
126
72
127
```
73
128
expect(element).toMatchScreenshot()
···
75
130
No existing reference screenshot found; a new one was created. Review it before running tests again.
76
131
77
132
Reference screenshot:
78
-
tests/__screenshots__/hero.test.ts/hero-section-chromium-darwin.png
133
+
tests/__screenshots__/button.vrt.test.ts/button-default-state-chromium-darwin.png
79
134
```
80
135
81
-
This is normal. Check that the screenshot looks right, then run the test again.
82
-
Vitest will now compare future runs against this baseline.
136
+
This is normal. Check that the screenshot looks right, then run the test again. Vitest will now compare future runs against this baseline.
83
137
84
138
::: tip
85
-
Reference screenshots live in `__screenshots__` folders next to your tests.
86
-
**Don't forget to commit them!**
139
+
Reference screenshots live in `__screenshots__` folders next to your tests. **Commit them to your repository.**
87
140
:::
88
141
89
-
### Screenshot Organization
142
+
### Screenshot organization
90
143
91
144
By default, screenshots are organized as:
92
145
93
146
```
94
147
.
95
148
├── __screenshots__
96
-
│ └── test-file.test.ts
149
+
│ └── test-file.vrt.test.ts
97
150
│ ├── test-name-chromium-darwin.png
98
151
│ ├── test-name-firefox-linux.png
99
152
│ └── test-name-webkit-win32.png
100
-
└── test-file.test.ts
153
+
└── test-file.vrt.test.ts
101
154
```
102
155
103
156
The naming convention includes:
104
-
- **Test name**: either the first argument of the `toMatchScreenshot()` call,
105
-
or automatically generated from the test's name.
106
-
- **Browser name**: `chrome`, `chromium`, `firefox` or `webkit`.
107
-
- **Platform**: `aix`, `darwin`, `freebsd`, `linux`, `openbsd`, `sunos`, or
108
-
`win32`.
157
+
- **Test name**: either the first argument of the `toMatchScreenshot()` call, or automatically generated from the test's name.
158
+
- **Browser name**: depends on the configured browser provider, for example `chrome`, `chromium`, `firefox` or `webkit`.
159
+
- **Platform**: `aix`, `darwin`, `freebsd`, `linux`, `openbsd`, `sunos`, or `win32`.
109
160
110
161
This ensures screenshots from different environments don't overwrite each other.
111
162
112
-
### Updating References
163
+
### Updating references
113
164
114
-
When you intentionally change your UI, you'll need to update the reference
115
-
screenshots:
165
+
When you intentionally change your UI, you'll need to update the reference screenshots just as you would update snapshots:
116
166
117
167
```bash
118
-
$ vitest --update
168
+
$ vitest --project vrt --update
119
169
```
120
170
121
-
Review updated screenshots before committing to make sure changes are
122
-
intentional.
171
+
Review updated screenshots before committing to make sure changes are intentional.
123
172
124
-
## How Visual Tests Work
173
+
::: warning Stale screenshots
174
+
Note that **screenshots for deleted or renamed tests aren't removed automatically**. Clean up the `__screenshots__` folder manually when you remove or rename tests, otherwise stale references will accumulate over time.
175
+
:::
125
176
126
-
Visual regression tests need stable screenshots to compare against. But pages aren't instantly stable as images load, animations finish, fonts render, and layouts settle.
177
+
### Debugging failed tests
127
178
128
-
Vitest handles this automatically through "Stable Screenshot Detection":
179
+
When a visual test fails, Vitest provides three images to help debug:
129
180
130
-
1. Vitest takes a first screenshot (or uses the reference screenshot if available) as baseline
131
-
1. It takes another screenshot and compares it with the baseline
132
-
- If the screenshots match, the page is stable and testing continues
133
-
- If they differ, Vitest uses the newest screenshot as the baseline and repeats
134
-
1. This continues until stability is achieved or the timeout is reached
181
+
1. **Reference screenshot**: the expected baseline image
182
+
1. **Actual screenshot**: what was captured during the test
183
+
1. **Diff image**: highlights the differences; only generated when the screenshots have the same dimensions (behavior may vary with custom matchers)
135
184
136
-
This ensures that transient visual changes (like loading spinners or animations) don't cause false failures. If something never stops animating though, you'll hit the timeout, so consider [disabling animations during testing](#disable-animations).
185
+
You'll see something like this in the CLI output:
137
186
138
-
If a stable screenshot is captured after retries (one or more) and a reference screenshot exists, Vitest performs a final comparison with the reference using `createDiff: true`. This will generate a diff image if they don't match.
187
+
```
188
+
expect(element).toMatchScreenshot()
139
189
140
-
During stability detection, Vitest calls comparators with `createDiff: false` since it only needs to know if screenshots match. This keeps the detection process fast.
190
+
Screenshot does not match the stored reference.
191
+
245 pixels (ratio 0.03) differ.
141
192
142
-
## Configuring Visual Tests
193
+
Reference screenshot:
194
+
tests/__screenshots__/button.vrt.test.ts/button-chromium-darwin.png
143
195
144
-
### Global Configuration
196
+
Actual screenshot:
197
+
tests/.vitest/attachments/button.vrt.test.ts/button-chromium-darwin-actual.png
145
198
146
-
Configure visual regression testing defaults in your
147
-
[Vitest config](/config/browser/expect#tomatchscreenshot):
199
+
Diff image:
200
+
tests/.vitest/attachments/button.vrt.test.ts/button-chromium-darwin-diff.png
201
+
```
148
202
149
-
```ts [vitest.config.ts]
203
+
While in UI mode, Vitest shows a tabbed diff view with an A/B slider as shown below.
204
+
205
+
<center>
206
+
<img alt="Animated demo of the visual regression diff view, switching tabs and using the slider to reveal differences" img-light src="/visual-regression/diff-view-light.avif">
207
+
<img alt="Animated demo of the visual regression diff view, switching tabs and using the slider to reveal differences" img-dark src="/visual-regression/diff-view-dark.avif">
208
+
209
+
<sup>An example of the visual regression diff UI, showing the "Diff", "Reference", "Actual", and "Slider" tabs, and how the slider reveals unexpected visual changes in a component.</sup>
210
+
</center>
211
+
212
+
#### Understanding the diff image
213
+
214
+
- **Red pixels** are areas that differ between reference and actual
215
+
- **Yellow pixels** are anti-aliasing differences (when anti-alias is not ignored)
216
+
- **Transparent/original** are unchanged areas
217
+
218
+
:::tip
219
+
If the diff is mostly red, something's really wrong. If it's speckled with a few red pixels around text, you probably just need to bump your threshold.
220
+
:::
221
+
222
+
## Configuring the `toMatchScreenshot` assertion
223
+
224
+
It's possible to configure the `toMatchScreenshot` assertion either globally, by changing its default options, or on a per-test basis.
225
+
226
+
To change the defaults, you have to change the [Vitest config](/config/browser/expect#tomatchscreenshot):
227
+
228
+
```ts{6-16} [vitest.config.ts]
150
229
import { defineConfig } from 'vitest/config'
151
230
152
231
export default defineConfig({
···
168
247
})
169
248
```
170
249
171
-
### Per-Test Configuration
250
+
For more fine-grained control, override global settings in specific tests by passing options directly to the assertion:
172
251
173
-
Override global settings for specific tests:
174
-
175
-
```ts
176
-
await expect(element).toMatchScreenshot('button-hover', {
252
+
```ts{2-6}
253
+
await expect(element).toMatchScreenshot('button', {
177
254
comparatorName: 'pixelmatch',
178
255
comparatorOptions: {
179
256
// more lax comparison for text-heavy elements
···
182
259
})
183
260
```
184
261
185
-
## Best Practices
262
+
## Third-party comparators
186
263
187
-
### Test Specific Elements
264
+
Vitest ships with `pixelmatch` as its built-in comparator. It's fast, compares images pixel-by-pixel, has no native dependencies, and handles the majority of cases well. Perceptual comparators aren't included by default because they bring heavier dependencies and there's no clear single "best one" to pick as different algorithms make different trade-offs, but the comparator API exists precisely to let you plug in whatever fits your needs. This decision may change as the ecosystem matures, though.
188
265
189
-
Unless you explicitly want to test the whole page, prefer capturing specific
190
-
components to reduce false positives:
266
+
For use cases where pixel-level diffing produces excessive noise, a perceptual or structural similarity comparator may be a better fit. These compare images more like a human would, tolerating minor rendering differences while still detecting meaningful visual changes.
267
+
268
+
There are many algorithms, so these are a useful starting point:
269
+
270
+
- [`@blazediff/ssim`](https://blazediff.dev/docs/ssim), [SSIM (Structural Similarity Index)](https://en.wikipedia.org/wiki/Structural_similarity_index_measure) implementations for perceptual image quality assessment. It offers standard SSIM, MS-SSIM (Multi-Scale SSIM), and Hitchhiker’s SSIM for various use cases
271
+
- [`@blazediff/gmsd`](https://blazediff.dev/docs/gmsd), a single-threaded GMSD (Gradient Magnitude Similarity Deviation) metric for perceptual image quality assessment, good for CI environments
272
+
273
+
To use one, install and register it:
274
+
275
+
```ts{5-11,18-46} [vitest.config.ts]
276
+
import ssim from '@blazediff/ssim/ssim'
277
+
import type { SsimOptionsExtended } from '@blazediff/ssim/ssim'
278
+
import { defineConfig } from 'vitest/config'
279
+
280
+
declare module 'vitest/browser' {
281
+
interface ScreenshotComparatorRegistry {
282
+
'standard-ssim': SsimOptionsExtended & {
283
+
threshold?: number
284
+
}
285
+
}
286
+
}
287
+
288
+
export default defineConfig({
289
+
test: {
290
+
browser: {
291
+
expect: {
292
+
toMatchScreenshot: {
293
+
comparators: {
294
+
// naive implementation, always check the library's docs
295
+
'standard-ssim': (
296
+
reference,
297
+
actual,
298
+
{ createDiff, ...options }
299
+
) => {
300
+
const diffBuffer = createDiff
301
+
? new Uint8Array(reference.data.length)
302
+
: undefined
303
+
304
+
const output = ssim(
305
+
reference.data,
306
+
actual.data,
307
+
diffBuffer,
308
+
reference.metadata.width,
309
+
reference.metadata.height,
310
+
options,
311
+
)
312
+
313
+
const pass = output >= (options.threshold ?? 0.95)
314
+
315
+
return {
316
+
pass,
317
+
diff: diffBuffer ?? null,
318
+
message: pass ? null : `SSIM score: ${output}.`,
319
+
}
320
+
},
321
+
},
322
+
},
323
+
},
324
+
},
325
+
},
326
+
})
327
+
```
328
+
329
+
Once registered, the comparator can be referenced by name in your config or on a per-test basis:
330
+
331
+
:::code-group
332
+
333
+
```ts{8} [vitest.config.ts]
334
+
import { defineConfig } from 'vitest/config'
335
+
336
+
export default defineConfig({
337
+
test: {
338
+
browser: {
339
+
expect: {
340
+
toMatchScreenshot: {
341
+
comparatorName: 'standard-ssim',
342
+
},
343
+
},
344
+
},
345
+
},
346
+
})
347
+
```
348
+
349
+
```ts{2} [button.vrt.test.tsx]
350
+
await expect(button).toMatchScreenshot('button', {
351
+
comparatorName: 'standard-ssim',
352
+
})
353
+
```
354
+
355
+
:::
356
+
357
+
## Best practices
358
+
359
+
### Test specific elements
360
+
361
+
Unless you explicitly want to test the whole page, prefer capturing specific components to reduce false positives:
191
362
192
363
```ts
193
364
// ❌ Captures entire page; prone to unrelated changes
194
365
await expect(page).toMatchScreenshot()
195
366
196
367
// ✅ Captures only the component under test
197
-
await expect(page.getByTestId('product-card')).toMatchScreenshot()
368
+
await expect(
369
+
page.getByRole('article', { name: 'Tote bag' })
370
+
).toMatchScreenshot()
198
371
```
199
372
200
-
### Handle Dynamic Content
373
+
### Handle dynamic content
201
374
202
-
Dynamic content like timestamps, user data, or random values will cause tests
203
-
to fail. You can either mock the sources of dynamic content or mask them when
204
-
using the Playwright provider by using the
205
-
[`mask` option](https://playwright.dev/docs/api/class-page#page-screenshot-option-mask)
206
-
in `screenshotOptions`.
375
+
Dynamic content like timestamps, user data, or random values will cause tests to fail. Either mock the underlying data sources or mask them using the [`mask` option](https://playwright.dev/docs/api/class-page#page-screenshot-option-mask) in `screenshotOptions` when using the Playwright provider.
207
376
208
-
```ts
209
-
await expect(page.getByTestId('profile')).toMatchScreenshot({
377
+
```ts{8}
378
+
const profile = page.getByRole(
379
+
'article',
380
+
{ name: 'Gracie\'s profile' },
381
+
)
382
+
383
+
await expect(profile).toMatchScreenshot({
210
384
screenshotOptions: {
211
-
mask: [page.getByTestId('last-seen')],
385
+
mask: [profile.getByRole('status')],
212
386
},
213
387
})
214
388
```
215
389
216
-
### Disable Animations
217
-
218
-
Animations can cause flaky tests. Disable them during testing by injecting
219
-
a custom CSS snippet:
220
-
221
-
```css
222
-
*, *::before, *::after {
223
-
animation-duration: 0s !important;
224
-
animation-delay: 0s !important;
225
-
transition-duration: 0s !important;
226
-
transition-delay: 0s !important;
227
-
}
228
-
```
390
+
### Disable animations
229
391
230
392
::: tip
231
-
When using the Playwright provider, animations are automatically disabled
232
-
when using the assertion: the `animations` option's value in `screenshotOptions`
233
-
is set to `"disabled"` by default.
393
+
When using the Playwright provider, animations are automatically disabled when using the built-in assertion: the `animations` option's value in `screenshotOptions` is set to `"disabled"` by default.
394
+
395
+
If you prefer to disable all animations to save some execution time, continue reading.
234
396
:::
235
397
236
-
### Set Appropriate Thresholds
237
-
238
-
Tuning thresholds is tricky. It depends on the content, test environment,
239
-
what's acceptable for your app, and might also change based on the test.
240
-
241
-
Vitest does not set a default for the mismatching pixels, that's up for the
242
-
user to decide based on their needs. The recommendation is to use
243
-
`allowedMismatchedPixelRatio`, so that the threshold is computed on the size
244
-
of the screenshot and not a fixed number.
245
-
246
-
When setting both `allowedMismatchedPixelRatio` and
247
-
`allowedMismatchedPixels`, Vitest uses whichever limit is stricter.
248
-
249
-
### Set consistent viewport sizes
250
-
251
-
As the browser instance might have a different default size, it's best to
252
-
set a specific viewport size, either on the test or the instance
253
-
configuration:
398
+
Animations can cause flaky tests. Disable them during testing by injecting a custom CSS snippet using [`setupFiles`](/config/setupfiles) or directly in your tests:
254
399
255
400
```ts
256
-
await page.viewport(1280, 720)
401
+
const stylesheet = document.createElement('style')
402
+
403
+
stylesheet.textContent = /* css */`
404
+
*, *::before, *::after {
405
+
animation-duration: 0s !important;
406
+
animation-delay: 0s !important;
407
+
transition-duration: 0s !important;
408
+
transition-delay: 0s !important;
409
+
}
410
+
`
411
+
412
+
document.head.appendChild(stylesheet)
257
413
```
258
414
259
-
```ts [vitest.config.ts]
260
-
import { playwright } from '@vitest/browser-playwright'
261
-
import { defineConfig } from 'vitest/config'
415
+
Alternatively, you can declare the CSS in a custom HTML template by using [`browser.testerHtmlPath`](/config/browser/testerhtmlpath).
262
416
263
-
export default defineConfig({
264
-
test: {
265
-
browser: {
266
-
enabled: true,
267
-
provider: playwright(),
268
-
instances: [
269
-
{
270
-
browser: 'chromium',
271
-
viewport: { width: 1280, height: 720 },
272
-
},
273
-
],
274
-
},
275
-
},
276
-
})
277
-
```
417
+
### Set appropriate thresholds
418
+
419
+
Tuning thresholds is tricky. It depends on the content, test environment, what's acceptable for your app, and might also change based on the test.
420
+
421
+
Vitest does not define a default tolerance for mismatched pixels. The appropriate value depends on your application and environment. The recommendation is to use `allowedMismatchedPixelRatio`, so that the threshold is computed on the size of the screenshot and not a fixed number.
422
+
423
+
When setting both `allowedMismatchedPixelRatio` and `allowedMismatchedPixels`, Vitest uses whichever limit is stricter.
278
424
279
425
### Use Git LFS
280
426
281
-
Store reference screenshots in
282
-
[Git LFS](https://github.com/git-lfs/git-lfs?tab=readme-ov-file) if you plan to
283
-
have a large test suite.
427
+
Store reference screenshots in [Git LFS](https://github.com/git-lfs/git-lfs?tab=readme-ov-file) if you plan to have a large test suite.
284
428
285
-
## Debugging Failed Tests
429
+
## Common issues and solutions
286
430
287
-
When a visual test fails, Vitest provides three images to help debug:
431
+
### False positives from font rendering
288
432
289
-
1. **Reference screenshot**: the expected baseline image
290
-
1. **Actual screenshot**: what was captured during the test
291
-
1. **Diff image**: highlights the differences, but this might not get generated
292
-
293
-
You'll see something like:
294
-
295
-
```
296
-
expect(element).toMatchScreenshot()
297
-
298
-
Screenshot does not match the stored reference.
299
-
245 pixels (ratio 0.03) differ.
300
-
301
-
Reference screenshot:
302
-
tests/__screenshots__/button.test.ts/button-chromium-darwin.png
303
-
304
-
Actual screenshot:
305
-
tests/.vitest/attachments/button.test.ts/button-chromium-darwin-actual.png
306
-
307
-
Diff image:
308
-
tests/.vitest/attachments/button.test.ts/button-chromium-darwin-diff.png
309
-
```
310
-
311
-
### Understanding the diff image
312
-
313
-
- **Red pixels** are areas that differ between reference and actual
314
-
- **Yellow pixels** are anti-aliasing differences (when anti-alias is not ignored)
315
-
- **Transparent/original** are unchanged areas
316
-
317
-
:::tip
318
-
If the diff is mostly red, something's really wrong. If it's speckled with a
319
-
few red pixels around text, you probably just need to bump your threshold.
320
-
:::
321
-
322
-
## Common Issues and Solutions
323
-
324
-
### False Positives from Font Rendering
325
-
326
-
Font availability and rendering varies significantly between systems. Some
327
-
possible solutions might be to:
433
+
Font availability and rendering varies significantly between systems. Some possible solutions might be to:
328
434
329
435
- Use web fonts and wait for them to load:
330
436
···
337
443
338
444
- Increase comparison threshold for text-heavy areas:
339
445
340
-
```ts
341
-
await expect(page.getByTestId('article-summary')).toMatchScreenshot({
446
+
```ts{6-7}
447
+
await expect(
448
+
page.getByRole('article', { name: 'How to grow tomatoes' })
449
+
).toMatchScreenshot({
342
450
comparatorName: 'pixelmatch',
343
451
comparatorOptions: {
344
452
// 10% of the pixels are allowed to change
···
347
455
})
348
456
```
349
457
350
-
- Use a cloud service or containerized environment for consistent font rendering.
458
+
- [Consider a shared environment setup](#visual-testing-for-teams) for consistent font rendering.
351
459
352
-
### Flaky Tests or Different Screenshot Sizes
460
+
### Flaky tests or different screenshot sizes
353
461
354
-
If tests pass and fail randomly, or if screenshots have different dimensions
355
-
between runs:
462
+
If tests pass and fail randomly, or if screenshots have different dimensions between runs:
356
463
357
464
- Wait for everything to load, including loading indicators
358
465
- Set explicit viewport sizes: `await page.viewport(1920, 1080)`
359
466
- Check for responsive behavior at viewport boundaries
360
467
- Check for unintended animations or transitions
361
468
- Increase test timeout for large screenshots
362
-
- Use a cloud service or containerized environment
469
+
- [Consider a shared environment setup](#visual-testing-for-teams)
363
470
364
-
## Visual Regression Testing for Teams
471
+
## Visual testing for teams
365
472
366
-
Remember when we mentioned visual tests need a stable environment? Well, here's
367
-
the thing: your local machine isn't it.
473
+
Even with a controlled local setup, references generated on one machine will often fail on another. This matters as soon as more than one person is running the suite.
368
474
369
-
For teams, you've basically got three options:
475
+
Running the visual regression suite in a shared environment solves this problem. There are three ways to do this:
370
476
371
-
1. **Self-hosted runners**, complex to set up, painful to maintain
372
-
1. **GitHub Actions**, free (for open source), works with any provider
373
-
1. **Cloud services**, like
374
-
[Azure App Testing](https://azure.microsoft.com/en-us/products/app-testing/),
375
-
built for this exact problem
477
+
1. **Self-hosted runners** (e.g., Docker images), complex to set up and maintain
478
+
1. **Generate references in CI**, which requires some setup
479
+
1. **Cloud services**, like [Azure App Testing](https://azure.microsoft.com/en-us/products/app-testing/), built to solve this exact problem, but usually restricted to specific providers and browsers
376
480
377
-
We'll focus on options 2 and 3 since they're the quickest to get running.
481
+
Options 2 and 3 are the quickest to get running, so those are covered below.
378
482
379
-
To be upfront, the main trade-offs for each are:
483
+
:::: tabs key:shared-environment-vrt
484
+
=== GitHub Actions (CI)
380
485
381
-
- **GitHub Actions**: visual tests only run in CI (developers can't run them
382
-
locally)
383
-
- **Microsoft's service**: works everywhere but costs money and only works
384
-
with Playwright
385
-
386
-
:::: tabs key:vrt-for-teams
387
-
=== GitHub Actions
388
-
389
-
The trick here is keeping visual tests separate from your regular tests,
390
-
otherwise, you'll waste hours checking failing logs of screenshot mismatches.
391
-
392
-
### Organizing Your Tests
393
-
394
-
First, isolate your visual tests. Stick them in a `visual` folder (or whatever
395
-
makes sense for your project):
396
-
397
-
```json [package.json]
398
-
{
399
-
"scripts": {
400
-
"test:unit": "vitest --exclude tests/visual/*.test.ts",
401
-
"test:visual": "vitest tests/visual/*.test.ts"
402
-
}
403
-
}
404
-
```
405
-
406
-
Now developers can run `npm run test:unit` locally without visual tests getting
407
-
in the way. Visual tests stay in CI where the environment is consistent.
408
-
409
-
::: tip Alternative
410
-
Not a fan of glob patterns? You could also use separate
411
-
[Test Projects](/guide/projects) instead and run them using:
412
-
413
-
- `vitest --project unit`
414
-
- `vitest --project visual`
415
-
:::
416
-
417
-
### CI Setup
418
-
419
-
Your CI needs browsers installed. How you do this depends on your provider:
486
+
GitHub runners don't have browsers preinstalled. Install them before running tests, using the steps for your provider:
420
487
421
488
::: tabs key:provider
422
489
== Playwright
423
490
424
-
[Playwright](https://npmx.dev/package/playwright) makes this easy. Just pin
425
-
your version and add this before running tests:
491
+
[Playwright](https://npmx.dev/package/playwright) makes this easy. Just pin your version and add this step before running tests:
426
492
427
493
```yaml [.github/workflows/ci.yml]
428
494
# ...the rest of the workflow
···
432
498
433
499
== WebdriverIO
434
500
435
-
[WebdriverIO](https://npmx.dev/package/webdriverio) expects you to bring
436
-
your own browsers. The folks at
437
-
[@browser-actions](https://github.com/browser-actions) have your back:
501
+
[WebdriverIO](https://npmx.dev/package/webdriverio) installs browsers automatically if none can be found when a test run starts, but it's recommended to decouple the installation process. To help with this, the folks at [@browser-actions](https://github.com/browser-actions) have packaged scripts to install [Chrome](https://github.com/browser-actions/setup-chrome), [Edge](https://github.com/browser-actions/setup-edge), and [Firefox](https://github.com/browser-actions/setup-firefox) in convenient reusable actions:
438
502
439
503
```yaml [.github/workflows/ci.yml]
440
504
# ...the rest of the workflow
···
445
509
446
510
:::
447
511
448
-
Then run your visual tests:
512
+
Then in your existing workflow run the visual tests:
449
513
450
514
```yaml [.github/workflows/ci.yml]
451
515
# ...the rest of the workflow
···
454
518
run: npm run test:visual
455
519
```
456
520
457
-
### The Update Workflow
521
+
### The update workflow
458
522
459
-
Here's where it gets interesting. You don't want to update screenshots on every
460
-
PR automatically <small>*(chaos!)*</small>. Instead, create a
461
-
manually-triggered workflow that developers can run when they intentionally
462
-
change the UI.
523
+
Running `vitest --update` locally would generate screenshots on your machine, defeating the whole point of a controlled environment. Instead, you need a way to trigger the update in CI where the environment matches the one that runs the tests.
524
+
525
+
You don't want this to happen automatically on every PR <small>*(chaos!)*</small>. Instead, create a manually-triggered workflow that runs when there are intentional changes to the UI.
463
526
464
527
The workflow below:
465
528
- Only runs on feature branches (never on main)
···
477
540
<img alt="Action summary after no updates" img-dark src="/vrt-gha-summary-no-update-dark.png">
478
541
479
542
::: tip
480
-
This is just one approach. Some teams prefer PR comments (`/update-screenshots`),
481
-
others use labels. Adjust it to fit your workflow!
543
+
This is just one approach. Some prefer PR comments (`/update-screenshots`), others use labels. Adjust it to fit your workflow.
482
544
483
-
The important part is having a controlled way to update baselines.
545
+
The important part is having a controlled way to update reference screenshots.
484
546
:::
485
547
486
548
```yaml [.github/workflows/update-screenshots.yml]
···
537
599
- name: Install Playwright Browsers
538
600
run: npx --no playwright install --with-deps --only-shell
539
601
540
-
# the magic happens below 🪄
541
602
- name: Update Visual Regression Screenshots
542
603
run: npm run test:visual --update
543
604
···
594
655
fi
595
656
```
596
657
597
-
=== Azure App Testing
658
+
=== Azure App Testing (Cloud service)
598
659
599
-
Your tests stay local, only the browsers run in the cloud. It's Playwright's
600
-
remote browser feature, but Microsoft handles all the infrastructure.
660
+
With this method, your tests stay local but the browsers run in the cloud. This is built on top of Playwright's remote browser feature and Azure handles all the infrastructure.
601
661
602
-
### Organizing Your Tests
662
+
Everyone uses the same cloud browsers, so references are consistent regardless of who runs them. Tests work locally, you pay only for what you use, and there's nothing to maintain.
603
663
604
-
Keep visual tests separate to control costs. Only tests that actually take
605
-
screenshots should use the service.
664
+
### Configuration
606
665
607
-
The cleanest approach is using [Test Projects](/guide/projects):
666
+
To have Playwright connect to the browsers spawned within the service, you have to update the provider configuration.
608
667
609
-
```ts [vitest.config.ts]
668
+
```ts{14-28} [vitest.config.ts]
610
669
import { env } from 'node:process'
611
670
import { defineConfig } from 'vitest/config'
612
671
import { playwright } from '@vitest/browser-playwright'
613
672
614
673
export default defineConfig({
615
-
// ...global Vite config
616
-
tests: {
617
-
// ...global Vitest config
674
+
test: {
675
+
// ...other configurations
618
676
projects: [
619
677
{
620
678
extends: true,
621
679
test: {
622
-
name: 'unit',
623
-
include: ['tests/**/*.test.ts'],
624
-
// regular config, can use local browsers
625
-
},
626
-
},
627
-
{
628
-
extends: true,
629
-
test: {
630
-
name: 'visual',
631
-
// or you could use a different suffix, e.g.,: `tests/**/*.visual.ts?(x)`
632
-
include: ['visual-regression-tests/**/*.test.ts?(x)'],
680
+
name: 'vrt',
633
681
browser: {
634
-
enabled: true,
635
682
provider: playwright({
636
683
connectOptions: {
637
684
wsEndpoint: `${env.PLAYWRIGHT_SERVICE_URL}?${new URLSearchParams({
···
650
697
headless: true,
651
698
instances: [
652
699
{
653
-
browser: 'chromium',
654
-
viewport: { width: 2560, height: 1440 },
700
+
browser: '[browser-name]',
701
+
viewport: { width: 1280, height: 720 },
655
702
},
656
703
],
657
704
},
705
+
include: [vrtPattern],
658
706
},
659
707
},
708
+
// ...other projects
660
709
],
661
710
},
662
711
})
663
712
```
664
713
665
-
Follow the [official guide to create a Playwright Workspace](https://learn.microsoft.com/en-us/azure/app-testing/playwright-workspaces/quickstart-run-end-to-end-tests?tabs=playwrightcli&pivots=playwright-test-runner#create-a-workspace).
714
+
To create a Playwright Workspace follow the [official guide](https://learn.microsoft.com/en-us/azure/app-testing/playwright-workspaces/quickstart-run-end-to-end-tests?tabs=playwrightcli&pivots=playwright-test-runner#create-a-workspace).
666
715
667
716
Once your workspace is created, configure Vitest to use it:
668
717
669
718
1. **Set the endpoint URL**: following the [official guide](https://learn.microsoft.com/en-us/azure/app-testing/playwright-workspaces/quickstart-run-end-to-end-tests?tabs=playwrightcli&pivots=playwright-test-runner#configure-the-browser-endpoint), retrieve the URL and set it as the `PLAYWRIGHT_SERVICE_URL` environment variable.
670
719
1. **Enable token authentication**: [enable access tokens](https://learn.microsoft.com/en-us/azure/app-testing/playwright-workspaces/how-to-manage-authentication?pivots=playwright-test-runner#enable-authentication-using-access-tokens) for your workspace, then [generate a token](https://learn.microsoft.com/en-us/azure/app-testing/playwright-workspaces/how-to-manage-access-tokens#generate-a-workspace-access-token) and set it as the `PLAYWRIGHT_SERVICE_ACCESS_TOKEN` environment variable.
671
720
672
-
::: danger Keep that Token Secret!
673
-
Never commit `PLAYWRIGHT_SERVICE_ACCESS_TOKEN` to your repository. Anyone with
674
-
the token can rack up your bill. Use environment variables locally and secrets
675
-
in CI.
721
+
::: danger Keep that token secret!
722
+
Never commit `PLAYWRIGHT_SERVICE_ACCESS_TOKEN` to your repository. Anyone with the token can rack up your bill. Use environment variables locally and secrets in CI.
676
723
:::
677
724
678
-
Then split your `test` script like this:
679
-
680
-
```json [package.json]
681
-
{
682
-
"scripts": {
683
-
"test:visual": "vitest --project visual",
684
-
"test:unit": "vitest --project unit"
685
-
}
686
-
}
687
-
```
688
-
689
-
### Running Tests
725
+
### Running tests
690
726
691
727
```bash
692
728
# Local development
693
-
npm run test:unit # free, runs locally
729
+
npm run test:unit # runs locally using your browsers
694
730
npm run test:visual # uses cloud browsers
695
731
696
732
# Update screenshots
697
733
npm run test:visual -- --update
698
734
```
699
735
700
-
The best part of this approach is that it just works:
736
+
### CI setup
701
737
702
-
- **Consistent screenshots**, everyone uses the same cloud browsers
703
-
- **Works locally**, developers can run and update visual tests on their machines
704
-
- **Pay for what you use**, only visual tests consume service minutes
705
-
- **No Docker or workflow setups needed**, nothing to manage or maintain
706
-
707
-
### CI Setup
708
-
709
-
In your CI, add the secrets:
738
+
Add the secrets to your CI configuration:
710
739
711
740
```yaml
712
741
env:
···
714
743
PLAYWRIGHT_SERVICE_ACCESS_TOKEN: ${{ secrets.PLAYWRIGHT_SERVICE_ACCESS_TOKEN }}
715
744
```
716
745
717
-
Then run your tests like normal. The service handles the rest.
746
+
Then run your tests like normal. The service handles the browser infrastructure.
718
747
719
748
::::
720
749
721
-
### So Which One?
750
+
### Picking the right option
722
751
723
-
Both approaches work. The real question is what pain points matter most to your
724
-
team.
752
+
All approaches work. The real question is what pain points matter most to you and your team.
725
753
726
-
If you're already deep in the GitHub ecosystem, GitHub Actions is hard to beat.
727
-
Free for open source, works with any browser provider, and you control
728
-
everything.
754
+
If you're comfortable with containerization, a self-hosted Docker setup gives you a controlled environment without any external dependencies or costs. The downside is maintenance as you own the setup, the browser versions, and any breakage.
729
755
730
-
The downside? That "works on my machine" conversation when someone generates
731
-
screenshots locally and they don't match CI expectations anymore.
756
+
CI runs work with any browser provider and give you full control, but screenshots can only be generated in CI. If someone runs `vitest --update` locally and commits the result, those references will likely fail on the next CI run. This is preventable by guarding the command behind a CI environment check.
732
757
733
-
A cloud service makes sense if developers need to run visual tests locally.
758
+
A cloud service makes sense if you want developers to be able to run and update visual tests locally without risking mismatched references. It becomes even more useful when designers are involved in reviewing changes, or when the push-wait-check-fix-push cycle becomes a real bottleneck.
734
759
735
-
Some teams have designers checking their work or developers who prefer catching
736
-
issues before pushing. It allows skipping the push-wait-check-fix-push cycle.
760
+
Still on the fence? Start with the CI workflow. You can always move to a container or cloud service later if it becomes a pain point.
737
761
738
-
Still on the fence? Start with GitHub Actions. You can always add a cloud
739
-
service later if local testing becomes a pain point.
762
+
## Going deeper
763
+
764
+
### How Vitest ensures screenshot stability
765
+
766
+
Visual regression tests rely on screenshots remaining stable across runs. In practice, pages are not instantly stable: images load asynchronously, animations finish at different times, fonts render, and layouts settle. To mitigate this, Vitest uses a "Stable Screenshot Detection" strategy:
767
+
768
+
1. It takes an initial screenshot (or uses the reference screenshot if available) as baseline
769
+
1. It takes another screenshot and compares it with the baseline
770
+
- If the screenshots match, the page is stable and testing continues
771
+
- If they differ, Vitest uses the newest screenshot as the baseline and repeats
772
+
1. This continues until stability is achieved or the timeout is reached
773
+
774
+
This ensures that transient visual changes (like loading spinners or animations) don't cause false positives. If something never stops animating, though, you'll hit the timeout, so consider [disabling animations during testing](#disable-animations).
775
+
776
+
If a stable screenshot is captured after one or more retries and a reference screenshot exists, Vitest performs a final comparison with the reference using `createDiff: true`. This will generate a diff image if they don't match.
777
+
778
+
During stability detection, Vitest calls comparators with `createDiff: false` since it only needs to know if screenshots match. This keeps the detection process fast.
docs/public/visual-regression/diff-view-dark.avif
docs/public/visual-regression/diff-view-dark.avif
This is a binary file and will not be displayed.
docs/public/visual-regression/diff-view-light.avif
docs/public/visual-regression/diff-view-light.avif
This is a binary file and will not be displayed.