[READ-ONLY] Mirror of https://github.com/vitest-dev/vitest. Next generation testing framework powered by Vite.
vitest.dev
test
testing-tools
vite
feat: introduce the new reporter API (#7069)
Co-authored-by: Ari Perkkiö <ari.perkkio@gmail.com>
authored by
Vladimir
Ari Perkkiö
and committed by
GitHub
(Jan 14, 2025, 4:46 PM +0100)
766624ab
faca4de8
+2485
-962
+4
docs/.vitepress/config.ts
+4
docs/.vitepress/config.ts
+18
-19
docs/advanced/metadata.md
+18
-19
docs/advanced/metadata.md
···
20
20
})
21
21
```
22
22
23
-
Once a test is completed, Vitest will send a task including the result and `meta` to the Node.js process using RPC. To intercept and process this task, you can utilize the `onTaskUpdate` method available in your reporter implementation:
23
+
Once a test is completed, Vitest will send a task including the result and `meta` to the Node.js process using RPC, and then report it in `onTestCaseResult` and other hooks that have access to tasks. To process this test case, you can utilize the `onTestCaseResult` method available in your reporter implementation:
24
24
25
25
```ts [custom-reporter.js]
26
-
export default {
27
-
// you can intercept packs if needed
28
-
onTaskUpdate(packs) {
29
-
const [id, result, meta] = packs[0]
30
-
},
31
-
// meta is located on every task inside "onFinished"
32
-
onFinished(files) {
33
-
files[0].meta.done === true
34
-
files[0].tasks[0].meta.custom === 'some-custom-handler'
35
-
}
36
-
}
37
-
```
26
+
import type { Reporter, TestCase, TestModule } from 'vitest/node'
38
27
39
-
::: warning
40
-
Vitest can send several tasks at the same time if several tests are completed in a short period of time.
41
-
:::
28
+
export default {
29
+
onTestCaseResult(testCase: TestCase) {
30
+
// custom === 'some-custom-handler' ✅
31
+
const { custom } = testCase.meta()
32
+
},
33
+
onTestRunEnd(testModule: TestModule) {
34
+
testModule.meta().done === true
35
+
testModule.children.at(0).meta().custom === 'some-custom-handler'
36
+
}
37
+
} satisfies Reporter
38
+
```
42
39
43
40
::: danger BEWARE
44
41
Vitest uses different methods to communicate with the Node.js process.
···
56
53
57
54
```ts
58
55
const vitest = await createVitest('test')
59
-
await vitest.start()
60
-
vitest.state.getFiles()[0].meta.done === true
61
-
vitest.state.getFiles()[0].tasks[0].meta.custom === 'some-custom-handler'
56
+
const { testModules } = await vitest.start()
57
+
58
+
const testModule = testModules[0]
59
+
testModule.meta().done === true
60
+
testModule.children.at(0).meta().custom === 'some-custom-handler'
62
61
```
63
62
64
63
It's also possible to extend type definitions when using TypeScript:
+10
docs/api/index.md
+10
docs/api/index.md
···
1179
1179
1180
1180
::: tip
1181
1181
This hook is always called in reverse order and is not affected by [`sequence.hooks`](/config/#sequence-hooks) option.
1182
+
1183
+
<!-- TODO: should it be called? https://github.com/vitest-dev/vitest/pull/7069 -->
1184
+
Note that this hook is not called if test was skipped with a dynamic `ctx.skip()` call:
1185
+
1186
+
```ts{2}
1187
+
test('skipped dynamically', (t) => {
1188
+
onTestFinished(() => {}) // not called
1189
+
t.skip()
1190
+
})
1191
+
```
1182
1192
:::
1183
1193
1184
1194
### onTestFailed
+311
docs/advanced/api/reporters.md
+311
docs/advanced/api/reporters.md
···
1
+
# Reporters
2
+
3
+
::: warning
4
+
This is an advanced API. If you just want to configure built-in reporters, read the ["Reporters"](/guide/reporters) guide.
5
+
:::
6
+
7
+
Vitest has its own test run lifecycle. These are represented by reporter's methods:
8
+
9
+
- [`onInit`](#oninit)
10
+
- [`onTestRunStart`](#ontestrunstart)
11
+
- [`onTestModuleQueued`](#ontestmodulequeued)
12
+
- [`onTestModuleCollected`](#ontestmodulecollected)
13
+
- [`onTestModuleStart`](#ontestmodulestart)
14
+
- [`onTestSuiteReady`](#ontestsuiteready)
15
+
- [`onHookStart(beforeAll)`](#onhookstart)
16
+
- [`onHookEnd(beforeAll)`](#onhookend)
17
+
- [`onTestCaseReady`](#ontestcaseready)
18
+
- [`onHookStart(beforeEach)`](#onhookstart)
19
+
- [`onHookEnd(beforeEach)`](#onhookend)
20
+
- [`onHookStart(afterEach)`](#onhookstart)
21
+
- [`onHookEnd(afterEach)`](#onhookend)
22
+
- [`onTestCaseResult`](#ontestcaseresult)
23
+
- [`onHookStart(afterAll)`](#onhookstart)
24
+
- [`onHookEnd(afterAll)`](#onhookend)
25
+
- [`onTestSuiteResult`](#ontestsuiteresult)
26
+
- [`onTestModuleEnd`](#ontestmoduleend)
27
+
- [`onTestRunEnd`](#ontestrunend)
28
+
29
+
Tests and suites within a single module will be reported in order unless they were skipped. All skipped tests are reported at the end of suite/module.
30
+
31
+
Note that since test modules can run in parallel, Vitest will report them in parallel.
32
+
33
+
This guide lists all supported reporter methods. However, don't forget that instead of creating your own reporter, you can [extend existing one](/advanced/reporters) instead:
34
+
35
+
```ts [custom-reporter.js]
36
+
import { BaseReporter } from 'vitest/reporters'
37
+
38
+
export default class CustomReporter extends BaseReporter {
39
+
onTestRunEnd(testModules, errors) {
40
+
console.log(testModule.length, 'tests finished running')
41
+
super.onTestRunEnd(testModules, errors)
42
+
}
43
+
}
44
+
```
45
+
46
+
## onInit
47
+
48
+
```ts
49
+
function onInit(vitest: Vitest): Awaitable<void>
50
+
```
51
+
52
+
This method is called when [Vitest](/advanced/api/vitest) was initiated or started, but before the tests were filtered.
53
+
54
+
::: info
55
+
Internally this method is called inside [`vitest.start`](/advanced/api/vitest#start), [`vitest.init`](/advanced/api/vitest#init) or [`vitest.mergeReports`](/advanced/api/vitest#mergereports). If you are using programmatic API, make sure to call either one dependning on your needs before calling [`vitest.runTestSpecifications`](/advanced/api/vitest#runtestspecifications), for example. Built-in CLI will always run methods in correct order.
56
+
:::
57
+
58
+
Note that you can also get access to `vitest` instance from test cases, suites and test modules via a [`project`](/advanced/api/test-project) property, but it might also be useful to store a reference to `vitest` in this method.
59
+
60
+
::: details Example
61
+
```ts
62
+
import type { Reporter, TestSpecification, Vitest } from 'vitest/node'
63
+
64
+
class MyReporter implements Reporter {
65
+
private vitest!: Vitest
66
+
67
+
onInit(vitest: Vitest) {
68
+
this.vitest = vitest
69
+
}
70
+
71
+
onTestRunStart(specifications: TestSpecification[]) {
72
+
console.log(
73
+
specifications.length,
74
+
'test files will run in',
75
+
this.vitest.config.root,
76
+
)
77
+
}
78
+
}
79
+
80
+
export default new MyReporter()
81
+
```
82
+
:::
83
+
84
+
## onTestRunStart
85
+
86
+
```ts
87
+
function onTestRunStart(
88
+
specifications: TestSpecification[]
89
+
): Awaitable<void>
90
+
```
91
+
92
+
This method is called when a new test run has started. It receives an array of [test specifications](/advanced/api/test-specification) scheduled to run. This array is readonly and available only for information purposes.
93
+
94
+
If Vitest didn't find any test files to run, this event will be invoked with an empty array, and then [`onTestRunEnd`](#ontestrunend) will be called immediately after.
95
+
96
+
::: details Example
97
+
```ts
98
+
import type { Reporter, TestSpecification } from 'vitest/node'
99
+
100
+
class MyReporter implements Reporter {
101
+
onTestRunStart(specifications: TestSpecification[]) {
102
+
console.log(specifications.length, 'test files will run')
103
+
}
104
+
}
105
+
106
+
export default new MyReporter()
107
+
```
108
+
:::
109
+
110
+
::: tip DEPRECATION NOTICE
111
+
This method was added in Vitest 3, replacing `onPathsCollected` and `onSpecsCollected`, both of which are now deprecated.
112
+
:::
113
+
114
+
## onTestRunEnd
115
+
116
+
```ts
117
+
function onTestRunEnd(
118
+
testModules: ReadonlyArray<TestModule>,
119
+
unhandledErrors: ReadonlyArray<SerializedError>,
120
+
reason: TestRunEndReason
121
+
): Awaitable<void>
122
+
```
123
+
124
+
This method is called after all tests have finished running and the coverage merged all reports, if it's enabled. Note that you can get the coverage information in [`onCoverage`](#oncoverage) hook.
125
+
126
+
It receives a readonly list of test modules. You can iterate over it via a [`testModule.children`](/advanced/api/test-collection) property to report the state and errors, if any.
127
+
128
+
The second argument is a readonly list of unhandled errors that Vitest wasn't able to attribute to any test. These can happen outside of the test run because of an error in a plugin, or inside the test run as a side-effect of a non-awaited function (for example, a timeout that threw an error after the test has finished running).
129
+
130
+
The third argument indicated why the test run was finished:
131
+
132
+
- `passed`: test run was finished normally and there are no errors
133
+
- `failed`: test run has at least one error (due to a syntax error during collection or an actual error during test execution)
134
+
- `interrupted`: test was interruped by [`vitest.cancelCurrentRun`](/advanced/api/vitest#cancelcurrentrun) call or `Ctrl+C` was pressed in the terminal (note that it's still possible to have failed tests in this case)
135
+
136
+
If Vitest didn't find any test files to run, this event will be invoked with empty arrays of modules and errors, and the state will depend on the value of [`config.passWithNoTests`](/config/#passwithnotests).
137
+
138
+
::: details Example
139
+
```ts
140
+
import type {
141
+
Reporter,
142
+
SerializedError,
143
+
TestModule,
144
+
TestRunEndReason,
145
+
TestSpecification
146
+
} from 'vitest/node'
147
+
148
+
class MyReporter implements Reporter {
149
+
onTestRunEnd(
150
+
testModules: ReadonlyArray<TestModule>,
151
+
unhandledErrors: ReadonlyArray<SerializedError>,
152
+
reason: TestRunEndReason,
153
+
) {
154
+
if (reason === 'passed') {
155
+
testModules.forEach(module => console.log(module.moduleId, 'succeeded'))
156
+
}
157
+
else if (reason === 'failed') {
158
+
// note that this will skip possible errors in suites
159
+
// you can get them from testSuite.errors()
160
+
for (const testCase of testModules.children.allTests()) {
161
+
if (testCase.result().state === 'failed') {
162
+
console.log(testCase.fullName, 'in', testCase.module.moduleId, 'failed')
163
+
console.log(testCase.result().errors)
164
+
}
165
+
}
166
+
}
167
+
else {
168
+
console.log('test run was interrupted, skipping report')
169
+
}
170
+
}
171
+
}
172
+
173
+
export default new MyReporter()
174
+
```
175
+
:::
176
+
177
+
::: tip DEPRECATION NOTICE
178
+
This method was added in Vitest 3, replacing `onFinished`, which is now deprecated.
179
+
:::
180
+
181
+
## onCoverage
182
+
183
+
```ts
184
+
function onCoverage(coverage: unknown): Awaitable<void>
185
+
```
186
+
187
+
This hook is called after coverage results have been processed. Coverage provider's reporters are called after this hook. The typings of `coverage` depends on the `coverage.provider`. For Vitest's default built-in providers you can import the types from `istanbul-lib-coverage` package:
188
+
189
+
```ts
190
+
import type { CoverageMap } from 'istanbul-lib-coverage'
191
+
192
+
declare function onCoverage(coverage: CoverageMap): Awaitable<void>
193
+
```
194
+
195
+
If Vitest didn't perform any coverage, this hook is not called.
196
+
197
+
## onTestModuleQueued
198
+
199
+
```ts
200
+
function onTestModuleQueued(testModule: TestModule): Awaitable<void>
201
+
```
202
+
203
+
This method is called right before Vitest imports the setup file and the test module itself. This means that `testModule` will have no [`children`](/advanced/api/test-suite#children) yet, but you can start reporting it as the next test to run.
204
+
205
+
## onTestModuleCollected
206
+
207
+
```ts
208
+
function onTestModuleCollected(testModule: TestModule): Awaitable<void>
209
+
```
210
+
211
+
This method is called when all tests inside the file were collected, meaning [`testModule.children`](/advanced/api/test-suite#children) collection is populated, but tests don't have any results yet.
212
+
213
+
## onTestModuleStart
214
+
215
+
```ts
216
+
function onTestModuleStart(testModule: TestModule): Awaitable<void>
217
+
```
218
+
219
+
This method is called right after [`onTestModuleCollected`](#ontestmodulecollected) unless Vitest runs in collection mode ([`vitest.collect()`](/advanced/api/vitest#collect) or `vitest collect` in the CLI), in this case it will not be called at all because there are no tests to run.
220
+
221
+
## onTestModuleEnd
222
+
223
+
```ts
224
+
function onTestModuleEnd(testModule: TestModule): Awaitable<void>
225
+
```
226
+
227
+
This method is called when every test in the module finished running. This means, every test inside [`testModule.children`](/advanced/api/test-suite#children) will have a `test.result()` that is not equal to `pending`.
228
+
229
+
## onHookStart
230
+
231
+
```ts
232
+
function onHookStart(context: ReportedHookContext): Awaitable<void>
233
+
```
234
+
235
+
This method is called when any of these hooks have started running:
236
+
237
+
- `beforeAll`
238
+
- `afterAll`
239
+
- `beforeEach`
240
+
- `afterEach`
241
+
242
+
If `beforeAll` or `afterAll` are started, the `entity` will be either [`TestSuite`](/advanced/api/test-suite) or [`TestModule`](/advanced/api/test-module).
243
+
244
+
If `beforeEach` or `afterEach` are started, the `entity` will always be [`TestCase`](/advanced/api/test-case).
245
+
246
+
::: warning
247
+
`onHookStart` method will not be called if the hook did not run during the test run.
248
+
:::
249
+
250
+
## onHookEnd
251
+
252
+
```ts
253
+
function onHookEnd(context: ReportedHookContext): Awaitable<void>
254
+
```
255
+
256
+
This method is called when any of these hooks have finished running:
257
+
258
+
- `beforeAll`
259
+
- `afterAll`
260
+
- `beforeEach`
261
+
- `afterEach`
262
+
263
+
If `beforeAll` or `afterAll` have finished, the `entity` will be either [`TestSuite`](/advanced/api/test-suite) or [`TestModule`](/advanced/api/test-module).
264
+
265
+
If `beforeEach` or `afterEach` have finished, the `entity` will always be [`TestCase`](/advanced/api/test-case).
266
+
267
+
::: warning
268
+
`onHookEnd` method will not be called if the hook did not run during the test run.
269
+
:::
270
+
271
+
## onTestSuiteReady
272
+
273
+
```ts
274
+
function onTestSuiteReady(testSuite: TestSuite): Awaitable<void>
275
+
```
276
+
277
+
This method is called before the suite starts to run its tests. This method is also called if the suite was skipped.
278
+
279
+
If the file doesn't have any suites, this method will not be called. Consider using `onTestModuleStart` to cover this use case.
280
+
281
+
## onTestSuiteResult
282
+
283
+
```ts
284
+
function onTestSuiteResult(testSuite: TestSuite): Awaitable<void>
285
+
```
286
+
287
+
This method is called after the suite has finished running tests. This method is also called if the suite was skipped.
288
+
289
+
If the file doesn't have any suites, this method will not be called. Consider using `onTestModuleEnd` to cover this use case.
290
+
291
+
## onTestCaseReady
292
+
293
+
```ts
294
+
function onTestCaseReady(testCase: TestCase): Awaitable<void>
295
+
```
296
+
297
+
This method is called before the test starts to run or it was skipped. Note that `beforeEach` and `afterEach` hooks are considered part of the test because they can influence the result.
298
+
299
+
::: warning
300
+
Notice that it's possible to have [`testCase.result()`](/advanced/api/test-case#result) with `passed` or `failed` state already when `onTestCaseReady` is called. This can happen if test was running too fast and both `onTestCaseReady` and `onTestCaseResult` were scheduled to run in the same microtask.
301
+
:::
302
+
303
+
## onTestCaseResult
304
+
305
+
```ts
306
+
function onTestCaseResult(testCase: TestCase): Awaitable<void>
307
+
```
308
+
309
+
This method is called when the test has finished running or was just skipped. Note that this will be called after the `afterEach` hook is finished, if there are any.
310
+
311
+
At this point, [`testCase.result()`](/advanced/api/test-case#result) will have non-pending state.
+41
-56
docs/advanced/api/test-case.md
+41
-56
docs/advanced/api/test-case.md
···
10
10
}
11
11
```
12
12
13
-
::: warning
14
-
We are planning to introduce a new Reporter API that will be using this API by default. For now, the Reporter API uses [runner tasks](/advanced/runner#tasks), but you can still access `TestCase` via `vitest.state.getReportedEntity` method:
15
-
16
-
```ts
17
-
import type { RunnerTestFile, TestModule, Vitest } from 'vitest/node'
18
-
19
-
class Reporter {
20
-
private vitest!: Vitest
21
-
22
-
onInit(vitest: Vitest) {
23
-
this.vitest = vitest
24
-
}
25
-
26
-
onFinished(files: RunnerTestFile[]) {
27
-
for (const file of files) {
28
-
const testModule = this.vitest.getReportedEntity(file) as TestModule
29
-
for (const test of testModule.children.allTests()) {
30
-
console.log(test) // TestCase
31
-
}
32
-
}
33
-
}
34
-
}
35
-
```
36
-
:::
37
-
38
13
## project
39
14
40
15
This references the [`TestProject`](/advanced/api/test-project) that the test belongs to.
···
124
99
125
100
```ts
126
101
interface TaskOptions {
127
-
each: boolean | undefined
128
-
concurrent: boolean | undefined
129
-
shuffle: boolean | undefined
130
-
retry: number | undefined
131
-
repeats: number | undefined
132
-
mode: 'run' | 'only' | 'skip' | 'todo'
102
+
readonly each: boolean | undefined
103
+
readonly fails: boolean | undefined
104
+
readonly concurrent: boolean | undefined
105
+
readonly shuffle: boolean | undefined
106
+
readonly retry: number | undefined
107
+
readonly repeats: number | undefined
108
+
readonly mode: 'run' | 'only' | 'skip' | 'todo'
133
109
}
134
110
```
135
111
···
142
118
```
143
119
144
120
Checks if the test did not fail the suite. If the test is not finished yet or was skipped, it will return `true`.
145
-
146
-
## skipped
147
-
148
-
```ts
149
-
function skipped(): boolean
150
-
```
151
-
152
-
Checks if the test was skipped during collection or dynamically with `ctx.skip()`.
153
121
154
122
## meta
155
123
···
174
142
## result
175
143
176
144
```ts
177
-
function result(): TestResult | undefined
145
+
function result(): TestResult
178
146
```
179
147
180
-
Test results. It will be `undefined` if test is skipped during collection, not finished yet or was just collected.
148
+
Test results. If test is not finished yet or was just collected, it will be equal to `TestResultPending`:
149
+
150
+
```ts
151
+
export interface TestResultPending {
152
+
/**
153
+
* The test was collected, but didn't finish running yet.
154
+
*/
155
+
readonly state: 'pending'
156
+
/**
157
+
* Pending tests have no errors.
158
+
*/
159
+
readonly errors: undefined
160
+
}
161
+
```
181
162
182
163
If the test was skipped, the return value will be `TestResultSkipped`:
183
164
···
187
168
* The test was skipped with `skip` or `todo` flag.
188
169
* You can see which one was used in the `options.mode` option.
189
170
*/
190
-
state: 'skipped'
171
+
readonly state: 'skipped'
191
172
/**
192
173
* Skipped tests have no errors.
193
174
*/
194
-
errors: undefined
175
+
readonly errors: undefined
195
176
/**
196
177
* A custom note passed down to `ctx.skip(note)`.
197
178
*/
198
-
note: string | undefined
179
+
readonly note: string | undefined
199
180
}
200
181
```
201
182
···
210
191
/**
211
192
* The test failed to execute.
212
193
*/
213
-
state: 'failed'
194
+
readonly state: 'failed'
214
195
/**
215
196
* Errors that were thrown during the test execution.
216
197
*/
217
-
errors: TestError[]
198
+
readonly errors: ReadonlyArray<TestError>
218
199
}
219
200
```
220
201
221
-
If the test passed, the retunr value will be `TestResultPassed`:
202
+
If the test passed, the return value will be `TestResultPassed`:
222
203
223
204
```ts
224
205
interface TestResultPassed {
225
206
/**
226
207
* The test passed successfully.
227
208
*/
228
-
state: 'passed'
209
+
readonly state: 'passed'
229
210
/**
230
211
* Errors that were thrown during the test execution.
231
212
*/
232
-
errors: TestError[] | undefined
213
+
readonly errors: ReadonlyArray<TestError> | undefined
233
214
}
234
215
```
235
216
···
250
231
/**
251
232
* If the duration of the test is above `slowTestThreshold`.
252
233
*/
253
-
slow: boolean
234
+
readonly slow: boolean
254
235
/**
255
236
* The amount of memory used by the test in bytes.
256
237
* This value is only available if the test was executed with `logHeapUsage` flag.
257
238
*/
258
-
heap: number | undefined
239
+
readonly heap: number | undefined
259
240
/**
260
241
* The time it takes to execute the test in ms.
261
242
*/
262
-
duration: number
243
+
readonly duration: number
263
244
/**
264
245
* The time in ms when the test started.
265
246
*/
266
-
startTime: number
247
+
readonly startTime: number
267
248
/**
268
249
* The amount of times the test was retried.
269
250
*/
270
-
retryCount: number
251
+
readonly retryCount: number
271
252
/**
272
253
* The amount of times the test was repeated as configured by `repeats` option.
273
254
* This value can be lower if the test failed during the repeat and no `retry` is configured.
274
255
*/
275
-
repeatCount: number
256
+
readonly repeatCount: number
276
257
/**
277
258
* If test passed on a second retry.
278
259
*/
279
-
flaky: boolean
260
+
readonly flaky: boolean
280
261
}
281
262
```
263
+
264
+
::: info
265
+
`diagnostic()` will return `undefined` if the test was not scheduled to run yet.
266
+
:::
+3
-7
docs/advanced/api/test-collection.md
+3
-7
docs/advanced/api/test-collection.md
···
57
57
## allTests
58
58
59
59
```ts
60
-
function allTests(
61
-
state?: TestResult['state'] | 'running'
62
-
): Generator<TestCase, undefined, void>
60
+
function allTests(state?: TestState): Generator<TestCase, undefined, void>
63
61
```
64
62
65
63
Filters all tests that are part of this collection and its children.
66
64
67
65
```ts
68
66
for (const test of module.children.allTests()) {
69
-
if (!test.result()) {
67
+
if (test.result().state === 'pending') {
70
68
console.log('test', test.fullName, 'did not finish')
71
69
}
72
70
}
···
77
75
## tests
78
76
79
77
```ts
80
-
function tests(
81
-
state?: TestResult['state'] | 'running'
82
-
): Generator<TestCase, undefined, void>
78
+
function tests(state?: TestState): Generator<TestCase, undefined, void>
83
79
```
84
80
85
81
Filters only the tests that are part of this collection. You can pass down a `state` value to filter tests by the state.
+21
-28
docs/advanced/api/test-module.md
+21
-28
docs/advanced/api/test-module.md
···
10
10
}
11
11
```
12
12
13
-
The `TestModule` inherits all methods and properties from the [`TestSuite`](/advanced/api/test-module). This guide will only list methods and properties unique to the `TestModule`
14
-
15
-
::: warning
16
-
We are planning to introduce a new Reporter API that will be using this API by default. For now, the Reporter API uses [runner tasks](/advanced/runner#tasks), but you can still access `TestModule` via `vitest.state.getReportedEntity` method:
17
-
18
-
```ts
19
-
import type { RunnerTestFile, TestModule, Vitest } from 'vitest/node'
20
-
21
-
class Reporter {
22
-
private vitest!: Vitest
23
-
24
-
onInit(vitest: Vitest) {
25
-
this.vitest = vitest
26
-
}
27
-
28
-
onFinished(files: RunnerTestFile[]) {
29
-
for (const file of files) {
30
-
const testModule = this.vitest.state.getReportedEntity(file) as TestModule
31
-
console.log(testModule) // TestModule
32
-
}
33
-
}
34
-
}
35
-
```
13
+
::: warning Extending Suite Methods
14
+
The `TestModule` class inherits all methods and properties from the [`TestSuite`](/advanced/api/test-suite). This guide will only list methods and properties unique to the `TestModule`.
36
15
:::
37
16
38
17
## moduleId
39
18
40
19
This is usually an absolute unix file path (even on Windows). It can be a virtual id if the file is not on the disk. This value corresponds to Vite's `ModuleGraph` id.
20
+
21
+
```ts
22
+
'C:/Users/Documents/project/example.test.ts' // ✅
23
+
'/Users/mac/project/example.test.ts' // ✅
24
+
'C:\\Users\\Documents\\project\\example.test.ts' // ❌
25
+
```
26
+
27
+
## state
28
+
29
+
```ts
30
+
function state(): TestModuleState
31
+
```
32
+
33
+
Works the same way as [`testSuite.state()`](/advanced/api/test-suite#state), but can also return `queued` if module wasn't executed yet.
41
34
42
35
## diagnostic
43
36
···
52
45
/**
53
46
* The time it takes to import and initiate an environment.
54
47
*/
55
-
environmentSetupDuration: number
48
+
readonly environmentSetupDuration: number
56
49
/**
57
50
* The time it takes Vitest to setup test harness (runner, mocks, etc.).
58
51
*/
59
-
prepareDuration: number
52
+
readonly prepareDuration: number
60
53
/**
61
54
* The time it takes to import the test module.
62
55
* This includes importing everything in the module and executing suite callbacks.
63
56
*/
64
-
collectDuration: number
57
+
readonly collectDuration: number
65
58
/**
66
59
* The time it takes to import the setup module.
67
60
*/
68
-
setupDuration: number
61
+
readonly setupDuration: number
69
62
/**
70
63
* Accumulated duration of all tests and hooks in the module.
71
64
*/
72
-
duration: number
65
+
readonly duration: number
73
66
}
74
67
```
+8
docs/advanced/api/test-specification.md
+8
docs/advanced/api/test-specification.md
···
13
13
14
14
`createSpecification` expects resolved module ID. It doesn't auto-resolve the file or check that it exists on the file system.
15
15
16
+
## taskId
17
+
18
+
[Test module's](/advanced/api/test-suite#id) identifier.
19
+
16
20
## project
17
21
18
22
This references the [`TestProject`](/advanced/api/test-project) that the test module belongs to.
···
26
30
'/Users/mac/project/example.test.ts' // ✅
27
31
'C:\\Users\\Documents\\project\\example.test.ts' // ❌
28
32
```
33
+
34
+
## testModule
35
+
36
+
Instance of [`TestModule`](/advanced/api/test-module) assosiated with the specification. If test wasn't queued yet, this will be `undefined`.
29
37
30
38
## pool <Badge type="warning">experimental</Badge> {#pool}
31
39
+35
-36
docs/advanced/api/test-suite.md
+35
-36
docs/advanced/api/test-suite.md
···
10
10
}
11
11
```
12
12
13
-
::: warning
14
-
We are planning to introduce a new Reporter API that will be using this API by default. For now, the Reporter API uses [runner tasks](/advanced/runner#tasks), but you can still access `TestSuite` via `vitest.state.getReportedEntity` method:
15
-
16
-
```ts
17
-
import type { RunnerTestFile, TestModule, Vitest } from 'vitest/node'
18
-
19
-
class Reporter {
20
-
private vitest!: Vitest
21
-
22
-
onInit(vitest: Vitest) {
23
-
this.vitest = vitest
24
-
}
25
-
26
-
onFinished(files: RunnerTestFile[]) {
27
-
for (const file of files) {
28
-
const testModule = this.vitest.state.getReportedEntity(file) as TestModule
29
-
for (const suite of testModule.children.allSuites()) {
30
-
console.log(suite) // TestSuite
31
-
}
32
-
}
33
-
}
34
-
}
35
-
```
36
-
:::
37
-
38
13
## project
39
14
40
15
This references the [`TestProject`](/advanced/api/test-project) that the test belongs to.
···
125
100
126
101
```ts
127
102
interface TaskOptions {
128
-
each: boolean | undefined
129
-
concurrent: boolean | undefined
130
-
shuffle: boolean | undefined
131
-
retry: number | undefined
132
-
repeats: number | undefined
133
-
mode: 'run' | 'only' | 'skip' | 'todo'
103
+
readonly each: boolean | undefined
104
+
readonly fails: boolean | undefined
105
+
readonly concurrent: boolean | undefined
106
+
readonly shuffle: boolean | undefined
107
+
readonly retry: number | undefined
108
+
readonly repeats: number | undefined
109
+
readonly mode: 'run' | 'only' | 'skip' | 'todo'
134
110
}
135
111
```
136
112
···
153
129
```
154
130
155
131
::: warning
156
-
Note that `suite.children` will only iterate the first level of nesting, it won't go deeper.
132
+
Note that `suite.children` will only iterate the first level of nesting, it won't go deeper. If you need to iterate over all tests or suites, use [`children.allTests()`](/advanced/api/test-collection#alltests) or [`children.allSuites()`](/advanced/api/test-collection#allsuites). If you need to iterate over everything, use recursive function:
133
+
134
+
```ts
135
+
function visit(collection: TestCollection) {
136
+
for (const task of collection) {
137
+
if (task.type === 'suite') {
138
+
// report a suite
139
+
visit(task.children)
140
+
}
141
+
else {
142
+
// report a test
143
+
}
144
+
}
145
+
}
146
+
```
157
147
:::
158
148
159
149
## ok
···
164
154
165
155
Checks if the suite has any failed tests. This will also return `false` if suite failed during collection. In that case, check the [`errors()`](#errors) for thrown errors.
166
156
167
-
## skipped
157
+
## state
168
158
169
159
```ts
170
-
function skipped(): boolean
160
+
function state(): TestSuiteState
171
161
```
172
162
173
-
Checks if the suite was skipped during collection.
163
+
Checks the running state of the suite. Possible return values:
164
+
165
+
- **pending**: the tests in this suite did not finish running yet.
166
+
- **failed**: this suite has failed tests or they couldn't be collected. If [`errors()`](#errors) is not empty, it means the suite failed to collect tests.
167
+
- **passed**: every test inside this suite has passed.
168
+
- **skipped**: this suite was skipped during collection.
169
+
170
+
::: warning
171
+
Note that [test module](/advanced/api/test-module) also has a `state` method that returns the same values, but it can also return an additional `queued` state if the module wasn't executed yet.
172
+
:::
174
173
175
174
## errors
176
175
···
189
188
```
190
189
191
190
::: warning
192
-
Note that errors are serialized into simple object: `instanceof Error` will always return `false`.
191
+
Note that errors are serialized into simple objects: `instanceof Error` will always return `false`.
193
192
:::
+2
-1
packages/runner/src/context.ts
+2
-1
packages/runner/src/context.ts
+2
packages/runner/src/hooks.ts
+2
packages/runner/src/hooks.ts
···
160
160
*
161
161
* **Note:** The `onTestFinished` hooks are running in reverse order of their registration. You can configure this by changing the `sequence.hooks` option in the config file.
162
162
*
163
+
* **Note:** The `onTestFinished` hook is not called if the test is canceled with a dynamic `ctx.skip()` call.
164
+
*
163
165
* @param {Function} fn - The callback function to be executed after a test finishes. The function can receive parameters providing details about the completed test, including its success or failure status.
164
166
* @param {number} [timeout] - Optional timeout in milliseconds for the hook. If not provided, the default hook timeout from the runner's configuration is used.
165
167
* @throws {Error} Throws an error if the function is not called within a test.
+62
-33
packages/runner/src/run.ts
+62
-33
packages/runner/src/run.ts
···
3
3
import type { FileSpecification, VitestRunner } from './types/runner'
4
4
import type {
5
5
File,
6
-
HookCleanupCallback,
7
6
HookListener,
8
7
SequenceHooks,
9
8
Suite,
···
13
12
TaskResult,
14
13
TaskResultPack,
15
14
TaskState,
15
+
TaskUpdateEvent,
16
16
Test,
17
17
TestContext,
18
18
} from './types/tasks'
···
31
31
const unixNow = Date.now
32
32
33
33
function updateSuiteHookState(
34
-
suite: Task,
34
+
task: Task,
35
35
name: keyof SuiteHooks,
36
36
state: TaskState,
37
37
runner: VitestRunner,
38
38
) {
39
-
if (!suite.result) {
40
-
suite.result = { state: 'run' }
39
+
if (!task.result) {
40
+
task.result = { state: 'run' }
41
41
}
42
-
if (!suite.result?.hooks) {
43
-
suite.result.hooks = {}
42
+
if (!task.result.hooks) {
43
+
task.result.hooks = {}
44
44
}
45
-
const suiteHooks = suite.result.hooks
45
+
const suiteHooks = task.result.hooks
46
46
if (suiteHooks) {
47
47
suiteHooks[name] = state
48
-
updateTask(suite, runner)
48
+
49
+
let event: TaskUpdateEvent = state === 'run' ? 'before-hook-start' : 'before-hook-end'
50
+
51
+
if (name === 'afterAll' || name === 'afterEach') {
52
+
event = state === 'run' ? 'after-hook-start' : 'after-hook-end'
53
+
}
54
+
55
+
updateTask(
56
+
event,
57
+
task,
58
+
runner,
59
+
)
49
60
}
50
61
}
51
62
···
113
124
name: T,
114
125
runner: VitestRunner,
115
126
args: SuiteHooks[T][0] extends HookListener<infer A, any> ? A : never,
116
-
): Promise<HookCleanupCallback[]> {
127
+
): Promise<unknown[]> {
117
128
const sequence = runner.config.sequence.hooks
118
129
119
-
const callbacks: HookCleanupCallback[] = []
130
+
const callbacks: unknown[] = []
120
131
// stop at file level
121
132
const parentSuite: Suite | null = 'filepath' in suite ? null : suite.suite || suite.file
122
133
···
126
137
)
127
138
}
128
139
129
-
updateSuiteHookState(currentTask, name, 'run', runner)
130
-
131
140
const hooks = getSuiteHooks(suite, name, sequence)
141
+
142
+
if (hooks.length > 0) {
143
+
updateSuiteHookState(currentTask, name, 'run', runner)
144
+
}
132
145
133
146
if (sequence === 'parallel') {
134
147
callbacks.push(
···
141
154
}
142
155
}
143
156
144
-
updateSuiteHookState(currentTask, name, 'pass', runner)
157
+
if (hooks.length > 0) {
158
+
updateSuiteHookState(currentTask, name, 'pass', runner)
159
+
}
145
160
146
161
if (name === 'afterEach' && parentSuite) {
147
162
callbacks.push(
···
153
168
}
154
169
155
170
const packs = new Map<string, [TaskResult | undefined, TaskMeta]>()
171
+
const eventsPacks: [string, TaskUpdateEvent][] = []
156
172
let updateTimer: any
157
173
let previousUpdate: Promise<void> | undefined
158
174
159
-
export function updateTask(task: Task, runner: VitestRunner): void {
175
+
export function updateTask(event: TaskUpdateEvent, task: Task, runner: VitestRunner): void {
176
+
eventsPacks.push([task.id, event])
160
177
packs.set(task.id, [task.result, task.meta])
161
178
162
179
const { clearTimeout, setTimeout } = getSafeTimers()
···
176
193
const taskPacks = Array.from(packs).map<TaskResultPack>(([id, task]) => {
177
194
return [id, task[0], task[1]]
178
195
})
179
-
const p = runner.onTaskUpdate?.(taskPacks)
196
+
const p = runner.onTaskUpdate?.(taskPacks, eventsPacks)
197
+
eventsPacks.length = 0
180
198
packs.clear()
181
199
return p
182
200
}
183
201
}
184
202
185
-
async function callCleanupHooks(cleanups: HookCleanupCallback[]) {
203
+
async function callCleanupHooks(cleanups: unknown[]) {
186
204
await Promise.all(
187
205
cleanups.map(async (fn) => {
188
206
if (typeof fn !== 'function') {
···
201
219
}
202
220
203
221
if (test.result?.state === 'fail') {
204
-
updateTask(test, runner)
222
+
// should not be possible to get here, I think this is just copy pasted from suite
223
+
// TODO: maybe someone fails tests in `beforeAll` hooks?
224
+
// https://github.com/vitest-dev/vitest/pull/7069
225
+
updateTask('test-failed-early', test, runner)
205
226
return
206
227
}
207
228
···
212
233
startTime: unixNow(),
213
234
retryCount: 0,
214
235
}
215
-
updateTask(test, runner)
236
+
updateTask('test-prepare', test, runner)
216
237
217
238
setCurrentTest(test)
218
239
···
222
243
for (let repeatCount = 0; repeatCount <= repeats; repeatCount++) {
223
244
const retry = test.retry ?? 0
224
245
for (let retryCount = 0; retryCount <= retry; retryCount++) {
225
-
let beforeEachCleanups: HookCleanupCallback[] = []
246
+
let beforeEachCleanups: unknown[] = []
226
247
try {
227
248
await runner.onBeforeTryTask?.(test, {
228
249
retry: retryCount,
···
271
292
}
272
293
273
294
// skipped with new PendingError
274
-
if (test.pending || test.result?.state === 'skip') {
295
+
if (test.result?.pending || test.result?.state === 'skip') {
275
296
test.mode = 'skip'
276
-
test.result = { state: 'skip', note: test.result?.note }
277
-
updateTask(test, runner)
297
+
test.result = { state: 'skip', note: test.result?.note, pending: true }
298
+
updateTask('test-finished', test, runner)
278
299
setCurrentTest(undefined)
279
300
return
280
301
}
···
309
330
)
310
331
}
311
332
312
-
delete test.onFailed
313
-
delete test.onFinished
333
+
test.onFailed = undefined
334
+
test.onFinished = undefined
314
335
315
336
if (test.result.state === 'pass') {
316
337
break
···
323
344
}
324
345
325
346
// update retry info
326
-
updateTask(test, runner)
347
+
updateTask('test-retried', test, runner)
327
348
}
328
349
}
329
350
···
346
367
347
368
await runner.onAfterRunTask?.(test)
348
369
349
-
updateTask(test, runner)
370
+
updateTask('test-finished', test, runner)
350
371
}
351
372
352
373
function failTask(result: TaskResult, err: unknown, diffOptions: DiffOptions | undefined) {
353
374
if (err instanceof PendingError) {
354
375
result.state = 'skip'
355
376
result.note = err.note
377
+
result.pending = true
356
378
return
357
379
}
358
380
···
369
391
suite.tasks.forEach((t) => {
370
392
t.mode = 'skip'
371
393
t.result = { ...t.result, state: 'skip' }
372
-
updateTask(t, runner)
394
+
updateTask('test-finished', t, runner)
373
395
if (t.type === 'suite') {
374
396
markTasksAsSkipped(t, runner)
375
397
}
···
381
403
382
404
if (suite.result?.state === 'fail') {
383
405
markTasksAsSkipped(suite, runner)
384
-
updateTask(suite, runner)
406
+
// failed during collection
407
+
updateTask('suite-failed-early', suite, runner)
385
408
return
386
409
}
387
410
388
411
const start = now()
389
412
413
+
const mode = suite.mode
414
+
390
415
suite.result = {
391
-
state: 'run',
416
+
state: mode === 'skip' || mode === 'todo' ? mode : 'run',
392
417
startTime: unixNow(),
393
418
}
394
419
395
-
updateTask(suite, runner)
420
+
updateTask('suite-prepare', suite, runner)
396
421
397
-
let beforeAllCleanups: HookCleanupCallback[] = []
422
+
let beforeAllCleanups: unknown[] = []
398
423
399
424
if (suite.mode === 'skip') {
400
425
suite.result.state = 'skip'
426
+
427
+
updateTask('suite-finished', suite, runner)
401
428
}
402
429
else if (suite.mode === 'todo') {
403
430
suite.result.state = 'todo'
431
+
432
+
updateTask('suite-finished', suite, runner)
404
433
}
405
434
else {
406
435
try {
···
476
505
}
477
506
}
478
507
479
-
updateTask(suite, runner)
480
-
481
508
suite.result.duration = now() - start
509
+
510
+
updateTask('suite-finished', suite, runner)
482
511
483
512
await runner.onAfterRunSuite?.(suite)
484
513
}
+2
packages/runner/src/types.ts
+2
packages/runner/src/types.ts
-26
test/benchmark/test/reporter.test.ts
-26
test/benchmark/test/reporter.test.ts
···
1
-
import type { RunnerTestCase } from 'vitest'
2
1
import * as pathe from 'pathe'
3
2
import { assert, expect, it } from 'vitest'
4
-
import { TaskParser } from 'vitest/src/node/reporters/task-parser.js'
5
3
import { runVitest } from '../../test-utils'
6
4
7
5
it('summary', async () => {
···
33
31
for (const [index, line] of expected.trim().split('\n').entries()) {
34
32
expect(lines[index]).toMatch(line)
35
33
}
36
-
})
37
-
38
-
it('reports passed tasks just once', async () => {
39
-
const passed: string[] = []
40
-
41
-
class CustomReporter extends TaskParser {
42
-
onTestFinished(_test: RunnerTestCase): void {
43
-
passed.push(_test.name)
44
-
}
45
-
}
46
-
47
-
await runVitest({
48
-
root: pathe.join(import.meta.dirname, '../fixtures/reporter'),
49
-
benchmark: {
50
-
reporters: new CustomReporter(),
51
-
},
52
-
}, ['multiple.bench.ts'], 'benchmark')
53
-
54
-
expect(passed).toMatchInlineSnapshot(`
55
-
[
56
-
"first",
57
-
"second",
58
-
]
59
-
`)
60
34
})
61
35
62
36
it.for([true, false])('includeSamples %s', async (includeSamples) => {
+42
-5
test/cli/test/reported-tasks.test.ts
+42
-5
test/cli/test/reported-tasks.test.ts
···
56
56
expect(testModule.location).toBeUndefined()
57
57
expect(testModule.moduleId).toBe(resolve(root, './1_first.test.ts'))
58
58
expect(testModule.project).toBe(project)
59
-
expect(testModule.children.size).toBe(16)
59
+
expect(testModule.children.size).toBe(17)
60
60
61
61
const tests = [...testModule.children.tests()]
62
-
expect(tests).toHaveLength(11)
62
+
expect(tests).toHaveLength(12)
63
63
const deepTests = [...testModule.children.allTests()]
64
-
expect(deepTests).toHaveLength(21)
64
+
expect(deepTests).toHaveLength(22)
65
65
66
-
expect.soft([...testModule.children.allTests('skipped')]).toHaveLength(7)
66
+
expect.soft([...testModule.children.allTests('skipped')]).toHaveLength(8)
67
67
expect.soft([...testModule.children.allTests('passed')]).toHaveLength(9)
68
68
expect.soft([...testModule.children.allTests('failed')]).toHaveLength(5)
69
-
expect.soft([...testModule.children.allTests('running')]).toHaveLength(0)
69
+
expect.soft([...testModule.children.allTests('pending')]).toHaveLength(0)
70
70
71
71
const suites = [...testModule.children.suites()]
72
72
expect(suites).toHaveLength(5)
···
161
161
expect(diagnostic.flaky).toBe(false)
162
162
expect(diagnostic.repeatCount).toBe(0)
163
163
expect(diagnostic.repeatCount).toBe(0)
164
+
})
165
+
166
+
it('correctly reports a skipped test', () => {
167
+
const optionTestCase = findTest(testModule.children, 'skips an option test')
168
+
expect(optionTestCase.result()).toEqual({
169
+
state: 'skipped',
170
+
note: undefined,
171
+
errors: undefined,
172
+
})
173
+
174
+
const modifierTestCase = findTest(testModule.children, 'skips a .modifier test')
175
+
expect(modifierTestCase.result()).toEqual({
176
+
state: 'skipped',
177
+
note: undefined,
178
+
errors: undefined,
179
+
})
180
+
181
+
const ctxSkippedTestCase = findTest(testModule.children, 'skips an ctx.skip() test')
182
+
expect(ctxSkippedTestCase.result()).toEqual({
183
+
state: 'skipped',
184
+
note: undefined,
185
+
errors: undefined,
186
+
})
187
+
188
+
const testOptionTodo = findTest(testModule.children, 'todos an option test')
189
+
expect(testOptionTodo.result()).toEqual({
190
+
state: 'skipped',
191
+
note: undefined,
192
+
errors: undefined,
193
+
})
194
+
195
+
const testModifierTodo = findTest(testModule.children, 'todos a .modifier test')
196
+
expect(testModifierTodo.result()).toEqual({
197
+
state: 'skipped',
198
+
note: undefined,
199
+
errors: undefined,
200
+
})
164
201
})
165
202
166
203
it('correctly reports multiple failures', () => {
+3
test/core/test/sequencers.test.ts
+3
test/core/test/sequencers.test.ts
-2
test/coverage-test/test/merge-reports.test.ts
-2
test/coverage-test/test/merge-reports.test.ts
···
4
4
test('--merge-reports', async () => {
5
5
for (const index of [1, 2, 3]) {
6
6
await runVitest({
7
-
name: `generate #${index} blob report`,
8
7
include: ['fixtures/test/merge-fixture-*.test.ts'],
9
8
reporters: 'blob',
10
9
shard: `${index}/3`,
···
13
12
}
14
13
15
14
await runVitest({
16
-
name: 'merge blob reports',
17
15
// Pass default value - this option is publicly only available via CLI so it's a bit hacky usage here
18
16
mergeReports: '.vitest-reports',
19
17
coverage: {
+3
test/reporters/tests/dot.test.ts
+3
test/reporters/tests/dot.test.ts
···
58
58
59
59
expect(stdout).toContain('✓ fixtures/ok.test.ts')
60
60
expect(stdout).toContain('Test Files 1 passed (1)')
61
+
expect(stdout).not.toContain('·')
61
62
62
63
expect(stderr).toBe('')
63
64
})
···
72
73
expect(stdout).toContain('❯ fixtures/some-failing.test.ts (2 tests | 1 failed)')
73
74
expect(stdout).toContain('✓ 2 + 3 = 5')
74
75
expect(stdout).toContain('× 3 + 3 = 7')
76
+
expect(stdout).not.toContain('\n·x\n')
75
77
76
78
expect(stdout).toContain('Test Files 1 failed (1)')
77
79
expect(stdout).toContain('Tests 1 failed | 1 passed')
···
89
91
expect(stdout).toContain('↓ fixtures/all-skipped.test.ts (2 tests | 2 skipped)')
90
92
expect(stdout).toContain('Test Files 1 skipped (1)')
91
93
expect(stdout).toContain('Tests 1 skipped | 1 todo')
94
+
expect(stdout).not.toContain('\n--\n')
92
95
93
96
expect(stderr).toContain('')
94
97
})
-156
test/reporters/tests/task-parser.test.ts
-156
test/reporters/tests/task-parser.test.ts
···
1
-
import type { File, Test } from '@vitest/runner'
2
-
import type { Reporter, TestSpecification } from 'vitest/node'
3
-
import type { HookOptions } from '../../../packages/vitest/src/node/reporters/task-parser'
4
-
import { expect, test } from 'vitest'
5
-
import { TaskParser } from '../../../packages/vitest/src/node/reporters/task-parser'
6
-
import { runVitest } from '../../test-utils'
7
-
8
-
test('tasks are reported in correct order', async () => {
9
-
const reporter = new TaskReporter()
10
-
11
-
const { stdout, stderr } = await runVitest({
12
-
config: false,
13
-
include: ['./fixtures/task-parser-tests/*.test.ts'],
14
-
fileParallelism: false,
15
-
reporters: [reporter],
16
-
sequence: { sequencer: Sorter },
17
-
})
18
-
19
-
expect(stdout).toBe('')
20
-
expect(stderr).toBe('')
21
-
22
-
expect(reporter.calls).toMatchInlineSnapshot(`
23
-
[
24
-
"|fixtures/task-parser-tests/example-1.test.ts| start",
25
-
"|fixtures/task-parser-tests/example-1.test.ts| beforeAll start (suite)",
26
-
"|fixtures/task-parser-tests/example-1.test.ts| beforeAll end (suite)",
27
-
"|fixtures/task-parser-tests/example-1.test.ts| beforeAll end (suite)",
28
-
"|fixtures/task-parser-tests/example-1.test.ts| start",
29
-
"|fixtures/task-parser-tests/example-1.test.ts| RUN some test",
30
-
"|fixtures/task-parser-tests/example-1.test.ts| beforeEach start (test)",
31
-
"|fixtures/task-parser-tests/example-1.test.ts| beforeEach end (test)",
32
-
"|fixtures/task-parser-tests/example-1.test.ts| RUN some test",
33
-
"|fixtures/task-parser-tests/example-1.test.ts| beforeEach end (test)",
34
-
"|fixtures/task-parser-tests/example-1.test.ts| afterEach start (test)",
35
-
"|fixtures/task-parser-tests/example-1.test.ts| beforeEach end (test)",
36
-
"|fixtures/task-parser-tests/example-1.test.ts| afterEach end (test)",
37
-
"|fixtures/task-parser-tests/example-1.test.ts| beforeAll end (suite)",
38
-
"|fixtures/task-parser-tests/example-1.test.ts| afterAll end (suite)",
39
-
"|fixtures/task-parser-tests/example-1.test.ts| beforeEach end (test)",
40
-
"|fixtures/task-parser-tests/example-1.test.ts| afterEach end (test)",
41
-
"|fixtures/task-parser-tests/example-1.test.ts| beforeEach end (test)",
42
-
"|fixtures/task-parser-tests/example-1.test.ts| beforeEach end (test)",
43
-
"|fixtures/task-parser-tests/example-1.test.ts| DONE some test",
44
-
"|fixtures/task-parser-tests/example-1.test.ts| DONE Fast test 1",
45
-
"|fixtures/task-parser-tests/example-1.test.ts| RUN parallel slow tests 1.1",
46
-
"|fixtures/task-parser-tests/example-1.test.ts| RUN parallel slow tests 1.2",
47
-
"|fixtures/task-parser-tests/example-1.test.ts| beforeEach end (test)",
48
-
"|fixtures/task-parser-tests/example-1.test.ts| afterEach end (test)",
49
-
"|fixtures/task-parser-tests/example-1.test.ts| beforeEach end (test)",
50
-
"|fixtures/task-parser-tests/example-1.test.ts| afterEach end (test)",
51
-
"|fixtures/task-parser-tests/example-1.test.ts| beforeAll end (suite)",
52
-
"|fixtures/task-parser-tests/example-1.test.ts| DONE parallel slow tests 1.1",
53
-
"|fixtures/task-parser-tests/example-1.test.ts| DONE parallel slow tests 1.2",
54
-
"|fixtures/task-parser-tests/example-1.test.ts| start",
55
-
"|fixtures/task-parser-tests/example-1.test.ts| afterAll start (suite)",
56
-
"|fixtures/task-parser-tests/example-1.test.ts| beforeAll end (suite)",
57
-
"|fixtures/task-parser-tests/example-1.test.ts| afterAll end (suite)",
58
-
"|fixtures/task-parser-tests/example-1.test.ts| DONE Skipped test 1",
59
-
"|fixtures/task-parser-tests/example-1.test.ts| finish",
60
-
"|fixtures/task-parser-tests/example-2.test.ts| start",
61
-
"|fixtures/task-parser-tests/example-2.test.ts| beforeAll start (suite)",
62
-
"|fixtures/task-parser-tests/example-2.test.ts| beforeAll end (suite)",
63
-
"|fixtures/task-parser-tests/example-2.test.ts| beforeAll end (suite)",
64
-
"|fixtures/task-parser-tests/example-2.test.ts| start",
65
-
"|fixtures/task-parser-tests/example-2.test.ts| RUN some test",
66
-
"|fixtures/task-parser-tests/example-2.test.ts| beforeEach start (test)",
67
-
"|fixtures/task-parser-tests/example-2.test.ts| beforeEach end (test)",
68
-
"|fixtures/task-parser-tests/example-2.test.ts| RUN some test",
69
-
"|fixtures/task-parser-tests/example-2.test.ts| beforeEach end (test)",
70
-
"|fixtures/task-parser-tests/example-2.test.ts| afterEach start (test)",
71
-
"|fixtures/task-parser-tests/example-2.test.ts| beforeEach end (test)",
72
-
"|fixtures/task-parser-tests/example-2.test.ts| afterEach end (test)",
73
-
"|fixtures/task-parser-tests/example-2.test.ts| beforeAll end (suite)",
74
-
"|fixtures/task-parser-tests/example-2.test.ts| afterAll end (suite)",
75
-
"|fixtures/task-parser-tests/example-2.test.ts| beforeEach end (test)",
76
-
"|fixtures/task-parser-tests/example-2.test.ts| afterEach end (test)",
77
-
"|fixtures/task-parser-tests/example-2.test.ts| beforeEach end (test)",
78
-
"|fixtures/task-parser-tests/example-2.test.ts| beforeEach end (test)",
79
-
"|fixtures/task-parser-tests/example-2.test.ts| DONE some test",
80
-
"|fixtures/task-parser-tests/example-2.test.ts| DONE Fast test 1",
81
-
"|fixtures/task-parser-tests/example-2.test.ts| RUN parallel slow tests 2.1",
82
-
"|fixtures/task-parser-tests/example-2.test.ts| RUN parallel slow tests 2.2",
83
-
"|fixtures/task-parser-tests/example-2.test.ts| beforeEach end (test)",
84
-
"|fixtures/task-parser-tests/example-2.test.ts| afterEach end (test)",
85
-
"|fixtures/task-parser-tests/example-2.test.ts| beforeEach end (test)",
86
-
"|fixtures/task-parser-tests/example-2.test.ts| afterEach end (test)",
87
-
"|fixtures/task-parser-tests/example-2.test.ts| beforeAll end (suite)",
88
-
"|fixtures/task-parser-tests/example-2.test.ts| DONE parallel slow tests 2.1",
89
-
"|fixtures/task-parser-tests/example-2.test.ts| DONE parallel slow tests 2.2",
90
-
"|fixtures/task-parser-tests/example-2.test.ts| start",
91
-
"|fixtures/task-parser-tests/example-2.test.ts| afterAll start (suite)",
92
-
"|fixtures/task-parser-tests/example-2.test.ts| beforeAll end (suite)",
93
-
"|fixtures/task-parser-tests/example-2.test.ts| afterAll end (suite)",
94
-
"|fixtures/task-parser-tests/example-2.test.ts| DONE Skipped test 1",
95
-
"|fixtures/task-parser-tests/example-2.test.ts| finish",
96
-
]
97
-
`)
98
-
})
99
-
100
-
class TaskReporter extends TaskParser implements Reporter {
101
-
calls: string[] = []
102
-
103
-
// @ts-expect-error -- not sure why
104
-
onInit(ctx) {
105
-
super.onInit(ctx)
106
-
}
107
-
108
-
onTestFilePrepare(file: File) {
109
-
this.calls.push(`|${file.name}| start`)
110
-
}
111
-
112
-
onTestFileFinished(file: File) {
113
-
this.calls.push(`|${file.name}| finish`)
114
-
}
115
-
116
-
onTestStart(test: Test) {
117
-
this.calls.push(`|${test.file.name}| RUN ${test.name}`)
118
-
}
119
-
120
-
onTestFinished(test: Test) {
121
-
this.calls.push(`|${test.file.name}| DONE ${test.name}`)
122
-
}
123
-
124
-
onHookStart(options: HookOptions) {
125
-
this.calls.push(`|${options.file.name}| ${options.name} start (${options.type})`)
126
-
}
127
-
128
-
onHookEnd(options: HookOptions) {
129
-
this.calls.push(`|${options.file.name}| ${options.name} end (${options.type})`)
130
-
}
131
-
}
132
-
133
-
class Sorter {
134
-
sort(files: TestSpecification[]) {
135
-
return files.sort((a, b) => {
136
-
const idA = Number.parseInt(
137
-
a.moduleId.match(/example-(\d*)\.test\.ts/)![1],
138
-
)
139
-
const idB = Number.parseInt(
140
-
b.moduleId.match(/example-(\d*)\.test\.ts/)![1],
141
-
)
142
-
143
-
if (idA > idB) {
144
-
return 1
145
-
}
146
-
if (idA < idB) {
147
-
return -1
148
-
}
149
-
return 0
150
-
})
151
-
}
152
-
153
-
shard(files: TestSpecification[]) {
154
-
return files
155
-
}
156
-
}
+1119
test/reporters/tests/test-run.test.ts
+1119
test/reporters/tests/test-run.test.ts
···
1
+
import type {
2
+
ReportedHookContext,
3
+
Reporter,
4
+
SerializedError,
5
+
TestCase,
6
+
TestModule,
7
+
TestRunEndReason,
8
+
TestSpecification,
9
+
TestSuite,
10
+
UserConfig,
11
+
} from 'vitest/node'
12
+
import { rmSync } from 'node:fs'
13
+
import { resolve, sep } from 'node:path'
14
+
import { describe, expect, onTestFinished, test } from 'vitest'
15
+
import { runInlineTests, ts } from '../../test-utils'
16
+
17
+
describe('TestRun', () => {
18
+
test('pass test run without files (no-watch)', async () => {
19
+
const report = await run(
20
+
{},
21
+
{
22
+
passWithNoTests: true,
23
+
watch: false,
24
+
},
25
+
{
26
+
printTestRunEvents: true,
27
+
failed: true,
28
+
},
29
+
)
30
+
31
+
expect(report).toMatchInlineSnapshot(`
32
+
"
33
+
onTestRunStart (0 specifications)
34
+
onTestRunEnd (passed, 0 modules, 0 errors)"
35
+
`)
36
+
})
37
+
38
+
test('pass test run without files (watch)', async () => {
39
+
const report = await run(
40
+
{},
41
+
{
42
+
passWithNoTests: true,
43
+
watch: true,
44
+
},
45
+
{
46
+
printTestRunEvents: true,
47
+
failed: true,
48
+
},
49
+
)
50
+
51
+
expect(report).toMatchInlineSnapshot(`
52
+
"
53
+
onTestRunStart (0 specifications)
54
+
onTestRunEnd (passed, 0 modules, 0 errors)"
55
+
`)
56
+
})
57
+
58
+
test('fail test run without files (no-watch)', async () => {
59
+
const report = await run(
60
+
{},
61
+
{
62
+
passWithNoTests: false,
63
+
watch: false,
64
+
},
65
+
{
66
+
printTestRunEvents: true,
67
+
failed: true,
68
+
},
69
+
)
70
+
71
+
expect(report).toMatchInlineSnapshot(`
72
+
"
73
+
onTestRunStart (0 specifications)
74
+
onTestRunEnd (failed, 0 modules, 0 errors)"
75
+
`)
76
+
})
77
+
})
78
+
79
+
describe('TestModule', () => {
80
+
test('single test module', async () => {
81
+
const report = await run({
82
+
'test-module.test.ts': ts`
83
+
test('example', () => {});
84
+
`,
85
+
})
86
+
87
+
expect(report).toMatchInlineSnapshot(`
88
+
"
89
+
onTestModuleQueued (test-module.test.ts)
90
+
onTestModuleCollected (test-module.test.ts)
91
+
onTestModuleStart (test-module.test.ts)
92
+
onTestCaseReady (test-module.test.ts) |example|
93
+
onTestCaseResult (test-module.test.ts) |example|
94
+
onTestModuleEnd (test-module.test.ts)"
95
+
`)
96
+
})
97
+
98
+
test('multiple test modules', async () => {
99
+
const report = await run({
100
+
'first.test.ts': ts`
101
+
test('first test case', () => {});
102
+
`,
103
+
'second.test.ts': ts`
104
+
test('second test case', () => {});
105
+
`,
106
+
})
107
+
108
+
expect(report).toMatchInlineSnapshot(`
109
+
"
110
+
onTestModuleQueued (first.test.ts)
111
+
onTestModuleCollected (first.test.ts)
112
+
onTestModuleStart (first.test.ts)
113
+
onTestCaseReady (first.test.ts) |first test case|
114
+
onTestCaseResult (first.test.ts) |first test case|
115
+
onTestModuleEnd (first.test.ts)
116
+
117
+
onTestModuleQueued (second.test.ts)
118
+
onTestModuleCollected (second.test.ts)
119
+
onTestModuleStart (second.test.ts)
120
+
onTestCaseReady (second.test.ts) |second test case|
121
+
onTestCaseResult (second.test.ts) |second test case|
122
+
onTestModuleEnd (second.test.ts)"
123
+
`)
124
+
})
125
+
126
+
test('test modules with delay', async () => {
127
+
const report = await run({
128
+
'first.test.ts': ts`
129
+
${delay()}
130
+
test('first test case', async () => { ${delay()} });
131
+
`,
132
+
'second.test.ts': ts`
133
+
${delay()}
134
+
test('second test case', async () => { ${delay()} });
135
+
`,
136
+
})
137
+
138
+
expect(report).toMatchInlineSnapshot(`
139
+
"
140
+
onTestModuleQueued (first.test.ts)
141
+
onTestModuleCollected (first.test.ts)
142
+
onTestModuleStart (first.test.ts)
143
+
onTestCaseReady (first.test.ts) |first test case|
144
+
onTestCaseResult (first.test.ts) |first test case|
145
+
onTestModuleEnd (first.test.ts)
146
+
147
+
onTestModuleQueued (second.test.ts)
148
+
onTestModuleCollected (second.test.ts)
149
+
onTestModuleStart (second.test.ts)
150
+
onTestCaseReady (second.test.ts) |second test case|
151
+
onTestCaseResult (second.test.ts) |second test case|
152
+
onTestModuleEnd (second.test.ts)"
153
+
`)
154
+
})
155
+
})
156
+
157
+
describe('TestCase', () => {
158
+
test('single test case', async () => {
159
+
const report = await run({
160
+
'example.test.ts': ts`
161
+
test('single test case', () => {});
162
+
`,
163
+
})
164
+
165
+
expect(report).toMatchInlineSnapshot(`
166
+
"
167
+
onTestModuleQueued (example.test.ts)
168
+
onTestModuleCollected (example.test.ts)
169
+
onTestModuleStart (example.test.ts)
170
+
onTestCaseReady (example.test.ts) |single test case|
171
+
onTestCaseResult (example.test.ts) |single test case|
172
+
onTestModuleEnd (example.test.ts)"
173
+
`)
174
+
})
175
+
176
+
test('multiple test cases', async () => {
177
+
const report = await run({
178
+
'example.test.ts': ts`
179
+
test('first', () => {});
180
+
test('second', () => {});
181
+
test('third', () => {});
182
+
`,
183
+
})
184
+
185
+
expect(report).toMatchInlineSnapshot(`
186
+
"
187
+
onTestModuleQueued (example.test.ts)
188
+
onTestModuleCollected (example.test.ts)
189
+
onTestModuleStart (example.test.ts)
190
+
onTestCaseReady (example.test.ts) |first|
191
+
onTestCaseResult (example.test.ts) |first|
192
+
onTestCaseReady (example.test.ts) |second|
193
+
onTestCaseResult (example.test.ts) |second|
194
+
onTestCaseReady (example.test.ts) |third|
195
+
onTestCaseResult (example.test.ts) |third|
196
+
onTestModuleEnd (example.test.ts)"
197
+
`)
198
+
})
199
+
200
+
test('multiple test cases with delay', async () => {
201
+
const report = await run({
202
+
'example.test.ts': ts`
203
+
${delay()}
204
+
test('first', async () => { ${delay()} });
205
+
test('second', async () => { ${delay()} });
206
+
test('third', async () => { ${delay()} });
207
+
`,
208
+
})
209
+
210
+
expect(report).toMatchInlineSnapshot(`
211
+
"
212
+
onTestModuleQueued (example.test.ts)
213
+
onTestModuleCollected (example.test.ts)
214
+
onTestModuleStart (example.test.ts)
215
+
onTestCaseReady (example.test.ts) |first|
216
+
onTestCaseResult (example.test.ts) |first|
217
+
onTestCaseReady (example.test.ts) |second|
218
+
onTestCaseResult (example.test.ts) |second|
219
+
onTestCaseReady (example.test.ts) |third|
220
+
onTestCaseResult (example.test.ts) |third|
221
+
onTestModuleEnd (example.test.ts)"
222
+
`)
223
+
})
224
+
225
+
test('failing test case', async () => {
226
+
const report = await run({
227
+
'example.test.ts': ts`
228
+
test('failing test case', () => {
229
+
expect(1).toBe(2)
230
+
});
231
+
`,
232
+
})
233
+
234
+
expect(report).toMatchInlineSnapshot(`
235
+
"
236
+
onTestModuleQueued (example.test.ts)
237
+
onTestModuleCollected (example.test.ts)
238
+
onTestModuleStart (example.test.ts)
239
+
onTestCaseReady (example.test.ts) |failing test case|
240
+
onTestCaseResult (example.test.ts) |failing test case|
241
+
onTestModuleEnd (example.test.ts)"
242
+
`)
243
+
})
244
+
245
+
test('skipped test case', async () => {
246
+
const report = await run({
247
+
'example.test.ts': ts`
248
+
test('running', () => {});
249
+
test.skip('skipped', () => {});
250
+
`,
251
+
})
252
+
253
+
expect(report).toMatchInlineSnapshot(`
254
+
"
255
+
onTestModuleQueued (example.test.ts)
256
+
onTestModuleCollected (example.test.ts)
257
+
onTestModuleStart (example.test.ts)
258
+
onTestCaseReady (example.test.ts) |running|
259
+
onTestCaseResult (example.test.ts) |running|
260
+
onTestCaseReady (example.test.ts) |skipped|
261
+
onTestCaseResult (example.test.ts) |skipped|
262
+
onTestModuleEnd (example.test.ts)"
263
+
`)
264
+
})
265
+
266
+
test('dynamically skipped test case', async () => {
267
+
const report = await run({
268
+
'example.test.ts': ts`
269
+
test('running', () => {});
270
+
test('skipped', (ctx) => { ctx.skip() });
271
+
`,
272
+
})
273
+
274
+
expect(report).toMatchInlineSnapshot(`
275
+
"
276
+
onTestModuleQueued (example.test.ts)
277
+
onTestModuleCollected (example.test.ts)
278
+
onTestModuleStart (example.test.ts)
279
+
onTestCaseReady (example.test.ts) |running|
280
+
onTestCaseResult (example.test.ts) |running|
281
+
onTestCaseReady (example.test.ts) |skipped|
282
+
onTestCaseResult (example.test.ts) |skipped|
283
+
onTestModuleEnd (example.test.ts)"
284
+
`)
285
+
})
286
+
287
+
test('skipped all test cases', async () => {
288
+
const report = await run({
289
+
'example.test.ts': ts`
290
+
test.skip('first', () => {});
291
+
test.skip('second', () => {});
292
+
`,
293
+
})
294
+
295
+
expect(report).toMatchInlineSnapshot(`
296
+
"
297
+
onTestModuleQueued (example.test.ts)
298
+
onTestModuleCollected (example.test.ts)
299
+
onTestModuleStart (example.test.ts)
300
+
onTestCaseReady (example.test.ts) |first|
301
+
onTestCaseResult (example.test.ts) |first|
302
+
onTestCaseReady (example.test.ts) |second|
303
+
onTestCaseResult (example.test.ts) |second|
304
+
onTestModuleEnd (example.test.ts)"
305
+
`)
306
+
})
307
+
})
308
+
309
+
describe('TestSuite', () => {
310
+
test('single test suite', async () => {
311
+
const report = await run({
312
+
'example.test.ts': ts`
313
+
describe("example suite", () => {
314
+
test('first test case', () => {});
315
+
});
316
+
`,
317
+
})
318
+
319
+
expect(report).toMatchInlineSnapshot(`
320
+
"
321
+
onTestModuleQueued (example.test.ts)
322
+
onTestModuleCollected (example.test.ts)
323
+
onTestModuleStart (example.test.ts)
324
+
onTestSuiteReady (example.test.ts) |example suite|
325
+
onTestCaseReady (example.test.ts) |first test case|
326
+
onTestCaseResult (example.test.ts) |first test case|
327
+
onTestSuiteResult (example.test.ts) |example suite|
328
+
onTestModuleEnd (example.test.ts)"
329
+
`)
330
+
})
331
+
332
+
test('multiple test suites', async () => {
333
+
const report = await run({
334
+
'example.test.ts': ts`
335
+
describe("first suite", () => {
336
+
test('first test case', () => {});
337
+
});
338
+
339
+
describe("second suite", () => {
340
+
test('second test case', () => {});
341
+
});
342
+
`,
343
+
})
344
+
345
+
expect(report).toMatchInlineSnapshot(`
346
+
"
347
+
onTestModuleQueued (example.test.ts)
348
+
onTestModuleCollected (example.test.ts)
349
+
onTestModuleStart (example.test.ts)
350
+
onTestSuiteReady (example.test.ts) |first suite|
351
+
onTestCaseReady (example.test.ts) |first test case|
352
+
onTestCaseResult (example.test.ts) |first test case|
353
+
onTestSuiteResult (example.test.ts) |first suite|
354
+
onTestSuiteReady (example.test.ts) |second suite|
355
+
onTestCaseReady (example.test.ts) |second test case|
356
+
onTestCaseResult (example.test.ts) |second test case|
357
+
onTestSuiteResult (example.test.ts) |second suite|
358
+
onTestModuleEnd (example.test.ts)"
359
+
`)
360
+
})
361
+
362
+
test('multiple test suites with delay', async () => {
363
+
const report = await run({
364
+
'example.test.ts': ts`
365
+
${delay()}
366
+
describe("first suite", async () => {
367
+
${delay()}
368
+
test('first test case', async () => { ${delay()} });
369
+
});
370
+
371
+
describe("second suite", async () => {
372
+
${delay()}
373
+
test('second test case', async () => { ${delay()} });
374
+
});
375
+
`,
376
+
})
377
+
378
+
expect(report).toMatchInlineSnapshot(`
379
+
"
380
+
onTestModuleQueued (example.test.ts)
381
+
onTestModuleCollected (example.test.ts)
382
+
onTestModuleStart (example.test.ts)
383
+
onTestSuiteReady (example.test.ts) |first suite|
384
+
onTestCaseReady (example.test.ts) |first test case|
385
+
onTestCaseResult (example.test.ts) |first test case|
386
+
onTestSuiteResult (example.test.ts) |first suite|
387
+
onTestSuiteReady (example.test.ts) |second suite|
388
+
onTestCaseReady (example.test.ts) |second test case|
389
+
onTestCaseResult (example.test.ts) |second test case|
390
+
onTestSuiteResult (example.test.ts) |second suite|
391
+
onTestModuleEnd (example.test.ts)"
392
+
`)
393
+
})
394
+
395
+
test('nested test suites', async () => {
396
+
const report = await run({
397
+
'example.test.ts': ts`
398
+
describe("first suite", () => {
399
+
test('first test case', () => {});
400
+
401
+
describe("second suite", () => {
402
+
test('second test case', () => {});
403
+
404
+
describe("third suite", () => {
405
+
test('third test case', () => {});
406
+
});
407
+
});
408
+
});
409
+
`,
410
+
})
411
+
412
+
expect(report).toMatchInlineSnapshot(`
413
+
"
414
+
onTestModuleQueued (example.test.ts)
415
+
onTestModuleCollected (example.test.ts)
416
+
onTestModuleStart (example.test.ts)
417
+
onTestSuiteReady (example.test.ts) |first suite|
418
+
onTestCaseReady (example.test.ts) |first test case|
419
+
onTestCaseResult (example.test.ts) |first test case|
420
+
onTestSuiteReady (example.test.ts) |second suite|
421
+
onTestCaseReady (example.test.ts) |second test case|
422
+
onTestCaseResult (example.test.ts) |second test case|
423
+
onTestSuiteReady (example.test.ts) |third suite|
424
+
onTestCaseReady (example.test.ts) |third test case|
425
+
onTestCaseResult (example.test.ts) |third test case|
426
+
onTestSuiteResult (example.test.ts) |third suite|
427
+
onTestSuiteResult (example.test.ts) |second suite|
428
+
onTestSuiteResult (example.test.ts) |first suite|
429
+
onTestModuleEnd (example.test.ts)"
430
+
`)
431
+
})
432
+
433
+
test('skipped test suite', async () => {
434
+
const report = await run({
435
+
'example.test.ts': ts`
436
+
describe.skip("skipped suite", () => {
437
+
test('first test case', () => {});
438
+
});
439
+
`,
440
+
})
441
+
442
+
expect(report).toMatchInlineSnapshot(`
443
+
"
444
+
onTestModuleQueued (example.test.ts)
445
+
onTestModuleCollected (example.test.ts)
446
+
onTestModuleStart (example.test.ts)
447
+
onTestSuiteReady (example.test.ts) |skipped suite|
448
+
onTestCaseReady (example.test.ts) |first test case|
449
+
onTestCaseResult (example.test.ts) |first test case|
450
+
onTestSuiteResult (example.test.ts) |skipped suite|
451
+
onTestModuleEnd (example.test.ts)"
452
+
`)
453
+
})
454
+
455
+
test('skipped double nested test suite', async () => {
456
+
const report = await run({
457
+
'example.test.ts': ts`
458
+
describe.skip("skipped suite", () => {
459
+
describe.skip("nested skipped suite", () => {
460
+
test('first nested case', () => {});
461
+
})
462
+
});
463
+
464
+
test('first test case', () => {});
465
+
`,
466
+
})
467
+
468
+
expect(report).toMatchInlineSnapshot(`
469
+
"
470
+
onTestModuleQueued (example.test.ts)
471
+
onTestModuleCollected (example.test.ts)
472
+
onTestModuleStart (example.test.ts)
473
+
onTestSuiteReady (example.test.ts) |skipped suite|
474
+
onTestSuiteReady (example.test.ts) |nested skipped suite|
475
+
onTestCaseReady (example.test.ts) |first nested case|
476
+
onTestCaseResult (example.test.ts) |first nested case|
477
+
onTestSuiteResult (example.test.ts) |nested skipped suite|
478
+
onTestSuiteResult (example.test.ts) |skipped suite|
479
+
onTestCaseReady (example.test.ts) |first test case|
480
+
onTestCaseResult (example.test.ts) |first test case|
481
+
onTestModuleEnd (example.test.ts)"
482
+
`)
483
+
})
484
+
485
+
test('skipped nested test suite', async () => {
486
+
const report = await run({
487
+
'example.test.ts': ts`
488
+
describe("first suite", () => {
489
+
test('first test case', () => {});
490
+
491
+
describe.skip("skipped suite", () => {
492
+
test('second test case', () => {});
493
+
});
494
+
});
495
+
`,
496
+
})
497
+
498
+
expect(report).toMatchInlineSnapshot(`
499
+
"
500
+
onTestModuleQueued (example.test.ts)
501
+
onTestModuleCollected (example.test.ts)
502
+
onTestModuleStart (example.test.ts)
503
+
onTestSuiteReady (example.test.ts) |first suite|
504
+
onTestCaseReady (example.test.ts) |first test case|
505
+
onTestCaseResult (example.test.ts) |first test case|
506
+
onTestSuiteReady (example.test.ts) |skipped suite|
507
+
onTestCaseReady (example.test.ts) |second test case|
508
+
onTestCaseResult (example.test.ts) |second test case|
509
+
onTestSuiteResult (example.test.ts) |skipped suite|
510
+
onTestSuiteResult (example.test.ts) |first suite|
511
+
onTestModuleEnd (example.test.ts)"
512
+
`)
513
+
})
514
+
})
515
+
516
+
describe('hooks', () => {
517
+
test('beforeEach', async () => {
518
+
const report = await run({
519
+
'example.test.ts': ts`
520
+
beforeEach(() => {});
521
+
522
+
test('first', () => {});
523
+
test('second', () => {});
524
+
`,
525
+
})
526
+
527
+
expect(report).toMatchInlineSnapshot(`
528
+
"
529
+
onTestModuleQueued (example.test.ts)
530
+
onTestModuleCollected (example.test.ts)
531
+
onTestModuleStart (example.test.ts)
532
+
onTestCaseReady (example.test.ts) |first|
533
+
onHookStart (example.test.ts) |first| [beforeEach]
534
+
onHookEnd (example.test.ts) |first| [beforeEach]
535
+
onTestCaseResult (example.test.ts) |first|
536
+
onTestCaseReady (example.test.ts) |second|
537
+
onHookStart (example.test.ts) |second| [beforeEach]
538
+
onHookEnd (example.test.ts) |second| [beforeEach]
539
+
onTestCaseResult (example.test.ts) |second|
540
+
onTestModuleEnd (example.test.ts)"
541
+
`)
542
+
})
543
+
544
+
test('afterEach', async () => {
545
+
const report = await run({
546
+
'example.test.ts': ts`
547
+
afterEach(() => {});
548
+
549
+
test('first', () => {});
550
+
test('second', () => {});
551
+
`,
552
+
})
553
+
554
+
expect(report).toMatchInlineSnapshot(`
555
+
"
556
+
onTestModuleQueued (example.test.ts)
557
+
onTestModuleCollected (example.test.ts)
558
+
onTestModuleStart (example.test.ts)
559
+
onTestCaseReady (example.test.ts) |first|
560
+
onHookStart (example.test.ts) |first| [afterEach]
561
+
onHookEnd (example.test.ts) |first| [afterEach]
562
+
onTestCaseResult (example.test.ts) |first|
563
+
onTestCaseReady (example.test.ts) |second|
564
+
onHookStart (example.test.ts) |second| [afterEach]
565
+
onHookEnd (example.test.ts) |second| [afterEach]
566
+
onTestCaseResult (example.test.ts) |second|
567
+
onTestModuleEnd (example.test.ts)"
568
+
`)
569
+
})
570
+
571
+
test('beforeEach and afterEach', async () => {
572
+
const report = await run({
573
+
'example.test.ts': ts`
574
+
beforeEach(() => {});
575
+
afterEach(() => {});
576
+
577
+
test('first', () => {});
578
+
test('second', () => {});
579
+
`,
580
+
})
581
+
582
+
expect(report).toMatchInlineSnapshot(`
583
+
"
584
+
onTestModuleQueued (example.test.ts)
585
+
onTestModuleCollected (example.test.ts)
586
+
onTestModuleStart (example.test.ts)
587
+
onTestCaseReady (example.test.ts) |first|
588
+
onHookStart (example.test.ts) |first| [beforeEach]
589
+
onHookEnd (example.test.ts) |first| [beforeEach]
590
+
onHookStart (example.test.ts) |first| [afterEach]
591
+
onHookEnd (example.test.ts) |first| [afterEach]
592
+
onTestCaseResult (example.test.ts) |first|
593
+
onTestCaseReady (example.test.ts) |second|
594
+
onHookStart (example.test.ts) |second| [beforeEach]
595
+
onHookEnd (example.test.ts) |second| [beforeEach]
596
+
onHookStart (example.test.ts) |second| [afterEach]
597
+
onHookEnd (example.test.ts) |second| [afterEach]
598
+
onTestCaseResult (example.test.ts) |second|
599
+
onTestModuleEnd (example.test.ts)"
600
+
`)
601
+
})
602
+
603
+
test('beforeAll', async () => {
604
+
const report = await run({
605
+
'example.test.ts': ts`
606
+
beforeAll(() => {});
607
+
608
+
test('first', () => {});
609
+
test('second', () => {});
610
+
`,
611
+
})
612
+
613
+
expect(report).toMatchInlineSnapshot(`
614
+
"
615
+
onTestModuleQueued (example.test.ts)
616
+
onTestModuleCollected (example.test.ts)
617
+
onTestModuleStart (example.test.ts)
618
+
onHookStart (example.test.ts) [beforeAll]
619
+
onHookEnd (example.test.ts) [beforeAll]
620
+
onTestCaseReady (example.test.ts) |first|
621
+
onTestCaseResult (example.test.ts) |first|
622
+
onTestCaseReady (example.test.ts) |second|
623
+
onTestCaseResult (example.test.ts) |second|
624
+
onTestModuleEnd (example.test.ts)"
625
+
`)
626
+
})
627
+
628
+
test('afterAll', async () => {
629
+
const report = await run({
630
+
'example.test.ts': ts`
631
+
afterAll(() => {});
632
+
633
+
test('first', () => {});
634
+
test('second', () => {});
635
+
`,
636
+
})
637
+
638
+
expect(report).toMatchInlineSnapshot(`
639
+
"
640
+
onTestModuleQueued (example.test.ts)
641
+
onTestModuleCollected (example.test.ts)
642
+
onTestModuleStart (example.test.ts)
643
+
onTestCaseReady (example.test.ts) |first|
644
+
onTestCaseResult (example.test.ts) |first|
645
+
onTestCaseReady (example.test.ts) |second|
646
+
onTestCaseResult (example.test.ts) |second|
647
+
onHookStart (example.test.ts) [afterAll]
648
+
onHookEnd (example.test.ts) [afterAll]
649
+
onTestModuleEnd (example.test.ts)"
650
+
`)
651
+
})
652
+
653
+
test('beforeAll and afterAll', async () => {
654
+
const report = await run({
655
+
'example.test.ts': ts`
656
+
beforeAll(() => {});
657
+
afterAll(() => {});
658
+
659
+
test('first', () => {});
660
+
test('second', () => {});
661
+
`,
662
+
})
663
+
664
+
expect(report).toMatchInlineSnapshot(`
665
+
"
666
+
onTestModuleQueued (example.test.ts)
667
+
onTestModuleCollected (example.test.ts)
668
+
onTestModuleStart (example.test.ts)
669
+
onHookStart (example.test.ts) [beforeAll]
670
+
onHookEnd (example.test.ts) [beforeAll]
671
+
onTestCaseReady (example.test.ts) |first|
672
+
onTestCaseResult (example.test.ts) |first|
673
+
onTestCaseReady (example.test.ts) |second|
674
+
onTestCaseResult (example.test.ts) |second|
675
+
onHookStart (example.test.ts) [afterAll]
676
+
onHookEnd (example.test.ts) [afterAll]
677
+
onTestModuleEnd (example.test.ts)"
678
+
`)
679
+
})
680
+
681
+
test('all hooks with delay', async () => {
682
+
const report = await run({
683
+
'example.test.ts': ts`
684
+
${delay()}
685
+
beforeAll(async () => { ${delay()} });
686
+
afterAll(async () => { ${delay()} });
687
+
beforeEach(async () => { ${delay()} });
688
+
afterEach(async () => { ${delay()} });
689
+
690
+
test('first', async () => { ${delay()} });
691
+
test('second', async () => { ${delay()} });
692
+
`,
693
+
})
694
+
695
+
expect(report).toMatchInlineSnapshot(`
696
+
"
697
+
onTestModuleQueued (example.test.ts)
698
+
onTestModuleCollected (example.test.ts)
699
+
onTestModuleStart (example.test.ts)
700
+
onHookStart (example.test.ts) [beforeAll]
701
+
onHookEnd (example.test.ts) [beforeAll]
702
+
onTestCaseReady (example.test.ts) |first|
703
+
onHookStart (example.test.ts) |first| [beforeEach]
704
+
onHookEnd (example.test.ts) |first| [beforeEach]
705
+
onHookStart (example.test.ts) |first| [afterEach]
706
+
onHookEnd (example.test.ts) |first| [afterEach]
707
+
onTestCaseResult (example.test.ts) |first|
708
+
onTestCaseReady (example.test.ts) |second|
709
+
onHookStart (example.test.ts) |second| [beforeEach]
710
+
onHookEnd (example.test.ts) |second| [beforeEach]
711
+
onHookStart (example.test.ts) |second| [afterEach]
712
+
onHookEnd (example.test.ts) |second| [afterEach]
713
+
onTestCaseResult (example.test.ts) |second|
714
+
onHookStart (example.test.ts) [afterAll]
715
+
onHookEnd (example.test.ts) [afterAll]
716
+
onTestModuleEnd (example.test.ts)"
717
+
`)
718
+
})
719
+
720
+
test('beforeAll on suite', async () => {
721
+
const report = await run({
722
+
'example.test.ts': ts`
723
+
describe("example", () => {
724
+
beforeAll(() => {});
725
+
726
+
test('first', () => {});
727
+
test('second', () => {});
728
+
})
729
+
`,
730
+
})
731
+
732
+
expect(report).toMatchInlineSnapshot(`
733
+
"
734
+
onTestModuleQueued (example.test.ts)
735
+
onTestModuleCollected (example.test.ts)
736
+
onTestModuleStart (example.test.ts)
737
+
onTestSuiteReady (example.test.ts) |example|
738
+
onHookStart (example.test.ts) |example| [beforeAll]
739
+
onHookEnd (example.test.ts) |example| [beforeAll]
740
+
onTestCaseReady (example.test.ts) |first|
741
+
onTestCaseResult (example.test.ts) |first|
742
+
onTestCaseReady (example.test.ts) |second|
743
+
onTestCaseResult (example.test.ts) |second|
744
+
onTestSuiteResult (example.test.ts) |example|
745
+
onTestModuleEnd (example.test.ts)"
746
+
`)
747
+
})
748
+
749
+
test('afterAll on suite', async () => {
750
+
const report = await run({
751
+
'example.test.ts': ts`
752
+
describe("example", () => {
753
+
afterAll(() => {});
754
+
755
+
test('first', () => {});
756
+
test('second', () => {});
757
+
})
758
+
`,
759
+
})
760
+
761
+
expect(report).toMatchInlineSnapshot(`
762
+
"
763
+
onTestModuleQueued (example.test.ts)
764
+
onTestModuleCollected (example.test.ts)
765
+
onTestModuleStart (example.test.ts)
766
+
onTestSuiteReady (example.test.ts) |example|
767
+
onTestCaseReady (example.test.ts) |first|
768
+
onTestCaseResult (example.test.ts) |first|
769
+
onTestCaseReady (example.test.ts) |second|
770
+
onTestCaseResult (example.test.ts) |second|
771
+
onHookStart (example.test.ts) |example| [afterAll]
772
+
onHookEnd (example.test.ts) |example| [afterAll]
773
+
onTestSuiteResult (example.test.ts) |example|
774
+
onTestModuleEnd (example.test.ts)"
775
+
`)
776
+
})
777
+
})
778
+
779
+
describe('merge reports', () => {
780
+
test('correctly reports events for a single test module', async () => {
781
+
const blobsOutputDirectory = resolve(import.meta.dirname, 'fixtures-blobs')
782
+
const blobOutputFile = resolve(blobsOutputDirectory, 'blob.json')
783
+
onTestFinished(() => {
784
+
rmSync(blobOutputFile)
785
+
})
786
+
787
+
const { root } = await runInlineTests({
788
+
'example.test.ts': ts`
789
+
test('first', () => {});
790
+
describe('suite', () => {
791
+
test('second', () => {});
792
+
});
793
+
`,
794
+
}, {
795
+
globals: true,
796
+
reporters: [['blob', { outputFile: blobOutputFile }]],
797
+
})
798
+
799
+
const report = await run(
800
+
{},
801
+
{
802
+
mergeReports: blobsOutputDirectory,
803
+
},
804
+
{
805
+
roots: [root],
806
+
},
807
+
)
808
+
809
+
expect(report).toMatchInlineSnapshot(`
810
+
"
811
+
onTestModuleQueued (example.test.ts)
812
+
onTestModuleCollected (example.test.ts)
813
+
onTestModuleStart (example.test.ts)
814
+
onTestCaseReady (example.test.ts) |first|
815
+
onTestCaseResult (example.test.ts) |first|
816
+
onTestSuiteReady (example.test.ts) |suite|
817
+
onTestCaseReady (example.test.ts) |second|
818
+
onTestCaseResult (example.test.ts) |second|
819
+
onTestSuiteResult (example.test.ts) |suite|
820
+
onTestModuleEnd (example.test.ts)"
821
+
`)
822
+
})
823
+
824
+
test('correctly reports multiple test modules', async () => {
825
+
const blobsOutputDirectory = resolve(import.meta.dirname, 'fixtures-blobs')
826
+
const blobOutputFile1 = resolve(blobsOutputDirectory, 'blob-1.json')
827
+
const blobOutputFile2 = resolve(blobsOutputDirectory, 'blob-2.json')
828
+
onTestFinished(() => {
829
+
rmSync(blobOutputFile1)
830
+
rmSync(blobOutputFile2)
831
+
})
832
+
833
+
const { root: root1 } = await runInlineTests({
834
+
'example-1.test.ts': ts`
835
+
test('first', () => {});
836
+
describe('suite', () => {
837
+
test('second', () => {});
838
+
});
839
+
`,
840
+
}, {
841
+
globals: true,
842
+
reporters: [['blob', { outputFile: blobOutputFile1 }]],
843
+
})
844
+
845
+
const { root: root2 } = await runInlineTests({
846
+
'example-2.test.ts': ts`
847
+
test('first', () => {});
848
+
describe.skip('suite', () => {
849
+
test('second', () => {});
850
+
test('third', () => {});
851
+
});
852
+
test.skip('fourth', () => {});
853
+
test('fifth', () => {});
854
+
`,
855
+
}, {
856
+
globals: true,
857
+
reporters: [['blob', { outputFile: blobOutputFile2 }]],
858
+
})
859
+
860
+
const report = await run({}, {
861
+
mergeReports: blobsOutputDirectory,
862
+
}, {
863
+
roots: [root1, root2],
864
+
})
865
+
866
+
expect(report).toMatchInlineSnapshot(`
867
+
"
868
+
onTestModuleQueued (example-1.test.ts)
869
+
onTestModuleCollected (example-1.test.ts)
870
+
onTestModuleStart (example-1.test.ts)
871
+
onTestCaseReady (example-1.test.ts) |first|
872
+
onTestCaseResult (example-1.test.ts) |first|
873
+
onTestSuiteReady (example-1.test.ts) |suite|
874
+
onTestCaseReady (example-1.test.ts) |second|
875
+
onTestCaseResult (example-1.test.ts) |second|
876
+
onTestSuiteResult (example-1.test.ts) |suite|
877
+
onTestModuleEnd (example-1.test.ts)
878
+
879
+
onTestModuleQueued (example-2.test.ts)
880
+
onTestModuleCollected (example-2.test.ts)
881
+
onTestModuleStart (example-2.test.ts)
882
+
onTestCaseReady (example-2.test.ts) |first|
883
+
onTestCaseResult (example-2.test.ts) |first|
884
+
onTestSuiteReady (example-2.test.ts) |suite|
885
+
onTestCaseReady (example-2.test.ts) |second|
886
+
onTestCaseResult (example-2.test.ts) |second|
887
+
onTestCaseReady (example-2.test.ts) |third|
888
+
onTestCaseResult (example-2.test.ts) |third|
889
+
onTestSuiteResult (example-2.test.ts) |suite|
890
+
onTestCaseReady (example-2.test.ts) |fifth|
891
+
onTestCaseResult (example-2.test.ts) |fifth|
892
+
onTestCaseReady (example-2.test.ts) |fourth|
893
+
onTestCaseResult (example-2.test.ts) |fourth|
894
+
onTestModuleEnd (example-2.test.ts)"
895
+
`)
896
+
})
897
+
})
898
+
899
+
describe('type checking', () => {
900
+
test('typechking is reported correctly', async () => {
901
+
const report = await run({
902
+
'example-1.test-d.ts': ts`
903
+
test('first', () => {});
904
+
describe('suite', () => {
905
+
test('second', () => {});
906
+
});
907
+
`,
908
+
'example-2.test-d.ts': ts`
909
+
test('first', () => {});
910
+
describe.skip('suite', () => {
911
+
test('second', () => {});
912
+
test('third', () => {});
913
+
});
914
+
test.skip('fourth', () => {});
915
+
test('fifth', () => {});
916
+
`,
917
+
'tsconfig.json': JSON.stringify({
918
+
compilerOptions: {
919
+
strict: true,
920
+
},
921
+
include: ['./*.test-d.ts'],
922
+
}),
923
+
}, {
924
+
typecheck: {
925
+
enabled: true,
926
+
},
927
+
}, { printTestRunEvents: true })
928
+
929
+
// NOTE: typechecker reports test modules in bulk, so the order of queued and collect
930
+
// is different from the normal test run, this is because the typechecker runs everything together
931
+
// this _might_ need to be changed in the future
932
+
expect(report).toMatchInlineSnapshot(`
933
+
"
934
+
onTestRunStart (2 specifications)
935
+
onTestModuleQueued (example-1.test-d.ts)
936
+
onTestModuleQueued (example-2.test-d.ts)
937
+
onTestModuleCollected (example-1.test-d.ts)
938
+
onTestModuleCollected (example-2.test-d.ts)
939
+
onTestModuleStart (example-1.test-d.ts)
940
+
onTestCaseReady (example-1.test-d.ts) |first|
941
+
onTestCaseResult (example-1.test-d.ts) |first|
942
+
onTestSuiteReady (example-1.test-d.ts) |suite|
943
+
onTestCaseReady (example-1.test-d.ts) |second|
944
+
onTestCaseResult (example-1.test-d.ts) |second|
945
+
onTestSuiteResult (example-1.test-d.ts) |suite|
946
+
onTestModuleEnd (example-1.test-d.ts)
947
+
948
+
onTestModuleStart (example-2.test-d.ts)
949
+
onTestCaseReady (example-2.test-d.ts) |first|
950
+
onTestCaseResult (example-2.test-d.ts) |first|
951
+
onTestSuiteReady (example-2.test-d.ts) |suite|
952
+
onTestCaseReady (example-2.test-d.ts) |second|
953
+
onTestCaseResult (example-2.test-d.ts) |second|
954
+
onTestCaseReady (example-2.test-d.ts) |third|
955
+
onTestCaseResult (example-2.test-d.ts) |third|
956
+
onTestSuiteResult (example-2.test-d.ts) |suite|
957
+
onTestCaseReady (example-2.test-d.ts) |fifth|
958
+
onTestCaseResult (example-2.test-d.ts) |fifth|
959
+
onTestCaseReady (example-2.test-d.ts) |fourth|
960
+
onTestCaseResult (example-2.test-d.ts) |fourth|
961
+
onTestModuleEnd (example-2.test-d.ts)
962
+
963
+
onTestRunEnd (failed, 2 modules, 0 errors)"
964
+
`)
965
+
})
966
+
})
967
+
968
+
interface ReporterOptions {
969
+
printTestRunEvents?: boolean
970
+
roots?: string[]
971
+
failed?: boolean
972
+
}
973
+
974
+
async function run(
975
+
structure: Parameters<typeof runInlineTests>[0],
976
+
customConfig?: UserConfig,
977
+
reporterOptions?: ReporterOptions,
978
+
) {
979
+
const reporter = new CustomReporter(reporterOptions)
980
+
981
+
const config: UserConfig = {
982
+
config: false,
983
+
fileParallelism: false,
984
+
globals: true,
985
+
reporters: [reporter],
986
+
sequence: {
987
+
sequencer: class Sorter {
988
+
sort(files: TestSpecification[]) {
989
+
return files.sort((a, b) => a.moduleId.localeCompare(b.moduleId))
990
+
}
991
+
992
+
shard(files: TestSpecification[]) {
993
+
return files
994
+
}
995
+
},
996
+
},
997
+
...customConfig,
998
+
}
999
+
1000
+
const { stdout, stderr } = await runInlineTests(structure, config)
1001
+
1002
+
if (reporterOptions?.printTestRunEvents && reporterOptions?.failed) {
1003
+
if (config.passWithNoTests) {
1004
+
expect(stdout).toContain('No test files found, exiting with code 0')
1005
+
}
1006
+
else {
1007
+
expect(stderr).toContain('No test files found, exiting with code 1')
1008
+
}
1009
+
}
1010
+
else if (!reporterOptions?.printTestRunEvents) {
1011
+
expect(stdout).toBe('')
1012
+
expect(stderr).toBe('')
1013
+
}
1014
+
1015
+
return `\n${reporter.calls.join('\n').trim()}`
1016
+
}
1017
+
1018
+
class CustomReporter implements Reporter {
1019
+
calls: string[] = []
1020
+
1021
+
constructor(private options: ReporterOptions = {}) {}
1022
+
1023
+
onTestRunStart(specifications: ReadonlyArray<TestSpecification>) {
1024
+
if (this.options.printTestRunEvents) {
1025
+
this.calls.push(`onTestRunStart (${specifications.length} specifications)`)
1026
+
}
1027
+
}
1028
+
1029
+
onTestRunEnd(modules: ReadonlyArray<TestModule>, errors: ReadonlyArray<SerializedError>, state: TestRunEndReason) {
1030
+
if (this.options.printTestRunEvents) {
1031
+
this.calls.push(`onTestRunEnd (${state}, ${modules.length} modules, ${errors.length} errors)`)
1032
+
}
1033
+
}
1034
+
1035
+
onTestModuleQueued(module: TestModule) {
1036
+
this.calls.push(`onTestModuleQueued (${this.normalizeFilename(module)})`)
1037
+
}
1038
+
1039
+
onTestModuleCollected(module: TestModule) {
1040
+
this.calls.push(`onTestModuleCollected (${this.normalizeFilename(module)})`)
1041
+
}
1042
+
1043
+
onTestSuiteReady(testSuite: TestSuite) {
1044
+
this.calls.push(`${padded(testSuite, 'onTestSuiteReady')} (${this.normalizeFilename(testSuite.module)}) |${testSuite.name}|`)
1045
+
}
1046
+
1047
+
onTestSuiteResult(testSuite: TestSuite) {
1048
+
this.calls.push(`${padded(testSuite, 'onTestSuiteResult')} (${this.normalizeFilename(testSuite.module)}) |${testSuite.name}|`)
1049
+
}
1050
+
1051
+
onTestModuleStart(module: TestModule) {
1052
+
this.calls.push(`onTestModuleStart (${this.normalizeFilename(module)})`)
1053
+
}
1054
+
1055
+
onTestModuleEnd(module: TestModule) {
1056
+
this.calls.push(`onTestModuleEnd (${this.normalizeFilename(module)})\n`)
1057
+
}
1058
+
1059
+
onTestCaseReady(test: TestCase) {
1060
+
this.calls.push(`${padded(test, 'onTestCaseReady')} (${this.normalizeFilename(test.module)}) |${test.name}|`)
1061
+
}
1062
+
1063
+
onTestCaseResult(test: TestCase) {
1064
+
this.calls.push(`${padded(test, 'onTestCaseResult')} (${this.normalizeFilename(test.module)}) |${test.name}|`)
1065
+
}
1066
+
1067
+
onHookStart(hook: ReportedHookContext) {
1068
+
const module = hook.entity.type === 'module' ? hook.entity : hook.entity.module
1069
+
const name = hook.entity.type !== 'module' ? ` |${hook.entity.name}|` : ''
1070
+
this.calls.push(` ${padded(hook.entity, 'onHookStart', 19)} (${this.normalizeFilename(module)})${name} [${hook.name}]`)
1071
+
}
1072
+
1073
+
onHookEnd(hook: ReportedHookContext) {
1074
+
const module = hook.entity.type === 'module' ? hook.entity : hook.entity.module
1075
+
const name = hook.entity.type !== 'module' ? ` |${hook.entity.name}|` : ''
1076
+
this.calls.push(` ${padded(hook.entity, 'onHookEnd', 19)} (${this.normalizeFilename(module)})${name} [${hook.name}]`)
1077
+
}
1078
+
1079
+
normalizeFilename(module: TestModule) {
1080
+
return normalizeFilename(module, this.options.roots)
1081
+
}
1082
+
}
1083
+
1084
+
function normalizeFilename(module: TestModule, roots?: string[]) {
1085
+
const relative = (roots || [module.project.config.root]).reduce((acc, root) => {
1086
+
return acc.replace(root, '')
1087
+
}, module.moduleId)
1088
+
return relative.replaceAll(sep, '/')
1089
+
.substring(1)
1090
+
}
1091
+
1092
+
function padded(entity: TestSuite | TestCase | TestModule, name: string, pad = 21) {
1093
+
return (' '.repeat(getDepth(entity)) + name).padEnd(pad)
1094
+
}
1095
+
1096
+
function getDepth(entity: TestSuite | TestCase | TestModule) {
1097
+
if (entity.type === 'module') {
1098
+
return 0
1099
+
}
1100
+
1101
+
let depth = 0
1102
+
let parent = entity.parent
1103
+
1104
+
while (parent) {
1105
+
depth += 2
1106
+
if (parent.type !== 'module') {
1107
+
parent = parent.parent
1108
+
}
1109
+
else {
1110
+
break
1111
+
}
1112
+
}
1113
+
1114
+
return depth
1115
+
}
1116
+
1117
+
function delay() {
1118
+
return `await new Promise(resolve => setTimeout(resolve, 100));`
1119
+
}
+6
-10
packages/browser/src/node/rpc.ts
+6
-10
packages/browser/src/node/rpc.ts
···
1
1
import type { Duplex } from 'node:stream'
2
2
import type { ErrorWithDiff } from 'vitest'
3
-
import type { BrowserCommandContext, ResolveSnapshotPathHandlerContext, TestModule, TestProject } from 'vitest/node'
3
+
import type { BrowserCommandContext, ResolveSnapshotPathHandlerContext, TestProject } from 'vitest/node'
4
4
import type { WebSocket } from 'ws'
5
5
import type { ParentBrowserProject } from './projectParent'
6
6
import type { BrowserServerState } from './state'
···
111
111
vitest.state.catchError(error, type)
112
112
},
113
113
async onQueued(file) {
114
-
vitest.state.collectFiles(project, [file])
115
-
const testModule = vitest.state.getReportedEntity(file) as TestModule
116
-
await vitest.report('onTestModuleQueued', testModule)
114
+
await vitest._testRun.enqueued(project, file)
117
115
},
118
116
async onCollected(files) {
119
-
vitest.state.collectFiles(project, files)
120
-
await vitest.report('onCollected', files)
117
+
await vitest._testRun.collected(project, files)
121
118
},
122
-
async onTaskUpdate(packs) {
123
-
vitest.state.updateTasks(packs)
124
-
await vitest.report('onTaskUpdate', packs)
119
+
async onTaskUpdate(packs, events) {
120
+
await vitest._testRun.updated(packs, events)
125
121
},
126
122
onAfterSuiteRun(meta) {
127
123
vitest.coverageProvider?.onAfterSuiteRun(meta)
128
124
},
129
125
sendLog(log) {
130
-
return vitest.report('onUserConsoleLog', log)
126
+
return vitest._testRun.log(log)
131
127
},
132
128
resolveSnapshotPath(testPath) {
133
129
return vitest.snapshot.resolvePath<ResolveSnapshotPathHandlerContext>(testPath, {
+4
-3
packages/browser/src/node/types.ts
+4
-3
packages/browser/src/node/types.ts
···
1
1
import type { ServerIdResolution, ServerMockResolution } from '@vitest/mocker/node'
2
+
import type { TaskEventPack, TaskResultPack } from '@vitest/runner'
2
3
import type { BirpcReturn } from 'birpc'
3
-
import type { AfterSuiteRunMeta, CancelReason, Reporter, RunnerTestFile, SnapshotResult, TaskResultPack, UserConsoleLog } from 'vitest'
4
+
import type { AfterSuiteRunMeta, CancelReason, Reporter, RunnerTestFile, SnapshotResult, UserConsoleLog } from 'vitest'
4
5
5
6
export interface WebSocketBrowserHandlers {
6
7
resolveSnapshotPath: (testPath: string) => string
7
8
resolveSnapshotRawPath: (testPath: string, rawPath: string) => string
8
9
onUnhandledError: (error: unknown, type: string) => Promise<void>
9
10
onQueued: (file: RunnerTestFile) => void
10
-
onCollected: (files?: RunnerTestFile[]) => Promise<void>
11
-
onTaskUpdate: (packs: TaskResultPack[]) => void
11
+
onCollected: (files: RunnerTestFile[]) => Promise<void>
12
+
onTaskUpdate: (packs: TaskResultPack[], events: TaskEventPack[]) => void
12
13
onAfterSuiteRun: (meta: AfterSuiteRunMeta) => void
13
14
onCancel: (reason: CancelReason) => void
14
15
getCountOfFailedTests: () => number
+2
-1
packages/runner/src/types/runner.ts
+2
-1
packages/runner/src/types/runner.ts
···
5
5
SequenceSetupFiles,
6
6
Suite,
7
7
Task,
8
+
TaskEventPack,
8
9
TaskResultPack,
9
10
Test,
10
11
TestContext,
···
128
129
/**
129
130
* Called, when a task is updated. The same as "onTaskUpdate" in a reporter, but this is running in the same thread as tests.
130
131
*/
131
-
onTaskUpdate?: (task: TaskResultPack[]) => Promise<void>
132
+
onTaskUpdate?: (task: TaskResultPack[], events: TaskEventPack[]) => Promise<void>
132
133
133
134
/**
134
135
* Called before running all tests in collected paths.
+29
-4
packages/runner/src/types/tasks.ts
+29
-4
packages/runner/src/types/tasks.ts
···
79
79
*/
80
80
file: File
81
81
/**
82
-
* Whether the task was skipped by calling `t.skip()`.
83
-
*/
84
-
pending?: boolean
85
-
/**
86
82
* Whether the task should succeed if it fails. If the task fails, it will be marked as passed.
87
83
*/
88
84
fails?: boolean
···
152
148
repeatCount?: number
153
149
/** @private */
154
150
note?: string
151
+
/**
152
+
* Whether the task was skipped by calling `t.skip()`.
153
+
* @internal
154
+
*/
155
+
pending?: boolean
155
156
}
156
157
157
158
/**
···
172
173
*/
173
174
meta: TaskMeta,
174
175
]
176
+
177
+
export type TaskEventPack = [
178
+
/**
179
+
* Unique task identifier from `task.id`.
180
+
*/
181
+
id: string,
182
+
/**
183
+
* The name of the event that triggered the update.
184
+
*/
185
+
event: TaskUpdateEvent,
186
+
]
187
+
188
+
export type TaskUpdateEvent =
189
+
| 'test-failed-early'
190
+
| 'suite-failed-early'
191
+
| 'test-prepare'
192
+
| 'test-finished'
193
+
| 'test-retried'
194
+
| 'suite-prepare'
195
+
| 'suite-finished'
196
+
| 'before-hook-start'
197
+
| 'before-hook-end'
198
+
| 'after-hook-start'
199
+
| 'after-hook-end'
175
200
176
201
export interface Suite extends TaskBase {
177
202
type: 'suite'
+2
-3
packages/vitest/src/api/setup.ts
+2
-3
packages/vitest/src/api/setup.ts
···
48
48
function setupClient(ws: WebSocket) {
49
49
const rpc = createBirpc<WebSocketEvents, WebSocketHandlers>(
50
50
{
51
-
async onTaskUpdate(packs) {
52
-
ctx.state.updateTasks(packs)
53
-
await ctx.report('onTaskUpdate', packs)
51
+
async onTaskUpdate(packs, events) {
52
+
await ctx._testRun.updated(packs, events)
54
53
},
55
54
getFiles() {
56
55
return ctx.state.getFiles()
+2
-2
packages/vitest/src/api/types.ts
+2
-2
packages/vitest/src/api/types.ts
···
1
-
import type { File, TaskResultPack } from '@vitest/runner'
1
+
import type { File, TaskEventPack, TaskResultPack } from '@vitest/runner'
2
2
import type { BirpcReturn } from 'birpc'
3
3
import type { SerializedConfig } from '../runtime/config'
4
4
import type { SerializedTestSpecification } from '../runtime/types/utils'
···
27
27
}
28
28
29
29
export interface WebSocketHandlers {
30
-
onTaskUpdate: (packs: TaskResultPack[]) => void
30
+
onTaskUpdate: (packs: TaskResultPack[], events: TaskEventPack[]) => void
31
31
getFiles: () => File[]
32
32
getTestFiles: () => Promise<SerializedTestSpecification[]>
33
33
getPaths: () => string[]
+38
-32
packages/vitest/src/node/core.ts
+38
-32
packages/vitest/src/node/core.ts
···
1
-
import type { CancelReason, File, TaskResultPack } from '@vitest/runner'
1
+
import type { CancelReason, File } from '@vitest/runner'
2
2
import type { Awaitable } from '@vitest/utils'
3
3
import type { Writable } from 'node:stream'
4
4
import type { ViteDevServer } from 'vite'
···
24
24
import { getCoverageProvider } from '../integrations/coverage'
25
25
import { distDir } from '../paths'
26
26
import { wildcardPatternToRegExp } from '../utils/base'
27
+
import { convertTasksToEvents } from '../utils/tasks'
27
28
import { BrowserSessions } from './browser/sessions'
28
29
import { VitestCache } from './cache'
29
30
import { resolveConfig } from './config/resolveConfig'
···
36
37
import { createBenchmarkReporters, createReporters } from './reporters/utils'
37
38
import { VitestSpecifications } from './specifications'
38
39
import { StateManager } from './state'
40
+
import { TestRun } from './test-run'
39
41
import { VitestWatcher } from './watcher'
40
42
import { resolveBrowserWorkspace, resolveWorkspace } from './workspace/resolveWorkspace'
41
43
···
94
96
/** @internal */ reporters: Reporter[] = undefined!
95
97
/** @internal */ vitenode: ViteNodeServer = undefined!
96
98
/** @internal */ runner: ViteNodeRunner = undefined!
99
+
/** @internal */ _testRun: TestRun = undefined!
97
100
98
101
private isFirstRun = true
99
102
private restartsCount = 0
···
214
217
this._state = new StateManager()
215
218
this._cache = new VitestCache(this.version)
216
219
this._snapshot = new SnapshotManager({ ...resolved.snapshotOptions })
220
+
this._testRun = new TestRun(this)
217
221
218
222
if (this.config.watch) {
219
223
this.watcher.registerWatcher()
···
448
452
await this.report('onInit', this)
449
453
await this.report('onPathsCollected', files.flatMap(f => f.filepath))
450
454
451
-
const workspaceSpecs = new Map<TestProject, File[]>()
455
+
const specifications: TestSpecification[] = []
452
456
for (const file of files) {
453
457
const project = this.getProjectByName(file.projectName || '')
454
-
const specs = workspaceSpecs.get(project) || []
455
-
specs.push(file)
456
-
workspaceSpecs.set(project, specs)
458
+
const specification = project.createSpecification(file.filepath, undefined, file.pool)
459
+
specifications.push(specification)
457
460
}
458
461
459
-
for (const [project, files] of workspaceSpecs) {
460
-
const filepaths = files.map(f => f.filepath)
461
-
this.state.clearFiles(project, filepaths)
462
-
files.forEach((file) => {
463
-
file.logs?.forEach(log => this.state.updateUserLog(log))
464
-
})
465
-
this.state.collectFiles(project, files)
466
-
}
467
-
468
-
await this.report('onCollected', files).catch(noop)
462
+
await this.report('onSpecsCollected', specifications.map(spec => spec.toJSON()))
463
+
await this._testRun.start(specifications).catch(noop)
469
464
470
465
for (const file of files) {
471
-
const logs: UserConsoleLog[] = []
472
-
const taskPacks: TaskResultPack[] = []
466
+
const project = this.getProjectByName(file.projectName || '')
467
+
await this._testRun.enqueued(project, file).catch(noop)
468
+
await this._testRun.collected(project, [file]).catch(noop)
473
469
474
-
const tasks = getTasks(file)
475
-
for (const task of tasks) {
470
+
const logs: UserConsoleLog[] = []
471
+
472
+
const { packs, events } = convertTasksToEvents(file, (task) => {
476
473
if (task.logs) {
477
474
logs.push(...task.logs)
478
475
}
479
-
taskPacks.push([task.id, task.result, task.meta])
480
-
}
476
+
})
477
+
481
478
logs.sort((log1, log2) => log1.time - log2.time)
482
479
483
480
for (const log of logs) {
484
-
await this.report('onUserConsoleLog', log).catch(noop)
481
+
await this._testRun.log(log).catch(noop)
485
482
}
486
483
487
-
await this.report('onTaskUpdate', taskPacks).catch(noop)
484
+
await this._testRun.updated(packs, events).catch(noop)
488
485
}
489
486
490
487
if (hasFailed(files)) {
···
492
489
}
493
490
494
491
this._checkUnhandledErrors(errors)
495
-
await this.report('onFinished', files, errors)
492
+
await this._testRun.end(specifications, errors).catch(noop)
496
493
await this.initCoverageProvider()
497
494
await this.coverageProvider?.mergeReports?.(coverages)
498
495
···
552
549
553
550
// if run with --changed, don't exit if no tests are found
554
551
if (!files.length) {
555
-
// Report coverage for uncovered files
552
+
const throwAnError = !this.config.watch || !(this.config.changed || this.config.related?.length)
553
+
554
+
await this._testRun.start([])
556
555
const coverage = await this.coverageProvider?.generateCoverage?.({ allTestsRun: true })
556
+
557
+
// set exit code before calling `onTestRunEnd` so the lifecycle is consistent
558
+
if (throwAnError) {
559
+
const exitCode = this.config.passWithNoTests ? 0 : 1
560
+
process.exitCode = exitCode
561
+
}
562
+
563
+
await this._testRun.end([], [], coverage)
564
+
// Report coverage for uncovered files
557
565
await this.reportCoverage(coverage, true)
558
566
559
567
this.logger.printNoTestFound(filters)
560
568
561
-
if (!this.config.watch || !(this.config.changed || this.config.related?.length)) {
562
-
const exitCode = this.config.passWithNoTests ? 0 : 1
563
-
process.exitCode = exitCode
569
+
if (throwAnError) {
564
570
throw new FilesNotFoundError(this.mode)
565
571
}
566
572
}
···
670
676
671
677
await this.report('onPathsCollected', filepaths)
672
678
await this.report('onSpecsCollected', specs.map(spec => spec.toJSON()))
679
+
await this._testRun.start(specs)
673
680
674
681
// previous run
675
682
await this.runningPromise
···
716
723
}
717
724
}
718
725
finally {
719
-
// can be duplicate files if different projects are using the same file
720
-
const files = Array.from(new Set(specs.map(spec => spec.moduleId)))
721
-
const errors = this.state.getUnhandledErrors()
726
+
// TODO: wait for coverage only if `onFinished` is defined
722
727
const coverage = await this.coverageProvider?.generateCoverage({ allTestsRun })
723
728
729
+
const errors = this.state.getUnhandledErrors()
724
730
this._checkUnhandledErrors(errors)
725
-
await this.report('onFinished', this.state.getFiles(files), errors, coverage)
731
+
await this._testRun.end(specs, errors, coverage)
726
732
await this.reportCoverage(coverage, allTestsRun)
727
733
}
728
734
})()
+29
packages/vitest/src/node/spec.ts
+29
packages/vitest/src/node/spec.ts
···
1
1
import type { SerializedTestSpecification } from '../runtime/types/utils'
2
2
import type { TestProject } from './project'
3
+
import type { TestModule } from './reporters/reported-tasks'
3
4
import type { Pool } from './types/pool-options'
5
+
import { generateFileHash } from '@vitest/runner/utils'
6
+
import { relative } from 'pathe'
4
7
5
8
export class TestSpecification {
6
9
/**
···
16
19
*/
17
20
public readonly 2: { pool: Pool }
18
21
22
+
/**
23
+
* The task ID associated with the test module.
24
+
*/
25
+
public readonly taskId: string
19
26
/**
20
27
* The test project that the module belongs to.
21
28
*/
···
43
50
this[0] = project
44
51
this[1] = moduleId
45
52
this[2] = { pool }
53
+
const name = project.config.name
54
+
const hashName = pool !== 'typescript'
55
+
? name
56
+
: name
57
+
// https://github.com/vitest-dev/vitest/blob/main/packages/vitest/src/typecheck/collect.ts#L58
58
+
? `${name}:__typecheck__`
59
+
: '__typecheck__'
60
+
this.taskId = generateFileHash(
61
+
relative(project.config.root, moduleId),
62
+
hashName,
63
+
)
46
64
this.project = project
47
65
this.moduleId = moduleId
48
66
this.pool = pool
49
67
this.testLines = testLines
68
+
}
69
+
70
+
/**
71
+
* Test module associated with the specification.
72
+
*/
73
+
get testModule(): TestModule | undefined {
74
+
const task = this.project.vitest.state.idMap.get(this.taskId)
75
+
if (!task) {
76
+
return undefined
77
+
}
78
+
return this.project.vitest.state.getReportedEntity(task) as TestModule | undefined
50
79
}
51
80
52
81
toJSON(): SerializedTestSpecification {
+170
packages/vitest/src/node/test-run.ts
+170
packages/vitest/src/node/test-run.ts
···
1
+
import type { File as RunnerTestFile, TaskEventPack, TaskResultPack, TaskUpdateEvent } from '@vitest/runner'
2
+
import type { SerializedError } from '../public/utils'
3
+
import type { UserConsoleLog } from '../types/general'
4
+
import type { Vitest } from './core'
5
+
import type { TestProject } from './project'
6
+
import type { ReportedHookContext, TestCollection, TestModule } from './reporters/reported-tasks'
7
+
import type { TestSpecification } from './spec'
8
+
import assert from 'node:assert'
9
+
import { serializeError } from '@vitest/utils/error'
10
+
11
+
export class TestRun {
12
+
constructor(private vitest: Vitest) {}
13
+
14
+
async start(specifications: TestSpecification[]) {
15
+
await this.vitest.report('onTestRunStart', [...specifications])
16
+
}
17
+
18
+
async enqueued(project: TestProject, file: RunnerTestFile) {
19
+
this.vitest.state.collectFiles(project, [file])
20
+
const testModule = this.vitest.state.getReportedEntity(file) as TestModule
21
+
await this.vitest.report('onTestModuleQueued', testModule)
22
+
}
23
+
24
+
async collected(project: TestProject, files: RunnerTestFile[]) {
25
+
this.vitest.state.collectFiles(project, files)
26
+
await Promise.all([
27
+
this.vitest.report('onCollected', files),
28
+
...files.map((file) => {
29
+
const testModule = this.vitest.state.getReportedEntity(file) as TestModule
30
+
return this.vitest.report('onTestModuleCollected', testModule)
31
+
}),
32
+
])
33
+
}
34
+
35
+
async log(log: UserConsoleLog) {
36
+
this.vitest.state.updateUserLog(log)
37
+
await this.vitest.report('onUserConsoleLog', log)
38
+
}
39
+
40
+
async updated(update: TaskResultPack[], events: TaskEventPack[]) {
41
+
this.vitest.state.updateTasks(update)
42
+
43
+
// TODO: what is the order or reports here?
44
+
// "onTaskUpdate" in parallel with others or before all or after all?
45
+
// TODO: error handling - what happens if custom reporter throws an error?
46
+
await this.vitest.report('onTaskUpdate', update)
47
+
48
+
for (const [id, event] of events) {
49
+
await this.reportEvent(id, event).catch((error) => {
50
+
this.vitest.state.catchError(serializeError(error), 'Unhandled Reporter Error')
51
+
})
52
+
}
53
+
}
54
+
55
+
async end(specifications: TestSpecification[], errors: unknown[], coverage?: unknown) {
56
+
// specification won't have the File task if they were filtered by the --shard command
57
+
const modules = specifications.map(spec => spec.testModule).filter(s => s != null)
58
+
const files = modules.map(m => m.task)
59
+
60
+
const state = this.vitest.isCancelling
61
+
? 'interrupted'
62
+
// by this point, the run will be marked as failed if there are any errors,
63
+
// should it be done by testRun.end?
64
+
: process.exitCode
65
+
? 'failed'
66
+
: 'passed'
67
+
68
+
try {
69
+
await Promise.all([
70
+
this.vitest.report('onTestRunEnd', modules, [...errors] as SerializedError[], state),
71
+
// TODO: in a perfect world, the coverage should be done in parallel to `onFinished`
72
+
this.vitest.report('onFinished', files, errors, coverage),
73
+
])
74
+
}
75
+
finally {
76
+
if (coverage) {
77
+
await this.vitest.report('onCoverage', coverage)
78
+
}
79
+
}
80
+
}
81
+
82
+
private async reportEvent(id: string, event: TaskUpdateEvent) {
83
+
const task = this.vitest.state.idMap.get(id)
84
+
const entity = task && this.vitest.state.getReportedEntity(task)
85
+
86
+
assert(task && entity, `Entity must be found for task ${task?.name || id}`)
87
+
88
+
if (event === 'suite-prepare' && entity.type === 'suite') {
89
+
return await this.vitest.report('onTestSuiteReady', entity)
90
+
}
91
+
92
+
if (event === 'suite-prepare' && entity.type === 'module') {
93
+
return await this.vitest.report('onTestModuleStart', entity)
94
+
}
95
+
96
+
if (event === 'suite-finished') {
97
+
assert(entity.type === 'suite' || entity.type === 'module', 'Entity type must be suite or module')
98
+
99
+
if (entity.state() === 'skipped') {
100
+
// everything inside suite or a module is skipped,
101
+
// so we won't get any children events
102
+
// we need to report everything manually
103
+
await this.reportChildren(entity.children)
104
+
}
105
+
else {
106
+
// skipped tests need to be reported manually once test module/suite has finished
107
+
for (const test of entity.children.tests('skipped')) {
108
+
if (test.task.result?.pending) {
109
+
// pending error tasks are reported normally
110
+
continue
111
+
}
112
+
await this.vitest.report('onTestCaseReady', test)
113
+
await this.vitest.report('onTestCaseResult', test)
114
+
}
115
+
}
116
+
117
+
if (entity.type === 'module') {
118
+
await this.vitest.report('onTestModuleEnd', entity)
119
+
}
120
+
else {
121
+
await this.vitest.report('onTestSuiteResult', entity)
122
+
}
123
+
124
+
return
125
+
}
126
+
127
+
if (event === 'test-prepare' && entity.type === 'test') {
128
+
return await this.vitest.report('onTestCaseReady', entity)
129
+
}
130
+
131
+
if (event === 'test-finished' && entity.type === 'test') {
132
+
return await this.vitest.report('onTestCaseResult', entity)
133
+
}
134
+
135
+
if (event.startsWith('before-hook') || event.startsWith('after-hook')) {
136
+
const isBefore = event.startsWith('before-hook')
137
+
138
+
const hook: ReportedHookContext = entity.type === 'test'
139
+
? {
140
+
name: isBefore ? 'beforeEach' : 'afterEach',
141
+
entity,
142
+
}
143
+
: {
144
+
name: isBefore ? 'beforeAll' : 'afterAll',
145
+
entity,
146
+
}
147
+
148
+
if (event.endsWith('-start')) {
149
+
await this.vitest.report('onHookStart', hook)
150
+
}
151
+
else {
152
+
await this.vitest.report('onHookEnd', hook)
153
+
}
154
+
}
155
+
}
156
+
157
+
private async reportChildren(children: TestCollection) {
158
+
for (const child of children) {
159
+
if (child.type === 'test') {
160
+
await this.vitest.report('onTestCaseReady', child)
161
+
await this.vitest.report('onTestCaseResult', child)
162
+
}
163
+
else {
164
+
await this.vitest.report('onTestSuiteReady', child)
165
+
await this.reportChildren(child.children)
166
+
await this.vitest.report('onTestSuiteResult', child)
167
+
}
168
+
}
169
+
}
170
+
}
+7
-1
packages/vitest/src/public/node.ts
+7
-1
packages/vitest/src/public/node.ts
···
34
34
35
35
export type {
36
36
ModuleDiagnostic,
37
-
38
37
TaskOptions,
38
+
39
39
TestCase,
40
40
TestCollection,
41
41
TestDiagnostic,
42
42
TestModule,
43
+
TestModuleState,
43
44
TestResult,
44
45
TestResultFailed,
45
46
TestResultPassed,
46
47
TestResultSkipped,
48
+
TestState,
47
49
TestSuite,
50
+
TestSuiteState,
48
51
} from '../node/reporters/reported-tasks'
49
52
export { BaseSequencer } from '../node/sequencers/BaseSequencer'
50
53
···
152
155
RunnerTestSuite,
153
156
} from './index'
154
157
export type {
158
+
ReportedHookContext,
155
159
Reporter,
160
+
TestRunEndReason,
156
161
} from './reporters'
157
162
export { generateFileHash } from '@vitest/runner/utils'
163
+
export type { SerializedError } from '@vitest/utils'
158
164
159
165
export {
160
166
esbuildVersion,
+2
packages/vitest/src/public/reporters.ts
+2
packages/vitest/src/public/reporters.ts
-1
packages/vitest/src/runtime/rpc.ts
-1
packages/vitest/src/runtime/rpc.ts
+21
-22
packages/vitest/src/typecheck/collect.ts
+21
-22
packages/vitest/src/typecheck/collect.ts
···
1
1
import type { File, RunMode, Suite, Test } from '@vitest/runner'
2
-
import type { Node } from 'estree'
3
2
import type { RawSourceMap } from 'vite-node'
4
3
import type { TestProject } from '../node/project'
5
4
import {
···
71
70
}
72
71
file.file = file
73
72
const definitions: LocalCallDefinition[] = []
74
-
const getName = (callee: Node): string | null => {
73
+
const getName = (callee: any): string | null => {
75
74
if (!callee) {
76
75
return null
77
76
}
···
85
84
return getName(callee.tag)
86
85
}
87
86
if (callee.type === 'MemberExpression') {
88
-
const object = callee.object as any
87
+
if (
88
+
callee.object?.type === 'Identifier'
89
+
&& ['it', 'test', 'describe', 'suite'].includes(callee.object.name)
90
+
) {
91
+
return callee.object?.name
92
+
}
89
93
// direct call as `__vite_ssr_exports_0__.test()`
90
-
if (object?.name?.startsWith('__vite_ssr_')) {
94
+
if (callee.object?.name?.startsWith('__vite_ssr_')) {
91
95
return getName(callee.property)
92
96
}
93
97
// call as `__vite_ssr__.test.skip()`
94
-
return getName(object?.property)
95
-
}
96
-
// unwrap (0, ...)
97
-
if (callee.type === 'SequenceExpression' && callee.expressions.length === 2) {
98
-
const [e0, e1] = callee.expressions
99
-
if (e0.type === 'Literal' && e0.value === 0) {
100
-
return getName(e1)
101
-
}
98
+
return getName(callee.object?.property)
102
99
}
103
100
return null
104
101
}
···
114
111
return
115
112
}
116
113
const property = callee?.property?.name
117
-
const mode = !property || property === name ? 'run' : property
118
-
// the test node for skipIf and runIf will be the next CallExpression
119
-
if (mode === 'each' || mode === 'skipIf' || mode === 'runIf' || mode === 'for') {
114
+
let mode = !property || property === name ? 'run' : property
115
+
// they will be picked up in the next iteration
116
+
if (['each', 'for', 'skipIf', 'runIf'].includes(mode)) {
120
117
return
121
118
}
122
119
123
120
let start: number
124
121
const end = node.end
125
-
122
+
// .each
126
123
if (callee.type === 'CallExpression') {
127
124
start = callee.end
128
125
}
···
137
134
arguments: [messageNode],
138
135
} = node
139
136
140
-
if (!messageNode) {
141
-
// called as "test()"
142
-
return
137
+
const isQuoted = messageNode?.type === 'Literal' || messageNode?.type === 'TemplateLiteral'
138
+
const message = isQuoted
139
+
? request.code.slice(messageNode.start + 1, messageNode.end - 1)
140
+
: request.code.slice(messageNode.start, messageNode.end)
141
+
142
+
// cannot statically analyze, so we always skip it
143
+
if (mode === 'skipIf' || mode === 'runIf') {
144
+
mode = 'skip'
143
145
}
144
-
145
-
const message = getNodeAsString(messageNode, request.code)
146
-
147
146
definitions.push({
148
147
start,
149
148
end,
+13
-7
packages/vitest/src/typecheck/typechecker.ts
+13
-7
packages/vitest/src/typecheck/typechecker.ts
···
1
1
import type { RawSourceMap } from '@ampproject/remapping'
2
-
import type { File, Task, TaskResultPack, TaskState } from '@vitest/runner'
2
+
import type { File, Task, TaskEventPack, TaskResultPack, TaskState } from '@vitest/runner'
3
3
import type { ParsedStack } from '@vitest/utils'
4
4
import type { EachMapping } from '@vitest/utils/source-map'
5
5
import type { ChildProcess } from 'node:child_process'
···
10
10
import type { TscErrorInfo } from './types'
11
11
import { rm } from 'node:fs/promises'
12
12
import { performance } from 'node:perf_hooks'
13
-
import { getTasks } from '@vitest/runner/utils'
14
13
import { eachMapping, generatedPositionFor, TraceMap } from '@vitest/utils/source-map'
15
14
import { basename, extname, resolve } from 'pathe'
16
15
import { x } from 'tinyexec'
16
+
import { convertTasksToEvents } from '../utils/tasks'
17
17
import { collectTests } from './collect'
18
18
import { getRawErrsMapFromTsCompile, getTsconfig } from './parse'
19
19
import { createIndexMap } from './utils'
···
358
358
return Object.values(this._tests || {}).map(i => i.file)
359
359
}
360
360
361
-
public getTestPacks() {
362
-
return Object.values(this._tests || {})
363
-
.map(({ file }) => getTasks(file))
364
-
.flat()
365
-
.map<TaskResultPack>(i => [i.id, i.result, { typecheck: true }])
361
+
public getTestPacksAndEvents() {
362
+
const packs: TaskResultPack[] = []
363
+
const events: TaskEventPack[] = []
364
+
365
+
for (const { file } of Object.values(this._tests || {})) {
366
+
const result = convertTasksToEvents(file)
367
+
packs.push(...result.packs)
368
+
events.push(...result.events)
369
+
}
370
+
371
+
return { packs, events }
366
372
}
367
373
}
368
374
+2
-3
packages/vitest/src/types/rpc.ts
+2
-3
packages/vitest/src/types/rpc.ts
···
1
-
import type { CancelReason, File, TaskResultPack } from '@vitest/runner'
1
+
import type { CancelReason, File, TaskEventPack, TaskResultPack } from '@vitest/runner'
2
2
import type { SnapshotResult } from '@vitest/snapshot'
3
3
import type { AfterSuiteRunMeta, TransformMode, UserConsoleLog } from './general'
4
4
···
35
35
force?: boolean
36
36
) => Promise<any>
37
37
38
-
onFinished: (files: File[], errors?: unknown[]) => void
39
38
onPathsCollected: (paths: string[]) => void
40
39
onUserConsoleLog: (log: UserConsoleLog) => void
41
40
onUnhandledError: (err: unknown, type: string) => void
42
41
onQueued: (file: File) => void
43
42
onCollected: (files: File[]) => Promise<void>
44
43
onAfterSuiteRun: (meta: AfterSuiteRunMeta) => void
45
-
onTaskUpdate: (pack: TaskResultPack[]) => Promise<void>
44
+
onTaskUpdate: (pack: TaskResultPack[], events: TaskEventPack[]) => Promise<void>
46
45
onCancel: (reason: CancelReason) => void
47
46
getCountOfFailedTests: () => number
48
47
+33
-1
packages/vitest/src/utils/tasks.ts
+33
-1
packages/vitest/src/utils/tasks.ts
···
1
-
import type { Suite, Task } from '@vitest/runner'
1
+
import type { File, Suite, Task, TaskEventPack, TaskResultPack } from '@vitest/runner'
2
2
import type { Arrayable } from '../types/general'
3
3
import { getTests } from '@vitest/runner/utils'
4
4
import { toArray } from '@vitest/utils'
···
17
17
&& e.message.match(/Snapshot .* mismatched/),
18
18
)
19
19
})
20
+
}
21
+
22
+
export function convertTasksToEvents(file: File, onTask?: (task: Task) => void): {
23
+
packs: TaskResultPack[]
24
+
events: TaskEventPack[]
25
+
} {
26
+
const packs: TaskResultPack[] = []
27
+
const events: TaskEventPack[] = []
28
+
29
+
function visit(suite: Suite | File) {
30
+
onTask?.(suite)
31
+
32
+
packs.push([suite.id, suite.result, suite.meta])
33
+
events.push([suite.id, 'suite-prepare'])
34
+
suite.tasks.forEach((task) => {
35
+
if (task.type === 'suite') {
36
+
visit(task)
37
+
}
38
+
else {
39
+
onTask?.(task)
40
+
packs.push([task.id, task.result, task.meta])
41
+
if (task.mode !== 'skip' && task.mode !== 'todo') {
42
+
events.push([task.id, 'test-prepare'], [task.id, 'test-finished'])
43
+
}
44
+
}
45
+
})
46
+
events.push([suite.id, 'suite-finished'])
47
+
}
48
+
49
+
visit(file)
50
+
51
+
return { packs, events }
20
52
}
+1
test/cli/fixtures/reported-tasks/1_first.test.ts
+1
test/cli/fixtures/reported-tasks/1_first.test.ts
-40
test/reporters/fixtures/task-parser-tests/example-1.test.ts
-40
test/reporters/fixtures/task-parser-tests/example-1.test.ts
···
1
-
import { beforeAll, beforeEach, afterEach, afterAll, test, describe } from "vitest";
2
-
import { setTimeout } from "node:timers/promises";
3
-
4
-
beforeAll(async () => {
5
-
await setTimeout(100);
6
-
});
7
-
8
-
afterAll(async () => {
9
-
await setTimeout(100);
10
-
});
11
-
12
-
describe("some suite", async () => {
13
-
beforeEach(async () => {
14
-
await setTimeout(100);
15
-
});
16
-
17
-
test("some test", async () => {
18
-
await setTimeout(100);
19
-
});
20
-
21
-
afterEach(async () => {
22
-
await setTimeout(100);
23
-
});
24
-
});
25
-
26
-
test("Fast test 1", () => {
27
-
//
28
-
});
29
-
30
-
test.skip("Skipped test 1", () => {
31
-
//
32
-
});
33
-
34
-
test.concurrent("parallel slow tests 1.1", async () => {
35
-
await setTimeout(100);
36
-
});
37
-
38
-
test.concurrent("parallel slow tests 1.2", async () => {
39
-
await setTimeout(100);
40
-
});
-40
test/reporters/fixtures/task-parser-tests/example-2.test.ts
-40
test/reporters/fixtures/task-parser-tests/example-2.test.ts
···
1
-
import { beforeAll, beforeEach, afterEach, afterAll, test, describe } from "vitest";
2
-
import { setTimeout } from "node:timers/promises";
3
-
4
-
beforeAll(async () => {
5
-
await setTimeout(100);
6
-
});
7
-
8
-
afterAll(async () => {
9
-
await setTimeout(100);
10
-
});
11
-
12
-
describe("some suite", async () => {
13
-
beforeEach(async () => {
14
-
await setTimeout(100);
15
-
});
16
-
17
-
test("some test", async () => {
18
-
await setTimeout(100);
19
-
});
20
-
21
-
afterEach(async () => {
22
-
await setTimeout(100);
23
-
});
24
-
});
25
-
26
-
test("Fast test 1", () => {
27
-
//
28
-
});
29
-
30
-
test.skip("Skipped test 1", () => {
31
-
//
32
-
});
33
-
34
-
test.concurrent("parallel slow tests 2.1", async () => {
35
-
await setTimeout(100);
36
-
});
37
-
38
-
test.concurrent("parallel slow tests 2.2", async () => {
39
-
await setTimeout(100);
40
-
});
-16
test/reporters/tests/__snapshots__/html.test.ts.snap
-16
test/reporters/tests/__snapshots__/html.test.ts.snap
···
17
17
"prepareDuration": 0,
18
18
"result": {
19
19
"duration": 0,
20
-
"hooks": {
21
-
"afterAll": "pass",
22
-
"beforeAll": "pass",
23
-
},
24
20
"startTime": 0,
25
21
"state": "fail",
26
22
},
···
67
63
"stackStr": "AssertionError: expected 2 to deeply equal 1",
68
64
},
69
65
],
70
-
"hooks": {
71
-
"afterEach": "pass",
72
-
"beforeEach": "pass",
73
-
},
74
66
"repeatCount": 0,
75
67
"retryCount": 0,
76
68
"startTime": 0,
···
134
126
"prepareDuration": 0,
135
127
"result": {
136
128
"duration": 0,
137
-
"hooks": {
138
-
"afterAll": "pass",
139
-
"beforeAll": "pass",
140
-
},
141
129
"startTime": 0,
142
130
"state": "pass",
143
131
},
···
155
143
"name": "2 + 3 = 5",
156
144
"result": {
157
145
"duration": 0,
158
-
"hooks": {
159
-
"afterEach": "pass",
160
-
"beforeEach": "pass",
161
-
},
162
146
"repeatCount": 0,
163
147
"retryCount": 0,
164
148
"startTime": 0,
+3
-3
packages/browser/src/client/tester/runner.ts
+3
-3
packages/browser/src/client/tester/runner.ts
···
1
-
import type { CancelReason, File, Suite, Task, TaskResultPack, VitestRunner } from '@vitest/runner'
1
+
import type { CancelReason, File, Suite, Task, TaskEventPack, TaskResultPack, VitestRunner } from '@vitest/runner'
2
2
import type { SerializedConfig, WorkerGlobalState } from 'vitest'
3
3
import type { VitestExecutor } from 'vitest/execute'
4
4
import type { VitestBrowserClientMocker } from './mocker'
···
131
131
return rpc().onCollected(files)
132
132
}
133
133
134
-
onTaskUpdate = (task: TaskResultPack[]): Promise<void> => {
135
-
return rpc().onTaskUpdate(task)
134
+
onTaskUpdate = (task: TaskResultPack[], events: TaskEventPack[]): Promise<void> => {
135
+
return rpc().onTaskUpdate(task, events)
136
136
}
137
137
138
138
importFile = async (filepath: string) => {
+2
-2
packages/vitest/src/node/cli/cli-api.ts
+2
-2
packages/vitest/src/node/cli/cli-api.ts
···
256
256
257
257
files.forEach((file) => {
258
258
for (const test of file.children.allTests()) {
259
-
if (test.skipped()) {
259
+
if (test.result().state === 'skipped') {
260
260
continue
261
261
}
262
262
const result: TestCollectJSONResult = {
···
280
280
281
281
testModules.forEach((testModule) => {
282
282
for (const test of testModule.children.allTests()) {
283
-
if (test.skipped()) {
283
+
if (test.result().state === 'skipped') {
284
284
continue
285
285
}
286
286
const fullName = `${test.module.task.name} > ${test.fullName}`
+9
-21
packages/vitest/src/node/pools/rpc.ts
+9
-21
packages/vitest/src/node/pools/rpc.ts
···
1
1
import type { RawSourceMap } from 'vite-node'
2
2
import type { RuntimeRPC } from '../../types/rpc'
3
3
import type { TestProject } from '../project'
4
-
import type { TestModule } from '../reporters/reported-tasks'
5
4
import type { ResolveSnapshotPathHandlerContext } from '../types/config'
6
5
import { mkdir, writeFile } from 'node:fs/promises'
7
6
import { join } from 'pathe'
···
15
14
}
16
15
17
16
export function createMethodsRPC(project: TestProject, options: MethodsOptions = {}): RuntimeRPC {
18
-
const ctx = project.ctx
17
+
const ctx = project.vitest
19
18
const cacheFs = options.cacheFs ?? false
20
19
return {
21
20
snapshotSaved(snapshot) {
···
79
78
ctx.state.collectPaths(paths)
80
79
return ctx.report('onPathsCollected', paths)
81
80
},
82
-
onQueued(file) {
83
-
ctx.state.collectFiles(project, [file])
84
-
const testModule = ctx.state.getReportedEntity(file) as TestModule
85
-
return ctx.report('onTestModuleQueued', testModule)
81
+
async onQueued(file) {
82
+
await ctx._testRun.enqueued(project, file)
86
83
},
87
-
onCollected(files) {
88
-
ctx.state.collectFiles(project, files)
89
-
return ctx.report('onCollected', files)
84
+
async onCollected(files) {
85
+
await ctx._testRun.collected(project, files)
90
86
},
91
87
onAfterSuiteRun(meta) {
92
88
ctx.coverageProvider?.onAfterSuiteRun(meta)
93
89
},
94
-
onTaskUpdate(packs) {
95
-
ctx.state.updateTasks(packs)
96
-
return ctx.report('onTaskUpdate', packs)
90
+
async onTaskUpdate(packs, events) {
91
+
await ctx._testRun.updated(packs, events)
97
92
},
98
-
onUserConsoleLog(log) {
99
-
ctx.state.updateUserLog(log)
100
-
ctx.report('onUserConsoleLog', log)
93
+
async onUserConsoleLog(log) {
94
+
await ctx._testRun.log(log)
101
95
},
102
96
onUnhandledError(err, type) {
103
97
ctx.state.catchError(err, type)
104
-
},
105
-
onFinished(files) {
106
-
const errors = ctx.state.getUnhandledErrors()
107
-
ctx._checkUnhandledErrors(errors)
108
-
109
-
return ctx.report('onFinished', files, errors)
110
98
},
111
99
onCancel(reason) {
112
100
ctx.cancelCurrentRun(reason)
+25
-10
packages/vitest/src/node/pools/typecheck.ts
+25
-10
packages/vitest/src/node/pools/typecheck.ts
···
19
19
) {
20
20
const checker = project.typechecker!
21
21
22
-
await ctx.report('onTaskUpdate', checker.getTestPacks())
22
+
const { packs, events } = checker.getTestPacksAndEvents()
23
+
await ctx._testRun.updated(packs, events)
23
24
24
25
if (!project.config.typecheck.ignoreSourceErrors) {
25
26
sourceErrors.forEach(error =>
···
62
63
checker.setFiles(files)
63
64
64
65
checker.onParseStart(async () => {
65
-
ctx.state.collectFiles(project, checker.getTestFiles())
66
-
await ctx.report('onCollected')
66
+
const files = checker.getTestFiles()
67
+
for (const file of files) {
68
+
await ctx._testRun.enqueued(project, file)
69
+
}
70
+
await ctx._testRun.collected(project, files)
67
71
})
68
72
69
73
checker.onParseEnd(result => onParseEnd(project, result))
···
81
85
}
82
86
83
87
await checker.collectTests()
84
-
ctx.state.collectFiles(project, checker.getTestFiles())
85
88
86
-
await ctx.report('onTaskUpdate', checker.getTestPacks())
87
-
await ctx.report('onCollected')
89
+
const testFiles = checker.getTestFiles()
90
+
for (const file of testFiles) {
91
+
await ctx._testRun.enqueued(project, file)
92
+
}
93
+
await ctx._testRun.collected(project, testFiles)
94
+
95
+
const { packs, events } = checker.getTestPacksAndEvents()
96
+
await ctx._testRun.updated(packs, events)
88
97
})
89
98
90
99
await checker.prepare()
···
108
117
const checker = await createWorkspaceTypechecker(project, files)
109
118
checker.setFiles(files)
110
119
await checker.collectTests()
111
-
ctx.state.collectFiles(project, checker.getTestFiles())
112
-
await ctx.report('onCollected')
120
+
const testFiles = checker.getTestFiles()
121
+
for (const file of testFiles) {
122
+
await ctx._testRun.enqueued(project, file)
123
+
}
124
+
await ctx._testRun.collected(project, testFiles)
113
125
}
114
126
}
115
127
···
136
148
})
137
149
const triggered = await _p
138
150
if (project.typechecker && !triggered) {
139
-
ctx.state.collectFiles(project, project.typechecker.getTestFiles())
140
-
await ctx.report('onCollected')
151
+
const testFiles = project.typechecker.getTestFiles()
152
+
for (const file of testFiles) {
153
+
await ctx._testRun.enqueued(project, file)
154
+
}
155
+
await ctx._testRun.collected(project, testFiles)
141
156
await onParseEnd(project, project.typechecker.getResult())
142
157
continue
143
158
}
+26
-7
packages/vitest/src/node/reporters/default.ts
+26
-7
packages/vitest/src/node/reporters/default.ts
···
1
-
import type { File, TaskResultPack } from '@vitest/runner'
1
+
import type { File } from '@vitest/runner'
2
2
import type { Vitest } from '../core'
3
3
import type { BaseOptions } from './base'
4
-
import type { TestModule } from './reported-tasks'
4
+
import type { ReportedHookContext, TestCase, TestModule } from './reported-tasks'
5
5
import { BaseReporter } from './base'
6
6
import { SummaryReporter } from './summary'
7
7
···
33
33
this.summary?.onTestModuleQueued(file)
34
34
}
35
35
36
+
onTestModuleCollected(module: TestModule) {
37
+
this.summary?.onTestModuleCollected(module)
38
+
}
39
+
40
+
onTestModuleEnd(module: TestModule) {
41
+
this.summary?.onTestModuleEnd(module)
42
+
}
43
+
44
+
onTestCaseReady(test: TestCase) {
45
+
this.summary?.onTestCaseReady(test)
46
+
}
47
+
48
+
onTestCaseResult(test: TestCase) {
49
+
this.summary?.onTestCaseResult(test)
50
+
}
51
+
52
+
onHookStart(hook: ReportedHookContext) {
53
+
this.summary?.onHookStart(hook)
54
+
}
55
+
56
+
onHookEnd(hook: ReportedHookContext) {
57
+
this.summary?.onHookEnd(hook)
58
+
}
59
+
36
60
onInit(ctx: Vitest) {
37
61
super.onInit(ctx)
38
62
this.summary?.onInit(ctx, { verbose: this.verbose })
···
50
74
}
51
75
52
76
this.summary?.onPathsCollected(paths)
53
-
}
54
-
55
-
onTaskUpdate(packs: TaskResultPack[]) {
56
-
this.summary?.onTaskUpdate(packs)
57
-
super.onTaskUpdate(packs)
58
77
}
59
78
60
79
onWatcherRerun(files: string[], trigger?: string) {
+44
-63
packages/vitest/src/node/reporters/dot.ts
+44
-63
packages/vitest/src/node/reporters/dot.ts
···
1
-
import type { File, TaskResultPack, TaskState, Test } from '@vitest/runner'
1
+
import type { File, Task, Test } from '@vitest/runner'
2
2
import type { Vitest } from '../core'
3
-
import { getTests } from '@vitest/runner/utils'
3
+
import type { TestCase, TestModule } from './reported-tasks'
4
4
import c from 'tinyrainbow'
5
5
import { BaseReporter } from './base'
6
6
import { WindowRenderer } from './renderers/windowedRenderer'
7
-
import { TaskParser } from './task-parser'
8
7
9
8
interface Icon {
10
9
char: string
11
10
color: (char: string) => string
12
11
}
13
12
13
+
type TestCaseState = ReturnType<TestCase['result']>['state']
14
+
14
15
export class DotReporter extends BaseReporter {
15
-
private summary?: DotSummary
16
+
private renderer?: WindowRenderer
17
+
private tests = new Map<Test['id'], TestCaseState>()
18
+
private finishedTests = new Set<TestCase['id']>()
16
19
17
20
onInit(ctx: Vitest) {
18
21
super.onInit(ctx)
19
22
20
23
if (this.isTTY) {
21
-
this.summary = new DotSummary()
22
-
this.summary.onInit(ctx)
24
+
this.renderer = new WindowRenderer({
25
+
logger: ctx.logger,
26
+
getWindow: () => this.createSummary(),
27
+
})
28
+
29
+
this.ctx.onClose(() => this.renderer?.stop())
23
30
}
24
31
}
25
32
26
-
onTaskUpdate(packs: TaskResultPack[]) {
27
-
this.summary?.onTaskUpdate(packs)
28
-
33
+
printTask(task: Task) {
29
34
if (!this.isTTY) {
30
-
super.onTaskUpdate(packs)
35
+
super.printTask(task)
31
36
}
32
37
}
33
38
34
39
onWatcherRerun(files: string[], trigger?: string) {
35
-
this.summary?.onWatcherRerun()
40
+
this.tests.clear()
41
+
this.renderer?.start()
36
42
super.onWatcherRerun(files, trigger)
37
43
}
38
44
39
45
onFinished(files?: File[], errors?: unknown[]) {
40
-
this.summary?.onFinished()
46
+
if (this.isTTY) {
47
+
const finalLog = formatTests(Array.from(this.tests.values()))
48
+
this.ctx.logger.log(finalLog)
49
+
}
50
+
51
+
this.tests.clear()
52
+
this.renderer?.finish()
53
+
41
54
super.onFinished(files, errors)
42
55
}
43
-
}
44
56
45
-
class DotSummary extends TaskParser {
46
-
private renderer!: WindowRenderer
47
-
private tests = new Map<Test['id'], TaskState>()
48
-
private finishedTests = new Set<Test['id']>()
49
-
50
-
onInit(ctx: Vitest): void {
51
-
this.ctx = ctx
52
-
53
-
this.renderer = new WindowRenderer({
54
-
logger: ctx.logger,
55
-
getWindow: () => this.createSummary(),
56
-
})
57
-
58
-
this.ctx.onClose(() => this.renderer.stop())
59
-
}
60
-
61
-
onWatcherRerun() {
62
-
this.tests.clear()
63
-
this.renderer.start()
64
-
}
65
-
66
-
onFinished() {
67
-
const finalLog = formatTests(Array.from(this.tests.values()))
68
-
this.ctx.logger.log(finalLog)
69
-
70
-
this.tests.clear()
71
-
this.renderer.finish()
72
-
}
73
-
74
-
onTestFilePrepare(file: File): void {
75
-
for (const test of getTests(file)) {
57
+
onTestModuleCollected(module: TestModule): void {
58
+
for (const test of module.children.allTests()) {
76
59
// Dot reporter marks pending tests as running
77
-
this.onTestStart(test)
60
+
this.onTestCaseReady(test)
78
61
}
79
62
}
80
63
81
-
onTestStart(test: Test) {
64
+
onTestCaseReady(test: TestCase) {
82
65
if (this.finishedTests.has(test.id)) {
83
66
return
84
67
}
85
-
86
-
this.tests.set(test.id, test.mode || 'run')
68
+
this.tests.set(test.id, test.result().state || 'run')
87
69
}
88
70
89
-
onTestFinished(test: Test) {
90
-
if (this.finishedTests.has(test.id)) {
91
-
return
92
-
}
93
-
71
+
onTestCaseResult(test: TestCase) {
94
72
this.finishedTests.add(test.id)
95
-
this.tests.set(test.id, test.result?.state || 'skip')
73
+
this.tests.set(test.id, test.result().state || 'skipped')
96
74
}
97
75
98
-
onTestFileFinished() {
76
+
onTestModuleEnd() {
77
+
if (!this.isTTY) {
78
+
return
79
+
}
80
+
99
81
const columns = this.ctx.logger.getColumns()
100
82
101
83
if (this.tests.size < columns) {
102
84
return
103
85
}
104
86
105
-
const finishedTests = Array.from(this.tests).filter(entry => entry[1] !== 'run')
87
+
const finishedTests = Array.from(this.tests).filter(entry => entry[1] !== 'pending')
106
88
107
89
if (finishedTests.length < columns) {
108
90
return
109
91
}
110
92
111
93
// Remove finished tests from state and render them in static output
112
-
const states: TaskState[] = []
94
+
const states: TestCaseState[] = []
113
95
let count = 0
114
96
115
97
for (const [id, state] of finishedTests) {
···
138
120
const pending: Icon = { char: '*', color: c.yellow }
139
121
const skip: Icon = { char: '-', color: (char: string) => c.dim(c.gray(char)) }
140
122
141
-
function getIcon(state: TaskState): Icon {
123
+
function getIcon(state: TestCaseState): Icon {
142
124
switch (state) {
143
-
case 'pass':
125
+
case 'passed':
144
126
return pass
145
-
case 'fail':
127
+
case 'failed':
146
128
return fail
147
-
case 'skip':
148
-
case 'todo':
129
+
case 'skipped':
149
130
return skip
150
131
default:
151
132
return pending
···
156
137
* Format test states into string while keeping ANSI escapes at minimal.
157
138
* Sibling icons with same color are merged into a single c.color() call.
158
139
*/
159
-
function formatTests(states: TaskState[]): string {
140
+
function formatTests(states: TestCaseState[]): string {
160
141
let currentIcon = pending
161
142
let count = 0
162
143
let output = ''
+4
-2
packages/vitest/src/node/reporters/index.ts
+4
-2
packages/vitest/src/node/reporters/index.ts
···
1
-
import type { Reporter } from '../types/reporter'
1
+
import type { Reporter, TestRunEndReason } from '../types/reporter'
2
2
import type { BaseOptions, BaseReporter } from './base'
3
3
import type { BlobOptions } from './blob'
4
4
import type { DefaultReporterOptions } from './default'
···
27
27
TapReporter,
28
28
VerboseReporter,
29
29
}
30
-
export type { BaseReporter, Reporter }
30
+
export type { BaseReporter, Reporter, TestRunEndReason }
31
31
32
32
export {
33
33
BenchmarkBuiltinReporters,
···
70
70
'hanging-process': never
71
71
'html': HTMLOptions
72
72
}
73
+
74
+
export type { ReportedHookContext } from './reported-tasks'
+126
-70
packages/vitest/src/node/reporters/reported-tasks.ts
+126
-70
packages/vitest/src/node/reporters/reported-tasks.ts
···
5
5
Suite as RunnerTestSuite,
6
6
TaskMeta,
7
7
} from '@vitest/runner'
8
-
import type { TestError } from '@vitest/utils'
8
+
import type { SerializedError, TestError } from '@vitest/utils'
9
9
import type { TestProject } from '../project'
10
10
11
11
class ReportedTaskImplementation {
···
122
122
}
123
123
124
124
/**
125
-
* Test results. Will be `undefined` if test is skipped, not finished yet or was just collected.
125
+
* Test results.
126
+
* - **pending**: Test was collected, but didn't finish running yet.
127
+
* - **passed**: Test passed successfully
128
+
* - **failed**: Test failed to execute
129
+
* - **skipped**: Test was skipped during collection or dynamically with `ctx.skip()`.
126
130
*/
127
-
public result(): TestResult | undefined {
131
+
public result(): TestResult {
128
132
const result = this.task.result
133
+
const mode = result?.state || this.task.mode
134
+
135
+
if (!result && (mode === 'skip' || mode === 'todo')) {
136
+
return {
137
+
state: 'skipped',
138
+
note: undefined,
139
+
errors: undefined,
140
+
}
141
+
}
142
+
129
143
if (!result || result.state === 'run' || result.state === 'queued') {
130
-
return undefined
144
+
return {
145
+
state: 'pending',
146
+
errors: undefined,
147
+
}
131
148
}
132
149
const state = result.state === 'fail'
133
150
? 'failed' as const
···
154
171
}
155
172
156
173
/**
157
-
* Checks if the test was skipped during collection or dynamically with `ctx.skip()`.
158
-
*/
159
-
public skipped(): boolean {
160
-
const mode = this.task.result?.state || this.task.mode
161
-
return mode === 'skip' || mode === 'todo'
162
-
}
163
-
164
-
/**
165
174
* Custom metadata that was attached to the test during its execution.
166
175
*/
167
176
public meta(): TaskMeta {
···
175
184
public diagnostic(): TestDiagnostic | undefined {
176
185
const result = this.task.result
177
186
// startTime should always be available if the test has properly finished
178
-
if (!result || result.state === 'run' || result.state === 'queued' || !result.startTime) {
187
+
if (!result || !result.startTime) {
179
188
return undefined
180
189
}
181
190
const duration = result.duration || 0
···
228
237
/**
229
238
* Filters all tests that are part of this collection and its children.
230
239
*/
231
-
*allTests(state?: TestResult['state'] | 'running'): Generator<TestCase, undefined, void> {
240
+
*allTests(state?: TestState): Generator<TestCase, undefined, void> {
232
241
for (const child of this) {
233
242
if (child.type === 'suite') {
234
243
yield * child.children.allTests(state)
235
244
}
236
245
else if (state) {
237
-
const testState = getTestState(child)
246
+
const testState = child.result().state
238
247
if (state === testState) {
239
248
yield child
240
249
}
···
248
257
/**
249
258
* Filters only the tests that are part of this collection.
250
259
*/
251
-
*tests(state?: TestResult['state'] | 'running'): Generator<TestCase, undefined, void> {
260
+
*tests(state?: TestState): Generator<TestCase, undefined, void> {
252
261
for (const child of this) {
253
262
if (child.type !== 'test') {
254
263
continue
255
264
}
256
265
257
266
if (state) {
258
-
const testState = getTestState(child)
267
+
const testState = child.result().state
259
268
if (state === testState) {
260
269
yield child
261
270
}
···
298
307
299
308
export type { TestCollection }
300
309
310
+
export type ReportedHookContext = {
311
+
readonly name: 'beforeAll' | 'afterAll'
312
+
readonly entity: TestSuite | TestModule
313
+
} | {
314
+
readonly name: 'beforeEach' | 'afterEach'
315
+
readonly entity: TestCase
316
+
}
317
+
301
318
abstract class SuiteImplementation extends ReportedTaskImplementation {
302
319
/** @internal */
303
320
declare public readonly task: RunnerTestSuite | RunnerTestFile
···
314
331
}
315
332
316
333
/**
317
-
* Checks if the suite was skipped during collection.
318
-
*/
319
-
public skipped(): boolean {
320
-
const mode = this.task.mode
321
-
return mode === 'skip' || mode === 'todo'
322
-
}
323
-
324
-
/**
325
334
* Errors that happened outside of the test run during collection, like syntax errors.
326
335
*/
327
-
public errors(): TestError[] {
328
-
return (this.task.result?.errors as TestError[] | undefined) || []
336
+
public errors(): SerializedError[] {
337
+
return (this.task.result?.errors as SerializedError[] | undefined) || []
329
338
}
330
339
}
331
340
···
379
388
declare public ok: () => boolean
380
389
381
390
/**
391
+
* Checks the running state of the suite.
392
+
*/
393
+
public state(): TestSuiteState {
394
+
return getSuiteState(this.task)
395
+
}
396
+
397
+
/**
382
398
* Full name of the suite including all parent suites separated with `>`.
383
399
*/
384
400
public get fullName(): string {
···
402
418
403
419
/**
404
420
* This is usually an absolute UNIX file path.
405
-
* It can be a virtual id if the file is not on the disk.
406
-
* This value corresponds to Vite's `ModuleGraph` id.
421
+
* It can be a virtual ID if the file is not on the disk.
422
+
* This value corresponds to the ID in the Vite's module graph.
407
423
*/
408
424
public readonly moduleId: string
409
425
···
414
430
}
415
431
416
432
/**
433
+
* Checks the running state of the test file.
434
+
*/
435
+
public state(): TestModuleState {
436
+
const state = this.task.result?.state
437
+
if (state === 'queued') {
438
+
return 'queued'
439
+
}
440
+
return getSuiteState(this.task)
441
+
}
442
+
443
+
/**
417
444
* Checks if the module has any failed tests.
418
445
* This will also return `false` if module failed during collection.
419
446
*/
420
447
declare public ok: () => boolean
421
-
422
-
/**
423
-
* Checks if the module was skipped and didn't run.
424
-
*/
425
-
declare public skipped: () => boolean
426
448
427
449
/**
428
450
* Useful information about the module like duration, memory usage, etc.
···
445
467
}
446
468
447
469
export interface TaskOptions {
448
-
each: boolean | undefined
449
-
concurrent: boolean | undefined
450
-
shuffle: boolean | undefined
451
-
retry: number | undefined
452
-
repeats: number | undefined
453
-
mode: 'run' | 'only' | 'skip' | 'todo' | 'queued'
470
+
readonly each: boolean | undefined
471
+
readonly fails: boolean | undefined
472
+
readonly concurrent: boolean | undefined
473
+
readonly shuffle: boolean | undefined
474
+
readonly retry: number | undefined
475
+
readonly repeats: number | undefined
476
+
readonly mode: 'run' | 'only' | 'skip' | 'todo'
454
477
}
455
478
456
479
function buildOptions(
457
-
task: RunnerTestCase | RunnerTestFile | RunnerTestSuite,
480
+
task: RunnerTestCase | RunnerTestSuite,
458
481
): TaskOptions {
459
482
return {
460
483
each: task.each,
484
+
fails: task.type === 'test' && task.fails,
461
485
concurrent: task.concurrent,
462
486
shuffle: task.shuffle,
463
487
retry: task.retry,
464
488
repeats: task.repeats,
465
-
mode: task.mode,
489
+
// runner types are too broad, but the public API should be more strict
490
+
// the queued state exists only on Files and this method is called
491
+
// only for tests and suites
492
+
mode: task.mode as TaskOptions['mode'],
466
493
}
467
494
}
468
495
469
-
export type TestResult = TestResultPassed | TestResultFailed | TestResultSkipped
496
+
export type TestSuiteState = 'skipped' | 'pending' | 'failed' | 'passed'
497
+
export type TestModuleState = TestSuiteState | 'queued'
498
+
export type TestState = TestResult['state']
499
+
500
+
export type TestResult =
501
+
| TestResultPassed
502
+
| TestResultFailed
503
+
| TestResultSkipped
504
+
| TestResultPending
505
+
506
+
export interface TestResultPending {
507
+
/**
508
+
* The test was collected, but didn't finish running yet.
509
+
*/
510
+
readonly state: 'pending'
511
+
/**
512
+
* Pending tests have no errors.
513
+
*/
514
+
readonly errors: undefined
515
+
}
470
516
471
517
export interface TestResultPassed {
472
518
/**
473
519
* The test passed successfully.
474
520
*/
475
-
state: 'passed'
521
+
readonly state: 'passed'
476
522
/**
477
523
* Errors that were thrown during the test execution.
478
524
*
479
525
* **Note**: If test was retried successfully, errors will still be reported.
480
526
*/
481
-
errors: TestError[] | undefined
527
+
readonly errors: ReadonlyArray<TestError> | undefined
482
528
}
483
529
484
530
export interface TestResultFailed {
485
531
/**
486
532
* The test failed to execute.
487
533
*/
488
-
state: 'failed'
534
+
readonly state: 'failed'
489
535
/**
490
536
* Errors that were thrown during the test execution.
491
537
*/
492
-
errors: TestError[]
538
+
readonly errors: ReadonlyArray<TestError>
493
539
}
494
540
495
541
export interface TestResultSkipped {
···
497
543
* The test was skipped with `only` (on another test), `skip` or `todo` flag.
498
544
* You can see which one was used in the `options.mode` option.
499
545
*/
500
-
state: 'skipped'
546
+
readonly state: 'skipped'
501
547
/**
502
548
* Skipped tests have no errors.
503
549
*/
504
-
errors: undefined
550
+
readonly errors: undefined
505
551
/**
506
552
* A custom note passed down to `ctx.skip(note)`.
507
553
*/
508
-
note: string | undefined
554
+
readonly note: string | undefined
509
555
}
510
556
511
557
export interface TestDiagnostic {
512
558
/**
513
559
* If the duration of the test is above `slowTestThreshold`.
514
560
*/
515
-
slow: boolean
561
+
readonly slow: boolean
516
562
/**
517
563
* The amount of memory used by the test in bytes.
518
564
* This value is only available if the test was executed with `logHeapUsage` flag.
519
565
*/
520
-
heap: number | undefined
566
+
readonly heap: number | undefined
521
567
/**
522
568
* The time it takes to execute the test in ms.
523
569
*/
524
-
duration: number
570
+
readonly duration: number
525
571
/**
526
572
* The time in ms when the test started.
527
573
*/
528
-
startTime: number
574
+
readonly startTime: number
529
575
/**
530
576
* The amount of times the test was retried.
531
577
*/
532
-
retryCount: number
578
+
readonly retryCount: number
533
579
/**
534
580
* The amount of times the test was repeated as configured by `repeats` option.
535
581
* This value can be lower if the test failed during the repeat and no `retry` is configured.
536
582
*/
537
-
repeatCount: number
583
+
readonly repeatCount: number
538
584
/**
539
585
* If test passed on a second retry.
540
586
*/
541
-
flaky: boolean
587
+
readonly flaky: boolean
542
588
}
543
589
544
590
export interface ModuleDiagnostic {
545
591
/**
546
592
* The time it takes to import and initiate an environment.
547
593
*/
548
-
environmentSetupDuration: number
594
+
readonly environmentSetupDuration: number
549
595
/**
550
596
* The time it takes Vitest to setup test harness (runner, mocks, etc.).
551
597
*/
552
-
prepareDuration: number
598
+
readonly prepareDuration: number
553
599
/**
554
600
* The time it takes to import the test module.
555
601
* This includes importing everything in the module and executing suite callbacks.
556
602
*/
557
-
collectDuration: number
603
+
readonly collectDuration: number
558
604
/**
559
605
* The time it takes to import the setup module.
560
606
*/
561
-
setupDuration: number
607
+
readonly setupDuration: number
562
608
/**
563
609
* Accumulated duration of all tests and hooks in the module.
564
610
*/
565
-
duration: number
566
-
}
567
-
568
-
function getTestState(test: TestCase): TestResult['state'] | 'running' {
569
-
if (test.skipped()) {
570
-
return 'skipped'
571
-
}
572
-
const result = test.result()
573
-
return result ? result.state : 'running'
611
+
readonly duration: number
574
612
}
575
613
576
614
function storeTask(
···
592
630
)
593
631
}
594
632
return reportedTask
633
+
}
634
+
635
+
function getSuiteState(task: RunnerTestSuite | RunnerTestFile): TestSuiteState {
636
+
const mode = task.mode
637
+
const state = task.result?.state
638
+
if (mode === 'skip' || mode === 'todo' || state === 'skip' || state === 'todo') {
639
+
return 'skipped'
640
+
}
641
+
if (state == null || state === 'run' || state === 'only') {
642
+
return 'pending'
643
+
}
644
+
if (state === 'fail') {
645
+
return 'failed'
646
+
}
647
+
if (state === 'pass') {
648
+
return 'passed'
649
+
}
650
+
throw new Error(`Unknown suite state: ${state}`)
595
651
}
+97
-127
packages/vitest/src/node/reporters/summary.ts
+97
-127
packages/vitest/src/node/reporters/summary.ts
···
1
-
import type { File, Test } from '@vitest/runner'
2
1
import type { Vitest } from '../core'
3
2
import type { Reporter } from '../types/reporter'
4
-
import type { TestModule } from './reported-tasks'
5
-
import type { HookOptions } from './task-parser'
6
-
import { getTests } from '@vitest/runner/utils'
3
+
import type { ReportedHookContext, TestCase, TestModule } from './reported-tasks'
7
4
import c from 'tinyrainbow'
8
5
import { F_POINTER, F_TREE_NODE_END, F_TREE_NODE_MIDDLE } from './renderers/figures'
9
6
import { formatProjectName, formatTime, formatTimeString, padSummaryTitle } from './renderers/utils'
10
7
import { WindowRenderer } from './renderers/windowedRenderer'
11
-
import { TaskParser } from './task-parser'
12
8
13
9
const DURATION_UPDATE_INTERVAL_MS = 100
14
10
const FINISHED_TEST_CLEANUP_TIME_MS = 1_000
···
34
30
hook?: Omit<SlowTask, 'hook'>
35
31
}
36
32
37
-
interface RunningTest extends Pick<Counter, 'total' | 'completed'> {
38
-
filename: File['name']
39
-
projectName: File['projectName']
33
+
interface RunningModule extends Pick<Counter, 'total' | 'completed'> {
34
+
filename: TestModule['task']['name']
35
+
projectName: TestModule['project']['name']
40
36
hook?: Omit<SlowTask, 'hook'>
41
-
tests: Map<Test['id'], SlowTask>
37
+
tests: Map<TestCase['id'], SlowTask>
38
+
typecheck: boolean
42
39
}
43
40
44
41
/**
45
42
* Reporter extension that renders summary and forwards all other logs above itself.
46
43
* Intended to be used by other reporters, not as a standalone reporter.
47
44
*/
48
-
export class SummaryReporter extends TaskParser implements Reporter {
45
+
export class SummaryReporter implements Reporter {
46
+
private ctx!: Vitest
49
47
private options!: Options
50
48
private renderer!: WindowRenderer
51
49
52
-
private suites = emptyCounters()
50
+
private modules = emptyCounters()
53
51
private tests = emptyCounters()
54
52
private maxParallelTests = 0
55
53
56
-
/** Currently running tests, may include finished tests too */
57
-
private runningTests = new Map<File['id'], RunningTest>()
54
+
/** Currently running test modules, may include finished test modules too */
55
+
private runningModules = new Map<TestModule['id'], RunningModule>()
58
56
59
-
/** ID of finished `this.runningTests` that are currently being shown */
60
-
private finishedTests = new Map<File['id'], NodeJS.Timeout>()
61
-
62
-
/** IDs of all finished tests */
63
-
private allFinishedTests = new Set<File['id']>()
57
+
/** ID of finished `this.runningModules` that are currently being shown */
58
+
private finishedModules = new Map<TestModule['id'], NodeJS.Timeout>()
64
59
65
60
private startTime = ''
66
61
private currentTime = 0
···
88
83
})
89
84
}
90
85
91
-
onTestModuleQueued(module: TestModule) {
92
-
this.onTestFilePrepare(module.task)
93
-
}
94
-
95
86
onPathsCollected(paths?: string[]) {
96
-
this.suites.total = (paths || []).length
87
+
this.modules.total = (paths || []).length
97
88
}
98
89
99
90
onWatcherRerun() {
100
-
this.runningTests.clear()
101
-
this.finishedTests.clear()
102
-
this.allFinishedTests.clear()
103
-
this.suites = emptyCounters()
91
+
this.runningModules.clear()
92
+
this.finishedModules.clear()
93
+
this.modules = emptyCounters()
104
94
this.tests = emptyCounters()
105
95
106
96
this.startTimers()
···
108
98
}
109
99
110
100
onFinished() {
111
-
this.runningTests.clear()
112
-
this.finishedTests.clear()
113
-
this.allFinishedTests.clear()
101
+
this.runningModules.clear()
102
+
this.finishedModules.clear()
114
103
this.renderer.finish()
115
104
clearInterval(this.durationInterval)
116
105
}
117
106
118
-
onTestFilePrepare(file: File) {
119
-
if (this.runningTests.has(file.id)) {
120
-
const stats = this.runningTests.get(file.id)!
121
-
// if there are no tests, it means the test was queued but not collected
122
-
if (!stats.total) {
123
-
const total = getTests(file).length
124
-
this.tests.total += total
125
-
stats.total = total
126
-
}
127
-
return
107
+
onTestModuleQueued(module: TestModule) {
108
+
// When new test module starts, take the place of previously finished test module, if any
109
+
if (this.finishedModules.size) {
110
+
const finished = this.finishedModules.keys().next().value
111
+
this.removeTestModule(finished)
128
112
}
129
113
130
-
if (this.allFinishedTests.has(file.id)) {
131
-
return
132
-
}
133
-
134
-
const total = getTests(file).length
135
-
this.tests.total += total
136
-
137
-
// When new test starts, take the place of previously finished test, if any
138
-
if (this.finishedTests.size) {
139
-
const finished = this.finishedTests.keys().next().value
140
-
this.removeTestFile(finished)
141
-
}
142
-
143
-
this.runningTests.set(file.id, {
144
-
total,
145
-
completed: 0,
146
-
filename: file.name,
147
-
projectName: file.projectName,
148
-
tests: new Map(),
149
-
})
150
-
151
-
this.maxParallelTests = Math.max(this.maxParallelTests, this.runningTests.size)
114
+
this.runningModules.set(module.id, initializeStats(module))
152
115
}
153
116
154
-
onHookStart(options: HookOptions) {
117
+
onTestModuleCollected(module: TestModule) {
118
+
let stats = this.runningModules.get(module.id)
119
+
120
+
if (!stats) {
121
+
stats = initializeStats(module)
122
+
this.runningModules.set(module.id, stats)
123
+
}
124
+
125
+
const total = Array.from(module.children.allTests()).length
126
+
this.tests.total += total
127
+
stats.total = total
128
+
129
+
this.maxParallelTests = Math.max(this.maxParallelTests, this.runningModules.size)
130
+
}
131
+
132
+
onHookStart(options: ReportedHookContext) {
155
133
const stats = this.getHookStats(options)
156
134
157
135
if (!stats) {
···
174
152
hook.onFinish = () => clearTimeout(timeout)
175
153
}
176
154
177
-
onHookEnd(options: HookOptions) {
155
+
onHookEnd(options: ReportedHookContext) {
178
156
const stats = this.getHookStats(options)
179
157
180
158
if (stats?.hook?.name !== options.name) {
···
185
163
stats.hook.visible = false
186
164
}
187
165
188
-
onTestStart(test: Test) {
166
+
onTestCaseReady(test: TestCase) {
189
167
// Track slow running tests only on verbose mode
190
168
if (!this.options.verbose) {
191
169
return
192
170
}
193
171
194
-
const stats = this.getTestStats(test)
172
+
const stats = this.runningModules.get(test.module.id)
195
173
196
174
if (!stats || stats.tests.has(test.id)) {
197
175
return
···
216
194
stats.tests.set(test.id, slowTest)
217
195
}
218
196
219
-
onTestFinished(test: Test) {
220
-
const stats = this.getTestStats(test)
197
+
onTestCaseResult(test: TestCase) {
198
+
const stats = this.runningModules.get(test.module.id)
221
199
222
200
if (!stats) {
223
201
return
···
227
205
stats.tests.delete(test.id)
228
206
229
207
stats.completed++
230
-
const result = test.result
208
+
const result = test.result()
231
209
232
-
if (result?.state === 'pass') {
210
+
if (result?.state === 'passed') {
233
211
this.tests.passed++
234
212
}
235
-
else if (result?.state === 'fail') {
213
+
else if (result?.state === 'failed') {
236
214
this.tests.failed++
237
215
}
238
-
else if (!result?.state || result?.state === 'skip' || result?.state === 'todo') {
216
+
else if (!result?.state || result?.state === 'skipped') {
239
217
this.tests.skipped++
240
218
}
241
219
}
242
220
243
-
onTestFileFinished(file: File) {
244
-
if (this.allFinishedTests.has(file.id)) {
245
-
return
221
+
onTestModuleEnd(module: TestModule) {
222
+
const state = module.state()
223
+
this.modules.completed++
224
+
225
+
if (state === 'passed') {
226
+
this.modules.passed++
227
+
}
228
+
else if (state === 'failed') {
229
+
this.modules.failed++
230
+
}
231
+
else if (module.task.mode === 'todo' && state === 'skipped') {
232
+
this.modules.todo++
233
+
}
234
+
else if (state === 'skipped') {
235
+
this.modules.skipped++
246
236
}
247
237
248
-
this.allFinishedTests.add(file.id)
249
-
this.suites.completed++
250
-
251
-
if (file.result?.state === 'pass') {
252
-
this.suites.passed++
253
-
}
254
-
else if (file.result?.state === 'fail') {
255
-
this.suites.failed++
256
-
}
257
-
else if (file.result?.state === 'skip') {
258
-
this.suites.skipped++
259
-
}
260
-
else if (file.result?.state === 'todo') {
261
-
this.suites.todo++
262
-
}
263
-
264
-
const left = this.suites.total - this.suites.completed
238
+
const left = this.modules.total - this.modules.completed
265
239
266
240
// Keep finished tests visible in summary for a while if there are more tests left.
267
-
// When a new test starts in onTestFilePrepare it will take this ones place.
241
+
// When a new test starts in onTestModuleQueued it will take this ones place.
268
242
// This reduces flickering by making summary more stable.
269
243
if (left > this.maxParallelTests) {
270
-
this.finishedTests.set(file.id, setTimeout(() => {
271
-
this.removeTestFile(file.id)
244
+
this.finishedModules.set(module.id, setTimeout(() => {
245
+
this.removeTestModule(module.id)
272
246
}, FINISHED_TEST_CLEANUP_TIME_MS).unref())
273
247
}
274
248
else {
275
249
// Run is about to end as there are less tests left than whole run had parallel at max.
276
-
// Remove finished test immediately.
277
-
this.removeTestFile(file.id)
250
+
// Remove finished test immediatelly.
251
+
this.removeTestModule(module.id)
278
252
}
279
253
}
280
254
281
-
private getTestStats(test: Test) {
282
-
const file = test.file
283
-
let stats = this.runningTests.get(file.id)
284
-
285
-
if (!stats || stats.total === 0) {
286
-
// It's possible that that test finished before it's preparation was even reported
287
-
this.onTestFilePrepare(test.file)
288
-
stats = this.runningTests.get(file.id)!
289
-
290
-
// It's also possible that this update came after whole test file was reported as finished
291
-
if (!stats) {
292
-
return
293
-
}
294
-
}
295
-
296
-
return stats
297
-
}
298
-
299
-
private getHookStats({ file, id, type }: HookOptions) {
255
+
private getHookStats({ entity }: ReportedHookContext) {
300
256
// Track slow running hooks only on verbose mode
301
257
if (!this.options.verbose) {
302
258
return
303
259
}
304
260
305
-
const stats = this.runningTests.get(file.id)
261
+
const module = entity.type === 'module' ? entity : entity.module
262
+
const stats = this.runningModules.get(module.id)
306
263
307
264
if (!stats) {
308
265
return
309
266
}
310
267
311
-
return type === 'suite' ? stats : stats?.tests.get(id)
268
+
return entity.type === 'test' ? stats.tests.get(entity.id) : stats
312
269
}
313
270
314
271
private createSummary() {
315
272
const summary = ['']
316
273
317
-
for (const testFile of Array.from(this.runningTests.values()).sort(sortRunningTests)) {
274
+
for (const testFile of Array.from(this.runningModules.values()).sort(sortRunningModules)) {
275
+
const typecheck = testFile.typecheck ? `${c.bgBlue(c.bold(' TS '))} ` : ''
318
276
summary.push(
319
277
c.bold(c.yellow(` ${F_POINTER} `))
320
278
+ formatProjectName(testFile.projectName)
279
+
+ typecheck
321
280
+ testFile.filename
322
281
+ c.dim(!testFile.completed && !testFile.total
323
282
? ' [queued]'
···
345
304
}
346
305
}
347
306
348
-
if (this.runningTests.size > 0) {
307
+
if (this.runningModules.size > 0) {
349
308
summary.push('')
350
309
}
351
310
352
-
summary.push(padSummaryTitle('Test Files') + getStateString(this.suites))
311
+
summary.push(padSummaryTitle('Test Files') + getStateString(this.modules))
353
312
summary.push(padSummaryTitle('Tests') + getStateString(this.tests))
354
313
summary.push(padSummaryTitle('Start at') + this.startTime)
355
314
summary.push(padSummaryTitle('Duration') + formatTime(this.duration))
···
369
328
}, DURATION_UPDATE_INTERVAL_MS).unref()
370
329
}
371
330
372
-
private removeTestFile(id?: File['id']) {
331
+
private removeTestModule(id?: TestModule['id']) {
373
332
if (!id) {
374
333
return
375
334
}
376
335
377
-
const testFile = this.runningTests.get(id)
336
+
const testFile = this.runningModules.get(id)
378
337
testFile?.hook?.onFinish()
379
338
testFile?.tests?.forEach(test => test.onFinish())
380
339
381
-
this.runningTests.delete(id)
340
+
this.runningModules.delete(id)
382
341
383
-
clearTimeout(this.finishedTests.get(id))
384
-
this.finishedTests.delete(id)
342
+
clearTimeout(this.finishedModules.get(id))
343
+
this.finishedModules.delete(id)
385
344
}
386
345
}
387
346
···
402
361
)
403
362
}
404
363
405
-
function sortRunningTests(a: RunningTest, b: RunningTest) {
364
+
function sortRunningModules(a: RunningModule, b: RunningModule) {
406
365
if ((a.projectName || '') > (b.projectName || '')) {
407
366
return 1
408
367
}
···
412
371
}
413
372
414
373
return a.filename.localeCompare(b.filename)
374
+
}
375
+
376
+
function initializeStats(module: TestModule): RunningModule {
377
+
return {
378
+
total: 0,
379
+
completed: 0,
380
+
filename: module.task.name,
381
+
projectName: module.project.name,
382
+
tests: new Map(),
383
+
typecheck: !!module.task.meta.typecheck,
384
+
}
415
385
}
-86
packages/vitest/src/node/reporters/task-parser.ts
-86
packages/vitest/src/node/reporters/task-parser.ts
···
1
-
import type { File, Task, TaskResultPack, Test } from '@vitest/runner'
2
-
import type { Vitest } from '../core'
3
-
import { getTests } from '@vitest/runner/utils'
4
-
5
-
export interface HookOptions {
6
-
name: string
7
-
file: File
8
-
id: File['id'] | Test['id']
9
-
type: Task['type']
10
-
}
11
-
12
-
export class TaskParser {
13
-
ctx!: Vitest
14
-
15
-
onInit(ctx: Vitest) {
16
-
this.ctx = ctx
17
-
}
18
-
19
-
onHookStart(_options: HookOptions) {}
20
-
onHookEnd(_options: HookOptions) {}
21
-
22
-
onTestStart(_test: Test) {}
23
-
onTestFinished(_test: Test) {}
24
-
25
-
onTestFilePrepare(_file: File) {}
26
-
onTestFileFinished(_file: File) {}
27
-
28
-
onTaskUpdate(packs: TaskResultPack[]) {
29
-
const startingTestFiles: File[] = []
30
-
const finishedTestFiles: File[] = []
31
-
32
-
const startingTests: Test[] = []
33
-
const finishedTests: Test[] = []
34
-
35
-
const startingHooks: HookOptions[] = []
36
-
const endingHooks: HookOptions[] = []
37
-
38
-
for (const pack of packs) {
39
-
const task = this.ctx.state.idMap.get(pack[0])
40
-
41
-
if (task?.type === 'suite' && 'filepath' in task && task.result?.state) {
42
-
if (task?.result?.state === 'run' || task?.result?.state === 'queued') {
43
-
startingTestFiles.push(task)
44
-
}
45
-
else {
46
-
// Skipped tests are not reported, do it manually
47
-
for (const test of getTests(task)) {
48
-
if (!test.result || test.result?.state === 'skip') {
49
-
finishedTests.push(test)
50
-
}
51
-
}
52
-
53
-
finishedTestFiles.push(task.file)
54
-
}
55
-
}
56
-
57
-
if (task?.type === 'test') {
58
-
if (task.result?.state === 'run' || task.result?.state === 'queued') {
59
-
startingTests.push(task)
60
-
}
61
-
else if (task.result?.hooks?.afterEach !== 'run') {
62
-
finishedTests.push(task)
63
-
}
64
-
}
65
-
66
-
if (task?.result?.hooks) {
67
-
for (const [hook, state] of Object.entries(task.result.hooks)) {
68
-
if (state === 'run' || state === 'queued') {
69
-
startingHooks.push({ name: hook, file: task.file, id: task.id, type: task.type })
70
-
}
71
-
else {
72
-
endingHooks.push({ name: hook, file: task.file, id: task.id, type: task.type })
73
-
}
74
-
}
75
-
}
76
-
}
77
-
78
-
endingHooks.forEach(hook => this.onHookEnd(hook))
79
-
finishedTests.forEach(test => this.onTestFinished(test))
80
-
finishedTestFiles.forEach(file => this.onTestFileFinished(file))
81
-
82
-
startingTestFiles.forEach(file => this.onTestFilePrepare(file))
83
-
startingTests.forEach(test => this.onTestStart(test))
84
-
startingHooks.forEach(hook => this.onHookStart(hook))
85
-
}
86
-
}
+85
-4
packages/vitest/src/node/types/reporter.ts
+85
-4
packages/vitest/src/node/types/reporter.ts
···
1
1
import type { File, TaskResultPack } from '@vitest/runner'
2
+
import type { SerializedError } from '@vitest/utils'
2
3
import type { SerializedTestSpecification } from '../../runtime/types/utils'
3
4
import type { Awaitable, UserConsoleLog } from '../../types/general'
4
5
import type { Vitest } from '../core'
5
-
import type { TestModule } from '../reporters/reported-tasks'
6
+
import type { ReportedHookContext, TestCase, TestModule, TestSuite } from '../reporters/reported-tasks'
7
+
import type { TestSpecification } from '../spec'
8
+
9
+
export type TestRunEndReason = 'passed' | 'interrupted' | 'failed'
6
10
7
11
export interface Reporter {
8
-
onInit?: (ctx: Vitest) => void
12
+
onInit?: (vitest: Vitest) => void
13
+
/**
14
+
* @deprecated use `onTestRunStart` instead
15
+
*/
9
16
onPathsCollected?: (paths?: string[]) => Awaitable<void>
17
+
/**
18
+
* @deprecated use `onTestRunStart` instead
19
+
*/
10
20
onSpecsCollected?: (specs?: SerializedTestSpecification[]) => Awaitable<void>
11
-
onTestModuleQueued?: (file: TestModule) => Awaitable<void>
12
-
onCollected?: (files?: File[]) => Awaitable<void>
21
+
/**
22
+
* @deprecated use `onTestModuleCollected` instead
23
+
*/
24
+
onCollected?: (files: File[]) => Awaitable<void>
25
+
/**
26
+
* @deprecated use `onTestRunEnd` instead
27
+
*/
13
28
onFinished?: (
14
29
files: File[],
15
30
errors: unknown[],
16
31
coverage?: unknown
17
32
) => Awaitable<void>
33
+
/**
34
+
* @deprecated use `onTestModuleQueued`, `onTestModuleStart`, `onTestModuleEnd`, `onTestCaseReady`, `onTestCaseResult` instead
35
+
*/
18
36
onTaskUpdate?: (packs: TaskResultPack[]) => Awaitable<void>
19
37
onTestRemoved?: (trigger?: string) => Awaitable<void>
20
38
onWatcherStart?: (files?: File[], errors?: unknown[]) => Awaitable<void>
···
22
40
onServerRestart?: (reason?: string) => Awaitable<void>
23
41
onUserConsoleLog?: (log: UserConsoleLog) => Awaitable<void>
24
42
onProcessTimeout?: () => Awaitable<void>
43
+
44
+
/**
45
+
* Called when the new test run starts.
46
+
*/
47
+
onTestRunStart?: (specifications: ReadonlyArray<TestSpecification>) => Awaitable<void>
48
+
/**
49
+
* Called when the test run is finished.
50
+
*/
51
+
onTestRunEnd?: (
52
+
testModules: ReadonlyArray<TestModule>,
53
+
unhandledErrors: ReadonlyArray<SerializedError>,
54
+
reason: TestRunEndReason
55
+
) => Awaitable<void>
56
+
57
+
/**
58
+
* Called when the module is enqueued for testing. The file itself is not loaded yet.
59
+
*/
60
+
onTestModuleQueued?: (testModule: TestModule) => Awaitable<void>
61
+
/**
62
+
* Called when the test file is loaded and the module is ready to run tests.
63
+
*/
64
+
onTestModuleCollected?: (testModule: TestModule) => Awaitable<void>
65
+
/**
66
+
* Called when starting to run tests of the test file
67
+
*/
68
+
onTestModuleStart?: (testModule: TestModule) => Awaitable<void>
69
+
/**
70
+
* Called when all tests of the test file have finished running.
71
+
*/
72
+
onTestModuleEnd?: (testModule: TestModule) => Awaitable<void>
73
+
74
+
/**
75
+
* Called when test case is ready to run.
76
+
* Called before the `beforeEach` hooks for the test are run.
77
+
*/
78
+
onTestCaseReady?: (testCase: TestCase) => Awaitable<void>
79
+
/**
80
+
* Called after the test and its hooks are finished running.
81
+
* The `result()` cannot be `pending`.
82
+
*/
83
+
onTestCaseResult?: (testCase: TestCase) => Awaitable<void>
84
+
85
+
/**
86
+
* Called when test suite is ready to run.
87
+
* Called before the `beforeAll` hooks for the test are run.
88
+
*/
89
+
onTestSuiteReady?: (testSuite: TestSuite) => Awaitable<void>
90
+
/**
91
+
* Called after the test suite and its hooks are finished running.
92
+
* The `state` cannot be `pending`.
93
+
*/
94
+
onTestSuiteResult?: (testSuite: TestSuite) => Awaitable<void>
95
+
96
+
/**
97
+
* Called before the hook starts to run.
98
+
*/
99
+
onHookStart?: (hook: ReportedHookContext) => Awaitable<void>
100
+
/**
101
+
* Called after the hook finished running.
102
+
*/
103
+
onHookEnd?: (hook: ReportedHookContext) => Awaitable<void>
104
+
105
+
onCoverage?: (coverage: unknown) => Awaitable<void>
25
106
}
+7
-6
packages/vitest/src/runtime/runners/benchmark.ts
+7
-6
packages/vitest/src/runtime/runners/benchmark.ts
···
1
1
import type {
2
2
Suite,
3
3
Task,
4
+
TaskUpdateEvent,
4
5
VitestRunner,
5
6
VitestRunnerImportSource,
6
7
} from '@vitest/runner'
···
59
60
startTime: start,
60
61
benchmark: createBenchmarkResult(suite.name),
61
62
}
62
-
updateTask(suite)
63
+
updateTask('suite-prepare', suite)
63
64
64
65
const addBenchTaskListener = (
65
66
task: InstanceType<typeof Task>,
···
82
83
if (!runner.config.benchmark?.includeSamples) {
83
84
result.samples.length = 0
84
85
}
85
-
updateTask(benchmark)
86
+
updateTask('test-finished', benchmark)
86
87
},
87
88
{
88
89
once: true,
···
122
123
123
124
for (const benchmark of benchmarkGroup) {
124
125
const task = benchmarkTasks.get(benchmark)!
125
-
updateTask(benchmark)
126
+
updateTask('test-prepare', benchmark)
126
127
await task.warmup()
127
128
tasks.push([
128
129
await new Promise<BenchTask>(resolve =>
···
137
138
suite.result!.duration = performance.now() - start
138
139
suite.result!.state = 'pass'
139
140
140
-
updateTask(suite)
141
+
updateTask('suite-finished', suite)
141
142
defer.resolve(null)
142
143
143
144
await defer
144
145
}
145
146
146
-
function updateTask(task: Task) {
147
-
updateRunnerTask(task, runner)
147
+
function updateTask(event: TaskUpdateEvent, task: Task) {
148
+
updateRunnerTask(event, task, runner)
148
149
}
149
150
}
150
151
+3
-3
packages/vitest/src/runtime/runners/index.ts
+3
-3
packages/vitest/src/runtime/runners/index.ts
···
62
62
63
63
// patch some methods, so custom runners don't need to call RPC
64
64
const originalOnTaskUpdate = testRunner.onTaskUpdate
65
-
testRunner.onTaskUpdate = async (task) => {
66
-
const p = rpc().onTaskUpdate(task)
67
-
await originalOnTaskUpdate?.call(testRunner, task)
65
+
testRunner.onTaskUpdate = async (task, events) => {
66
+
const p = rpc().onTaskUpdate(task, events)
67
+
await originalOnTaskUpdate?.call(testRunner, task, events)
68
68
return p
69
69
}
70
70
+7
-3
test/cli/fixtures/custom-pool/pool/custom-pool.ts
+7
-3
test/cli/fixtures/custom-pool/pool/custom-pool.ts
···
1
1
import type { RunnerTestFile, RunnerTestCase } from 'vitest'
2
2
import type { ProcessPool, Vitest } from 'vitest/node'
3
3
import { createMethodsRPC } from 'vitest/node'
4
-
import { getTasks } from '@vitest/runner/utils'
4
+
import { getTasks, generateFileHash } from '@vitest/runner/utils'
5
5
import { normalize, relative } from 'pathe'
6
6
7
7
export default (vitest: Vitest): ProcessPool => {
···
20
20
vitest.logger.console.warn('[pool] running tests for', project.name, 'in', normalize(file).toLowerCase().replace(normalize(process.cwd()).toLowerCase(), ''))
21
21
const path = relative(project.config.root, file)
22
22
const taskFile: RunnerTestFile = {
23
-
id: `${path}${project.name}`,
23
+
id: generateFileHash(path, project.config.name),
24
24
name: path,
25
25
mode: 'run',
26
26
meta: {},
···
49
49
}
50
50
taskFile.tasks.push(taskTest)
51
51
await methods.onCollected([taskFile])
52
-
await methods.onTaskUpdate(getTasks(taskFile).map(task => [task.id, task.result, task.meta]))
52
+
await methods.onTaskUpdate(getTasks(taskFile).map(task => [
53
+
task.id,
54
+
task.result,
55
+
task.meta,
56
+
]), [])
53
57
}
54
58
},
55
59
close() {