chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
feat: support any, all and not component/tag filters on Query (#3380)
Improves `ex.Query` to support "any" and "not" filters for components, as well as all/any/not tags filters. The intention is for this to replace `ex.TagQuery`, but that is not (yet) in this PR.
With this change, this would much easily allow systems to exclude entities with tags. For example, GraphicsSystem could now exclude any Entity with an `ex.offscreen` tag instead of manually checking in its update. This could also be used for #3076 by adding a tag to an Entity while paused that excludes it from systems.
I also removed the restriction of throwing on empty component queries, as I think it could make sense to have a system that applies for every Entity. I can change this, though.
---
There's also some code quality changes here.
- I added an eslint rule to prevent `fit` tests (as I've accidentally committed those before)
- an easy way to enable console logs in karma by doing `CAPTURE_CONSOLE=true npm run test` as I'd previously have to edit the karma config file.
- fixed some ts errors in collider tests
===:clipboard: PR Checklist :clipboard:===
- [ ] :pushpin: issue exists in github for these changes
- [x] :microscope: existing tests still pass
- [x] :see_no_evil: code conforms to the [style guide](https://github.com/excaliburjs/Excalibur/blob/main/STYLEGUIDE.md)
- [x] :triangular_ruler: new tests written and passing / old tests updated with new scenario(s)
- [x] :page_facing_up: changelog entry added (or not needed)
==================
<!-- If you're closing an issue with this pull request, or contributing a significant change, please include your changes in the appropriate section of CHANGELOG.md as outlined in https://github.com/excaliburjs/Excalibur/blob/main/.github/CONTRIBUTING.md#creating-a-pull-request. -->
<!--Please format your pull request title according to our commit message styleguide: https://github.com/excaliburjs/Excalibur/blob/main/.github/CONTRIBUTING.md#commit-messages -->
<!-- Thanks again! -->
<!--------------------------------------------------------------------------------------------->
Closes #
## Changes:
- Queries can now take additional options to filter in/out by components or tags.
```ts
const query = new Query({
// all fields are optional
components: {
all: [ComponentA, ComponentB] as const, // important for type safety!
any: [ComponentC, ComponentD] as const, // important for type safety!
not: [ComponentE]
},
tags: {
all: ['tagA', 'tagB'],
any: ['tagC', 'tagD'],
not: ['tagE']
}
})
// previous constructor type still works and is shorthand for components.all
new Query([ComponentA, ComponentB] as const)
```
- Queries can now match all entities by specifying no filters
```ts
const query = new Query({})
```
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
feat: support any, all and not component/tag filters on Query (#3380)
Improves `ex.Query` to support "any" and "not" filters for components, as well as all/any/not tags filters. The intention is for this to replace `ex.TagQuery`, but that is not (yet) in this PR.
With this change, this would much easily allow systems to exclude entities with tags. For example, GraphicsSystem could now exclude any Entity with an `ex.offscreen` tag instead of manually checking in its update. This could also be used for #3076 by adding a tag to an Entity while paused that excludes it from systems.
I also removed the restriction of throwing on empty component queries, as I think it could make sense to have a system that applies for every Entity. I can change this, though.
---
There's also some code quality changes here.
- I added an eslint rule to prevent `fit` tests (as I've accidentally committed those before)
- an easy way to enable console logs in karma by doing `CAPTURE_CONSOLE=true npm run test` as I'd previously have to edit the karma config file.
- fixed some ts errors in collider tests
===:clipboard: PR Checklist :clipboard:===
- [ ] :pushpin: issue exists in github for these changes
- [x] :microscope: existing tests still pass
- [x] :see_no_evil: code conforms to the [style guide](https://github.com/excaliburjs/Excalibur/blob/main/STYLEGUIDE.md)
- [x] :triangular_ruler: new tests written and passing / old tests updated with new scenario(s)
- [x] :page_facing_up: changelog entry added (or not needed)
==================
<!-- If you're closing an issue with this pull request, or contributing a significant change, please include your changes in the appropriate section of CHANGELOG.md as outlined in https://github.com/excaliburjs/Excalibur/blob/main/.github/CONTRIBUTING.md#creating-a-pull-request. -->
<!--Please format your pull request title according to our commit message styleguide: https://github.com/excaliburjs/Excalibur/blob/main/.github/CONTRIBUTING.md#commit-messages -->
<!-- Thanks again! -->
<!--------------------------------------------------------------------------------------------->
Closes #
## Changes:
- Queries can now take additional options to filter in/out by components or tags.
```ts
const query = new Query({
// all fields are optional
components: {
all: [ComponentA, ComponentB] as const, // important for type safety!
any: [ComponentC, ComponentD] as const, // important for type safety!
not: [ComponentE]
},
tags: {
all: ['tagA', 'tagB'],
any: ['tagC', 'tagD'],
not: ['tagE']
}
})
// previous constructor type still works and is shorthand for components.all
new Query([ComponentA, ComponentB] as const)
```
- Queries can now match all entities by specifying no filters
```ts
const query = new Query({})
```
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
chore: migrate tests to vitest (#3381)
managed to migrate the engine leak & memory reporters, although it took a bit of rigamarole. Mainly because, as opposed to karma, the reporter runs in node and not the browser. So I have to track the data needed separately in some global hooks _within_ the browser environment, which the reporter then reads and creates the logs if needed.
However, it seems the memory tracking is regularly reporting >1 mb, so I'm not sure if it's tracking properly or if things have changed with the new browsers. My only theory is that because the timing where memory is read is during an `afterEach` _before_ the test's `afterEach`, it's running before any potential cleanup. The timing of this is not something that's easy to control, unfortunately. I did try to prove this theory by doing the memory analysis on the next test's `beforeEach`, but it didnt seem to change the results, so it may not be an issue.
Implementation is done in `src/spec/vitest/__reporters__/memory.ts` and `src/spec/vitest/__reporters/memory.setup.ts`
---
I noticed CouroutineSpec was an offender for >10mb memory, which spins up additional engines. I added a short 100ms wait after the engine.dispose and the memory usage did drop, so it does seem like this is prone to scheduled garbage collecting.
---
I was able to run the garbage collector (if exposed, currently only on chrome) and this makes the reports more accurate
chore: split out vitest tests into visual & unit (#3508)
This splits out our vitest test suite into `unit` and `visual` tests. `unit` will run on chromium, while `visual` will run on chrome. I've removed firefox/webkit browsers as we were not running these in CI, and there's a discussion to be had about whether they even make sense to use in unit tests, or are they actually only valuable for visual tests. If the latter, we need to figure out how to not rely on Chrome for consistent screenshotting.
Tests that are visual simply need to contain `@visual` somewhere in it's test name or `describe` name. Unit tests will run for all tests without `@visual`, while visual tests will run all that contain `@visual`. I had considered splitting them out into different files under different folders, but it's nice to re-use the same setup/teardown logic. Otherwise I had to copy & paste all of that in both places.
I also made some other tweaks to our vitest config:
- I've removed the conditional exemption of `--disable-gpu` flag for Chromium on macOS, which although slower it ensures the visual tests pass. I suppose this is because it uses a software renderer.
- Changed the test pooling from `thread` to the default. I never saw a difference in speed but in theory the default (`fork`) is supposed to be faster
- Enabled fileParallelism for tests. I had run into WebGL context issues before, but now they seem to be gone, so this should lead to faster tests
- I've switched from Chrome to actual Chromium for the visual tests, and then lowered the tolerance on the TextSpec so that it could pass both in Windows/Ubuntu CI as well as macOS locally. I'm fine to undo this, but a lower tolerance on font tests might make sense to do anyway and this frees us from relying on Windows+Chrome to run these tests.