Skipped Tests #6376
Replies: 2 comments 2 replies
-
I haven't gone through all the skipped tests but the main reason seems to be that those are non deterministic / flaky. We could consider replacing the default skip method with a custom one that only skips those tests if run in the CI. Alternatively, we have to take some time and review skipped tests, maybe a resolution could be to add retries to make them consistently pass. If a test does not pass after multiple runs it's rather an issue with the test itself and should consider rewriting or even removing it. |
Beta Was this translation helpful? Give feedback.
-
If there is a skipped test without a definite comment when to bring it back or reference to an issue, then it's better to just remove it. |
Beta Was this translation helpful? Give feedback.
-
@nazarhussain brought up a good point that we have a ton of skipped tests and it pollutes the eslint output. He attempted to remove the warnings in #6311 but we migrated those changes out of that PR to limit the scope a bit (was a big PR). @tuyennhv just put up a PR #6368 that is a valid case where a test wants to be skipped.
Figured I would open this up to the floor for discussion and if we want to formalize/memorialize how we skip tests.
A penny for your thoughts? 🪙
Beta Was this translation helpful? Give feedback.
All reactions