File shows in indirect change

[vbmade2000]

I am using cargo-llvm-cov utility with nextest. During multiple runs, a Rust source file shows uncovered (even though it is covered). This happens intermittently.

Some times even if change contains unrelated files (markdown files), this file shows change. Because of these reasons, the issue is hard to reproducible. Need help in understanding why this might occur.

[taiki-e]

One possibility is if your test relies on some kind of randomness. For example, tests that using random generators such as rand, property tests such as quickcheck, concurrent/parallel tests with not enough iterations, tests that depend on timing and succeed even if they are early or late, etc.

Also, if there are components that depend on some global state, the parallelism of the test itself may be causing problems. In that case, this could be resolved by setting RUST_TEST_THREADS=1 and/or NEXTEST_TEST_THREADS=1 env vars.

Also, can it be reproduced without nextest? Or is it specific to using it with nextest?

[vbmade2000]

1. The code is not public so won't be able to provide the repo.
2. There is no randomness. No use of any random data generation.
3. No use of property tests such as quickcheck
4. Concurrent/parallel tests with not enough iterations.
   No sure. Tests are isolated and executed using #[tokio::test(flavor = "current_thread", start_paused = true)] so I assume they run on single thread. Uses mock objects with predefined output.
5. Testing using RUST_TEST_THREADS=1 and/or NEXTEST_TEST_THREADS=1 env vars.
   Not checked yet. The issue is so random that to check each possibility I have to run them many times. At one time if I run them 10 times, it will show the issue in one of the iterations and another time if I run it 20 and everything will be fine.
6. Also, can it be reproduced without nextest? Or is it specific to using it with nextest?
   Not sure if it is specific to nextest. Will run without nextest.