-
Notifications
You must be signed in to change notification settings - Fork 267
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Added support for dynamic seed generation.
- Loading branch information
1 parent
50df180
commit 10edb1c
Showing
32 changed files
with
559 additions
and
110 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
<div class="highlight"><pre><span></span><span class="k">constant</span><span class="w"> </span><span class="n">global_seed</span><span class="w"> </span><span class="o">:</span><span class="w"> </span><span class="kt">string</span><span class="w"> </span><span class="o">:=</span><span class="w"> </span><span class="n">get_seed</span><span class="p">(</span><span class="n">runner_cfg</span><span class="p">,</span><span class="w"> </span><span class="s">"Optional salt"</span><span class="p">);</span> | ||
</pre></div> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
<div class="highlight"><pre><span></span><span class="n">randomizing_process</span><span class="w"> </span><span class="o">:</span><span class="w"> </span><span class="k">process</span><span class="w"> </span><span class="k">is</span> | ||
<span class="w"> </span><span class="k">variable</span><span class="w"> </span><span class="n">local_seed</span><span class="w"> </span><span class="o">:</span><span class="w"> </span><span class="n">string_seed_t</span><span class="p">;</span><span class="w"> </span><span class="c1">-- string_seed_t = string(1 to 16)</span> | ||
<span class="k">begin</span> | ||
<span class="w"> </span><span class="n">get_seed</span><span class="p">(</span><span class="n">local_seed</span><span class="p">,</span><span class="w"> </span><span class="n">salt</span><span class="w"> </span><span class="o">=></span><span class="w"> </span><span class="n">randomizing_process</span><span class="na">'path_name</span><span class="p">);</span> | ||
</pre></div> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
<div class="highlight"><pre><span></span><span class="n">get_uniform_seed</span><span class="p">(</span><span class="n">seed1</span><span class="p">,</span><span class="w"> </span><span class="n">seed2</span><span class="p">,</span><span class="w"> </span><span class="s">"Optional salt"</span><span class="p">);</span> | ||
|
||
<span class="n">uniform</span><span class="p">(</span><span class="n">seed1</span><span class="p">,</span><span class="w"> </span><span class="n">seed2</span><span class="p">,</span><span class="w"> </span><span class="n">a_random_value</span><span class="p">);</span> | ||
<span class="n">uniform</span><span class="p">(</span><span class="n">seed1</span><span class="p">,</span><span class="w"> </span><span class="n">seed2</span><span class="p">,</span><span class="w"> </span><span class="n">another_random_value</span><span class="p">);</span> | ||
</pre></div> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
<div class="highlight" style="background: #002b36; color: #eee8d5;"><pre style="line-height: 125%; background: #002b36; color: #eee8d5;">> python run.py "lib.tb_seed.Test that fails" --seed fb19f3cca859d69c</pre></div> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,18 +1,19 @@ | ||
<div class="highlight" style="background: #002b36; color: #eee8d5;"><pre style="line-height: 125%; background: #002b36; color: #eee8d5;">> python run.py -v | ||
Starting lib.tb_magic_paths.all | ||
Output file: C:\github\vunit\docs\run\src\vunit_out\test_output\lib.tb_magic_paths.all_243b3c717ce1d4e82490245d1b7e8fe8797f5e94\output.txt | ||
Seed for lib.tb_magic_paths.all: 3680f2e2321cdac3 | ||
0 fs - default - INFO - Directory containing testbench: C:/github/vunit/docs/run/src/ | ||
0 fs - default - INFO - Test output directory: C:/github/vunit/docs/run/src/vunit_out/test_output/lib.tb_magic_paths.all_243b3c717ce1d4e82490245d1b7e8fe8797f5e94/ | ||
simulation stopped @0ms with status 0 | ||
<span style="color: #008000; ">pass</span> (P=1 S=0 F=0 T=1) lib.tb_magic_paths.all (0.5 seconds) | ||
<span style="color: #008000; ">pass</span> (P=1 S=0 F=0 T=1) lib.tb_magic_paths.all (0.6 s) | ||
|
||
==== Summary ================================== | ||
<span style="color: #008000; ">pass</span> lib.tb_magic_paths.all (0.5 seconds) | ||
<span style="color: #008000; ">pass</span> lib.tb_magic_paths.all (0.6 s) | ||
=============================================== | ||
<span style="color: #008000; ">pass</span> 1 of 1 | ||
=============================================== | ||
Total time was 0.5 seconds | ||
Elapsed time was 0.5 seconds | ||
Total time was 0.6 s | ||
Elapsed time was 0.6 s | ||
=============================================== | ||
<span style="color: #008000; ">All passed!</span> | ||
</pre></div> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,15 +1,15 @@ | ||
<div class="highlight" style="background: #002b36; color: #eee8d5;"><pre style="line-height: 125%; background: #002b36; color: #eee8d5;">> python run.py | ||
Starting lib.tb_minimal.all | ||
Output file: C:\github\vunit\docs\run\src\vunit_out\test_output\lib.tb_minimal.all_42aa262c7c96c708ab3f3960f033f2328c642136\output.txt | ||
<span style="color: #008000; ">pass</span> (P=1 S=0 F=0 T=1) lib.tb_minimal.all (0.5 seconds) | ||
<span style="color: #008000; ">pass</span> (P=1 S=0 F=0 T=1) lib.tb_minimal.all (0.6 s) | ||
|
||
==== Summary ============================== | ||
<span style="color: #008000; ">pass</span> lib.tb_minimal.all (0.5 seconds) | ||
<span style="color: #008000; ">pass</span> lib.tb_minimal.all (0.6 s) | ||
=========================================== | ||
<span style="color: #008000; ">pass</span> 1 of 1 | ||
=========================================== | ||
Total time was 0.5 seconds | ||
Elapsed time was 0.5 seconds | ||
Total time was 0.6 s | ||
Elapsed time was 0.6 s | ||
=========================================== | ||
<span style="color: #008000; ">All passed!</span> | ||
</pre></div> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,25 +1,25 @@ | ||
<div class="highlight" style="background: #002b36; color: #eee8d5;"><pre style="line-height: 125%; background: #002b36; color: #eee8d5;">> python run.py | ||
Starting lib.tb_running_test_case.Test scenario A | ||
Output file: C:\github\vunit\docs\run\src\vunit_out\test_output\lib.tb_running_test_case.Test_scenario_A_b118fcdf82c6ba5772e038ce7455962692f50c2a\output.txt | ||
<span style="color: #008000; ">pass</span> (P=1 S=0 F=0 T=3) lib.tb_running_test_case.Test scenario A (0.5 seconds) | ||
<span style="color: #008000; ">pass</span> (P=1 S=0 F=0 T=3) lib.tb_running_test_case.Test scenario A (0.6 s) | ||
|
||
Starting lib.tb_running_test_case.Test scenario B | ||
(22:55:49) Starting lib.tb_running_test_case.Test scenario B | ||
Output file: C:\github\vunit\docs\run\src\vunit_out\test_output\lib.tb_running_test_case.Test_scenario_B_8cd407ec92589901f9b4594c0f27835478242f2c\output.txt | ||
<span style="color: #008000; ">pass</span> (P=2 S=0 F=0 T=3) lib.tb_running_test_case.Test scenario B (0.5 seconds) | ||
<span style="color: #008000; ">pass</span> (P=2 S=0 F=0 T=3) lib.tb_running_test_case.Test scenario B (0.6 s) | ||
|
||
Starting lib.tb_running_test_case.Test something else | ||
(22:55:50) Starting lib.tb_running_test_case.Test something else | ||
Output file: C:\github\vunit\docs\run\src\vunit_out\test_output\lib.tb_running_test_case.Test_something_else_27dcc1aa8d44993b6b2d0b0a017fa6001b4c2aa7\output.txt | ||
<span style="color: #008000; ">pass</span> (P=3 S=0 F=0 T=3) lib.tb_running_test_case.Test something else (0.5 seconds) | ||
<span style="color: #008000; ">pass</span> (P=3 S=0 F=0 T=3) lib.tb_running_test_case.Test something else (0.6 s) | ||
|
||
==== Summary ======================================================== | ||
<span style="color: #008000; ">pass</span> lib.tb_running_test_case.Test scenario A (0.5 seconds) | ||
<span style="color: #008000; ">pass</span> lib.tb_running_test_case.Test scenario B (0.5 seconds) | ||
<span style="color: #008000; ">pass</span> lib.tb_running_test_case.Test something else (0.5 seconds) | ||
<span style="color: #008000; ">pass</span> lib.tb_running_test_case.Test scenario A (0.6 s) | ||
<span style="color: #008000; ">pass</span> lib.tb_running_test_case.Test scenario B (0.6 s) | ||
<span style="color: #008000; ">pass</span> lib.tb_running_test_case.Test something else (0.6 s) | ||
===================================================================== | ||
<span style="color: #008000; ">pass</span> 3 of 3 | ||
===================================================================== | ||
Total time was 1.6 seconds | ||
Elapsed time was 1.6 seconds | ||
Total time was 1.8 s | ||
Elapsed time was 1.8 s | ||
===================================================================== | ||
<span style="color: #008000; ">All passed!</span> | ||
</pre></div> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,26 +1,23 @@ | ||
<div class="highlight" style="background: #002b36; color: #eee8d5;"><pre style="line-height: 125%; background: #002b36; color: #eee8d5;">> python run.py | ||
Starting lib.tb_stop_level.Test that fails multiple times but doesn't stop | ||
Output file: C:\github\vunit\docs\run\src\vunit_out\test_output\lib.tb_stop_level.Test_that_fails_multiple_times_but_doesn't_stop_d08f48d859442d0bc71e2bcdd8b429119f7cc17c\output.txt | ||
Seed for lib.tb_stop_level.Test that fails multiple times but doesn't stop: 7a0e979de335b966 | ||
0 fs - check - ERROR - Equality check failed - Got 17. Expected 18. | ||
0 fs - check - ERROR - Equality check failed - Got 17. Expected 19. | ||
FAILURE - Logger check has 2 errors | ||
C:\github\vunit\vunit\vhdl\core\src\core_pkg.vhd:84:7:@0ms:(report failure): Final log check failed | ||
C:\ghdl\bin\ghdl.exe:error: report failed | ||
in process .tb_stop_level(tb).test_runner | ||
from: vunit_lib.logger_pkg.final_log_check at logger_pkg-body.vhd:1249 | ||
from: vunit_lib.run_pkg.test_runner_cleanup at run.vhd:114 | ||
from: process lib.tb_stop_level(tb).test_runner at tb_stop_level.vhd:29 | ||
C:\ghdl\bin\ghdl.exe:error: simulation failed | ||
<span style="color: #cb4b16; ">fail</span> (P=0 S=0 F=1 T=1) lib.tb_stop_level.Test that fails multiple times but doesn't stop (0.5 seconds) | ||
C:\github\vunit\vunit\vhdl\core\src\core_pkg.vhd:85:7:@0ms:(report failure): Final log check failed | ||
ghdl:error: report failed | ||
ghdl:error: simulation failed | ||
<span style="color: #cb4b16; ">fail</span> (P=0 S=0 F=1 T=1) lib.tb_stop_level.Test that fails multiple times but doesn't stop (0.6 s) | ||
|
||
==== Summary ============================================================================= | ||
<span style="color: #cb4b16; ">fail</span> lib.tb_stop_level.Test that fails multiple times but doesn't stop (0.5 seconds) | ||
<span style="color: #cb4b16; ">fail</span> lib.tb_stop_level.Test that fails multiple times but doesn't stop (0.6 s) | ||
========================================================================================== | ||
<span style="color: #008000; ">pass</span> 0 of 1 | ||
<span style="color: #cb4b16; ">fail</span> 1 of 1 | ||
========================================================================================== | ||
Total time was 0.5 seconds | ||
Elapsed time was 0.5 seconds | ||
Total time was 0.6 s | ||
Elapsed time was 0.6 s | ||
========================================================================================== | ||
<span style="color: #cb4b16; ">Some failed!</span> | ||
</pre></div> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,40 +1,43 @@ | ||
<div class="highlight" style="background: #002b36; color: #eee8d5;"><pre style="line-height: 125%; background: #002b36; color: #eee8d5;">> python run.py | ||
Starting lib.tb_stopping_failure.Test that fails on an assert | ||
Output file: C:\repos\vunit\docs\run\src\vunit_out\test_output\lib.tb_stopping_failure.Test_that_fails_on_an_assert_f53b930e2c7649bc33253af52f8ea89a9c05f07b\output.txt | ||
C:\repos\vunit\docs\run\src\tb_stopping_failure.vhd:24:9:@0ms:(assertion error): Assertion violation | ||
Output file: C:\github\vunit\docs\run\src\vunit_out\test_output\lib.tb_stopping_failure.Test_that_fails_on_an_assert_f53b930e2c7649bc33253af52f8ea89a9c05f07b\output.txt | ||
Seed for lib.tb_stopping_failure.Test that fails on an assert: a866d0986a663e6b | ||
C:\github\vunit\docs\run\src\tb_stopping_failure.vhd:24:9:@0ms:(assertion error): Assertion violation | ||
ghdl:error: assertion failed | ||
ghdl:error: simulation failed | ||
<span style="color: #cb4b16; ">fail</span> (P=0 S=0 F=1 T=4) lib.tb_stopping_failure.Test that fails on an assert (0.5 s) | ||
<span style="color: #cb4b16; ">fail</span> (P=0 S=0 F=1 T=4) lib.tb_stopping_failure.Test that fails on an assert (0.6 s) | ||
|
||
(11:35:31) Starting lib.tb_stopping_failure.Test that crashes on boundary problems | ||
Output file: C:\repos\vunit\docs\run\src\vunit_out\test_output\lib.tb_stopping_failure.Test_that_crashes_on_boundary_problems_b53105615efefaa16d0cf9ee1bad37b5d3369e95\output.txt | ||
ghdl:error: index (314) out of bounds (1 to 17) at C:\repos\vunit\docs\run\src\tb_stopping_failure.vhd:26 | ||
(22:55:55) Starting lib.tb_stopping_failure.Test that crashes on boundary problems | ||
Output file: C:\github\vunit\docs\run\src\vunit_out\test_output\lib.tb_stopping_failure.Test_that_crashes_on_boundary_problems_b53105615efefaa16d0cf9ee1bad37b5d3369e95\output.txt | ||
Seed for lib.tb_stopping_failure.Test that crashes on boundary problems: cbcb543764149e78 | ||
ghdl:error: index (340) out of bounds (1 to 17) at C:\github\vunit\docs\run\src\tb_stopping_failure.vhd:26 | ||
ghdl:error: simulation failed | ||
<span style="color: #cb4b16; ">fail</span> (P=0 S=0 F=2 T=4) lib.tb_stopping_failure.Test that crashes on boundary problems (0.5 s) | ||
<span style="color: #cb4b16; ">fail</span> (P=0 S=0 F=2 T=4) lib.tb_stopping_failure.Test that crashes on boundary problems (0.6 s) | ||
|
||
(11:35:31) Starting lib.tb_stopping_failure.Test that fails on VUnit check procedure | ||
Output file: C:\repos\vunit\docs\run\src\vunit_out\test_output\lib.tb_stopping_failure.Test_that_fails_on_VUnit_check_procedure_717a6f8ff044e3d5fa7d7d3ec5a32971d74864dd\output.txt | ||
(22:55:55) Starting lib.tb_stopping_failure.Test that fails on VUnit check procedure | ||
Output file: C:\github\vunit\docs\run\src\vunit_out\test_output\lib.tb_stopping_failure.Test_that_fails_on_VUnit_check_procedure_717a6f8ff044e3d5fa7d7d3ec5a32971d74864dd\output.txt | ||
Seed for lib.tb_stopping_failure.Test that fails on VUnit check procedure: 6581894e9ea614b1 | ||
0 fs - check - ERROR - Equality check failed - Got 17. Expected 18. | ||
C:\repos\vunit\vunit\vhdl\core\src\core_pkg.vhd:85:7:@0ms:(report failure): Stop simulation on log level error | ||
C:\github\vunit\vunit\vhdl\core\src\core_pkg.vhd:85:7:@0ms:(report failure): Stop simulation on log level error | ||
ghdl:error: report failed | ||
ghdl:error: simulation failed | ||
<span style="color: #cb4b16; ">fail</span> (P=0 S=0 F=3 T=4) lib.tb_stopping_failure.Test that fails on VUnit check procedure (0.5 s) | ||
<span style="color: #cb4b16; ">fail</span> (P=0 S=0 F=3 T=4) lib.tb_stopping_failure.Test that fails on VUnit check procedure (0.6 s) | ||
|
||
(11:35:32) Starting lib.tb_stopping_failure.Test that a warning passes | ||
Output file: C:\repos\vunit\docs\run\src\vunit_out\test_output\lib.tb_stopping_failure.Test_that_a_warning_passes_7db91f3b27aea5f89e74e39ea51ce6d61558674e\output.txt | ||
<span style="color: #008000; ">pass</span> (P=1 S=0 F=3 T=4) lib.tb_stopping_failure.Test that a warning passes (0.4 s) | ||
(22:55:56) Starting lib.tb_stopping_failure.Test that a warning passes | ||
Output file: C:\github\vunit\docs\run\src\vunit_out\test_output\lib.tb_stopping_failure.Test_that_a_warning_passes_7db91f3b27aea5f89e74e39ea51ce6d61558674e\output.txt | ||
<span style="color: #008000; ">pass</span> (P=1 S=0 F=3 T=4) lib.tb_stopping_failure.Test that a warning passes (0.6 s) | ||
|
||
==== Summary ============================================================================ | ||
<span style="color: #008000; ">pass</span> lib.tb_stopping_failure.Test that a warning passes (0.4 s) | ||
<span style="color: #cb4b16; ">fail</span> lib.tb_stopping_failure.Test that fails on an assert (0.5 s) | ||
<span style="color: #cb4b16; ">fail</span> lib.tb_stopping_failure.Test that crashes on boundary problems (0.5 s) | ||
<span style="color: #cb4b16; ">fail</span> lib.tb_stopping_failure.Test that fails on VUnit check procedure (0.5 s) | ||
<span style="color: #008000; ">pass</span> lib.tb_stopping_failure.Test that a warning passes (0.6 s) | ||
<span style="color: #cb4b16; ">fail</span> lib.tb_stopping_failure.Test that fails on an assert (0.6 s) | ||
<span style="color: #cb4b16; ">fail</span> lib.tb_stopping_failure.Test that crashes on boundary problems (0.6 s) | ||
<span style="color: #cb4b16; ">fail</span> lib.tb_stopping_failure.Test that fails on VUnit check procedure (0.6 s) | ||
========================================================================================= | ||
<span style="color: #008000; ">pass</span> 1 of 4 | ||
<span style="color: #cb4b16; ">fail</span> 3 of 4 | ||
========================================================================================= | ||
Total time was 1.8 s | ||
Elapsed time was 1.8 s | ||
Total time was 2.4 s | ||
Elapsed time was 2.4 s | ||
========================================================================================= | ||
<span style="color: #cb4b16; ">Some failed!</span> | ||
</pre></div> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,20 +1,20 @@ | ||
<div class="highlight" style="background: #002b36; color: #eee8d5;"><pre style="line-height: 125%; background: #002b36; color: #eee8d5;">> python run.py | ||
Starting lib.tb_with_lower_level_control.Test something | ||
Output file: C:\github\vunit\docs\run\src\vunit_out\test_output\lib.tb_with_lower_level_control.Test_something_a280708dc2f527dff5e45e7a7d2b48df39330b4f\output.txt | ||
<span style="color: #008000; ">pass</span> (P=1 S=0 F=0 T=2) lib.tb_with_lower_level_control.Test something (0.5 seconds) | ||
<span style="color: #008000; ">pass</span> (P=1 S=0 F=0 T=2) lib.tb_with_lower_level_control.Test something (0.6 s) | ||
|
||
Starting lib.tb_with_lower_level_control.Test something else | ||
(22:55:51) Starting lib.tb_with_lower_level_control.Test something else | ||
Output file: C:\github\vunit\docs\run\src\vunit_out\test_output\lib.tb_with_lower_level_control.Test_something_else_e47dc199cab8c612d9a0f46b8be7d141576fc970\output.txt | ||
<span style="color: #008000; ">pass</span> (P=2 S=0 F=0 T=2) lib.tb_with_lower_level_control.Test something else (0.5 seconds) | ||
<span style="color: #008000; ">pass</span> (P=2 S=0 F=0 T=2) lib.tb_with_lower_level_control.Test something else (0.6 s) | ||
|
||
==== Summary =============================================================== | ||
<span style="color: #008000; ">pass</span> lib.tb_with_lower_level_control.Test something (0.5 seconds) | ||
<span style="color: #008000; ">pass</span> lib.tb_with_lower_level_control.Test something else (0.5 seconds) | ||
<span style="color: #008000; ">pass</span> lib.tb_with_lower_level_control.Test something (0.6 s) | ||
<span style="color: #008000; ">pass</span> lib.tb_with_lower_level_control.Test something else (0.6 s) | ||
============================================================================ | ||
<span style="color: #008000; ">pass</span> 2 of 2 | ||
============================================================================ | ||
Total time was 1.1 seconds | ||
Elapsed time was 1.1 seconds | ||
Total time was 1.2 s | ||
Elapsed time was 1.2 s | ||
============================================================================ | ||
<span style="color: #008000; ">All passed!</span> | ||
</pre></div> |
Oops, something went wrong.