Skip to content

Commit

Permalink
Added support for dynamic seed generation.
Browse files Browse the repository at this point in the history
  • Loading branch information
LarsAsplund committed Jan 7, 2025
1 parent 50df180 commit 10edb1c
Show file tree
Hide file tree
Showing 32 changed files with 559 additions and 110 deletions.
2 changes: 2 additions & 0 deletions docs/run/img/get_seed_and_runner_cfg.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
<div class="highlight"><pre><span></span><span class="k">constant</span><span class="w"> </span><span class="n">global_seed</span><span class="w"> </span><span class="o">:</span><span class="w"> </span><span class="kt">string</span><span class="w"> </span><span class="o">:=</span><span class="w"> </span><span class="n">get_seed</span><span class="p">(</span><span class="n">runner_cfg</span><span class="p">,</span><span class="w"> </span><span class="s">&quot;Optional salt&quot;</span><span class="p">);</span>
</pre></div>
5 changes: 5 additions & 0 deletions docs/run/img/get_seed_wo_runner_cfg.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
<div class="highlight"><pre><span></span><span class="n">randomizing_process</span><span class="w"> </span><span class="o">:</span><span class="w"> </span><span class="k">process</span><span class="w"> </span><span class="k">is</span>
<span class="w"> </span><span class="k">variable</span><span class="w"> </span><span class="n">local_seed</span><span class="w"> </span><span class="o">:</span><span class="w"> </span><span class="n">string_seed_t</span><span class="p">;</span><span class="w"> </span><span class="c1">-- string_seed_t = string(1 to 16)</span>
<span class="k">begin</span>
<span class="w"> </span><span class="n">get_seed</span><span class="p">(</span><span class="n">local_seed</span><span class="p">,</span><span class="w"> </span><span class="n">salt</span><span class="w"> </span><span class="o">=&gt;</span><span class="w"> </span><span class="n">randomizing_process</span><span class="na">&#39;path_name</span><span class="p">);</span>
</pre></div>
5 changes: 5 additions & 0 deletions docs/run/img/get_uniform_seed.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
<div class="highlight"><pre><span></span><span class="n">get_uniform_seed</span><span class="p">(</span><span class="n">seed1</span><span class="p">,</span><span class="w"> </span><span class="n">seed2</span><span class="p">,</span><span class="w"> </span><span class="s">&quot;Optional salt&quot;</span><span class="p">);</span>

<span class="n">uniform</span><span class="p">(</span><span class="n">seed1</span><span class="p">,</span><span class="w"> </span><span class="n">seed2</span><span class="p">,</span><span class="w"> </span><span class="n">a_random_value</span><span class="p">);</span>
<span class="n">uniform</span><span class="p">(</span><span class="n">seed1</span><span class="p">,</span><span class="w"> </span><span class="n">seed2</span><span class="p">,</span><span class="w"> </span><span class="n">another_random_value</span><span class="p">);</span>
</pre></div>
8 changes: 7 additions & 1 deletion docs/run/img/list.html
Original file line number Diff line number Diff line change
@@ -1,16 +1,22 @@
<div class="highlight" style="background: #002b36; color: #eee8d5;"><pre style="line-height: 125%; background: #002b36; color: #eee8d5;">> python run.py --list
lib.tb_fail_on_warning.Test that fails on an assert
lib.tb_fail_on_warning.Test that crashes on boundary problems
lib.tb_fail_on_warning.Test that fails on VUnit check procedure
lib.tb_fail_on_warning.Test that a warning passes
lib.tb_magic_paths.all
lib.tb_minimal.all
lib.tb_running_test_case.Test scenario A
lib.tb_running_test_case.Test scenario B
lib.tb_running_test_case.Test something else
lib.tb_run_all_in_same_sim.Test to_string for integer again
lib.tb_run_all_in_same_sim.Test to_string for boolean again
lib.tb_seed.all
lib.tb_standalone.Test that fails on VUnit check procedure
lib.tb_standalone.Test to_string for boolean
lib.tb_stopping_failure.Test that fails on an assert
lib.tb_stopping_failure.Test that crashes on boundary problems
lib.tb_stopping_failure.Test that fails on VUnit check procedure
lib.tb_stopping_failure.Test that a warning passes
lib.tb_stop_level.Test that fails multiple times but doesn't stop
lib.tb_with_lower_level_control.Test something
lib.tb_with_lower_level_control.Test something else
Expand All @@ -19,5 +25,5 @@
lib.tb_with_watchdog.Test that stalls
lib.tb_with_watchdog.Test to_string for boolean
lib.tb_with_watchdog.Test that needs longer timeout
Listed 20 tests
Listed 26 tests
</pre></div>
1 change: 1 addition & 0 deletions docs/run/img/seed_option.html
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
<div class="highlight" style="background: #002b36; color: #eee8d5;"><pre style="line-height: 125%; background: #002b36; color: #eee8d5;">> python run.py "lib.tb_seed.Test that fails" --seed fb19f3cca859d69c</pre></div>
9 changes: 5 additions & 4 deletions docs/run/img/tb_magic_paths_stdout.html
Original file line number Diff line number Diff line change
@@ -1,18 +1,19 @@
<div class="highlight" style="background: #002b36; color: #eee8d5;"><pre style="line-height: 125%; background: #002b36; color: #eee8d5;">> python run.py -v
Starting lib.tb_magic_paths.all
Output file: C:\github\vunit\docs\run\src\vunit_out\test_output\lib.tb_magic_paths.all_243b3c717ce1d4e82490245d1b7e8fe8797f5e94\output.txt
Seed for lib.tb_magic_paths.all: 3680f2e2321cdac3
0 fs - default - INFO - Directory containing testbench: C:/github/vunit/docs/run/src/
0 fs - default - INFO - Test output directory: C:/github/vunit/docs/run/src/vunit_out/test_output/lib.tb_magic_paths.all_243b3c717ce1d4e82490245d1b7e8fe8797f5e94/
simulation stopped @0ms with status 0
<span style="color: #008000; ">pass</span> (P=1 S=0 F=0 T=1) lib.tb_magic_paths.all (0.5 seconds)
<span style="color: #008000; ">pass</span> (P=1 S=0 F=0 T=1) lib.tb_magic_paths.all (0.6 s)

==== Summary ==================================
<span style="color: #008000; ">pass</span> lib.tb_magic_paths.all (0.5 seconds)
<span style="color: #008000; ">pass</span> lib.tb_magic_paths.all (0.6 s)
===============================================
<span style="color: #008000; ">pass</span> 1 of 1
===============================================
Total time was 0.5 seconds
Elapsed time was 0.5 seconds
Total time was 0.6 s
Elapsed time was 0.6 s
===============================================
<span style="color: #008000; ">All passed!</span>
</pre></div>
8 changes: 4 additions & 4 deletions docs/run/img/tb_minimal_stdout.html
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
<div class="highlight" style="background: #002b36; color: #eee8d5;"><pre style="line-height: 125%; background: #002b36; color: #eee8d5;">> python run.py
Starting lib.tb_minimal.all
Output file: C:\github\vunit\docs\run\src\vunit_out\test_output\lib.tb_minimal.all_42aa262c7c96c708ab3f3960f033f2328c642136\output.txt
<span style="color: #008000; ">pass</span> (P=1 S=0 F=0 T=1) lib.tb_minimal.all (0.5 seconds)
<span style="color: #008000; ">pass</span> (P=1 S=0 F=0 T=1) lib.tb_minimal.all (0.6 s)

==== Summary ==============================
<span style="color: #008000; ">pass</span> lib.tb_minimal.all (0.5 seconds)
<span style="color: #008000; ">pass</span> lib.tb_minimal.all (0.6 s)
===========================================
<span style="color: #008000; ">pass</span> 1 of 1
===========================================
Total time was 0.5 seconds
Elapsed time was 0.5 seconds
Total time was 0.6 s
Elapsed time was 0.6 s
===========================================
<span style="color: #008000; ">All passed!</span>
</pre></div>
20 changes: 10 additions & 10 deletions docs/run/img/tb_running_test_case_stdout.html
Original file line number Diff line number Diff line change
@@ -1,25 +1,25 @@
<div class="highlight" style="background: #002b36; color: #eee8d5;"><pre style="line-height: 125%; background: #002b36; color: #eee8d5;">> python run.py
Starting lib.tb_running_test_case.Test scenario A
Output file: C:\github\vunit\docs\run\src\vunit_out\test_output\lib.tb_running_test_case.Test_scenario_A_b118fcdf82c6ba5772e038ce7455962692f50c2a\output.txt
<span style="color: #008000; ">pass</span> (P=1 S=0 F=0 T=3) lib.tb_running_test_case.Test scenario A (0.5 seconds)
<span style="color: #008000; ">pass</span> (P=1 S=0 F=0 T=3) lib.tb_running_test_case.Test scenario A (0.6 s)

Starting lib.tb_running_test_case.Test scenario B
(22:55:49) Starting lib.tb_running_test_case.Test scenario B
Output file: C:\github\vunit\docs\run\src\vunit_out\test_output\lib.tb_running_test_case.Test_scenario_B_8cd407ec92589901f9b4594c0f27835478242f2c\output.txt
<span style="color: #008000; ">pass</span> (P=2 S=0 F=0 T=3) lib.tb_running_test_case.Test scenario B (0.5 seconds)
<span style="color: #008000; ">pass</span> (P=2 S=0 F=0 T=3) lib.tb_running_test_case.Test scenario B (0.6 s)

Starting lib.tb_running_test_case.Test something else
(22:55:50) Starting lib.tb_running_test_case.Test something else
Output file: C:\github\vunit\docs\run\src\vunit_out\test_output\lib.tb_running_test_case.Test_something_else_27dcc1aa8d44993b6b2d0b0a017fa6001b4c2aa7\output.txt
<span style="color: #008000; ">pass</span> (P=3 S=0 F=0 T=3) lib.tb_running_test_case.Test something else (0.5 seconds)
<span style="color: #008000; ">pass</span> (P=3 S=0 F=0 T=3) lib.tb_running_test_case.Test something else (0.6 s)

==== Summary ========================================================
<span style="color: #008000; ">pass</span> lib.tb_running_test_case.Test scenario A (0.5 seconds)
<span style="color: #008000; ">pass</span> lib.tb_running_test_case.Test scenario B (0.5 seconds)
<span style="color: #008000; ">pass</span> lib.tb_running_test_case.Test something else (0.5 seconds)
<span style="color: #008000; ">pass</span> lib.tb_running_test_case.Test scenario A (0.6 s)
<span style="color: #008000; ">pass</span> lib.tb_running_test_case.Test scenario B (0.6 s)
<span style="color: #008000; ">pass</span> lib.tb_running_test_case.Test something else (0.6 s)
=====================================================================
<span style="color: #008000; ">pass</span> 3 of 3
=====================================================================
Total time was 1.6 seconds
Elapsed time was 1.6 seconds
Total time was 1.8 s
Elapsed time was 1.8 s
=====================================================================
<span style="color: #008000; ">All passed!</span>
</pre></div>
19 changes: 8 additions & 11 deletions docs/run/img/tb_stop_level_stdout.html
Original file line number Diff line number Diff line change
@@ -1,26 +1,23 @@
<div class="highlight" style="background: #002b36; color: #eee8d5;"><pre style="line-height: 125%; background: #002b36; color: #eee8d5;">> python run.py
Starting lib.tb_stop_level.Test that fails multiple times but doesn't stop
Output file: C:\github\vunit\docs\run\src\vunit_out\test_output\lib.tb_stop_level.Test_that_fails_multiple_times_but_doesn't_stop_d08f48d859442d0bc71e2bcdd8b429119f7cc17c\output.txt
Seed for lib.tb_stop_level.Test that fails multiple times but doesn't stop: 7a0e979de335b966
0 fs - check - ERROR - Equality check failed - Got 17. Expected 18.
0 fs - check - ERROR - Equality check failed - Got 17. Expected 19.
FAILURE - Logger check has 2 errors
C:\github\vunit\vunit\vhdl\core\src\core_pkg.vhd:84:7:@0ms:(report failure): Final log check failed
C:\ghdl\bin\ghdl.exe:error: report failed
in process .tb_stop_level(tb).test_runner
from: vunit_lib.logger_pkg.final_log_check at logger_pkg-body.vhd:1249
from: vunit_lib.run_pkg.test_runner_cleanup at run.vhd:114
from: process lib.tb_stop_level(tb).test_runner at tb_stop_level.vhd:29
C:\ghdl\bin\ghdl.exe:error: simulation failed
<span style="color: #cb4b16; ">fail</span> (P=0 S=0 F=1 T=1) lib.tb_stop_level.Test that fails multiple times but doesn't stop (0.5 seconds)
C:\github\vunit\vunit\vhdl\core\src\core_pkg.vhd:85:7:@0ms:(report failure): Final log check failed
ghdl:error: report failed
ghdl:error: simulation failed
<span style="color: #cb4b16; ">fail</span> (P=0 S=0 F=1 T=1) lib.tb_stop_level.Test that fails multiple times but doesn't stop (0.6 s)

==== Summary =============================================================================
<span style="color: #cb4b16; ">fail</span> lib.tb_stop_level.Test that fails multiple times but doesn't stop (0.5 seconds)
<span style="color: #cb4b16; ">fail</span> lib.tb_stop_level.Test that fails multiple times but doesn't stop (0.6 s)
==========================================================================================
<span style="color: #008000; ">pass</span> 0 of 1
<span style="color: #cb4b16; ">fail</span> 1 of 1
==========================================================================================
Total time was 0.5 seconds
Elapsed time was 0.5 seconds
Total time was 0.6 s
Elapsed time was 0.6 s
==========================================================================================
<span style="color: #cb4b16; ">Some failed!</span>
</pre></div>
43 changes: 23 additions & 20 deletions docs/run/img/tb_stopping_failure_stdout.html
Original file line number Diff line number Diff line change
@@ -1,40 +1,43 @@
<div class="highlight" style="background: #002b36; color: #eee8d5;"><pre style="line-height: 125%; background: #002b36; color: #eee8d5;">> python run.py
Starting lib.tb_stopping_failure.Test that fails on an assert
Output file: C:\repos\vunit\docs\run\src\vunit_out\test_output\lib.tb_stopping_failure.Test_that_fails_on_an_assert_f53b930e2c7649bc33253af52f8ea89a9c05f07b\output.txt
C:\repos\vunit\docs\run\src\tb_stopping_failure.vhd:24:9:@0ms:(assertion error): Assertion violation
Output file: C:\github\vunit\docs\run\src\vunit_out\test_output\lib.tb_stopping_failure.Test_that_fails_on_an_assert_f53b930e2c7649bc33253af52f8ea89a9c05f07b\output.txt
Seed for lib.tb_stopping_failure.Test that fails on an assert: a866d0986a663e6b
C:\github\vunit\docs\run\src\tb_stopping_failure.vhd:24:9:@0ms:(assertion error): Assertion violation
ghdl:error: assertion failed
ghdl:error: simulation failed
<span style="color: #cb4b16; ">fail</span> (P=0 S=0 F=1 T=4) lib.tb_stopping_failure.Test that fails on an assert (0.5 s)
<span style="color: #cb4b16; ">fail</span> (P=0 S=0 F=1 T=4) lib.tb_stopping_failure.Test that fails on an assert (0.6 s)

(11:35:31) Starting lib.tb_stopping_failure.Test that crashes on boundary problems
Output file: C:\repos\vunit\docs\run\src\vunit_out\test_output\lib.tb_stopping_failure.Test_that_crashes_on_boundary_problems_b53105615efefaa16d0cf9ee1bad37b5d3369e95\output.txt
ghdl:error: index (314) out of bounds (1 to 17) at C:\repos\vunit\docs\run\src\tb_stopping_failure.vhd:26
(22:55:55) Starting lib.tb_stopping_failure.Test that crashes on boundary problems
Output file: C:\github\vunit\docs\run\src\vunit_out\test_output\lib.tb_stopping_failure.Test_that_crashes_on_boundary_problems_b53105615efefaa16d0cf9ee1bad37b5d3369e95\output.txt
Seed for lib.tb_stopping_failure.Test that crashes on boundary problems: cbcb543764149e78
ghdl:error: index (340) out of bounds (1 to 17) at C:\github\vunit\docs\run\src\tb_stopping_failure.vhd:26
ghdl:error: simulation failed
<span style="color: #cb4b16; ">fail</span> (P=0 S=0 F=2 T=4) lib.tb_stopping_failure.Test that crashes on boundary problems (0.5 s)
<span style="color: #cb4b16; ">fail</span> (P=0 S=0 F=2 T=4) lib.tb_stopping_failure.Test that crashes on boundary problems (0.6 s)

(11:35:31) Starting lib.tb_stopping_failure.Test that fails on VUnit check procedure
Output file: C:\repos\vunit\docs\run\src\vunit_out\test_output\lib.tb_stopping_failure.Test_that_fails_on_VUnit_check_procedure_717a6f8ff044e3d5fa7d7d3ec5a32971d74864dd\output.txt
(22:55:55) Starting lib.tb_stopping_failure.Test that fails on VUnit check procedure
Output file: C:\github\vunit\docs\run\src\vunit_out\test_output\lib.tb_stopping_failure.Test_that_fails_on_VUnit_check_procedure_717a6f8ff044e3d5fa7d7d3ec5a32971d74864dd\output.txt
Seed for lib.tb_stopping_failure.Test that fails on VUnit check procedure: 6581894e9ea614b1
0 fs - check - ERROR - Equality check failed - Got 17. Expected 18.
C:\repos\vunit\vunit\vhdl\core\src\core_pkg.vhd:85:7:@0ms:(report failure): Stop simulation on log level error
C:\github\vunit\vunit\vhdl\core\src\core_pkg.vhd:85:7:@0ms:(report failure): Stop simulation on log level error
ghdl:error: report failed
ghdl:error: simulation failed
<span style="color: #cb4b16; ">fail</span> (P=0 S=0 F=3 T=4) lib.tb_stopping_failure.Test that fails on VUnit check procedure (0.5 s)
<span style="color: #cb4b16; ">fail</span> (P=0 S=0 F=3 T=4) lib.tb_stopping_failure.Test that fails on VUnit check procedure (0.6 s)

(11:35:32) Starting lib.tb_stopping_failure.Test that a warning passes
Output file: C:\repos\vunit\docs\run\src\vunit_out\test_output\lib.tb_stopping_failure.Test_that_a_warning_passes_7db91f3b27aea5f89e74e39ea51ce6d61558674e\output.txt
<span style="color: #008000; ">pass</span> (P=1 S=0 F=3 T=4) lib.tb_stopping_failure.Test that a warning passes (0.4 s)
(22:55:56) Starting lib.tb_stopping_failure.Test that a warning passes
Output file: C:\github\vunit\docs\run\src\vunit_out\test_output\lib.tb_stopping_failure.Test_that_a_warning_passes_7db91f3b27aea5f89e74e39ea51ce6d61558674e\output.txt
<span style="color: #008000; ">pass</span> (P=1 S=0 F=3 T=4) lib.tb_stopping_failure.Test that a warning passes (0.6 s)

==== Summary ============================================================================
<span style="color: #008000; ">pass</span> lib.tb_stopping_failure.Test that a warning passes (0.4 s)
<span style="color: #cb4b16; ">fail</span> lib.tb_stopping_failure.Test that fails on an assert (0.5 s)
<span style="color: #cb4b16; ">fail</span> lib.tb_stopping_failure.Test that crashes on boundary problems (0.5 s)
<span style="color: #cb4b16; ">fail</span> lib.tb_stopping_failure.Test that fails on VUnit check procedure (0.5 s)
<span style="color: #008000; ">pass</span> lib.tb_stopping_failure.Test that a warning passes (0.6 s)
<span style="color: #cb4b16; ">fail</span> lib.tb_stopping_failure.Test that fails on an assert (0.6 s)
<span style="color: #cb4b16; ">fail</span> lib.tb_stopping_failure.Test that crashes on boundary problems (0.6 s)
<span style="color: #cb4b16; ">fail</span> lib.tb_stopping_failure.Test that fails on VUnit check procedure (0.6 s)
=========================================================================================
<span style="color: #008000; ">pass</span> 1 of 4
<span style="color: #cb4b16; ">fail</span> 3 of 4
=========================================================================================
Total time was 1.8 s
Elapsed time was 1.8 s
Total time was 2.4 s
Elapsed time was 2.4 s
=========================================================================================
<span style="color: #cb4b16; ">Some failed!</span>
</pre></div>
14 changes: 7 additions & 7 deletions docs/run/img/tb_with_lower_level_control_stdout.html
Original file line number Diff line number Diff line change
@@ -1,20 +1,20 @@
<div class="highlight" style="background: #002b36; color: #eee8d5;"><pre style="line-height: 125%; background: #002b36; color: #eee8d5;">> python run.py
Starting lib.tb_with_lower_level_control.Test something
Output file: C:\github\vunit\docs\run\src\vunit_out\test_output\lib.tb_with_lower_level_control.Test_something_a280708dc2f527dff5e45e7a7d2b48df39330b4f\output.txt
<span style="color: #008000; ">pass</span> (P=1 S=0 F=0 T=2) lib.tb_with_lower_level_control.Test something (0.5 seconds)
<span style="color: #008000; ">pass</span> (P=1 S=0 F=0 T=2) lib.tb_with_lower_level_control.Test something (0.6 s)

Starting lib.tb_with_lower_level_control.Test something else
(22:55:51) Starting lib.tb_with_lower_level_control.Test something else
Output file: C:\github\vunit\docs\run\src\vunit_out\test_output\lib.tb_with_lower_level_control.Test_something_else_e47dc199cab8c612d9a0f46b8be7d141576fc970\output.txt
<span style="color: #008000; ">pass</span> (P=2 S=0 F=0 T=2) lib.tb_with_lower_level_control.Test something else (0.5 seconds)
<span style="color: #008000; ">pass</span> (P=2 S=0 F=0 T=2) lib.tb_with_lower_level_control.Test something else (0.6 s)

==== Summary ===============================================================
<span style="color: #008000; ">pass</span> lib.tb_with_lower_level_control.Test something (0.5 seconds)
<span style="color: #008000; ">pass</span> lib.tb_with_lower_level_control.Test something else (0.5 seconds)
<span style="color: #008000; ">pass</span> lib.tb_with_lower_level_control.Test something (0.6 s)
<span style="color: #008000; ">pass</span> lib.tb_with_lower_level_control.Test something else (0.6 s)
============================================================================
<span style="color: #008000; ">pass</span> 2 of 2
============================================================================
Total time was 1.1 seconds
Elapsed time was 1.1 seconds
Total time was 1.2 s
Elapsed time was 1.2 s
============================================================================
<span style="color: #008000; ">All passed!</span>
</pre></div>
Loading

0 comments on commit 10edb1c

Please sign in to comment.