Netlist Simulator and Waveform Viewer
The simulator framework enables the reverse engineer to simulate selected parts of the netlist in a cycle-accurate fashion and analyze the results in a graphical viewer. The simulator framework is made out of several plugins, like controller plugin to provide common functionality and API, Verilator simulator plugin and waveform viewer plugin. These plugins are not build automatically. It is recommended to force the build of all available plugins by issuing the flag -DBUILD_ALL_PLUGINS=1 flag when calling cmake.
The simulation can be controlled from waveform viewer GUI or via Python script or Python command line. In all cases the following steps are necessary
- Instantiate a simulation controller
- Select parts of the current netlist to be simulated
- Prepare simulation
- Select the clock(s) driving the simulation
- Select the simulation engine
- Load or create data for the inputs of the selected partial netlisst.
- Run the simulation
- Analyze the results
Python:
from hal_plugins import netlist_simulator_controller
pl_sim = hal_py.plugin_manager.get_plugin_instance("netlist_simulator_controller")
sim_ctrl = pl_sim.create_simulator_controller(<optional_name>)Immediately when creating a new controller instance a working directory for the waveform database is also created. The optional controller name and a random string will become part of the working directory name. The user can choose any variable name for the Python controller instance. In this tutorial, we call it sim_ctrl, which is used in the following Python command lines.
When creating a new controller instance a log message will be written on console revealing the autogenerated controller-ID.
GUI:
[Create simulation controller] from toolbar
Python:
sim_ctrl.add_gates(<list_of_gates>)The list of gates defining the partial netlist for simulation can be prepared using the well documented functions get_gates from hal netlist or module.
GUI:
[Simulation settings] from toolbar -> 'Select gates for simulation'
The user has the choice to pick the gates for the partial netlist individually, to select the entire netlist, or to use a selection previously made in the graph view from the netlist.
All subsequent preparation steps depend on the selected partial netlist for simulation. For this reason, only the selection of the gates can be made. If the user changes his mind about the selected gates, a new controller can be set up with the correct selection.
The sequence of the following steps is not important, they can be done in any order. However, the GUI allows the actual simulation only when all the steps below are completed.
Setting the clock is a convenient method to define a clock with a given period for a for a specific time span. All times are given as integer values. In most cases the unit 'picosecond' is assumed, but that is not important for the 'ideal' simulation, which does not simulate time jitter or delay.
However, it is possible to provide user-generated input data for the clock as for any other input network. In this case, it must be announced that no clock is used to skip this setup step.
Python:
sim_ctrl.add_clock_period(<clock_net>, <period>, <start_at_zero>, <duration>)
#---or---
sim_ctrl.set_no_clock_used()GUI:
[Simulation settings] from toolbar -> 'Select clock net'
Setup parameter for selected clock as above or check 'Do not use clock in simulation'.
While the menu offers three different engines the user should always select the external Verilator engine for most accurate simulation. The built-in HAL-simulator has known issues and is kept only for testing and comparison with simulation results from early HAL days. The dummy engine is for debugging purpose only.
Python:
eng = sim_ctrl.create_simulation_engine("verilator")GUI:
[Simulation settings] from toolbar -> 'Select engine ...' -> 'verilator'
The Verilator engine will be started as a separate process. For special cases (number of threads, additional input, remote execution) the engine might need additional parameter.
Python:
eng.set_engine_property("num_of_threads", "8")GUI:
[Simulation settings] from toolbar -> 'Settings ...' -> 'Engine properties'-Tab
The output and error messages generated by the engine during the simulation are output to the console. GUI users can get a slightly enhanced view of the engines output. GUI:
[Simulation settings] from toolbar -> 'Show output of engine'
Python:
sim_ctrl.import_vcd(<filename>,<filter>) # import .vcd input
sim_ctrl.import_csv(<filename>,<filter>,<timescale>) # import .csv input
sim_ctrl.import_saleae(<directory>,<lookupDictionary>,<timescale>) # import .csv inputGUI:
[Open input file] from toolbar -> select one of (*.vcd) (*.csv) (saleae.json)
The regular way providing input data to drive the simulation is by importing a waveform data file. Supported file formats are .VCD, .CSV, and Saleae. GUI user can double-click on the input data to open an editor and modify individual values. In python individual values can be set by 'sim_ctrl.set_input()' command but performance would be lousy since every input requires synchronization with database.
While the name (or ID) of the assigned net from the netlist should be found in header data of VCD or CSV files, this information is missing in binary Saleae data format. Therefore, a file named saleae.json must be present in hal in the same directory where the binary files containing the link to the nets from the netlist are located. GUI users have to pick that file in the file selection dialog when importing Saleae data. In Python, a lookup dictionary containing this information must be passed as a parameter.
Python:
sim_ctrl.run_simulation()
while eng.get_state() > 0 and eng.get_state() != 2:
print ("eng state running %d" % (eng.get_state())) # Output: 1
time.sleep(1)
if not eng.get_state()==0:
# ... Error handling when simulation engine failed to complete
if not sim_ctrl.get_results():
# ... Error handling when simulation results cannot be parsedSimulation will be started in a separate thread. Therefore a loop is mandatory to check the engines state (1=Simulation running, 0=Ended successfully, -1=Some error). Once the engine completed successfully, the sim_ctrl.get_results() method must be called to parse engines output into waveform database. Waveform results are not loaded into the viewer immediately (See 'Analyze the results').
GUI:
[Run simulation] from toolbar
A status message at the bottom of the waveform viewer widget informs the user whether the simulation is still running, has ended successfully or crashed with some error. On success the controller will try to parse the results and create the waveform database. Dependent on the size of the simulated network, this may take a while.
Tree view controls the order of the displayed waveforms. Reorder or assign to groups by drag'n drop. Invoke context menu to remove items, rename, or create groups.
Waveform diagram shows graphical representation of waveform. Use mouse wheel to zoom in or out. You can also drag over region to zoom in.
Cursor reports waveform value for selected cursor time. The cursor can be moved to a new position by dragging or double-clicking.
1 Example for 'regular waveform' : CLK, START, DONE.
2 Example for 'waveform group' : OUTPUT (combining 16 bit OUTPUT_0 ... OUTPUT_15 to value displayed as hex number).
3 Example for 'boolean waveform' : A^B (performing XOR operation on waveform A and B).
4 Example for 'trigger time set' : trigger1 (time stamps whenever waveform plusOp_0 has a raising edge).
All results parsed from simulation engine are 'regular' waveforms which can be considered as a function of values 0,1,X,Z over simulated time. Typical analysis steps are the formation of groups from signals representing a single bit, the combination of waveforms by logical operations, or the extraction of a series of "trigger" timestamps at events of interest.
While the commands and options for analyzing waveform results from simulation are different in Python and GUI, the user can combine the most convenient tools from both "worlds". Once a waveform is loaded or assigned to a group, its graphical representation becomes visible in the waveform viewer and the user can interact with it through the GUI. If the user wants to work with elements created by GUI commands, he can access them through the netlist controller with the command "pl_sim.get_controller_by_id()".
Python: Get a waveform object and load waveform to viewer:
waveform = sim_ctrl.get_waveform_by_net(net)Get value at time t as integer value. The undecided state 'X' will be represented by -1.
value = waveform.get_value_at(t)It is possible to retrieve all transition events of a waveform using the waveform.get_events(t0) method. However, a single call must not consume all the computers memory by generating an extremely large list. Therefore, the list size is limited, and for waveforms with a very higt number of transitions, multiple calls must be issued according to the following scheme:
waveform = s.get_waveform_by_net(n)
t0 = 0 # start time for requested event data buffer (list of tuples)
while (evts := waveform.get_events(t0)): # as long as there are events …
t0 = evts[-1][0] + 1 # next start time := maxtime + 1
for tuple in evts: # dump events in console
print ("%8d %4d" % (tuple))Sometimes the user want to retrieve waveform values whenever an other waveform signal toggles or reaches a certain value. For this particular case, a set of trigger times must first be created (see below). Once this set is created, it can be used with the following command:
evts = waveform.get_triggered_events(trigger,t0)The user can customize the name of a waveform. Note that the renaming method belongs to the controller, as the controller may send signals to the graphical user interface to make the new name known to the viewer.
sim_ctrl.rename_waveform(waveform,<new name>)Waveforms to be analyzed must first be loaded into the viewer.
GUI:
[Load waveform] from toolbar -> select waveforms to analyze or ...
- [All waveform items] button
- [Current GUI selection] button
When a HAL project is saved to disk and reopened, the simulation results are also restored. Note that the core waveform data is implemented as a database, so that Changes are saved without an explicit ''save'' command. It is also possible to manually open results from previous simulations, if the simulation was performed with the same netlist.
Python:
# based on plugin instance pl_sim (see above)
sim_ctrl = pl_sim.restore_simulator(netlist,"<path_to_external_simulation_workdir>/netlist_simulator_controller.json")GUI:
[Open input file] from toolbar
Select file 'netlist_simulator_controller.json' in working directory of external simulation
[Simulation settings] from toolbar -> 'settings ...'
The waveform viewer plugin has its own file with user settings. The settings made are saved and restored at the next session. The settings items are:
Global settings
- Load waveform to memory if number transitions <
- Maximum number of values to load into editor
- Custom base directory instead of project dir:
Enging properties
- Engine property name and value
Color settings
- Color for regular waveform in waveform diagram
- Color for selected waveform in waveform diagram
- Color for waveform with undefined value
- Value (0,1,X) specific color both for background of bullet icon and net in graphical netlist.
In hal import netlist <hal-directory>/plugins/simulator/netlist_simulator_controller/test/netlists/toycipher/cipher_flat.vhd into project. It is ok to unfold the top_module.
The module has a total of 34 inputs: the clock CLK, START, 16 KEY pins, and 16 PLAINTEXT pins. The following Python script generates sample input for simulation.
def toBinary(hex):
retval = []
for cc in hex[::-1]:
hexdigit = int(cc,16)
mask = 1
for j in range(4):
bit = 0 if (hexdigit&mask==0) else 1
retval.append(bit)
mask <<= 1
return retval
class Waveform:
def __init__(self,name,trans):
self.tr = trans
self.name = name
def value(self,t):
count = 0
while count < len(self.tr):
if self.tr[count] > t:
return count % 2
count += 1
return count % 2
#Input values for cipher converted to LSB list of binary values
key = "ABCD"
txt = "1337"
print ("key..............<%s>" % (key))
print ("plaintext....... <%s>" % (txt))
key_bits = toBinary(key)
txt_bits = toBinary(txt)
#Transition times (unit=100ps) for input waveform
strt = Waveform("START",[10,20,130,140,250,260])
inputs = [strt]
for i in range(16):
inputs.append(Waveform("KEY_%d"%(i), [] if key_bits[i] == 0 else [5,240]))
inputs.append(Waveform("PLAINTEXT_%d"%(i), [] if txt_bits[i] == 0 else [120,240]))
#Generate CSV header, net names must be enclosed in quotes
of=open("cipher_simul_input.csv","w")
of.write("time,\"CLK\"")
for inp in inputs:
of.write (",\"" + inp.name + "\"")
of.write("\n")
#Generate values for nets, first column is time (unit ms)
t = 0
clk = 0
while t < 290:
of.write("%8.7f,%d" % (t/10000000.0,clk))
for inp in inputs:
of.write(",%d" % (inp.value(t)))
t += 5
clk = 1 if clk==0 else 0
of.write("\n")
of.close()Running the simple Python script above (with hard-coded key and plaintext values and build-in output file name) will produce the workbench file 'cipher_simul_input.csv'.
[Create simulation controller] from toolbar.
[Simulation settings] from toolbar -> 'Select gates for simulation' -> 'All gates' -> 'Ok'
[Simulation settings] from toolbar -> 'Select clock net' -> 'Do not use clock in simulation' -> 'Ok'
[Simulation settings] from toolbar -> 'Select engine ...' -> 'verilator'
[Open input file] from toolbar -> files of type : CSV files (*.csv) -> locate the generated workbench file 'cipher_simul_input.csv'
[Run simulation] from toolbar
[Add waveform by net] from toolbar -> 'All waveform items' -> 'Ok'
Might be a good time to detach waveform viewer widget and enlarge its viewport.
[Toggle waveform resolution]
Moving cursor by drag'n drop or double click reveals simulated value at any time
[Simulation settings] from toolbar -> Visualize net state by color
Nets are colored according to simulated value (0=blue, 1=red, X=grey)
This part describes the hal built-in simulator, which should no longer be used. However, for the moment it is kept for testing and proving backward compatibility. For this purpose the documentation behind this item is kept ... but will be removed eventually.
The user must specify the subset of the netlist that he wants to simulate. For this purpose, a list of gates must be specified and passed to the simulator instance sim so that it can automatically determine the inputs and outputs nets of the subgraph. This can be done with the function add_gates as shown below.
gate_list = [g1, g2, g3, g4] # collect gates to simulate in a list
sim_ctrl.add_gates(gate_list) # add the gates to the simulation instanceAs stated before, the simulator will automatically detect all inputs and outputs to the simulated subgraph. Using get_input_nets and get_output_nets, the user can retrieve those nets and may use them later on to, e.g., set input values during simulation.
sim_ctrl.get_input_nets() # get all input nets
sim_ctrl.get_output_nets() # get all output netsNext, the clock signal needs to be specified. This can be done using either add_clock_frequency or add_clock_period, which take the net connected to the clock and a clock frequency or period as input. Hence, the function assigns a specific clock frequency to the given net and thus allows for different clock domains.
sim_ctrl.add_clock_frequency(clk, 1000000) # assign a clock frequency of 1 MHz to input net 'clk'To avoid getting stuck in infinite loops, it may be helpful to set a iteration timeout that stops the simulation after a pre-defined number of iterations. The timeout is given in number of iterations and can be controlled using set_iteration_timeout and get_iteration_timeout. It defaults to 10000000 iterations and assigning a value of 0 disables the timeout.
sim_ctrl.set_iteration_timeout(1000) # set the timeout to 1000 iterationsIn some cases, sequential gates may come with an initial value that is assigned to their internal state during device startup. For example, this is the case for flip-flops within FPGA netlists, which may retrieve their initial state from the netlist or rather bitstream file. To accommodate this during simulation, these initial values can be loaded into the internal state of such sequential elements using load_initial_values_from_netlist. Further, load_initial_values may be used to specify an initial value to load into all sequential elements before running the simulation.
sim_ctrl.load_initial_values_from_netlist() # load initial values into sequential gates if available
sim_ctrl.load_initial_values(hal_py.BooleanFunction.Value.ZERO) # load logical 0 into all FFs before running simulationTo finally run the simulation, logical values should be assigned to all of the input nets of the simulated subgraph. The function set_input provides exactly this functionality and takes a net as well as a value as input. It may also be used during simulation to change input values between simulation cycles. The simulation itself is started using simulate and proceeds for the specified number of picoseconds. An example is given below.
sim_ctrl.set_input(in1, hal_py.BooleanFunction.Value.ONE) # assign a logical 1 to input net in1
sim_ctrl.set_input(in2, hal_py.BooleanFunction.Value.ZERO) # assign a logical 0 to input net in2
sim_ctrl.simulate(3000) # simulate for 3000 ps
sim_ctrl.set_input(in1, hal_py.BooleanFunction.Value.ZERO) # change the value of in1 to a logical 0
sim_ctrl.simulate(2000) # simulate for another 2000 psTo get the results of the simulation, the function get_simulation_state can be called to retrieve the whole simulation result. It returns a simulation state that contains detailed information about the simulation including all events that occurred. By calling get_net_value on this instance, the user can retrieve the value of each net included in the subgraph at any point in the simulation.
state = sim_ctrl.get_simulation_state() # retrieve the simulation state
val = state.get_net_value(net, 1000) # get the value of net 'net' after 1000 psFurthermore, the simulation state holds a list of all events incurred during simulation that can be obtained using get_events. This list may also be expanded by the user by creating and adding a new event using add_event. The current simulation state of the simulator instance may be overwritten using set_simulation_state. More on these operations can be found in the official Python API documentation.
To assist the simulation process, HAL can generate waveform files using the VCD file format. For this purpose, the function generate_vcd can be called to write the current simulation instance to a waveform file. The user may specify a start and an end time (in ps) to avoid the generation of unnecessarily large traces. The VCD file can then be viewed using popular open-source tools like gtkwave. An example looks like this:
sim_ctrl.generate_vcd("trace.vcd", 0, 300000)In our tests we provide a netlist for a simple counter. The following code simulates the netlist:
from hal_plugins import netlist_simulator
pl_sim = hal_py.plugin_manager.get_plugin_instance("netlist_simulator")
sim_ctrl = pl_sim.create_simulator_controller()
sim_ctrl.add_gates(netlist.get_gates())
sim_ctrl.load_initial_values(hal_py.BooleanFunction.Value.ZERO)
clock_enable_B = netlist.get_net_by_id(8)
clock = netlist.get_net_by_id(9)
reset = netlist.get_net_by_id(7)
output_0 = netlist.get_net_by_id(6)
output_1 = netlist.get_net_by_id(5)
output_2 = netlist.get_net_by_id(4)
output_3 = netlist.get_net_by_id(3)
sim_ctrl.add_clock_period(clock, 10000)
sim_ctrl.set_input(clock_enable_B, hal_py.BooleanFunction.Value.ONE)
sim_ctrl.set_input(reset, hal_py.BooleanFunction.Value.ZERO)
sim_ctrl.simulate(40 * 1000)
sim_ctrl.set_input(clock_enable_B, hal_py.BooleanFunction.Value.ZERO)
sim_ctrl.simulate(110 * 1000)
sim_ctrl.set_input(reset, hal_py.BooleanFunction.Value.ONE)
sim_ctrl.simulate(20 * 1000)
sim_ctrl.set_input(reset, hal_py.BooleanFunction.Value.ZERO)
sim_ctrl.simulate(70 * 1000)
sim_ctrl.set_input(clock_enable_B, hal_py.BooleanFunction.Value.ONE)
sim_ctrl.simulate(23 * 1000)
sim_ctrl.set_input(reset, hal_py.BooleanFunction.Value.ONE)
sim_ctrl.simulate(20 * 1000)
sim_ctrl.simulate(20 * 1000)
sim_ctrl.simulate(5 * 1000)
sim_ctrl.generate_vcd("counter.vcd", 0, 300000)The generated vcd can be viewed using gtkwave.
