Board testing results

[StephanvanSchaik]

This is related to the testing part of issue #128. I have written a Python script that tries to import all the defined Platform classes in nmigen-boards and then checks if building blinky using the corresponding toolchain works. If the build succeeds, it writes the checksum of the board file and the filename to a file, such that the test can be skipped if the board file hasn't been modified when running the tests again. The only thing that is currently missing is to try and specify different toolchains by setting the PATH (e.g. Intel Quartus Lite 20.1 doesn't support Cyclone III and Quartus II Web Edition 13.1 is required to test the board instead). Once I have that part figured out, I can share a gist of the Python script, because it might be useful for automation, but the amount of disk storage required to have the toolchains installed is perhaps a bit problematic.

I have the following toolchains installed:

• Quartus II Web Edition 13.1
• Quartus Lite Edition 20.1
• Xilinx Vivado WebPACK 2020.1
• Xilinx ISE WebPACK 14.7 (nothing seems to depend on this)
• yosys, nextpnr, icestorm, prjtrellis

I have the following programming tools installed:

• mercpcl (Mercury board) (Git)
• OpenFPGAloader (Git)
• OpenOCD (Git)
• xc3sprog (Git)

The following boards currently fail this test:

• Atlys
• Genesys2Platform
• KC705
• KCU105
• MachXO3SK (depends on Lattice Diamond)
• Mercury (some issue in the generated Verilog)
• MicroZedZ010, MicroZedZ020 (only defines the package + two connectors)
• Numato Mimas (no toolchain_program)
• Quickfeather
• SK XC6SLX9 (no toolchain_program)
• TinyFPGA AX1 and AX2 (only defines the package + one connector)
• ZTurnLiteZ007S and ZTurnLiteZ010 (only defines the package + one connector)

All other boards that are currently in the repository (commit 068d9fe) pass this test. If nothing is mentioned between parentheses after the board name, then I am not entirely sure what causes it to fail.

[StephanvanSchaik]

I worked out the PATH environment variable. The test script that I am using can be found at nmigen-boards-test. Finally, I also updated the list of boards as the pull requests got merged.

[whitequark]

Thank you for doing this work! I've been planning to do it for a while, but other things were more important.

The aim was always to have something like this integrated in the upstream repository, though it might still take a while to figure out all the details.

[StephanvanSchaik]

I am glad to help as I really like the nmigen project myself.

Thank you for doing this work! I've been planning to do it for a while, but other things were more important.

The aim was always to have something like this integrated in the upstream repository, though it might still take a while to figure out all the details.

I am glad I could help out here. Yes, this is still pretty basic and there are a lot of details that we have to figure out to have it upstream.

One issue is the amount of disk storage required to have the toolchains installed: 17G for Quartus Lite 20.1, 32G for Vivado 2020.1, 18G for Xilinx ISE 14.7, 24G for Quartus II WebPACK 13.1, which totals up to 91G for the proprietary toolchains (and I don't have Lattice Diamond installed myself, at the moment). The FOSS toolchain (yosys + nextpnr + icestorm + prjtrellis) sits at about 550M, which is much more reasonable.

Another issue is that, at least for Intel/Altera FPGAs, blinky does not generate all the constraints, but just those are needed for blinky, which makes it great as an initial test to check whether things work, but it would be nice if we can get better coverage for the constraint files generated by nmigen.

One idea I had for next weekend is to implement a parser for the QSF (Intel/Altera) and XDC (Xilinx) constraint files (and in the future others). As most FPGAs boards either come with a project template to start from or with multiple examples, this could be useful to generate the Python file for nmigen-boards from the constraints, given that the pin naming is somewhat sensible, and in addition this could be helpful to ensure the board definitions in nmigen-boards have the right attributes set, especially in terms of I/O standards. The generated board files would not be an immediate replacement to doing it by hand, but it would simply be a helpful tool to get started much quicker, as I would rather focus on making sure the details are right, rather than making sure I typed over the pin assignments correctly.

For newcomers, this could also lower the barrier to use nmigen if their board is currently not supported, as they don't have to figure out how to write the board definition from scratch.

[mithro]

@StephanvanSchaik - There has been a lot of discussion around things like constraint files at https://github.com/hdl/constraints and https://hdl.github.io/constraints/ -- I'm unsure what the conclusion was however.

Having Python parsers for the XDC / QSF files would be something of interest to the SymbiFlow project. I believe we already have some hacked together stuff at https://github.com/SymbiFlow/symbiflow-arch-defs/blob/master/xc/common/utils/prjxray_pcf_to_xdc.py and https://github.com/SymbiFlow/symbiflow-arch-defs/blob/master/utils/lib/parse_pcf.py -- The SymbiFlow project also has SDC / XDC parsers as Yosys plugins at https://github.com/SymbiFlow/yosys-symbiflow-plugins#sdc-plugin and https://github.com/SymbiFlow/yosys-symbiflow-plugins#xdc-plugin

[StephanvanSchaik]

@mithro Thanks! I will have a look.