ZkVM Benchmarks

A suite of benchmarks to compare the performance of different ZkVMs.

Setup

Install Rust:

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
rustup install nightly

Install the SP1 toolchain:

curl -L https://sp1.succinct.xyz | bash
source ~/.bashrc
sp1up
cargo prove --version

Install the Risc0 toolchain:

curl -L https://risczero.com/install | bash
source ~/.bashrc
rzup
cargo risczero --version

Install Docker.

• https://docs.docker.com/engine/install/ubuntu/
• https://docs.docker.com/engine/install/linux-postinstall/

Install the NVIDIA Container Toolkit if you are using NVIDIA GPUs.

• https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html

Note: When benchmarking, make sure that you run 1 round of Fibonacci (or some other small program) to download the R0 docker image to make sure it doesn't contribute to benchmarking time.

Note: On Ubuntu 22.04, you might need to install libssl1.0 to use the Risc0 toolchain. You can follow these instructions (from here).

Usage

You can edit the sweep.py file and execute it to run a sweep of benchmarks (results will be under the benchmarks folder):

python3 sweep.py

To run a single benchmark, you can run:

./eval.sh (loop|fibonacci|ssz_withdrawals|tendermint) (sp1|risc0|powdr-plonky3|powdr-estark) (poseidon|sha256|blake3|...)

Note that right now only poseidion is supported for all zkVMs, since we are interested in also profiling recursion.

Example SP1:

./eval.sh fibonacci sp1 poseidon 22 benchmark

Example Risc0:

./eval.sh fibonacci risc0 poseidon 22 benchmark

Example Powdr:

./eval.sh fibonacci powdr-plonky3 poseidon 22 benchmark

Note for benchmarking the Reth program, you must also pass in a block number:

./eval.sh reth sp1 poseidon 22 benchmark 19409768

The inputs for these blocks have already been generated here. You can add more blocks by using the SP1-Reth script.

Common Issues

Because Risc0 uses C++ for their prover, you may need to install the C++ compiler and libraries.

For Ubuntu/Debian-based systems:

# Update system repositories and installed packages
sudo apt update
sudo apt upgrade

# Install the latest version of GCC and libraries
sudo apt install build-essential libc6

For CentOS/RHEL-based systems:

# Update system repositories and installed packages
sudo yum update

# Install the Development Tools group which includes GCC
sudo yum groupinstall "Development Tools"

# Install or update g++
sudo yum install -y gcc-c++

For Fedora:

# Update system repositories and installed packages
sudo dnf update

# Install the Development Tools group which includes GCC
sudo dnf groupinstall "Development Tools"

Setting up NVIDIA:

sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker