Skip to content
This repository has been archived by the owner on Oct 31, 2024. It is now read-only.

Latest commit

 

History

History
343 lines (241 loc) · 10.6 KB

README.md

File metadata and controls

343 lines (241 loc) · 10.6 KB
Raw

Zero Bin

A composition of paladin and plonky-block-proof-gen. Given the proof generation protocol as input, generate a proof. The project is instrumented with paladin, and as such can distribute proof generation across multiple worker machines.

Project layout

ops
├── Cargo.toml
└── src
   └── lib.rs
worker
├── Cargo.toml
└── src
   └── main.rs
leader
├── Cargo.toml
└── src
   └── main.rs
rpc
├── Cargo.toml
└── src
   └── main.rs
verifier
├── Cargo.toml
└── src
   └── main.rs

Ops

Defines the proof operations that can be distributed to workers.

Worker

The worker process. Receives proof operations from the leader, and returns the result.

Leader

The leader process. Receives proof generation requests, and distributes them to workers.

RPC

A binary to generate the block trace format expected by the leader.

Verifier

A binary to verify the correctness of the generated proof.

Leader Usage

The leader has various subcommands for different io modes. The leader binary arguments are as follows:

cargo r --release --bin leader -- --help

Usage: leader [OPTIONS] <COMMAND>

Commands:
  stdio    Reads input from stdin and writes output to stdout
  jerigon  Reads input from a Jerigon node and writes output to stdout
  http     Reads input from HTTP and writes output to a directory
  help     Print this message or the help of the given subcommand(s)

Options:
  -h, --help
          Print help (see a summary with '-h')

Paladin options:
  -t, --task-bus-routing-key <TASK_BUS_ROUTING_KEY>
          Specifies the routing key for publishing task messages. In most cases, the default value should suffice

          [default: task]

  -s, --serializer <SERIALIZER>
          Determines the serialization format to be used

          [default: postcard]
          [possible values: postcard, cbor]

  -r, --runtime <RUNTIME>
          Specifies the runtime environment to use

          [default: amqp]
          [possible values: amqp, in-memory]

  -n, --num-workers <NUM_WORKERS>
          Specifies the number of worker threads to spawn (in memory runtime only)

      --amqp-uri <AMQP_URI>
          Provides the URI for the AMQP broker, if the AMQP runtime is selected

          [env: AMQP_URI=amqp://localhost:5672]

Table circuit sizes:
      --persistence <PERSISTENCE>
          [default: disk]

          Possible values:
          - none: Do not persist the processed circuits
          - disk: Persist the processed circuits to disk

      --arithmetic <CIRCUIT_BIT_RANGE>
          The min/max size for the arithmetic table circuit.

          [env: ARITHMETIC_CIRCUIT_SIZE=16..22]

      --byte-packing <CIRCUIT_BIT_RANGE>
          The min/max size for the byte packing table circuit.

          [env: BYTE_PACKING_CIRCUIT_SIZE=10..22]

      --cpu <CIRCUIT_BIT_RANGE>
          The min/max size for the cpu table circuit.

          [env: CPU_CIRCUIT_SIZE=15..22]

      --keccak <CIRCUIT_BIT_RANGE>
          The min/max size for the keccak table circuit.

          [env: KECCAK_CIRCUIT_SIZE=14..22]

      --keccak-sponge <CIRCUIT_BIT_RANGE>
          The min/max size for the keccak sponge table circuit.

          [env: KECCAK_SPONGE_CIRCUIT_SIZE=9..22]

      --logic <CIRCUIT_BIT_RANGE>
          The min/max size for the logic table circuit.

          [env: LOGIC_CIRCUIT_SIZE=12..22]

      --memory <CIRCUIT_BIT_RANGE>
          The min/max size for the memory table circuit.

          [env: MEMORY_CIRCUIT_SIZE=18..22]

Note that both paladin and plonky2 table circuit sizes are configurable via command line arguments and environment variables. The command line arguments take precedence over the environment variables.

TABLE CIRCUIT SIZES ARE ONLY RELEVANT FOR THE LEADER WHEN RUNNING IN in-memory MODE.

If you want to configure the table circuit sizes when running in a distributed environment, you must configure the table circuit sizes on the worker processes (the command line arguments are the same).

stdio

The stdio command reads proof input from stdin and writes output to stdout.

cargo r --release --bin leader stdio --help

Reads input from stdin and writes output to stdout

Usage: leader stdio [OPTIONS]

Options:
  -f, --previous-proof <PREVIOUS_PROOF>  The previous proof output
  -h, --help                             Print help

Pull prover input from the rpc binary.

cargo r --release --bin rpc fetch --rpc-url <RPC_URL> -b 6 > ./input/block_6.json

Pipe the block input to the leader binary.

cat ./input/block_6.json | cargo r --release --bin leader -- -r in-memory stdio > ./output/proof_6.json

Jerigon

The Jerigon command reads proof input from a Jerigon node and writes output to stdout.

cargo r --release --bin leader jerigon --help

Reads input from a Jerigon node and writes output to stdout

Usage: leader jerigon [OPTIONS] --rpc-url <RPC_URL> --block-number <BLOCK_NUMBER>

Options:
  -u, --rpc-url <RPC_URL>

  -b, --block-number <BLOCK_NUMBER>
          The block number for which to generate a proof
  -c, --checkpoint-block-number <CHECKPOINT_BLOCK_NUMBER>
          The checkpoint block number [default: 0]
  -f, --previous-proof <PREVIOUS_PROOF>
          The previous proof output
  -o, --proof-output-path <PROOF_OUTPUT_PATH>
          If provided, write the generated proof to this file instead of stdout
  -h, --help
          Print help

Prove a block.

cargo r --release --bin leader -- -r in-memory jerigon -u <RPC_URL> -b 16 > ./output/proof_16.json

HTTP

The HTTP command reads proof input from HTTP and writes output to a directory.

cargo r --release --bin leader http --help

Reads input from HTTP and writes output to a directory

Usage: leader http [OPTIONS] --output-dir <OUTPUT_DIR>

Options:
  -p, --port <PORT>              The port on which to listen [default: 8080]
  -o, --output-dir <OUTPUT_DIR>  The directory to which output should be written
  -h, --help                     Print help

Pull prover input from the rpc binary.

cargo r --release --bin rpc fetch -u <RPC_URL> -b 6 > ./input/block_6.json

Start the server.

RUST_LOG=debug cargo r --release --bin leader http --output-dir ./output

Note that HTTP mode requires a slightly modified input format from the rest of the commands. In particular, the previous proof is expected to be part of the payload. This is due to the fact that the HTTP mode may handle multiple requests concurrently, and thus the previous proof cannot reasonably be given by a command line argument like the other modes.

Using jq we can merge the previous proof and the block input into a single JSON object.

jq -s '{prover_input: .[0], previous: .[1]}' ./input/block_6.json ./output/proof_5.json | curl -X POST -H "Content-Type: application/json" -d @- http://localhost:8080/prove

Paladin Runtime

Paladin supports both an AMQP and in-memory runtime. The in-memory runtime will emulate a cluster in memory within a single process, and is useful for testing. The AMQP runtime is geared for a production environment. The AMQP runtime requires a running AMQP broker and spinning up worker processes. The AMQP uri can be specified with the --amqp-uri flag or be set with the AMQP_URI environment variable.

Starting an AMQP enabled cluster

Start rabbitmq

docker run --name rabbitmq -p 5672:5672 -p 15672:15672 rabbitmq:3-management
Start worker(s)

Start worker process(es). The default paladin runtime is AMQP, so no additional flags are required to enable it.

RUST_LOG=debug cargo r --release --bin worker
Start leader

Start the leader process with the desired command. The default paladin runtime is AMQP, so no additional flags are required to enable it.

RUST_LOG=debug cargo r --release --bin leader jerigon -u <RPC_URL> -b 16 > ./output/proof_16.json

Starting an in-memory (single process) cluster

Paladin can emulate a cluster in memory within a single process. Useful for testing purposes.

cat ./input/block_6.json | cargo r --release --bin leader -- -r in-memory stdio > ./output/proof_6.json

Verifier Usage

A verifier binary is provided to verify the correctness of the generated proof. The verifier expects output in the format generated by the leader. The verifier binary arguments are as follows:

cargo r --bin verifier -- --help

Usage: verifier --file-path <FILE_PATH>

Options:
  -f, --file-path <FILE_PATH>  The file containing the proof to verify
  -h, --help                   Print help

Example:

cargo r --release --bin verifier -- -f ./output/proof_16.json

RPC Usage

An rpc binary is provided to generate the block trace format expected by the leader.

cargo r --bin rpc -- --help

Usage: rpc <COMMAND>

Commands:
  fetch  Fetch and generate prover input from the RPC endpoint
  help   Print this message or the help of the given subcommand(s)

Options:
  -h, --help  Print help

Example:

cargo r --release --bin rpc fetch --rpc-url <RPC_URL> --block-number 16 > ./output/block-16.json

Docker

Docker images are provided for both the leader and worker binaries.

License

Licensed under either of

at your option.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.