Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

is_prime on sp1 #814

Draft
wants to merge 8 commits into
base: master
Choose a base branch
from

Conversation

mcalancea
Copy link
Collaborator

@mcalancea mcalancea commented Jan 7, 2025

Sandbox branch, shouldn't be merged.

Based on #813 . Hardcodes is_prime into bin/ and adds the is_prime/ sp1 project at the top level. I've been running:

cd ~/ceno/is_prime/program
cargo prove build
cd ~/ceno/is_prime/script
time (RUST_LOG=info cargo run --release -- --n=5000 --prove)

and

cd ~/ceno
time (cargo run --release --bin e2e -- --profiling=1 --n=5000 .)

Results are very interesting 👀 👀 . But can we confirm them @matthiasgoergens ?

Later Edit:
Ignore the Fibonacci names, it's auto-generated by sp1.
Consult https://docs.succinct.xyz/docs/getting-started/quickstart for more.

Copy link
Collaborator

@matthiasgoergens matthiasgoergens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you post the interesting results you mentioned, please?

is_prime/program/src/main.rs Outdated Show resolved Hide resolved
is_prime/rust-toolchain Show resolved Hide resolved
is_prime/script/Cargo.toml Outdated Show resolved Hide resolved
@@ -0,0 +1,127 @@
//! An end-to-end example of using the SP1 SDK to generate a proof of a program that can have an
//! EVM-Compatible proof generated which can be verified on-chain.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we going with evm compatible proofs?

Could we just do the simplest thing that SP1 allows? Or is there a good reason to go with something more complicated?

(I don't think our proof is evm-compatible? So it would be fair to require that of SP1.)

Copy link
Collaborator Author

@mcalancea mcalancea Jan 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I followed the instructions here. I generated the project via cargo prove new --bare is_prime.

  • I don't know why there's anything pertaining to evm there, it surprised me too.
  • It generated a Fibonacci project in a folder called is_prime :). I modified the logic, but didn't bother with all the modules, paths, etc. I changed them now.

@mcalancea
Copy link
Collaborator Author

mcalancea commented Jan 7, 2025

My results for sp1:

~/ceno/is_prime/script$ time (RUST_LOG=info cargo run --release -- --n=5000 --prove)

[... warnings ...]

    Finished `release` profile [optimized] target(s) in 0.20s
     Running `/home/mihai/ceno/is_prime/target/release/is_prime --n=5000 --prove`
2025-01-07T14:35:37.885026Z  INFO vk verification: true
n: 5000
2025-01-07T14:35:39.387542Z  INFO prove_core:clone: close time.busy=75.5µs time.idle=2.04µs
2025-01-07T14:35:39.410105Z  INFO prove_core:execute: close time.busy=22.5ms time.idle=971ns
2025-01-07T14:35:39.410837Z  INFO prove_core:create memory checkpoint: close time.busy=713µs time.idle=1.20µs
2025-01-07T14:35:39.595312Z  INFO prove_core: generated 1 records    
2025-01-07T14:35:39.595331Z  INFO prove_core: waiting for turn 0    
2025-01-07T14:35:39.595390Z  INFO prove_core: deferred 1 records    
2025-01-07T14:35:39.595394Z  INFO prove_core: collecting checkpoints    
2025-01-07T14:35:39.595398Z  INFO prove_core: fixing shape
2025-01-07T14:35:39.595415Z  INFO prove_core: fixing shape
2025-01-07T14:35:41.541968Z  INFO prove_core: generated 1 records    
2025-01-07T14:35:41.542045Z  INFO prove_core: deferred 1 records    
2025-01-07T14:36:30.824526Z  INFO prove_core: execution report (totals): total_cycles=397728, total_syscall_cycles=20, touched_memory_addresses=15075
2025-01-07T14:36:30.824543Z  INFO prove_core: execution report (opcode counts):
2025-01-07T14:36:30.824552Z  INFO prove_core:   91931 beq
2025-01-07T14:36:30.824555Z  INFO prove_core:   83425 add
2025-01-07T14:36:30.824558Z  INFO prove_core:   56634 bgeu
2025-01-07T14:36:30.824560Z  INFO prove_core:   50965 sltu
2025-01-07T14:36:30.824562Z  INFO prove_core:   45957 remu
2025-01-07T14:36:30.824564Z  INFO prove_core:   41631 mul
2025-01-07T14:36:30.824566Z  INFO prove_core:   10037 jalr
2025-01-07T14:36:30.824568Z  INFO prove_core:    5039 bltu
2025-01-07T14:36:30.824570Z  INFO prove_core:    5023 auipc
2025-01-07T14:36:30.824572Z  INFO prove_core:    1410 xor
2025-01-07T14:36:30.824574Z  INFO prove_core:    1370 srl
2025-01-07T14:36:30.824576Z  INFO prove_core:    1290 or
2025-01-07T14:36:30.824579Z  INFO prove_core:    1278 sll
2025-01-07T14:36:30.824581Z  INFO prove_core:     584 lw
2025-01-07T14:36:30.824583Z  INFO prove_core:     483 sw
2025-01-07T14:36:30.824585Z  INFO prove_core:     441 and
2025-01-07T14:36:30.824587Z  INFO prove_core:     137 lbu
2025-01-07T14:36:30.824590Z  INFO prove_core:      42 sb
2025-01-07T14:36:30.824592Z  INFO prove_core:      20 ecall
2025-01-07T14:36:30.824594Z  INFO prove_core:      16 sub
2025-01-07T14:36:30.824596Z  INFO prove_core:      15 bne
2025-01-07T14:36:30.824598Z  INFO prove_core:       0 sra
2025-01-07T14:36:30.824600Z  INFO prove_core:       0 slt
2025-01-07T14:36:30.824602Z  INFO prove_core:       0 lb
2025-01-07T14:36:30.824604Z  INFO prove_core:       0 lh
2025-01-07T14:36:30.824606Z  INFO prove_core:       0 lhu
2025-01-07T14:36:30.824608Z  INFO prove_core:       0 sh
2025-01-07T14:36:30.824610Z  INFO prove_core:       0 blt
2025-01-07T14:36:30.824612Z  INFO prove_core:       0 bge
2025-01-07T14:36:30.824628Z  INFO prove_core:       0 jal
2025-01-07T14:36:30.824630Z  INFO prove_core:       0 ebreak
2025-01-07T14:36:30.824632Z  INFO prove_core:       0 mulh
2025-01-07T14:36:30.824634Z  INFO prove_core:       0 mulhu
2025-01-07T14:36:30.824636Z  INFO prove_core:       0 mulhsu
2025-01-07T14:36:30.824638Z  INFO prove_core:       0 div
2025-01-07T14:36:30.824641Z  INFO prove_core:       0 divu
2025-01-07T14:36:30.824643Z  INFO prove_core:       0 rem
2025-01-07T14:36:30.824645Z  INFO prove_core:       0 unimp
2025-01-07T14:36:30.824647Z  INFO prove_core: execution report (syscall counts):
2025-01-07T14:36:30.824652Z  INFO prove_core:   8 commit
2025-01-07T14:36:30.824654Z  INFO prove_core:   8 commit_deferred_proofs
2025-01-07T14:36:30.824657Z  INFO prove_core:   1 halt
2025-01-07T14:36:30.824659Z  INFO prove_core:   1 write
2025-01-07T14:36:30.824661Z  INFO prove_core:   1 hint_len
2025-01-07T14:36:30.824663Z  INFO prove_core:   1 hint_read
2025-01-07T14:36:30.824665Z  INFO prove_core:   0 enter_unconstrained
2025-01-07T14:36:30.824667Z  INFO prove_core:   0 exit_unconstrained
2025-01-07T14:36:30.824669Z  INFO prove_core:   0 verify_sp1_proof
2025-01-07T14:36:30.824671Z  INFO prove_core:   0 ed_decompress
2025-01-07T14:36:30.824673Z  INFO prove_core:   0 secp256k1_double
2025-01-07T14:36:30.824675Z  INFO prove_core:   0 secp256k1_decompress
2025-01-07T14:36:30.824677Z  INFO prove_core:   0 bn254_double
2025-01-07T14:36:30.824680Z  INFO prove_core:   0 bls12381_decompress
2025-01-07T14:36:30.824682Z  INFO prove_core:   0 bls12381_double
2025-01-07T14:36:30.824684Z  INFO prove_core:   0 sha_compress
2025-01-07T14:36:30.824686Z  INFO prove_core:   0 ed_add
2025-01-07T14:36:30.824688Z  INFO prove_core:   0 keccak_permute
2025-01-07T14:36:30.824690Z  INFO prove_core:   0 secp256k1_add
2025-01-07T14:36:30.824692Z  INFO prove_core:   0 bn254_add
2025-01-07T14:36:30.824694Z  INFO prove_core:   0 uint256_mul
2025-01-07T14:36:30.824696Z  INFO prove_core:   0 bls12381_add
2025-01-07T14:36:30.824717Z  INFO prove_core:   0 bls12381_fp_add
2025-01-07T14:36:30.824720Z  INFO prove_core:   0 bls12381_fp_sub
2025-01-07T14:36:30.824722Z  INFO prove_core:   0 bls12381_fp_mul
2025-01-07T14:36:30.824724Z  INFO prove_core:   0 bls12381_fp2_add
2025-01-07T14:36:30.824726Z  INFO prove_core:   0 bls12381_fp2_sub
2025-01-07T14:36:30.824728Z  INFO prove_core:   0 bls12381_fp2_mul
2025-01-07T14:36:30.824730Z  INFO prove_core:   0 bn254_fp_add
2025-01-07T14:36:30.824732Z  INFO prove_core:   0 bn254_fp_sub
2025-01-07T14:36:30.824734Z  INFO prove_core:   0 bn254_fp_mul
2025-01-07T14:36:30.824736Z  INFO prove_core:   0 bn254_fp2_add
2025-01-07T14:36:30.824739Z  INFO prove_core:   0 bn254_fp2_sub
2025-01-07T14:36:30.824741Z  INFO prove_core:   0 bn254_fp2_mul
2025-01-07T14:36:30.824743Z  INFO prove_core:   0 sha_extend
2025-01-07T14:36:31.069952Z  INFO prove_core: summary: cycles=397728, gas=497216, e2e=51.437448647s, khz=7.73, proofSize=2656912
2025-01-07T14:36:31.083914Z  INFO prove_core: close time.busy=51.7s time.idle=111ms
Successfully generated proof!
2025-01-07T14:36:31.240078Z  INFO verify: close time.busy=155ms time.idle=1.38µs
Successfully verified proof!

real    0m53,977s
user    14m47,180s
sys     0m5,308s

and Ceno

~/ceno$ time (cargo run --release --bin e2e -- --profiling=1 --n=5000 .)

[... warnings ...]

    Finished `release` profile [optimized] target(s) in 0.14s
     Running `target/release/e2e --profiling=1 --n=5000 .`
Cycles: 316640
TRACE    ZKVM_create_proof [ 2.97s | 0.00% / 100.00% ]
TRACE    ┝━ commit_to_fixed_commit [ 114µs | 0.00% ] profiling_1: true
TRACE    ┝━ commit_to_traces [ 1.41s | 47.33% ] profiling_1: true
TRACE    ┕━ main_proofs [ 1.57s | 52.66% ] profiling_1: true
e2e proof stat: proof size = 19115894, hashes count = 42012

real    0m13,210s
user    1m10,816s
sys     0m2,447s

You can see the number of cycles differs (sp1 has about 25% more), but I don't think this can account for the performance gap. And the gap widens, for example, for --n=15000.

@matthiasgoergens
Copy link
Collaborator

matthiasgoergens commented Jan 8, 2025

If I enable link time optimisation for SP1, I get 325_334 cycles. That takes care of most of the difference in cycles, since it's only about 2.7% more than Ceno. (We can investigate that a bit more.)

diff --git a/is_prime/Cargo.toml b/is_prime/Cargo.toml
index 3915b209..9d246545 100644
--- a/is_prime/Cargo.toml
+++ b/is_prime/Cargo.toml
@@ -8,3 +8,14 @@ resolver = "2"
 
 [workspace.dependencies]
 alloy-sol-types = "0.7.7"
+
+[profile.release]
+lto = "fat"
+panic = "abort"
+codegen-units = 1
+
+[profile.host]
+inherits = "release"
+lto = "thin"
+panic = "abort"
+codegen-units = 16
cargo run --profile=host -- --n=5000  --execute

Let me actually run the proving next.

@matthiasgoergens
Copy link
Collaborator

matthiasgoergens commented Jan 8, 2025

When proving on my machine with LTO turned on for SP1 (see comment above), I get the following:

SP1

$ time cargo run --profile=host -- --n=5000  --prove
n: 5000
Successfully generated proof!
Successfully verified proof!

real    0m15.722s
user    5m11.674s
sys     0m4.221s

Ceno

$ time (cargo run --release --bin e2e -- --profiling=1 --n=5000 .)
Cycles: 316640
TRACE    ZKVM_create_proof [ 1.46s | 0.00% / 100.00% ]
TRACE    ┝━ commit_to_fixed_commit [ 67.5µs | 0.00% ] profiling_1: true
TRACE    ┝━ commit_to_traces [ 570ms | 38.97% ] profiling_1: true
TRACE    ┕━ main_proofs [ 893ms | 61.03% ] profiling_1: true
e2e proof stat: proof size = 19115894, hashes count = 42012

real    0m5.965s
user    0m49.337s
sys     0m2.761s

Further results

With n = 20_000 I get this for Ceno

real    0m11.363s
user    2m49.726s
sys     0m8.519s

And this for SP1:

real    0m53.418s
user    23m29.074s
sys     0m11.957s

@matthiasgoergens
Copy link
Collaborator

@mcalancea Could you please do the sorting benchmark for SP1 and ceno? Not the quadratic one, just the one that uses Vec's built-in sorting, please. I suspect that's more representative of real world workloads than 'is-prime' or the quadratic sorting. Thanks!

@mcalancea mcalancea mentioned this pull request Jan 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants