Skip to content

Commit

Permalink
New feature release (#27)
Browse files Browse the repository at this point in the history
Add support for NTT, Update configs initialisation, fix module visibility
  • Loading branch information
seemenkina authored Aug 17, 2023
1 parent 097ba53 commit 98226ea
Show file tree
Hide file tree
Showing 30 changed files with 1,140 additions and 441 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -13,3 +13,4 @@ Cargo.lock
*DS_Store*
*.csv
.env
test_data
46 changes: 22 additions & 24 deletions Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,40 +1,38 @@
[package]
name = "ingo-blaze"

This comment has been minimized.

Copy link
@wdnm8848

wdnm8848 Feb 19, 2024

So mu

version = "0.3.0"
edition = "2021"
authors = [ "Ingonyama" ]
authors = ["Ingonyama"]
description = "Library for ZK acceleration on Xilinx FPGAs."
edition = "2021"
homepage = "https://www.ingonyama.com"
name = "ingo-blaze"
repository = "https://github.com/ingonyama-zk/blaze"
version = "0.4.0"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
libc = "0.2.126"
ark-algebra-intro = "0.3.0"
ark-ec = "0.3.0"
ark-std = "0.3.0"
ark-ff = "0.3.0"
anyhow = "1.0.68"
ark-bls12-377 = "0.3.0"
ark-bn254 = "0.3.0"
ark-bls12-381 = "0.3.0"
ark-bn254 = "0.3.0"
ark-ec = "0.3.0"
ark-ff = "0.3.0"
ark-std = "0.3.0"
criterion = "0.4.0"
csv = "1.1"
dotenv = "0.15.0"
env_logger = "0.10.0"
libc = "0.2.126"
log = "0.4.0"
packed_struct = "0.10"
anyhow = "1.0.68"
csv = "1.1"
serde = { version = "1", features = ["derive"] }
num = "0.4"
strum = "0.24"
strum_macros = "0.24"
num-bigint = "0.4"
num-traits = "0.2.15"
packed_struct = "0.10"
rand = "0.8.5"
criterion = "0.4.0"
lazy_static = "1.4.0"
bindgen = "0.64.0"
dotenv = "0.15.0"
rayon = "1.6.1"
serde_json = { version = "1.0" }
serde-hex = "0.1.0"
strum = "0.24"
strum_macros = "0.24"
thiserror = "1.0"
num-bigint = "0.4"
num-traits = "0.2.15"

[[bench]]
harness = false
name = "ntt_bench"
86 changes: 68 additions & 18 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# BLAZE

<div align="center">blaze is a Rust library for ZK acceleration on Xilinx FPGAs.</div>

![ingo_BlazeFire_5d](https://github.com/ingonyama-zk/blaze/assets/2446179/6460173b-02af-4023-b055-c8274a0cbc21)
Expand All @@ -24,14 +25,15 @@ On the connection side, we can provide an API to retrieve any necessary data (in

The [DriverClient](src/driver_client/) module is designed to establish a connection between the FPGA/AWS and a known type of card, such as the C1100 card. It does not possess any knowledge about primitives.

The [DriverClient](src/driver_client/) provides basic IO methods and can load a binary, as well as provide specific and debug information about current HW. For a specific card type, the [DriverConfig](src/driver_client/dclient.rs) remains the same and can be accessed using the `driver_client_c1100_cfg` function.
The [DriverClient](src/driver_client/) provides basic IO methods and can load a binary, as well as provide specific and debug information about current HW. For a specific card type, the [DriverConfig](src/driver_client/dclient.rs) remains the same and can be accessed using the `driver_client_cfg` function.

It is important to note that the high-level management layer determines which client and primitive should be used. The [DriverClient](src/driver_client/) can be overused in this process.

How to create a new connection:

```rust
let dclient = DriverClient::new(&id, DriverConfig::driver_client_c1100_cfg());
let dclient = DriverClient::new(&id,
DriverConfig::driver_client_cfg(CardType::C1100));
```

### DriverPrimitive
Expand All @@ -43,14 +45,14 @@ The configuration (e.g. for msm there are addresses space and curve description)
To create a new primitive instance for MSM, for example, one would use the following code:

```rust
let dclient = DriverClient::new(&id, DriverConfig::driver_client_c1100_cfg());
let driver = msm_api::MSMClient::new(
msm_api::MSMInit {
mem_type: msm_api::PointMemoryType::DMA,
is_precompute: false,
curve: msm_api::Curve::BLS381,
},
dclient,
let dclient = DriverClient::new(&id, DriverConfig::driver_client_cfg(CardType::C1100));
let driver = MSMClient::new(
MSMInit {
mem_type: PointMemoryType::DMA,
is_precompute: true,
curve: Curve::BLS381,
},
dclient,
);
```

Expand All @@ -60,14 +62,11 @@ For data encapsulation, methods specific to each primitive can be divided into p

### General Example of usage

We will refer to any type of primitive as `DriverPrimitiveClient` to show generality. And any abbreviation for a specific primitive will be replaced by `dpc` (e.g. `dpc_api` can be `msm_api` )
We will refer to any type of primitive as `DriverPrimitiveClient` to show generality.

```rust
let dclient = DriverClient::new(&id, DriverConfig::driver_client_c1100_cfg());
let driver = dpc_api:: DriverPrimitiveClient::new(dpc_api::dpc_type, dclient);

let params = driver.get_loaded_binary_parameters();
let params_parse = dpc_api::DPCImageParametrs::parse_image_params(params[1]);
let dclient = DriverClient::new(&id, DriverConfig::driver_client_cfg(CardType::C1100));
let driver = DriverPrimitiveClient::new(dpc_type, dclient);

let _ = driver.initialize(dpc_param);
let _ = driver.set_data(dpc_input);
Expand Down Expand Up @@ -114,6 +113,30 @@ MSMInput = {
}
```

## NTT (Number Theoretic Transform) Module

This module implements the calculation of NTT of size `2^27`. To use it, the input byte vector of elements must be specified. Each element must be represented in little-endian. The result will be a similar byte vector.

It is worth noting that the data transfer process is slightly different from other modules. The following is an example of how to use NTT. More details can be found here: [LINK TO BLOG]

```rust
let dclient = DriverClient::new(&id, DriverConfig::driver_client_cfg(CardType::C1100));
let driver = NTTClient::new(NTT::Ntt, dclient);
let buf_host = 0;
let buf_kernel = 0;
driver.set_data(NTTInput {
buf_host,
data: in_vec,
})?;
driver.driver_client.initialize_cms()?;
driver.driver_client.reset_sensor_data()?;

driver.initialize(NttInit {})?;
driver.start_process(Some(buf_kernel))?;
driver.wait_result()?;
let res = driver.result(Some(buf_kernel))?.unwrap();
```

## Poseidon Module

## Running tests and benchmark
Expand All @@ -125,7 +148,6 @@ To run tests for the MSM primitive, use the following command:
```
RUST_LOG=<LEVEL_LOG> cargo test -- <TEST_FILE> -- <TEST_NAME>
```

Also, different tests can require additional parameters:
Expand All @@ -137,8 +159,36 @@ Also, it's possible to set up a number of points in MSM in the `MSM_SIZE` variab

If the values of `ID` and `MSM_SIZE` are not provided, they will be defaulted to `ID=0` and `MSM_SIZE=8192`.

### Poseidon tests
### NTT tests

To run tests for the NTT primitive, use the following command:

```
INFNAME=<INPUT_VEC_FILE> OUTFNAME=<REFERENCE_OUT_VEC> RUST_LOG=<LEVEL_LOG> cargo test -- integration_ntt
```

Also, different tests can require additional parameters:
`ID` `INFNAME`, and `OUTFNAME`.

Replace `<LEVEL_LOG>` with the desired log level (e.g. info, debug). Set `INFNAME` with the path to the input vector in little-endian byte format. Since we are testing correctness, set the path to the file with which you want to compare the result for the `OUTFNAME` variable. It should also be a little-endian byte vector
file and `ID` with the number of the FPGA slot.

If the value of `ID` is not provided, they will be defaulted to `ID=0`.

### NTT benchmark

Benchmarks for NTT are located in the benches directory, it's worth clarifying that there is no correctness check inside the benchmark - for that use the tests.

To run bench for the NTT primitive, use the following command:

```
INFNAME=<INPUT_VEC_FILE> RUST_LOG=<LEVEL_LOG> cargo bench
```

Also, bench can require additional parameters: `ID` and `INFNAME`. Set `INFNAME` with the path to the input vector in little-endian byte format.

If the value of `ID` is not provided, they will be defaulted to `ID=0`.

### Poseidon tests
54 changes: 54 additions & 0 deletions benches/ntt_bench.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
use std::{env, fs::File, io::Read};

use criterion::*;
use ingo_blaze::{driver_client::*, ingo_ntt::*};
use log::info;

fn bench_ntt_calc(c: &mut Criterion) {
env_logger::try_init().expect("Invalid logger initialisation");
let id = env::var("ID").unwrap_or_else(|_| 0.to_string());
let fname = env::var("FNAME").unwrap();
let mut f = File::open(fname).expect("no file found");
let mut in_vec: Vec<u8> = Default::default();
let _ = f.read_to_end(&mut in_vec);

let buf_host = 0;
let buf_kernel = 0;

info!("Create Driver API instance");
let dclient = DriverClient::new(&id, DriverConfig::driver_client_cfg(CardType::C1100));
let driver = NTTClient::new(NTT::Ntt, dclient);
log::info!("Starting set NTT data");
let _ = driver.set_data(NTTInput {
buf_host,
data: in_vec,
});
log::info!("Successfully set NTT data");
let _ = driver.driver_client.initialize_cms();
let _ = driver.driver_client.reset_sensor_data();

let mut group = c.benchmark_group("NTT computation");

log::info!("Starting NTT");
group.bench_function("NTT", |b| {
b.iter(|| {
let _ = driver.initialize(NttInit {});
let _ = driver.start_process(Some(buf_kernel));
let _ = driver.wait_result();
let _ = driver.driver_client.reset();
})
});
group.finish();
log::info!("Finishing NTT");

log::info!("Try to get NTT result");
let res = driver.result(Some(buf_kernel)).unwrap();
log::info!("NTT result: {:?}", res.unwrap().len());
}

criterion_group! {
name = benches;
config = Criterion::default().sample_size(10);
targets = bench_ntt_calc
}
criterion_main!(benches);
12 changes: 0 additions & 12 deletions configs/c1100_cfg.json

This file was deleted.

7 changes: 0 additions & 7 deletions configs/msm_bls377_dma_cfg.json

This file was deleted.

6 changes: 0 additions & 6 deletions configs/msm_bls377_hbm_cfg.json

This file was deleted.

7 changes: 0 additions & 7 deletions configs/msm_bls381_dma_cfg.json

This file was deleted.

6 changes: 0 additions & 6 deletions configs/msm_bls381_hbm_cfg.json

This file was deleted.

7 changes: 0 additions & 7 deletions configs/msm_bn254_dma_cfg.json

This file was deleted.

Loading

0 comments on commit 98226ea

Please sign in to comment.