Run benchmarks locally with criterion

[orottier]

Fixes #188

You can now run cargo bench locally to run the same benches as we run in CI.
It's a lot of boilerplate currently, I'm still looking for improvements.
Also we need to move to unify examples/benchmarks.rs in here.

[b-ma]

maybe a macro would be justified to generate the code from here to the bottom of the file

[b-ma]

Cool!

Also we need to move to unify examples/benchmarks.rs in here.

Do you want to port all the benches? If yes I can do it once you are satisfied with the general layout (...I would like to have more time to contribute, but at least I can do the small things :)

In any case, I think it's nice to keep examples/benchmarks.rs as it allows to compare with browser's perfs (even if it's quite noisy)

[github-actions]

Benchmark result:


bench_ctor              time:   [3.5576 ms 3.5637 ms 3.5728 ms]
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) high mild
  1 (1.00%) high severe

bench_audio_buffer_decode
                        time:   [9.7278 µs 9.7292 µs 9.7309 µs]
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high mild

bench_sine              time:   [13.530 ms 13.550 ms 13.573 ms]
Found 4 outliers among 100 measurements (4.00%)
  3 (3.00%) high mild
  1 (1.00%) high severe

bench_sine_gain         time:   [14.015 ms 14.037 ms 14.060 ms]
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high mild

bench_sine_gain_delay   time:   [22.345 ms 22.362 ms 22.381 ms]
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild

bench_buffer_src        time:   [4.1488 ms 4.1539 ms 4.1593 ms]
Found 8 outliers among 100 measurements (8.00%)
  7 (7.00%) high mild
  1 (1.00%) high severe

bench_buffer_src_delay  time:   [12.306 ms 12.325 ms 12.344 ms]
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

bench_buffer_src_iir    time:   [7.7479 ms 7.7603 ms 7.7727 ms]

bench_buffer_src_biquad time:   [7.2028 ms 7.2133 ms 7.2243 ms]
Found 5 outliers among 100 measurements (5.00%)
  5 (5.00%) high mild

bench_stereo_positional time:   [7.2202 ms 7.2313 ms 7.2427 ms]

bench_stereo_panning_automation
                        time:   [5.9868 ms 5.9967 ms 6.0076 ms]
Found 9 outliers among 100 measurements (9.00%)
  6 (6.00%) high mild
  3 (3.00%) high severe

bench_analyser_node     time:   [6.7329 ms 6.7423 ms 6.7522 ms]
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high mild

bench_hrtf_panners      time:   [31.189 ms 31.197 ms 31.206 ms]
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

[orottier]

Do you want to port all the benches?

Yes and no.
Yes, because criterion helps us to accurately run these benches for academic publishing. No because some of the benchmarks are way too heavy to run often and in CI. Maybe there's some middle ground?

In any case the current CI benches are not bad. Although I would like to have some coverage on resampling, upmixing and granular synth.

Any help is appreciated, maybe you can help with the macro's? and maybe copy over 3 or 4 of the lightweight benches from examples/ to benches/?

Another interesting observation is that iai and criterion don't seem to agree always. Both agreed that #379 was a performance win, but criterion tells me that #279 is also beneficial, whereas iai reports +10% instruction count.. To be investigated

[orottier]

#380 (comment)

hah, you triggered the bench action with your previous comment.
It ran criterion in CI because I believe the bench command (which now needs the --features iai flag) is invoked from the main branch but is then run on the local checkout. This should be fixed after merging the PR; CI should run iai