Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

go/extra/conbench: Add consensus benchmark command #3183

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

abukosek
Copy link
Contributor

No description provided.

@abukosek abukosek force-pushed the andrej/feature/consensus-bench branch 3 times, most recently from fc5d9c3 to 524ed49 Compare August 11, 2020 11:16
@codecov
Copy link

codecov bot commented Aug 11, 2020

Codecov Report

Merging #3183 into master will decrease coverage by 0.22%.
The diff coverage is 0.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #3183      +/-   ##
==========================================
- Coverage   66.02%   65.80%   -0.23%     
==========================================
  Files         371      371              
  Lines       33285    33287       +2     
==========================================
- Hits        21978    21906      -72     
- Misses       8074     8134      +60     
- Partials     3233     3247      +14     
Impacted Files Coverage Δ
go/oasis-node/cmd/common/pprof/pprof.go 19.64% <0.00%> (-0.73%) ⬇️
go/worker/compute/executor/committee/batch.go 66.66% <0.00%> (-16.67%) ⬇️
go/worker/compute/executor/committee/state.go 71.42% <0.00%> (-14.29%) ⬇️
go/consensus/tendermint/abci/state/state.go 54.54% <0.00%> (-9.10%) ⬇️
go/consensus/tendermint/epochtime/epochtime.go 85.18% <0.00%> (-7.41%) ⬇️
go/worker/common/committee/runtime_host.go 65.04% <0.00%> (-4.86%) ⬇️
go/worker/compute/executor/committee/node.go 62.19% <0.00%> (-3.08%) ⬇️
go/storage/mkvs/lookup.go 72.63% <0.00%> (-2.11%) ⬇️
go/consensus/tendermint/abci/mux.go 56.74% <0.00%> (-2.06%) ⬇️
go/storage/client/client.go 69.85% <0.00%> (-1.92%) ⬇️
... and 21 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bb1aa48...b487942. Read the comment docs.

@abukosek abukosek force-pushed the andrej/feature/consensus-bench branch 13 times, most recently from 4ceebbc to 387784c Compare August 12, 2020 17:54
@peterjgilbert peterjgilbert requested a review from pro-wh August 12, 2020 20:07
Copy link
Contributor

@pro-wh pro-wh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so far so good

@pro-wh
Copy link
Contributor

pro-wh commented Aug 12, 2020

a little reminiscent of https://github.com/oasisprotocol/oasis-core/blob/master/go/oasis-node/cmd/debug/txsource/workload/parallel.go the parallel workload. this is more robust to failures and adds measurements so we can get a transaction result out. a long time ago we were using a scheme where we'd take the middle 80% of the transactions to get the peak rate.

// Cut 10% at the beginning and the end.

@abukosek abukosek force-pushed the andrej/feature/consensus-bench branch 3 times, most recently from 7ca7793 to 32764f8 Compare August 13, 2020 14:47
@abukosek
Copy link
Contributor Author

Thanks for taking a look at this. I can add the transaction cutting, but IMO it's not needed -- in my test runs, only the first block at the start of the benchmark had fewer transactions, the other blocks were always full. As long as there are enough samples (so that the duration of the benchmark is long enough), it won't significantly affect the results.

@abukosek abukosek marked this pull request as ready for review August 13, 2020 15:17
@abukosek abukosek force-pushed the andrej/feature/consensus-bench branch 2 times, most recently from 9f8f23c to 2fd24b3 Compare August 14, 2020 15:47
@abukosek abukosek force-pushed the andrej/feature/consensus-bench branch 13 times, most recently from 780ea81 to 396a9d3 Compare September 28, 2020 11:36
@abukosek abukosek force-pushed the andrej/feature/consensus-bench branch 10 times, most recently from 322c811 to c21c1f6 Compare October 5, 2020 06:24
@abukosek abukosek force-pushed the andrej/feature/consensus-bench branch 3 times, most recently from 1976c1d to 2c80821 Compare October 7, 2020 13:16
@abukosek abukosek force-pushed the andrej/feature/consensus-bench branch from 2c80821 to b487942 Compare October 13, 2020 09:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants