-
Notifications
You must be signed in to change notification settings - Fork 8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Introduce CSV related tools to master #69
Changes from all commits
0ee207e
a05cd73
d113cd6
390b9a1
caaed2c
201206a
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -4,10 +4,10 @@ This directory contains docker related scripts and configurations to compile and | |
|
||
The environment is created as Docker Image (by default named and tagged as `smart-bench:latest`) which leverages multi-stage Docker builds. Helper scripts available in this directory streamline operations to build ([build.sh](./build.sh)) and run ([run.sh](./run.sh)) of the image. | ||
|
||
At the first stage of docker build, based upon `docker.io/paritytech/ci-linux:production` as rust enabled compilation env, smart-bench software is being compiled from current sources (this git repository). Resulting binary is copied into second stage / final docker image based on `ubuntu:20.04` image where also other dependencies are configured and installed. | ||
At the first stage of docker build, based upon `docker.io/paritytech/ci-unified:bullseye-1.74.0` as rust enabled compilation env, smart-bench software is being compiled from current sources (this git repository). Resulting binary is copied into second stage / final docker image based on `ubuntu:20.04` image where also other dependencies are configured and installed. | ||
|
||
Final image consists of few pieces that are integrated into it: | ||
- `polkadot`, `zombienet` and `polkadot-parachain` pre-compiled binaries taken from offical releases | ||
- `polkadot`, `polkadot-parachain` and `zombienet` pre-compiled binaries taken from offical releases | ||
- `moonbeam` binary provided by custom release created on [this fork](https://github.com/karolk91/moonbeam) (custom fork was required to enable dev rpc module which is not supported by offical releases) | ||
- pre-compiled smart contracts available within this repository at [contracts](../contracts/) dir | ||
- [entrypoint.sh](./entrypoint.sh) script that orchestrates running of the above to eventually receive benchmarking results from `smart-bench` itself. The script accepts set of parameters to be passed as-is to `smart-bench` app. It runs zombienet as a background job and starts smart-bench while providing it with correct parameters to reach to zombienet nodes. | ||
|
@@ -28,7 +28,7 @@ Usage scenarios above are possible by providing volume mounts arguments to `dock | |
|
||
a) downloads dependencies | ||
``` | ||
./downloads-bins.sh | ||
./download-bins.sh | ||
``` | ||
|
||
b) build smart-bench:latest image | ||
|
@@ -43,7 +43,7 @@ VERSION=1.0 ./build.sh | |
|
||
|
||
### Step 2. Run image | ||
## `run.sh` help screen: | ||
#### `run.sh` help screen: | ||
``` | ||
Usage: ./run.sh OPTION -- ARGUMENTS_TO_SMART_BENCH | ||
|
||
|
@@ -130,17 +130,18 @@ docker run --rm -it --init -v $PWD/configs:/usr/local/smart-bench/config smart-b | |
--- | ||
|
||
## Miscellaneous | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We are not using prefunded account on master - master is compatible with polkadot 1.0.0 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. will remove the section, thanks for spotting this |
||
### Moonbeam with Dev RPC module enabled build recipe | ||
|
||
Following is an example recipe how to build moonbeam binary with Dev RPC module enabled | ||
``` | ||
git clone https://github.com/PureStake/moonbeam.git | ||
cd moonbeam && git fetch https://github.com/karolk91/moonbeam master && git cherry-pick decd877 | ||
docker run -it --rm -v $PWD:/moonbeam docker.io/paritytech/ci-linux:production /bin/bash | ||
docker run -it --rm -v $PWD:/moonbeam docker.io/paritytech/ci-unified:bullseye-1.74.0 /bin/bash | ||
cd /moonbeam | ||
rustup toolchain install 1.69 | ||
rustup default 1.69 | ||
rustup override set 1.69 | ||
rustup target add wasm32-unknown-unknown --toolchain 1.69 | ||
rustup toolchain install 1.74 | ||
rustup default 1.74 | ||
rustup override set 1.74 | ||
rustup target add wasm32-unknown-unknown --toolchain 1.74 | ||
cargo build --release | ||
``` |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,75 @@ | ||
#!/bin/bash | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. maybe we could add output-csv option to smart-bench? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. sure, thats good idea, we can think of this pull request as some intermediate step towards better solution There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. OK, lets create an issue for improvement There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I agree this might be a better idea just to add the option to output as csv to smart-bench itself. |
||
|
||
set -euo pipefail | ||
|
||
SCRIPT_NAME="${BASH_SOURCE[0]}" | ||
SCRIPT_PATH=$(dirname "$(realpath -s "${BASH_SOURCE[0]}")") | ||
TIMESTAMP=$(date +%s) | ||
|
||
if [ -p /dev/stdin ]; then | ||
STATS=$(</dev/stdin) | ||
else | ||
echo "No input was found on stdin, skipping!" | ||
fi | ||
|
||
function echoerr() { echo "$@" 1>&2; } | ||
|
||
function usage { | ||
cat << EOF | ||
Script to translate smart bench stdout data into csv formatted benchmarking results and append to given output file | ||
Script expects data to be piped from smart-bench application into this script | ||
|
||
Usage: smart-bench ..... | ${SCRIPT_NAME} ARGS | ||
|
||
ARGS | ||
-o, --csv-output (Required) CSV formatted output of smart-bench | ||
-t, --timestamp (Optional) Timestamp to use for benchmark results - if not provided, current time is used | ||
-h, --help Print this help message | ||
|
||
EXAMPLE | ||
cargo run --release -- evm flipper --instance-count 1 --call-count 1500 --url ws://localhost:9988 | ${SCRIPT_NAME} --csv-output=benchmark-result.csv --timestamp=1714515934 | ||
EOF | ||
} | ||
|
||
function parse_args { | ||
function needs_arg { | ||
if [ -z "${OPTARG}" ]; then | ||
echoerr "No arg for --${OPT} option" | ||
exit 2 | ||
fi | ||
} | ||
|
||
# shellcheck disable=SC2214 | ||
while getopts o:t:h-: OPT; do | ||
# support long options: https://stackoverflow.com/a/28466267/519360 | ||
if [ "$OPT" = "-" ]; then # long option: reformulate OPT and OPTARG | ||
OPT="${OPTARG%%=*}" # extract long option name | ||
OPTARG="${OPTARG#"$OPT"}" # extract long option argument (may be empty) | ||
OPTARG="${OPTARG#=}" # if long option argument, remove assigning `=` | ||
fi | ||
case "$OPT" in | ||
o | csv-output) needs_arg && CSV_OUTPUT="${OPTARG}";; | ||
t | timestamp ) needs_arg && TIMESTAMP="${OPTARG}";; | ||
h | help ) usage; exit 0;; | ||
??* ) echoerr "Illegal option --$OPT"; exit 2;; # bad long option | ||
? ) exit 2 ;; # bad short option (error reported via getopts) | ||
esac | ||
done | ||
shift $((OPTIND-1)) # remove parsed options and args from $@ list | ||
|
||
[ -n "${CSV_OUTPUT-}" ] || { | ||
echoerr "missing -c/--csv-output arg" | ||
echoerr "" | ||
usage | ||
exit 2 | ||
} | ||
} | ||
|
||
parse_args "$@" | ||
|
||
|
||
platform=$(echo ${STATS} | grep -o 'Platform: [a-z0-9-]*' | awk '{print $2}') | ||
contract_types=$(echo ${STATS} | grep -o 'Contracts: [+a-z0-9-]*' | awk '{print $2}') | ||
tps=$(echo ${STATS} | grep -o 'sTPS: [0-9]\+\.[0-9]\{2\}' | awk '{print $2}') | ||
|
||
echo "${TIMESTAMP}, ${platform}, n/a, ${contract_types}, ${tps}, n/a, n/a" >> "${CSV_OUTPUT}" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
which moonbeam branch is compatible with polkadot v1.0.0?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
its the same moonbeam version for both branches