From f1ebb6d0b04f7adaae2364de3fa44e5b7367e131 Mon Sep 17 00:00:00 2001 From: Mak Date: Wed, 24 Jul 2024 14:08:06 +0100 Subject: [PATCH] Update weight-generation.md --- docs/weight-generation.md | 110 +++++++++----------------------------- 1 file changed, 25 insertions(+), 85 deletions(-) diff --git a/docs/weight-generation.md b/docs/weight-generation.md index 6904db0ca3..07ac2a3df3 100644 --- a/docs/weight-generation.md +++ b/docs/weight-generation.md @@ -2,100 +2,40 @@ To generate weights for a runtime -1. Build `chain-spec-generator` with `--profile production --features runtime-benchmarks` -2. Use it to build a chain spec for your runtime, e.g. `./target/production/chain-spec-generator --raw polkadot-local > polkadot-chain-spec.json` -3. Create `file_header.txt` +In a PR run the actions through comment: -```text -// Copyright (C) Parity Technologies and the various Polkadot contributors, see Contributions.md -// for a list of specific contributors. -// SPDX-License-Identifier: Apache-2.0 +```sh +/cmd bench --help # outputs the actual usage documentation with examples and supported runtimes -// Licensed under the Apache License, Version 2.0 (the "License"); -// you may not use this file except in compliance with the License. -// You may obtain a copy of the License at -// -// http://www.apache.org/licenses/LICENSE-2.0 -// -// Unless required by applicable law or agreed to in writing, software -// distributed under the License is distributed on an "AS IS" BASIS, -// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -// See the License for the specific language governing permissions and -// limitations under the License. -``` - -4. `rsync` chain spec/s and the file header to a benchmark machine - -5. Build `polkadot` binary from the latest release of `polkadot-sdk` with `--profile production --features runtime-benchmarks --bin polkadot` on the benchmark machine - -6. Run on the benchmark machine: - -```bash -#!/bin/bash - -# Default value is 'polkadot', but you can override it by passing a different value as an argument -CHAIN=${1:-polkadot} - -pallets=($( - ./target/production/polkadot benchmark pallet --list \ - --chain=./$CHAIN-chain-spec.json | - tail -n+2 | - cut -d',' -f1 | - sort | - uniq -)); +# or -mkdir -p ./$CHAIN-weights -for pallet in "${pallets[@]}"; do - output_file=./$CHAIN-weights/ - # a little hack for pallet_xcm_benchmarks - we want to output them to a nested directory - if [[ "$pallet" == "pallet_xcm_benchmarks::generic" ]] || [[ "$pallet" == "pallet_xcm_benchmarks::fungible" ]]; then - mkdir -p ./$CHAIN-weights/xcm - output_file="${output_file}xcm/${pallet//::/_}.rs" - fi - echo "Running benchmarks for $pallet to $output_file" - ./target/production/polkadot benchmark pallet \ - --chain=./$CHAIN-chain-spec.json \ - --steps=50 \ - --repeat=20 \ - --pallet=$pallet \ - --extrinsic=* \ - --wasm-execution=compiled \ - --heap-pages=4096 \ - --output="$output_file" \ - --header=./file_header.txt -done +/cmd --help # to see all available commands ``` -You probably want to do this inside a `tmux` session or something similar (e.g., `nohup &`), as it will take a while (several hours). - -7. `rsync` the weights back to your local machine, replacing the existing weights. - -8. Manually fix XCM weights by -- Replacing `impl xxx::yyy::WeightInfo for WeightInfo {` with `impl WeightInfo {` -- Marking all functions `pub(crate)` -- Removing any unused functions - -9. Commit the weight changes. +To generate weights for all pallets in a particular runtime(s), run the following command: +```sh +/cmd bench --runtime kusama polkadot +``` -10. Ensure the changes are reasonable. If not installed, `cargo install subweight`, check the weight changes: - ``` - subweight compare commits \ - --path-pattern "./**/weights/**/*.rs" \ - --method asymptotic \ - --ignore-errors \ - \ - ` - ``` - _Hint1: Add `--format markdown --no-color` for markdown-compatible results._ +> **📝 Note**: The action is not being run right-away, it will be queued and run in the next available runner. So might be quick, but might also take up to 10 mins (That's in control of Github). +Once the action is run, you'll see reaction 👀 on original comment, and if you didn't pass `--quiet` - it will also send a link to a pipeline when started, and link to whole workflow when finished. - _Hint2: Change `--path-pattern "./**/weights/**/*.rs"` to e.g. `--path-pattern "./relay/polkadot/weights/**/*.rs"` for a specific runtime._ +> **💡Hint #1** : if you run all runtimes or all pallets, it might be that some pallet in the middle is failed to generate weights, thus it stops (fails) the whole pipeline. +> If you want, you can make it to continue running, even if some pallets are failed, add `--continue-on-fail` flag to the command. The report will include which runtimes/pallets have failed, then you can re-run them separately after all is done. - _Hint3: Add `--change added changed` to include only relevant changes._ +This way it runs all possible runtimes for the specified pallets, if it finds them in the runtime +```sh +/cmd bench --pallet pallet_balances pallet_xcm_benchmarks::generic pallet_xcm_benchmarks::fungible +``` -## FAQ +If you want to run all specific pallet(s) for specific runtime(s), you can do it like this: +```sh +/cmd bench --runtime bridge-hub-polkadot --pallet pallet_xcm_benchmarks::generic pallet_xcm_benchmarks::fungible +``` -### What benchmark machine spec should I use? -See the [Polkadot Wiki Reference Hardware](https://wiki.polkadot.network/docs/maintain-guides-how-to-validate-polkadot#standard-hardware). +> **💡Hint #2** : Sometimes when you run too many commands, or they keep failing and you're rerunning them again, it's handy to add `--clean` flag to the command. This will clean up all yours and bot's comments in PR relevant to /cmd commands. +```sh +/cmd bench --runtime kusama polkadot --pallet=pallet_balances --clean --continue-on-fail +```