Skip to content

Commit

Permalink
edit pipelining
Browse files Browse the repository at this point in the history
  • Loading branch information
@filippoweb3
filippoweb3 committed Nov 7, 2024
1 parent 181e857 commit a1b20ab
Show file tree
Hide file tree
Showing 2 changed files with 118 additions and 119 deletions.
69 changes: 34 additions & 35 deletions docs/learn/learn-async-backing.md
View file
Original file line number Diff line number Diff line change
@@ -1,14 +1,30 @@
---
id: learn-async-backing
title: Asynchronous Backing
sidebar_label: Asynchronous Backing
description: A brief overview of asynchronous backing, and how it affects Polkadot's scalability.
keywords: [parachains, backing, parablock, perspective parachains, unincluded segments]
title: Pipelining
sidebar_label: Pipelining
description: How the Polkadot Cloud achieves pipelining to improve scalability.
keywords: [parachains, backing, parablock, perspective parachains, unincluded segments, pipelining]
slug: ../learn-async-backing
---

import RPC from "./../../components/RPC-Connection";

[Pipelining](<https://en.wikipedia.org/wiki/Pipeline_(computing)>) is a technique for processing
multiple stages of a task simultaneously by breaking it into smaller steps. This allows the next
step to start before the previous one is completely finished. This is often used in processors and
computer architectures to increase throughput.

Polkadot introduces pipelining to the parachain block
[generation, backing, and inclusion](./learn-parachains-protocol.md) via **asynchronous backing**.
It is analogous to the logical pipelining of processor instruction in "traditional" architectures,
where some instructions may be executed before others are complete.

Bundles of state transitions represented as blocks may be processed similarly. In the context of
Polkadot, pipelining aims to increase the throughput of the entire network by completing the backing
and inclusion steps for different blocks simultaneously. Asynchronous backing does not just allow
for pipelining within a single pipe (or core). It lays the foundation for a large number of pipes
(or cores) to run for the same parachain at the same time.

:::tip Asynchronous Backing Guide for Parachains

For upgrading a parachain for Asynchronous Backing compatibility, follow the instructions on
Expand Down Expand Up @@ -57,11 +73,11 @@ relay chain's progression:

Because of (1) parablocks can be generated every other relay chain block (i.e., every 12 seconds).
Because of (2) generation of parablock `P` can only start when `P - 1` is included (there is no
[pipelining](#pipelining)). Because of (3) execution time can take maximum 0.5 seconds as parablock
`P` is rushing to be backed in the next 5.5 seconds (2 seconds needed for backing and the rest for
gossiping). Every parablock is backed in 6 seconds (one relay chain block) and included in the next
6 seconds (next relay chain block). The time from generation to inclusion is 12 seconds. This limits
the amount of data a collator can add to each parablock.
pipelining). Because of (3) execution time can take maximum 0.5 seconds as parablock `P` is rushing
to be backed in the next 5.5 seconds (2 seconds needed for backing and the rest for gossiping).
Every parablock is backed in 6 seconds (one relay chain block) and included in the next 6 seconds
(next relay chain block). The time from generation to inclusion is 12 seconds. This limits the
amount of data a collator can add to each parablock.

Parablock generation will choose the most recently received relay block as a relay parent, although
with an imperfect network that may differ from the true most recent relay block. So, in general, if
Expand Down Expand Up @@ -124,9 +140,8 @@ In synchronous backing, collators generate parablocks using context entirely pul
chain. While in asynchronous backing, collators use additional context from the
[unincluded segment](#unincluded-segments). Parablocks are included every 6 seconds because backing
of parablock `N + 1` and inclusion of parablock `N` can happen on the same relay chain bock
([pipelining](#pipelining)). However, as for synchronous backing, a parablock takes 12 seconds to
get backed and included, and from inclusion to finality there is an additional 30-second time
window.
(pipelining). However, as for synchronous backing, a parablock takes 12 seconds to get backed and
included, and from inclusion to finality there is an additional 30-second time window.

Because the throughput is increased by 2x and parachains have 4x more execution time, asynchronous
backing is expected to deliver 8x more blockspace to parachains.
Expand Down Expand Up @@ -168,13 +183,13 @@ for execution. And so on, P3 can be generated while backing groups check P2, and
while P3 undergoing backing. In 24 seconds, P1 to P3 are included in the relay chain.

Note how there are always three unincluded parablocks at all times, i.e. compared to synchronous
backing there can be multiple unincluded parablocks (i.e. [pipelining](#pipelining)). For example,
when P1 is undergoing inclusion, P2 and P3 are undergoing backing. Collators were able to generate
multiple unincluded parablocks because on their end they have the
[unincluded segment](#unincluded-segments), a local storage of not-included parablock ancestors that
they can use to fetch information to build new parablocks. On the relay chain side,
[perspective parachains](#prospective-parachains) repeats the work each unincluded segment does in
tracking candidates (as validators cannot trust the record kept on parachains).
backing there can be multiple unincluded parablocks (i.e. pipelining). For example, when P1 is
undergoing inclusion, P2 and P3 are undergoing backing. Collators were able to generate multiple
unincluded parablocks because on their end they have the [unincluded segment](#unincluded-segments),
a local storage of not-included parablock ancestors that they can use to fetch information to build
new parablocks. On the relay chain side, [perspective parachains](#prospective-parachains) repeats
the work each unincluded segment does in tracking candidates (as validators cannot trust the record
kept on parachains).

The 6-second relay chain block delay includes a backing execution timeout (2 seconds) and some time
for network latency (the time it takes to gossip messages across the entire network). The limit
Expand Down Expand Up @@ -203,22 +218,6 @@ state roots, and ID info is placed on the parent block on the relay chain. The r
access the entire state of a parachain but only the values that changed during that block and the
merkelized hashes of the unchanged values.

### Pipelining

Asynchronous backing is a feature that introduces
[pipelining](https://www.techtarget.com/whatis/definition/pipelining) to the parachain block
[generation, backing and inclusion](./learn-parachains-protocol.md). It is analogous to the logical
pipelining of processor instruction in "traditional" architectures, where some instructions may be
executed before others are complete. Instructions may also be executed in parallel, enabling
multiple processor parts to work on potentially different instructions simultaneously.

Bundles of state transitions represented as blocks may be processed similarly. In the context of
Polkadot, pipelining aims to increase the throughput of the entire network by completing the backing
and inclusion steps for different blocks at the same time. Asynchronous backing does not just allow
for pipelining within a single pipe (or core). It lays the foundation for a large number of pipes
(or cores) to run for the same parachain at the same time. In that way, we have two distinct new
forms of parallel computation.

### Unincluded Segments

Unincluded segments are chains of candidate parablocks that have yet to be included in the relay
Expand Down
168 changes: 84 additions & 84 deletions polkadot-wiki/sidebars.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -436,90 +436,6 @@ module.exports = {
slug: '/learn-advanced-index',
},
items: [
{
type: "category",
label: "Polkadot-JS",
description: 'Overview of Polkadot-JS.',
link: {
type: 'doc',
id: "learn/learn-polkadotjs",
},
items: [
"general/polkadotjs",
"general/polkadotjs-ui",
{
type: "category",
label: "Polkadot-JS Guides",
description: 'Polkadot-JS Guides about Staking, Asset Hub, Vault App, and more.',
link: {
type: 'doc',
id: "learn/learn-polkadot-js-guides",
},
items: [
{
type: "category",
label: "Account Guides",
description: 'Polkadot-JS Guides about Accounts.',
link: {
type: 'doc',
id: "learn/learn-guides-accounts",
},
items: [
"learn/learn-guides-accounts-multisig",
"learn/learn-guides-accounts-proxy",
"learn/learn-guides-accounts-proxy-pure",
],
},
"learn/learn-guides-transfers",
{
type: "category",
label: "Staking Guides",
description: 'Polkadot-JS Guides about Staking.',
link: {
type: 'doc',
id: 'learn/learn-guides-staking',
},
items: [
"learn/learn-guides-nominator",
"learn/learn-guides-staking-pools",
],
},
"learn/learn-guides-polkadot-opengov",
"learn/learn-guides-treasury",
"learn/learn-guides-bounties",
"learn/learn-guides-identity",
"learn/learn-guides-ledger",
"learn/learn-guides-vault",
{
type: "category",
label: "Asset Hub Guides",
description: 'Polkadot-JS Guides about Asset Hub.',
link: {
type: 'doc',
id: 'learn/learn-guides-assets',
},
items: [
"learn/learn-guides-assets-create",
"learn/learn-guides-asset-conversion",
],
},
{
type: "category",
label: "Bridge Hub Guides",
description: 'Polkadot-JS Guides for Bridge Hub.',
link: {
type: 'doc',
id: 'learn/learn-guides-bridges',
},
items: [
"learn/learn-guides-dot-ksm-bridge",
],
},
"learn/learn-guides-claims",
],
},
],
},
{
type: "category",
label: "Architecture",
Expand Down Expand Up @@ -604,6 +520,90 @@ module.exports = {
"learn/learn-nft-pallets",
"learn/learn-cryptography",
'learn/learn-phragmen',
{
type: "category",
label: "Polkadot-JS",
description: 'Overview of Polkadot-JS.',
link: {
type: 'doc',
id: "learn/learn-polkadotjs",
},
items: [
"general/polkadotjs",
"general/polkadotjs-ui",
{
type: "category",
label: "Polkadot-JS Guides",
description: 'Polkadot-JS Guides about Staking, Asset Hub, Vault App, and more.',
link: {
type: 'doc',
id: "learn/learn-polkadot-js-guides",
},
items: [
{
type: "category",
label: "Account Guides",
description: 'Polkadot-JS Guides about Accounts.',
link: {
type: 'doc',
id: "learn/learn-guides-accounts",
},
items: [
"learn/learn-guides-accounts-multisig",
"learn/learn-guides-accounts-proxy",
"learn/learn-guides-accounts-proxy-pure",
],
},
"learn/learn-guides-transfers",
{
type: "category",
label: "Staking Guides",
description: 'Polkadot-JS Guides about Staking.',
link: {
type: 'doc',
id: 'learn/learn-guides-staking',
},
items: [
"learn/learn-guides-nominator",
"learn/learn-guides-staking-pools",
],
},
"learn/learn-guides-polkadot-opengov",
"learn/learn-guides-treasury",
"learn/learn-guides-bounties",
"learn/learn-guides-identity",
"learn/learn-guides-ledger",
"learn/learn-guides-vault",
{
type: "category",
label: "Asset Hub Guides",
description: 'Polkadot-JS Guides about Asset Hub.',
link: {
type: 'doc',
id: 'learn/learn-guides-assets',
},
items: [
"learn/learn-guides-assets-create",
"learn/learn-guides-asset-conversion",
],
},
{
type: "category",
label: "Bridge Hub Guides",
description: 'Polkadot-JS Guides for Bridge Hub.',
link: {
type: 'doc',
id: 'learn/learn-guides-bridges',
},
items: [
"learn/learn-guides-dot-ksm-bridge",
],
},
"learn/learn-guides-claims",
],
},
],
},
],
},
{
Expand Down

0 comments on commit a1b20ab

Please sign in to comment.