The collection node is responsible for accepting transactions from users, packaging them into collections to ease the load on consensus nodes, and storing transaction texts for the duration they are needed by the network.
This document provides a high-level overview of the collection node. Each section includes links to the appropriate package, which may contain more detailed documentation.
- Collection - a set of transactions proposed by a cluster of collection nodes.
- Guaranteed Collection - a collection that a quorum of nodes in the cluster has committed to storing.
- Collection Guarantee - an attestation to a collection that has been guaranteed.
Concretely, this is a hash over the collection and signatures from a qualified
majority of cluster members. (Sometimes simply referred to as
guarantee.) - Cluster - a group of collection nodes that work together to create collections. Each cluster is responsible for a different subset of transactions.
- Transactions are received by a collection node (typically via an Access Node).
- Transactions are propagated to collection nodes in the responsible cluster.
- Transactions are introduced into the memory pool.
- Transactions are included in a collection proposal.
- Transactions are removed from the memory pool when the collection is guaranteed.
- A collection is proposed by a cluster member.
- The collection is finalized by the core consensus algorithm.
- A guarantee for the finalized collection is submitted to consensus nodes.
- The guarantee is included in a block.
- The block is propagated to execution nodes.
- Execution nodes request the full collection from collection nodes.
The ingest engine is responsible for accepting, validating, and storing new transactions.
Once a transaction has been ingested, it can be included in a new collection via the proposal engine.
In general, collection nodes cannot fully ensure that a transaction is valid. Consequently, the validation performed at this stage is best-effort.
The proposal engine is responsible for handling the consensus process for the cluster.
It runs an instance of HotStuff within the cluster.
This results in each cluster building a secondary blockchain, where each block
represents a proposed collection.
The synchronization engine is responsible for keeping the node in sync with its cluster.
It periodically polls cluster members for their latest finalized collection, and handles
requesting ranges of finalized collections when the node is behind. It also handles
requesting specific collections when the proposal engine receives a new proposal for
which the parent is not known.
The provider engine is responsible for responding to requests for resources stored
by the collection node, typically full collections or individual transactions. In
particular, execution nodes and verification nodes request collections in order to
execute the constituent transactions.
The cluster state implements a storage layer for the blockchain produced by the
proposal engine. The design mirrors the Protocol State,
which is the storage layer for the main Flow blockchain produced by consensus nodes.
Since the core HotStuff logic is only aware of blocks, we sometimes use the term "cluster block" to refer to a block within a cluster's auxiliary blockchain. Each cluster block contains a single proposed collection.
The collection builder implements the logic for building new, valid collections to propose to the cluster.
Valid collections contain transactions that have passed basic validation, are not expired, and do not contain any transactions that already exist in a parent collection.
The collection finalizer marks a collection as finalized (aka guaranteed). This is invoked by the core HotStuff logic when it determines that a collection has been finalized by the cluster.
When a collection is finalized, the corresponding collection guarantee is sent to consensus nodes for inclusion in a block.