From 2d812c32a128ea3e3a59f529dfe2abcd3c5222ce Mon Sep 17 00:00:00 2001 From: marbar3778 Date: Fri, 19 Jul 2024 18:09:58 +0200 Subject: [PATCH 1/7] start adr75 for server v2 --- docs/architecture/adr-75-v2.md | 115 +++++++++++++++++++++++++++++++++ 1 file changed, 115 insertions(+) create mode 100644 docs/architecture/adr-75-v2.md diff --git a/docs/architecture/adr-75-v2.md b/docs/architecture/adr-75-v2.md new file mode 100644 index 000000000000..e2f00ae12163 --- /dev/null +++ b/docs/architecture/adr-75-v2.md @@ -0,0 +1,115 @@ +# ADR 75: V2 Upgrades + +## Changelog + +* 2023-11-07: Initial Draft + +## Status + +DRAFT + +## Abstract + +V2 is a reset in the Cosmos SDK architecture. It is a complete rewrite of the SDK, with a focus on modularity, extensibility, and performance. The V2 SDK breaks apart the core SDK into smaller modular components allowing users to pick and choose the components they need for their specific use case. This document outlines the changes and migration path for users of the V1 SDK. + +> "If you can't explain it simply, you don't understand it well enough." Provide +> a simplified and layman-accessible explanation of the ADR. +> A short (~200 word) description of the issue being addressed. + +## Context + +The Cosmos SDK began in 2016, at this time the software was written with the direct use case of the Cosmos Hub.Since then we have seen the SDK evolve and grow, with new features and improvements being added over time. The SDK today is used by over 100 different projects, with more users joining the ecosystem every day. This has led to a number of challenges, including: + +* The SDK is becoming increasingly complex, with many different components and dependencies. +* The SDK is becoming more difficult to maintain, with frequent breaking changes and compatibility issues. + +V2 is a complete rewrite of the Cosmos SDK, with a focus on modularity, extensibility, and performance. The goal is to make the Cosmos SDK easier to use for the various use cases that we have seen emerging in the ecosystem. + +```mermaid +graph TD + Z[CometBFT] --> A[Baseapp] + A[BaseApp] --> B[ABCI Methods] + A --> C[State Management] + A --> D[Transaction Processing] + A --> E[Query Handling] + + B --> B1[InitChain] + B --> B2[Info] + B --> B3[CheckTx] + B --> B4[PrepareProposal] + B --> B5[ProcessProposal] + B --> B6[FinalizeBlock] + B --> B7[Commit] + + C --> C1[Store] + C1 --> C2[Iavl] + + D --> D1[runTx] + D1 --> D2[Module] + + E --> E1[Query] + E1 --> D2[Module] +``` + +This is a high-level overview of Baseapp today. As we can see baseapp houses all the logic for the ABCI methods, state management, transaction processing, and query handling. This has led baseapp to be a very large monolith. + + + + +## Alternatives + +The alternative to doing a rewrite is to spend more time cleaning up baseapp. This would not fix issues around forking the repository to make changes like we see today. Keeping the current codebase does not allow the project to progress and reduce the maintenance burden on the project. + +> This section describes alternative designs to the chosen design. This section +> is important and if an adr does not have any alternatives then it should be +> considered that the ADR was not thought through. + +## Decision + +> This section describes our response to these forces. It is stated in full +> sentences, with active voice. "We will ..." +> {decision body} + +## Consequences + +> This section describes the resulting context, after applying the decision. All +> consequences should be listed here, not just the "positive" ones. A particular +> decision may have positive, negative, and neutral consequences, but all of them +> affect the team and project in the future. + +### Backwards Compatibility + +> All ADRs that introduce backwards incompatibilities must include a section +> describing these incompatibilities and their severity. The ADR must explain +> how the author proposes to deal with these incompatibilities. ADR submissions +> without a sufficient backwards compatibility treatise may be rejected outright. + +### Positive + +> {positive consequences} + +### Negative + +> {negative consequences} + +### Neutral + +> {neutral consequences} + +## Further Discussions + +> While an ADR is in the DRAFT or PROPOSED stage, this section should contain a +> summary of issues to be solved in future iterations (usually referencing comments +> from a pull-request discussion). +> +> Later, this section can optionally list ideas or improvements the author or +> reviewers found during the analysis of this ADR. + +## Test Cases [optional] + +Test cases for an implementation are mandatory for ADRs that are affecting consensus +changes. Other ADRs can choose to include links to test cases if applicable. + +## References + +* {reference link} From 7cba3f82dc292056fb3cdf2130f2938b62bd893f Mon Sep 17 00:00:00 2001 From: marbar3778 Date: Mon, 29 Jul 2024 14:09:16 +0200 Subject: [PATCH 2/7] add some diagrams --- docs/architecture/adr-75-v2.md | 39 +++++++++++++++++++++++++--------- 1 file changed, 29 insertions(+), 10 deletions(-) diff --git a/docs/architecture/adr-75-v2.md b/docs/architecture/adr-75-v2.md index e2f00ae12163..60ae2950c85b 100644 --- a/docs/architecture/adr-75-v2.md +++ b/docs/architecture/adr-75-v2.md @@ -12,10 +12,6 @@ DRAFT V2 is a reset in the Cosmos SDK architecture. It is a complete rewrite of the SDK, with a focus on modularity, extensibility, and performance. The V2 SDK breaks apart the core SDK into smaller modular components allowing users to pick and choose the components they need for their specific use case. This document outlines the changes and migration path for users of the V1 SDK. -> "If you can't explain it simply, you don't understand it well enough." Provide -> a simplified and layman-accessible explanation of the ADR. -> A short (~200 word) description of the issue being addressed. - ## Context The Cosmos SDK began in 2016, at this time the software was written with the direct use case of the Cosmos Hub.Since then we have seen the SDK evolve and grow, with new features and improvements being added over time. The SDK today is used by over 100 different projects, with more users joining the ecosystem every day. This has led to a number of challenges, including: @@ -53,19 +49,42 @@ graph TD This is a high-level overview of Baseapp today. As we can see baseapp houses all the logic for the ABCI methods, state management, transaction processing, and query handling. This has led baseapp to be a very large monolith. - - - ## Alternatives The alternative to doing a rewrite is to spend more time cleaning up baseapp. This would not fix issues around forking the repository to make changes like we see today. Keeping the current codebase does not allow the project to progress and reduce the maintenance burden on the project. -> This section describes alternative designs to the chosen design. This section -> is important and if an adr does not have any alternatives then it should be -> considered that the ADR was not thought through. ## Decision +The Descision is to rewrite the core componenets (baseapp, server, store) of the SDK into smaller modules. + +These components will be broken into separate go.mods. The modules consist of the following: + +* Cometbft +* Appmanager +* STF (State Transition Function) +* Server/v2 +* Store/v2 +* Runtime/v2 + +```mermaid +graph TD + subgraph Server + Mempool + E[Vote Extensions] + F[Prepare & Process Proposal] + Consensus + end +Server <--> A[AppManager] +A[AppManager] <--> B["STF(State Transition Function)"] +B <--> C[Bank] +B <--> D[Auth] +B <--> G[Staking] +Server --> H[Storage] +H --> I[State Storage] +H --> J[State Commitment] +``` + > This section describes our response to these forces. It is stated in full > sentences, with active voice. "We will ..." > {decision body} From df3262ff9e72afd1fe0fff34761d706bc2b5c881 Mon Sep 17 00:00:00 2001 From: marbar3778 Date: Tue, 30 Jul 2024 10:28:22 +0200 Subject: [PATCH 3/7] ++ --- docs/architecture/adr-75-v2.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/architecture/adr-75-v2.md b/docs/architecture/adr-75-v2.md index 60ae2950c85b..6e989bb0b73b 100644 --- a/docs/architecture/adr-75-v2.md +++ b/docs/architecture/adr-75-v2.md @@ -67,6 +67,8 @@ These components will be broken into separate go.mods. The modules consist of th * Store/v2 * Runtime/v2 +The flow of all the components was designed to be as simple as possible, and the flow is as follows: + ```mermaid graph TD subgraph Server @@ -85,6 +87,8 @@ H --> I[State Storage] H --> J[State Commitment] ``` +In the above diagram we do not mention runtime/v2 because it is the componend that is responsible for combining all the other components into a single application. + > This section describes our response to these forces. It is stated in full > sentences, with active voice. "We will ..." > {decision body} From c3b4847f526444f44d1a5d9cda20e0a8e796349d Mon Sep 17 00:00:00 2001 From: marbar3778 Date: Tue, 30 Jul 2024 17:53:59 +0200 Subject: [PATCH 4/7] ++ --- docs/architecture/adr-75-v2.md | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-) diff --git a/docs/architecture/adr-75-v2.md b/docs/architecture/adr-75-v2.md index 6e989bb0b73b..46c72be29ef6 100644 --- a/docs/architecture/adr-75-v2.md +++ b/docs/architecture/adr-75-v2.md @@ -60,7 +60,7 @@ The Descision is to rewrite the core componenets (baseapp, server, store) of the These components will be broken into separate go.mods. The modules consist of the following: -* Cometbft +* Consensus * Appmanager * STF (State Transition Function) * Server/v2 @@ -89,6 +89,23 @@ H --> J[State Commitment] In the above diagram we do not mention runtime/v2 because it is the componend that is responsible for combining all the other components into a single application. +### Consensus + +Consensus is the component that handles communication to the Consensus Engine (Networking & Consensus). The default implementation will be CometBFT, but other consensus engines can be used with v2. + +```mermaid +graph TD + subgraph Consensus + Mempool + E[Vote Extensions] + F[Prepare & Process Proposal] + end +Consensus <-->|ABCI| A[CometBFT] +``` + +Consensus handles the connection to the consensus engine, is the only component in the V2 design with write access to the state, is decoding transactions, passing the information to the appmanager to be processed and + + > This section describes our response to these forces. It is stated in full > sentences, with active voice. "We will ..." > {decision body} From c0af7124a2cc4a5e86c27ee43415cc61b12a1997 Mon Sep 17 00:00:00 2001 From: marbar3778 Date: Mon, 16 Sep 2024 15:10:24 +0200 Subject: [PATCH 5/7] add information --- docs/architecture/adr-75-v2.md | 112 +++++++++++++++++++++++---------- 1 file changed, 78 insertions(+), 34 deletions(-) diff --git a/docs/architecture/adr-75-v2.md b/docs/architecture/adr-75-v2.md index 46c72be29ef6..c54aeffbcdf3 100644 --- a/docs/architecture/adr-75-v2.md +++ b/docs/architecture/adr-75-v2.md @@ -61,7 +61,6 @@ The Descision is to rewrite the core componenets (baseapp, server, store) of the These components will be broken into separate go.mods. The modules consist of the following: * Consensus -* Appmanager * STF (State Transition Function) * Server/v2 * Store/v2 @@ -77,8 +76,7 @@ graph TD F[Prepare & Process Proposal] Consensus end -Server <--> A[AppManager] -A[AppManager] <--> B["STF(State Transition Function)"] +Server <--> B["STF(State Transition Function)"] B <--> C[Bank] B <--> D[Auth] B <--> G[Staking] @@ -91,7 +89,28 @@ In the above diagram we do not mention runtime/v2 because it is the componend th ### Consensus -Consensus is the component that handles communication to the Consensus Engine (Networking & Consensus). The default implementation will be CometBFT, but other consensus engines can be used with v2. +Consensus is the component that handles communication to the Consensus Engine (Networking & Consensus). The default implementation will be CometBFT, but other consensus engines can be used with v2. The goal of consensus is not to offer a consensus API to meet all needs, but a way to allow developers to swap out the server for a different consensus engine. An application developer should not assume that the cometbftserver will work with other consensus engines. + +Consensus is the component that controls the interaction with the consensus engine, concurrency and execution models. For context, there are three execution models we have identified: + +* **Immediate**: + * Immediate execution differs from what Cosmos SDK utilizes today. In CometBFT, consensus at height N is executed at height N+1. + * Immediate execution refers to coming to consensus at height N for the transactions in the same block. +* **Optimistic** + * Optimistic execution can mean different things to different applications. THe way we imagine it working is that consensus may not be made on every block. Instead consensus is made after execution. This design favors a fast chain as it will not slow down for execution until the optimistic window may be exceeded. +* **Delayed** + * Delayed execution is the default execution model in CometBFT. Consensus is made after execution, but the execution may be delayed until the next block. + +The consensus server is responsible for handling the execution model of the state machine. The state machine executes transactions when it is told, it is unaware of the execution model. + +Since consensus servers can be swapped there are certain features features specific to consensus engines need to be implemented in the server. In the CometBFT server we have implemented the following features: + +* Mempool +* Prepare & Process Proposal +* Vote Extensions +* Snapshots + +If another consensus server would like to utilize the above features they can be copied or implmented in the server. ```mermaid graph TD @@ -103,52 +122,77 @@ graph TD Consensus <-->|ABCI| A[CometBFT] ``` -Consensus handles the connection to the consensus engine, is the only component in the V2 design with write access to the state, is decoding transactions, passing the information to the appmanager to be processed and - - -> This section describes our response to these forces. It is stated in full -> sentences, with active voice. "We will ..." -> {decision body} +### State Transition Function + +The state transition function is the component that handles the execution of transactions. It is responsible for calling the correct message handler for the transaction. The state transition function is stateless, it is handed a read only view of state and returns state changes, the changes returned can be handled by consensus in anyway needed. + +```go +type StateTransitionFunction[T transaction.Tx] interface { + // DeliverBlock executes a block of transactions. + DeliverBlock( + ctx context.Context, + block *server.BlockRequest[T], + state store.ReaderMap, + ) (blockResult *server.BlockResponse, newState store.WriterMap, err error) + + // ValidateTx validates a transaction. + ValidateTx( + ctx context.Context, + state store.ReaderMap, + gasLimit uint64, + tx T, + ) server.TxResult + + // Simulate executes a transaction in simulation mode. + Simulate( + ctx context.Context, + state store.ReaderMap, + gasLimit uint64, + tx T, + ) (server.TxResult, store.WriterMap) + + // Query executes a query on the application. + Query( + ctx context.Context, + state store.ReaderMap, + gasLimit uint64, + req transaction.Msg, + ) (transaction.Msg, error) +} +``` ## Consequences -> This section describes the resulting context, after applying the decision. All -> consequences should be listed here, not just the "positive" ones. A particular -> decision may have positive, negative, and neutral consequences, but all of them -> affect the team and project in the future. +The design of the node comes with a number of tradeoffs. + +* Maintenance cost can be the same as existing Baseapp as handling many go.mods is a overhead. +* Modularity means different layers of abstractions, abstractions always have a cost. ### Backwards Compatibility -> All ADRs that introduce backwards incompatibilities must include a section -> describing these incompatibilities and their severity. The ADR must explain -> how the author proposes to deal with these incompatibilities. ADR submissions -> without a sufficient backwards compatibility treatise may be rejected outright. +The state machine was made to not affect modules that are not using the state transition function. If a user would like to migrate to v2 they will need to migrate to `appmodule.Environment` from `sdk.Context`. `sdk.Context` is a struct which is a global in the state machine, this desing limits the concurrency. + +V2 will have a breaking changes in regards to how CometBFT handles certain fields in ABCI. Previously, the Cosmos SDK paniced and recovered in the case of out of gas, providing an error to CometBFT which we do not return in the new design. + +V2 only works with `Store/v2`, `IAVL V1` can be used with `Store/v2`. This allows chains to continue with existing databases. There will be a migration happening to convert the database to the separation of Storage and Commitment. Once the migration is completed the state machine will query information from the rawDB unless otherwise specified. ### Positive -> {positive consequences} +* Ability to add new features to the SDK without forking the entire repository. +* Ability to create custom node configurations. +* Reduced maintenance cost burden. +* State machine is more performant. +* Isolated components allow for easier testing. +* Allow the team to delete a lot of code in `github.com/cosmos/cosmos-sdk`. ### Negative -> {negative consequences} +* Modularity and abstractions can be a burden. -### Neutral - -> {neutral consequences} ## Further Discussions -> While an ADR is in the DRAFT or PROPOSED stage, this section should contain a -> summary of issues to be solved in future iterations (usually referencing comments -> from a pull-request discussion). -> -> Later, this section can optionally list ideas or improvements the author or -> reviewers found during the analysis of this ADR. - -## Test Cases [optional] - -Test cases for an implementation are mandatory for ADRs that are affecting consensus -changes. Other ADRs can choose to include links to test cases if applicable. +* After reducing the feature set of Cosmos SDK, we can more easily look into rewriting the core into rust. This is dependent on crosslang. ## References From d99d189c3bc6fcf4708d90d368be24d2102da253 Mon Sep 17 00:00:00 2001 From: marbar3778 Date: Fri, 4 Oct 2024 12:10:24 +0200 Subject: [PATCH 6/7] add diagram for stf --- docs/architecture/adr-75-v2.md | 38 ++++++++++++++++++++++++++++++++-- 1 file changed, 36 insertions(+), 2 deletions(-) diff --git a/docs/architecture/adr-75-v2.md b/docs/architecture/adr-75-v2.md index c54aeffbcdf3..307a7340fb11 100644 --- a/docs/architecture/adr-75-v2.md +++ b/docs/architecture/adr-75-v2.md @@ -97,7 +97,7 @@ Consensus is the component that controls the interaction with the consensus engi * Immediate execution differs from what Cosmos SDK utilizes today. In CometBFT, consensus at height N is executed at height N+1. * Immediate execution refers to coming to consensus at height N for the transactions in the same block. * **Optimistic** - * Optimistic execution can mean different things to different applications. THe way we imagine it working is that consensus may not be made on every block. Instead consensus is made after execution. This design favors a fast chain as it will not slow down for execution until the optimistic window may be exceeded. + * Optimistic execution means different things to different applications. The way we imagine it working is that consensus may not be made on every block. Instead consensus is made after execution. This design favors a fast chain as it will not slow down for execution until the optimistic window may be exceeded. * **Delayed** * Delayed execution is the default execution model in CometBFT. Consensus is made after execution, but the execution may be delayed until the next block. @@ -110,7 +110,7 @@ Since consensus servers can be swapped there are certain features features speci * Vote Extensions * Snapshots -If another consensus server would like to utilize the above features they can be copied or implmented in the server. +If another consensus server would like to utilize the above features they can be copied or implemented in the server. ```mermaid graph TD @@ -122,10 +122,44 @@ graph TD Consensus <-->|ABCI| A[CometBFT] ``` +:::Note +ABCI, Vote Extensions, and Prepare & Process Proposal are primitives of cometbft, V2 is not tied to these features, teams do not have to adhere to them if they implement their own consensus engine. +::: + ### State Transition Function The state transition function is the component that handles the execution of transactions. It is responsible for calling the correct message handler for the transaction. The state transition function is stateless, it is handed a read only view of state and returns state changes, the changes returned can be handled by consensus in anyway needed. +The state transition function interface is simple and meant to be as light weight as possible. This is the only interface that is required to be implemented by bespoke consensus engines. + +```mermaid +graph TD + subgraph STF[State Transition Funciton] + BR --> DB + subgraph DB[DeliverBlock] + PB[PreBlock] + BB[BeginBlock] + DT["Deliver Transaction(s)"] + EB[EndBlock] + end + BR[BlockRequest] + Q[Query] + S[Simulate] + VT[Validate TX] + end + C -->|Decoded Transactions| STF + DBR --> STF + STF --> |ChangeSets, Events, Transaction Results| C2 + C2 -->|Commit To DB| D + + C[Consensus] + C2[Consensus] + D[DataBase] + DBR[DataBase Reader] +``` + +State Transition function interface: + ```go type StateTransitionFunction[T transaction.Tx] interface { // DeliverBlock executes a block of transactions. From d51cc4a5426585c9aa17dcea101866f29c066908 Mon Sep 17 00:00:00 2001 From: marbar3778 Date: Wed, 16 Oct 2024 17:42:03 +0200 Subject: [PATCH 7/7] add api --- docs/architecture/adr-75-v2.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/architecture/adr-75-v2.md b/docs/architecture/adr-75-v2.md index 307a7340fb11..44eecc0d2411 100644 --- a/docs/architecture/adr-75-v2.md +++ b/docs/architecture/adr-75-v2.md @@ -75,6 +75,7 @@ graph TD E[Vote Extensions] F[Prepare & Process Proposal] Consensus + API[API] end Server <--> B["STF(State Transition Function)"] B <--> C[Bank]