Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

init: fast query DB #229

Closed
wants to merge 18 commits into from
Closed

init: fast query DB #229

wants to merge 18 commits into from

Conversation

emidev98
Copy link
Contributor

@emidev98 emidev98 commented Dec 7, 2023

Implementation

This solution is based on the previous implementation from Mantlemint, StreamingServices and the possibility of setting a query database in a CosmosSDK app. Querying data from the IAVL CosmosSDK can be slow because the checks that have to be done, the MerkleTree re-balances and different write-read processes that are executed in the node. To data access we decided to stream stores data each time a block is commited to a new LevelDB database with multithreading optimizations and without fraudproof-checks because the block is already validated before being commited.

Downsides

There are two main downsides for this implementation, (1) the data will be duplicated if old history of IAVL node is not deleted and (2) the data in this DB does not check merkle tree validity (because there is no merkle tree) so you have to trust the node provider that didn't manipulated the data.

@emidev98 emidev98 changed the base branch from release/v2.8 to release/v2.9 December 7, 2023 17:19
@emidev98 emidev98 changed the title init: fast query DB init(v2.9): fast query DB Dec 7, 2023
@emidev98 emidev98 changed the base branch from release/v2.9 to release/v2.10 January 5, 2024 08:13
@emidev98 emidev98 changed the title init(v2.9): fast query DB init(v2.10): fast query DB Jan 5, 2024
@javiersuweijie javiersuweijie changed the base branch from release/v2.10 to feat/fast_db January 16, 2024 04:25
Copy link

codecov bot commented Jan 16, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

❗ No coverage uploaded for pull request base (feat/fast_db@2cfffef). Click here to learn what that means.

Additional details and impacted files

Impacted file tree graph

@@               Coverage Diff               @@
##             feat/fast_db     #229   +/-   ##
===============================================
  Coverage                ?   69.92%           
===============================================
  Files                   ?       45           
  Lines                   ?     2108           
  Branches                ?        0           
===============================================
  Hits                    ?     1474           
  Misses                  ?      466           
  Partials                ?      168           

@emidev98 emidev98 changed the title init(v2.10): fast query DB init: fast query DB Mar 6, 2024
@gregnuj gregnuj deleted the branch feat/fast_db March 14, 2024 17:23
@gregnuj gregnuj closed this Mar 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants