Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: ope poc for store #204

Conversation

dudong2
Copy link

@dudong2 dudong2 commented Oct 17, 2024

Description

Closes: #XXXX


Author Checklist

All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.

I have...

  • included the correct type prefix in the PR title
  • added ! to the type prefix if API or client breaking change
  • targeted the correct branch (see PR Targeting)
  • provided a link to the relevant issue or specification
  • followed the guidelines for building modules
  • included the necessary unit and integration tests
  • added a changelog entry to CHANGELOG.md
  • included comments for documenting Go code
  • updated the relevant documentation or specification
  • reviewed "Files changed" and left comments if necessary
  • confirmed all CI checks have passed

Reviewers Checklist

All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.

I have...

  • confirmed the correct type prefix in the PR title
  • confirmed ! in the type prefix if API or client breaking change
  • confirmed all author checklist items have been addressed
  • reviewed state machine logic
  • reviewed API design and naming
  • reviewed documentation is accurate
  • reviewed tests and test coverage
  • manually tested (if applicable)

yihuang and others added 11 commits October 15, 2024 18:30
generic interface

generic btree

generic cachekv

generic transient store

support ObjStore

changelog

Update CHANGELOG.md

Signed-off-by: yihuang <[email protected]>

object store key

Apply review suggestions

fix merge conflict

fix snapshot

revert dependers
Solution:
- remove the api

changelog
* Problem: MultiStore interface is bloated

Solution:
- Split out specialied methods from it, keeping the MultiStore generic

* Update store/CHANGELOG.md

Signed-off-by: yihuang <[email protected]>

---------

Signed-off-by: yihuang <[email protected]>
* Problem: nested cache store not efficient

Solution:
- introduce copy-on-write btree based cache store

temp

* changelog

* rename

* Update store/cachekv/store.go

Signed-off-by: yihuang <[email protected]>

---------

Signed-off-by: yihuang <[email protected]>
Solution:
- init cachestore on cachestore lazily.
- cleanup some unused stuff.

Update store/CHANGELOG.md

Signed-off-by: yihuang <[email protected]>
* Support RunAtomic API

* add unit test
Solution:
- fix and add test to support the usage pattern in ethermint

add Discard method to CacheWrap

better testing
…258)

Solution:
- add API NewFromParent to cache multistore.

Update store/CHANGELOG.md

Signed-off-by: yihuang <[email protected]>

fix test

fix lint
@dudong2 dudong2 self-assigned this Oct 17, 2024
Comment on lines +148 to +150
for _, store := range cms.stores {
store.Discard()
}

Check warning

Code scanning / CodeQL

Iteration over map Warning

Iteration over map may be a possible source of non-determinism
store = cms.initStore(key, cms.parentStore(key))
}
if key == nil || store == nil {
panic(fmt.Sprintf("kv store with key %v has not been registered in stores", key))

Check warning

Code scanning / CodeQL

Panic in BeginBock or EndBlock consensus methods Warning

Possible panics in BeginBock- or EndBlock-related consensus methods could cause a chain halt
panic(fmt.Sprintf("kv store with key %v has not been registered in stores", key))
store, ok := cms.getCacheWrapper(key).(types.KVStore)
if !ok {
panic(fmt.Sprintf("store with key %v is not KVStore", key))

Check warning

Code scanning / CodeQL

Panic in BeginBock or EndBlock consensus methods Warning

Possible panics in BeginBock- or EndBlock-related consensus methods could cause a chain halt
Comment on lines +225 to +227
for k, v := range cms.stores {
stores[k] = v.(types.BranchStore).Clone().(types.CacheWrap)
}

Check warning

Code scanning / CodeQL

Iteration over map Warning

Iteration over map may be a possible source of non-determinism
Comment on lines +252 to +259
for k, v := range other.stores {
store, ok := cms.stores[k]
if !ok {
store = cms.initStore(k, cms.parentStore(k))
}

store.(types.BranchStore).Restore(v.(types.BranchStore))
}

Check warning

Code scanning / CodeQL

Iteration over map Warning

Iteration over map may be a possible source of non-determinism
func (ts *BTreeStore[V]) Iterator(start, end []byte) types.GIterator[V] {
it, err := ts.BTree.Iterator(start, end)
if err != nil {
panic(err)

Check warning

Code scanning / CodeQL

Panic in BeginBock or EndBlock consensus methods Warning

Possible panics in BeginBock- or EndBlock-related consensus methods could cause a chain halt
func (ts *BTreeStore[V]) ReverseIterator(start, end []byte) types.GIterator[V] {
it, err := ts.BTree.ReverseIterator(start, end)
if err != nil {
panic(err)

Check warning

Code scanning / CodeQL

Panic in BeginBock or EndBlock consensus methods Warning

Possible panics in BeginBock- or EndBlock-related consensus methods could cause a chain halt
store := types.KVStore(s)
store, ok := s.(types.KVStore)
if !ok {
panic(fmt.Sprintf("store with key %v is not KVStore", key))

Check warning

Code scanning / CodeQL

Panic in BeginBock or EndBlock consensus methods Warning

Possible panics in BeginBock- or EndBlock-related consensus methods could cause a chain halt
// AssertValidValueLength checks if the value length is within length limit
func AssertValidValueLength(l int) {
if l > MaxValueLength {
panic(errors.New("value is too large"))

Check warning

Code scanning / CodeQL

Panic in BeginBock or EndBlock consensus methods Warning

Possible panics in BeginBock- or EndBlock-related consensus methods could cause a chain halt
Comment on lines +233 to +238
for k, v := range cms.stores {
if _, ok := other.stores[k]; !ok {
// clear the cache store if it's not in the other
v.Discard()
}
}

Check warning

Code scanning / CodeQL

Iteration over map Warning

Iteration over map may be a possible source of non-determinism
yihuang and others added 5 commits October 23, 2024 13:57
* feat: save some memory allocations on un-used cache store

* Update CHANGELOG.md

Signed-off-by: yihuang <[email protected]>

* Update store/internal/btree/btree.go

Signed-off-by: yihuang <[email protected]>

---------

Signed-off-by: yihuang <[email protected]>
…#588)

* Problem: versiondb/memiavl don't compatible with upstream sdk

Solution:
- avoid api breaking change

* fix build

Co-Authored-By: yihuang <[email protected]>
Copy link

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@github-actions github-actions bot added the Stale label Nov 23, 2024
@github-actions github-actions bot closed this Nov 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants