indexer-alt: index chain identifier #20148
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
3 Skipped Deployments
|
amnn
temporarily deployed
to
sui-typescript-aws-kms-test-env
GitHub Actions
Inactive
This was referenced Nov 2, 2024
wlmyng
approved these changes
Nov 2, 2024
| ); | ||
|
|
||
| -- Index to ensure there can only be one row in the genesis table. | ||
| CREATE UNIQUE INDEX IF NOT EXISTS kv_genesis_unique |
amnn
force-pushed
the
amnn/idx-refactors
branch
from
d1ebdf9
to
6016da8
Compare
amnn
force-pushed
the
amnn/idx-chain-id
branch
from
846cc2a
to
d0bde05
Compare
amnn
temporarily deployed
to
sui-typescript-aws-kms-test-env
GitHub Actions
Inactive
## Description Pipeline for indexing Display VersionUpdated events, so readers can render Display for Objects. ## Test plan Run the indexer on the first ~1.6M checkpoints (the first display events happen around checkpoint 1.5M), and then inspect the database: ``` sui$ cargo run -p sui-indexer-alt --release -- \ --database-url "postgres://postgres:postgrespw@localhost:5432/sui_indexer_alt" \ indexer --remote-store-url https://checkpoints.mainnet.sui.io \ --consistent-range 3600 ``` ``` sui_indexer_alt=# SELECT object_type, display_id FROM sum_displays LIMIT 10; object_type | display_id ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------- \x72f9c76421170b5a797432ba9e1b3b2e2b7cf6faa26eb955396c773af2479e1e0967616d655f383139320847616d653831393200 | \xb06e4607642f6c08e196f8bd84b653c784864a6badca2066e63db42e846fdd4c \x72ad8f46d4fc3bbc24df15f6c62a98044c2649a9a31895a957254b2cf16e0cb905646d656e7305446d656e7300 | \x831cd05a3de12ae3dcea4382e9f56eadded8efd83c742036acd789fc41f1a03f \x72ad8f46d4fc3bbc24df15f6c62a98044c2649a9a31895a957254b2cf16e0cb905646d656e7309446d656e734d65746100 | \x3e8acb139222c9e335980f8b427270e92d276152140545bfde46c700472a2e11 \xee496a0cc04d06a345982ba6697c90c619020de9e274408c7819f787ff66e1a1087375696672656e73075375694672656e0107ee496a0cc04d06a345982ba6697c90c619020de9e274408c7819f787ff66e1a10463617079044361707900 | \x3d53effd472191435e7735ff9ed0b3083ef10dee4793db51df92038cea48b139 \x5b45da03d42b064f5e051741b6fed3b29eb817c7923b83b92f37a1d2abf4fbab036e6674034e667400 | \x23453afb61ba4e0c9f040839c0d620c2cab774491656c086e10871aae8f8b3e9 \xbb1531504c9c3235d3cd637ed9573cbe18461255b4175a1cb1e1b07b8aa8e11b0873686f77636173650853686f776361736500 | \xa306db0c6d3f5cbb8169143c22da5b61caf3576f1b3f3e77537aef8d4ca4b515 \xbb1531504c9c3235d3cd637ed9573cbe18461255b4175a1cb1e1b07b8aa8e11b0473756961054d6564616c00 | \x70e764f567b976d8e654064c14a7fbd94f0994a229fb0a7e119d80b4f1e4b760 \xbb1531504c9c3235d3cd637ed9573cbe18461255b4175a1cb1e1b07b8aa8e11b04737569610d506572736f6e616c4d6564616c00 | \xb438c4cbda18b0890de8ca2a1813be70fc3b64fedf171ea7c658d43605a11ece \xbb1531504c9c3235d3cd637ed9573cbe18461255b4175a1cb1e1b07b8aa8e11b08746573745f6e66740b53756961546573744e465400 | \xa37afca51a40bc90e30991631515f2ab8b9ecef90e51fbf4af12aa0ea5413250 \xf1e7b4093872df432b2adb50533a1fa6f9e4b20856217657242fe7d9d4ac42e80b6574686f735f73717561640a4574686f7353717561640107f1e7b4093872df432b2adb50533a1fa6f9e4b20856217657242fe7d9d4ac42e80b6574686f735f73717561640b4574686f7353717561643100 | \x7c330d61f99af7983e8714b4e63437d93813316ac54e8f83103b2ab2a021997e (10 rows) ```
## Description Make it possible to access the ingestion client on the ingestion service and indexer. This will be used to bootstrap the indexer which may need to wait for the genesis checkpoint to initialise certain tables and state before indexing proper starts. ## Test plan CI
## Description Factor out the logic used by the ingestion broadcaster to retry fetching a checkpoint if it is not found. This change is behaviour preserving but the new `wait_for` function will be re-used in some bootstrapping logic that waits for the existence of the genesis checkpoint. ## Test plan CI
## Description ...to be used in a bootstrapping script.
## Description Add explicit `NOT NULL`s to the key columns inside the event filter tables. This is not a behavioural change as these columns are already `NOT NULL` because they are part of the primary key. As per an earlier PR comment, relying on the implicit `NOT NULL` behaviour was confusing when reading migrations (and other migration scripts use the explicit pattern). ## Test plan CI
## Description Allow `graceful_shutdown` to accept tasks that return something other than `()`. ## Test plan CI
## Description Index the chain identifier and initial protocol version in a `kv_genesis` table. This takes the place of chain identifier indexing and will also be used by indexers for protocol configs and feature flags, which need to know the initial protocol version to index and the chain identifier. ## Test plan Run the indexer twice, and note from the tracing messages that the first time, the indexer is bootstrapped, and the second time it reads from the bootstrapped table.
amnn
force-pushed
the
amnn/idx-refactors
branch
from
6016da8
to
cbd76fc
Compare
amnn
force-pushed
the
amnn/idx-chain-id
branch
from
d0bde05
to
719db00
Compare
amnn
temporarily deployed
to
sui-typescript-aws-kms-test-env
GitHub Actions
Inactive
8 tasks
amnn
added a commit
that referenced
this pull request
Nov 5, 2024
## Description Index the chain identifier and initial protocol version in a `kv_genesis` table. This takes the place of chain identifier indexing and will also be used by indexers for protocol configs and feature flags, which need to know the initial protocol version to index and the chain identifier. ## Test plan Run the indexer twice, and note from the tracing messages that the first time, the indexer is bootstrapped, and the second time it reads from the bootstrapped table. ## Stack - #20118 - #20132 - #20147 --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API:
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Index the chain identifier and initial protocol version in a
kv_genesistable. This takes the place of chain identifier indexing and will also be used by indexers for protocol configs and feature flags, which need to know the initial protocol version to index and the chain identifier.Test plan
Run the indexer twice, and note from the tracing messages that the first time, the indexer is bootstrapped, and the second time it reads from the bootstrapped table.
Stack
Release notes
Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required.
For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates.