Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

indexer: index new blocks #797

Closed
p4u opened this issue Feb 6, 2023 · 3 comments
Closed

indexer: index new blocks #797

p4u opened this issue Feb 6, 2023 · 3 comments

Comments

@p4u
Copy link
Member

p4u commented Feb 6, 2023

Blockchain blocks are currently not being handled by the indexer. Instead, we just retrieve them from the Tendermint blockstore once requested by the API endpoint /chain/blocks/height/:height.

There is a need on the Vocdoni explorer side (https://dev.explorer.vote) to retrieve a summary list of blocks; thus they can be listed. This summary needs to include little information, so we can aggregate several blocks on the same page.

Ideally, the indexer SQL database would store the following information.

  • height (primary key)
  • size of the block
  • dataHash
  • number of transactions included in the block
  • number of validator signatures
  • validator proposer address (indexed)
  • number of rounds

Currently, the Vochain application does not have an even handler (https://github.com/vocdoni/vocdoni-node/blob/master/vochain/state/eventlistener.go#L20) for allowing subscription to new blocks, but it might be added. Somehow, this even callback should be executed from vochain.BeginNewBlock().

Finally, the vochain/indexer can use this event call to store new block indxes on its SQL database.

An example on how a block looks like.

{
  "data": {
    "txs": [
      "CioiKAgCEAEaIMXSRgGG9xLCIDZlc5dzbPS+jVmyn3DANgm4AgAAAAAAIAISQWSK4t2xAl9k9PHUZUhUmqdDJKVWKq461fhTYxOnC3U4N0ct29IUhO9bT0KkeM0lFBQLiySB64z9JeR9oWKStlIA"
    ]
  },
  "evidence": {
    "evidence": []
  },
  "header": {
    "appHash": "994F3A12A64EF57F798B453B285177B245738ED562589F958711DB4D950B0779",
    "chainId": "vocdoni-development-76",
    "consensusHash": "9C51FE2637CF9460E108B5008DDEE1D25ECBC8C77926296836786D39C64686E4",
    "dataHash": "5458374E6E80F97E434A7E1AC9619A965E287F37A11CF7F80D656469C3D8F0C3",
    "evidenceHash": "E3B0C44298FC1C149AFBF4C8996FB92427AE41E4649B934CA495991B7852B855",
    "height": 244,
    "lastBlockId": {
      "hash": "0A75E30D4DB8A45E0FEE477448F7DEED1A50ABBB98C0BE4E96E50791F4A26776",
      "parts": {
        "hash": "D2803563C09030D4F89F65350AA71A8C5D661C797F789847B41D3E73262A84B3",
        "total": 1
      }
    },
    "lastCommitHash": "3FE9CD63FE6778877F9338908FB0FAC0741FE2085F901011F006245B995FE252",
    "lastResultsHash": "E333A249A29410449E16B8999E09A9AD5B6F7A24609B3AC8A96EF7488A82CF87",
    "nextValidatorsHash": "E28E7EFDF1EAD64EE3349052D6A1D8547EF0F482B4E467AB4B1250B9E5ED7F86",
    "proposerAddress": "ECCC09A0DF8F4E5554A9C58F634E9D6AFD5F1598",
    "time": "2023-02-06T13:08:22.90530923Z",
    "validatorsHash": "E28E7EFDF1EAD64EE3349052D6A1D8547EF0F482B4E467AB4B1250B9E5ED7F86",
    "version": {
      "app": 0,
      "block": 11
    }
  },
  "lastCommit": {
    "blockId": {
      "hash": "0A75E30D4DB8A45E0FEE477448F7DEED1A50ABBB98C0BE4E96E50791F4A26776",
      "parts": {
        "hash": "D2803563C09030D4F89F65350AA71A8C5D661C797F789847B41D3E73262A84B3",
        "total": 1
      }
    },
    "height": 243,
    "round": 0,
    "signatures": [
      {
        "blockIdFlag": 2,
        "signature": "W8kL6zvKW7HEynYFz4gK1s6mB5jMTqB6FWlJHo1q4PKQqksKUD3OgIoi2ujfbBhWab1qKbjE6kvl4RxbIGG9Cw==",
        "timestamp": "2023-02-06T13:08:22.896071844Z",
        "validatorAddress": "2E1B244B84E223747126EF621C022D5CEFC56F69"
      },
      {
        "blockIdFlag": 2,
        "signature": "ZtuSs8hKth2OFp1xixdzSAtExTXJCxO8mjlm6LDnpAzD3Q74KGPOkr6YCJ1P0+ppy3Gib7Dk4oODgcrPykIVAw==",
        "timestamp": "2023-02-06T13:08:22.90530923Z",
        "validatorAddress": "4EF00A8C18BD472167E67F28694F31451A195581"
      },
      {
        "blockIdFlag": 2,
        "signature": "RaGStdQHt13sxNMq3qkWHVq4QI+DSVAT676QacOgZSfC9R7Y/E4Y24pEwI3qVFlVbrerCilC689H1PtrUkmfBg==",
        "timestamp": "2023-02-06T13:08:22.906692497Z",
        "validatorAddress": "5C69093136E0CB84E5CFA8E958DADB33C0D0CCCF"
      },
      {
        "blockIdFlag": 2,
        "signature": "pcGs6VJgMKBxEXkF1Zu4Cxpa6iLcOGFb2l1RBu5dM4f3mH3ycn3UkDW5FiSJizzOAZcol1AEket8koGpKxeMAQ==",
        "timestamp": "2023-02-06T13:08:22.91373842Z",
        "validatorAddress": "ECCC09A0DF8F4E5554A9C58F634E9D6AFD5F1598"
      }
    ]
  }
}
@p4u
Copy link
Member Author

p4u commented Jul 18, 2023

I understand this is done, right @mvdan ?

@mvdan
Copy link
Contributor

mvdan commented Jul 21, 2023

Yes, this was #993, I forgot to link the issue.

Up to you whether we mark it as fully fixed or not. Note that we currently only index time and data_hash and nothing else, since those were the ones we immediately needed. My opinion is that we should add more as we need them. See https://github.com/vocdoni/vocdoni-node/blob/827f38491c86bbae68bbc2c891094ad8dba519bc/vochain/indexer/migrations/0006_create_table_blocks.sql.

@mvdan
Copy link
Contributor

mvdan commented Sep 11, 2024

#1329 finished this.

@mvdan mvdan closed this as completed Sep 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants