Skip to content

Commit

Permalink
Merge pull request #22 from rarimo/feature/better-cscalist-cli
Browse files Browse the repository at this point in the history 
Feature: better cscalist cli
  • Loading branch information
@violog
violog authored May 6, 2024
2 parents bad658f + 9056364 commit b6d3b39
Show file tree
Hide file tree
Showing 4 changed files with 281 additions and 77 deletions.
87 changes: 45 additions & 42 deletions docs/common/core/003-csca-list-proposal-guide.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ When obtained the passport signer's root certificate, you can check its presence
stored on Rarimo chain in a decentralized manner.
This is how you ensure that the passport is signed by an eligible entity.

Technical details can be found in [CSCA List Rarimo Core module](https://github.com/rarimo/rarimo-core/x/cscalist/README.md).
Technical details can be found in [CSCA List Rarimo Core module](../../../x/cscalist/README.md).

The keys of CSCA signers can be compromised or updated, ergo list updates must be handled.
This is done via proposals. The guide will help you in creating and verifying proposals on list changes.
Expand All @@ -38,10 +38,11 @@ You can call the CLI from that dir, or add it to your `$PATH`:
export PATH=$PATH:$(go env GOPATH)/bin
```

<!-- TODO: where the user can get RPC? -->
Some commands require network RPC in order to fetch state.
You can put it into environment variable for convenient command calls:
All commands require Rarimo home directory set.
Also, some of them require network RPC in order to fetch state.
You can put these into environment variable for convenient command calls:
```bash
export RARIMO_HOME=/path/to/your/rarimo/home
export RPC=https://your-rpc
```

Expand All @@ -59,79 +60,81 @@ This is why you have to monitor the list manually if you are interested in up-to
### Compare with state

```bash
rarimo-cored --node $RPC query cscalist ldif-tree-diff your-file.ldif
rarimo-cored --home $RARIMO_HOME --node $RPC query cscalist ldif-tree-diff your-file.ldif
```
If roots are different, the list was changed and the proposal must be created to update the tree.

## Create proposal

Most likely, you have already filled basic proposal data in the block scan (title, description, etc.). You should have the file with CSCA list, see **Download the list** subsection. Assuming the file name is `your-file.ldif`.

Obtain primary data for your proposal:
Next, obtain primary data for your proposal, which depends on the proposal type.

### ReplaceCSCAListProposal

```bash
cd ~/Downloads # or wherever you have the file
rarimo-cored query cscalist parse-ldif --output-format hash your-file.ldif your-output-file.txt
rarimo-cored --home $RARIMO_HOME query cscalist parse-ldif your-file.ldif your-output-file.txt
```

`your-output-file.txt` will contain the big list of hashes of public keys from the LDIF file. Example:
```
0x1d4dd579478a38c00f58a4d94263ff2bb0459992c073ebb7a6991194e44157f2
0x0dcc4019fccc7ad4fbb535a40633cc32f99a18096a736b21e695e35e964209ae
0x01a3d79b678d79a8f912b693c4d57b38cf0e44ef413b7684e92a664e98c911ed
"0x1d4dd579478a38c00f58a4d94263ff2bb0459992c073ebb7a6991194e44157f2",
"0x0dcc4019fccc7ad4fbb535a40633cc32f99a18096a736b21e695e35e964209ae",
"0x01a3d79b678d79a8f912b693c4d57b38cf0e44ef413b7684e92a664e98c911ed"
```
If this is `ReplaceCSCAListProposal`, just copy the hashes to the proposal `leaves` field.

For `EditCSCAListProposal` the process is more complicated:
These are the hashes of compatible public keys from the provided LDIF file.
Just copy them to the `leaves` field of your draft proposal.

### Method 1
### EditCSCAListProposal

1. Obtain the old Master List LDIF file, which tree is stored on-chain at the moment.
If you don't have it, use **Method 2** instead.
2. Optional: ensure that the list from file is the same as on-chain:
```bash
rarimo-cored --node $RPC query cscalist ldif-tree-diff your-old-file.ldif
cd ~/Downloads # or wherever you have the file
rarimo-cored --home $RARIMO_HOME query cscalist prepare-proposal your-file.ldif your-output-file.txt
```
You expect to see message that the trees' roots are the same.
Otherwise, seek for an actual `your-old-file.ldif`.
3. Extract the hashes from your old file:
```bash
rarimo-cored query cscalist parse-ldif --output-format hash your-old-file.ldif old-output-file.txt

`your-output-file.txt` will contain two lists of hashes to add and remove. Example:
```
=== To add ===
"0x1d4dd579478a38c00f58a4d94263ff2bb0459992c073ebb7a6991194e44157f2",
"0x0dcc4019fccc7ad4fbb535a40633cc32f99a18096a736b21e695e35e964209ae"
=== To remove ===
"0x01a3d79b678d79a8f912b693c4d57b38cf0e44ef413b7684e92a664e98c911ed"
```
4. Compare `your-output-file.txt` and `old-output-file.txt` to find the differences.
You can use `diff`, `vimdiff` or any preferable tool.
- hashes present only in `your-output-file.txt` are to be added
- hashes present only in `old-output-file.txt` are to be removed
5. Put acquired hashes into respective fields: `toAdd`, `toRemove`
The command has fetched the current state, compared it to the tree from file and provided the differences to consume.
Copy the respective hashes to the `toAdd` and `toRemove` fields of your draft proposal.

### Method 2
### Submit proposal

1. Fetch the current hashes from the Merkle tree:
Basically, you need a command:
```bash
rarimo-cored --node $RPC query cscalist tree > current-tree.txt
rarimo-cored --home $RARIMO_HOME --node $RPC tx gov submit-proposal draft_proposal.json --from <your-rarimo-account>
```
2. Iterate over the `current-tree.txt` manually and collect `key` values.
You may put them into `old-output-file.txt`.
<!-- TODO: manual iteration is very bad -->
3. Perform steps 4 and 5 from **Method 1**.

Congratulations! You should now have your proposal ready for submission.
Refer to Cosmos SDK documentation and other guides to get more details about:
- Account creation
- Generating and managing your keys
- Obtaining `draft_proposal.json`

## Verify proposal

In order to vote for the proposal, you need to ensure that its content corresponds to the actual state of CSCA List.

Firstly, download the list from [ICAO website](https://pkddownloadsg.icao.int/)
and prepare the list of hashes, like described in **Create proposal** section.

Secondly, check whether the proposal makes any difference to the current state:
```bash
rarimo-cored --node $RPC query cscalist ldif-tree-diff your-file.ldif
rarimo-cored --home $RARIMO_HOME --node $RPC query cscalist ldif-tree-diff your-file.ldif
```
If there is no difference, the proposal does not make sense, and you should vote 'No' or 'NoWithVeto'.
If there is no difference, the proposal either does not make sense, or is fraudulent, and you should vote 'No' or 'NoWithVeto'.
Again, the deeper validation depends on proposal type.

If this is `ReplaceCSCAListProposal`, just compare the `leaves` field with `your-output-file.txt`. It is possible that you could have the different order of hashes due to ICAO change of order. Therefore, you may wish to sort both lists, e.g. with `sort` command.

For `EditCSCAListProposal`, you need to perform the same steps as in **Create proposal** section.
Then compare the `toAdd` and `toRemove` fields with the differences between `your-output-file.txt` and `old-output-file.txt`.
For `EditCSCAListProposal`, you should additionally perform:
```bash
rarimo-cored --home $RARIMO_HOME --node $RPC query cscalist prepare-proposal your-file.ldif your-output-file.txt
```
Then compare the `toAdd` and `toRemove` fields with the differences between `your-output-file.txt` and someone's proposal.

If you get the same results as in the proposal, it is correct, and you can vote for it.
If you get the same results as in the proposal (of any type), it is correct, and you should support it.
100 changes: 65 additions & 35 deletions x/cscalist/client/cli/ldif.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import (
"errors"
"fmt"
"os"
"strings"

"github.com/cosmos/cosmos-sdk/client"
"github.com/cosmos/cosmos-sdk/client/flags"
Expand All @@ -21,12 +22,13 @@ const outputFormatFlag = "output-format"

func cmdParseLDIF() *cobra.Command {
const (
rawFormat = "raw"
hashFormat = "hash"
rootFormat = "root"
defaultFormat = hashFormat
rawFormat = "raw"
hashFormat = "hash"
proposalFormat = "proposal"
rootFormat = "root"
defaultFormat = proposalFormat
)
outputFormatValues := fmt.Sprintf("%s, %s, %s", rawFormat, hashFormat, rootFormat)
outputFormatValues := strings.Join([]string{rawFormat, hashFormat, proposalFormat, rootFormat}, ", ")

cmd := &cobra.Command{
Use: "parse-ldif <file> [output-file]",
Expand Down Expand Up @@ -80,12 +82,10 @@ automatically, see ldif-tree-diff)`,
}

case hashFormat:
for _, pk := range pubKeys {
hash := keccak256.Hash(pk)
if _, err = fmt.Fprintln(dst, hexutil.Encode(hash)); err != nil {
return fmt.Errorf("write to destination: %w", err)
}
}
return printHashes(pubKeys, dst, false)

case proposalFormat:
return printHashes(pubKeys, dst, true)

case rootFormat:
sPubKeys := make([]string, len(pubKeys))
Expand Down Expand Up @@ -115,11 +115,10 @@ automatically, see ldif-tree-diff)`,
func cmdLDIFTreeDiff() *cobra.Command {
cmd := &cobra.Command{
Use: "ldif-tree-diff <file>",
Short: "Build one tree from LDIF file, query another tree from chain and compare roots",
Short: "Build one tree from LDIF file, query tree root from chain and compare roots",
Long: `Provide a LDIF file to parse. The list can be manually retrieved from
https://pkddownloadsg.icao.int/. When the file is parsed, one tree is built,
another one is queried. Roots of 2 trees are compared. Use parse-ldif and tree
subcommands to find out the exact differences of data.
https://pkddownloadsg.icao.int/. Use parse-ldif and tree subcommands to find out
the exact differences of data.
Use cases:
- Automatically validate proposals: ensure that stored and proposed list differ
Expand All @@ -132,34 +131,20 @@ Use cases:
return fmt.Errorf("parse LDIF from file: %w", err)
}

pubKeys, err := data.RawPubKeys()
if err != nil {
return fmt.Errorf("extract raw public keys: %w", err)
}
sPubKeys := make([]string, len(pubKeys))
for i, pk := range pubKeys {
sPubKeys[i] = string(pk)
}

tree, err := mt.BuildFromRaw(sPubKeys)
if err != nil {
return fmt.Errorf("build Merkle tree: %w", err)
}

cliCtx := client.GetClientContextFromCmd(cmd)
cli := types.NewQueryClient(cliCtx)
root, err := getRootNode(cli)

built, stored, err := getBuiltAndStoredRoots(data, cli)
if err != nil {
return fmt.Errorf("get root node: %w", err)
return fmt.Errorf("get built and stored roots: %w", err)
}

treeRoot := hexutil.Encode(tree.Root())
if treeRoot != root.Hash {
fmt.Printf("Trees differ: built_root=%s stored_root=%s\n", treeRoot, root.Hash)
if built != stored {
fmt.Printf("Trees differ: built_root=%s stored_root=%s\n", built, stored)
return nil
}

fmt.Printf("Tree are same: root=%s\n", treeRoot)
fmt.Printf("Tree are same: root=%s\n", built)
return nil
},
}
Expand All @@ -168,6 +153,51 @@ Use cases:
return cmd
}

func printHashes(pubKeys [][]byte, dst *os.File, forProposal bool) error {
for i, pk := range pubKeys {
hash := keccak256.Hash(pk)

out := hexutil.Encode(hash)
if forProposal {
var comma string
if i < len(pubKeys)-1 {
comma = ","
}
out = fmt.Sprintf("%q%s", out, comma)
}

if _, err := fmt.Fprintln(dst, out); err != nil {
return fmt.Errorf("write to destination: %w", err)
}
}

return nil
}

func getBuiltAndStoredRoots(data ldif.LDIF, cli types.QueryClient) (string, string, error) {
pubKeys, err := data.RawPubKeys()
if err != nil {
return "", "", fmt.Errorf("extract raw public keys: %w", err)
}

sPubKeys := make([]string, len(pubKeys))
for i, pk := range pubKeys {
sPubKeys[i] = string(pk)
}

tree, err := mt.BuildFromRaw(sPubKeys)
if err != nil {
return "", "", fmt.Errorf("build Merkle tree: %w", err)
}

root, err := getRootNode(cli)
if err != nil {
return "", "", fmt.Errorf("get root node: %w", err)
}

return hexutil.Encode(tree.Root()), root.Hash, nil
}

func getRootNode(cli types.QueryClient) (*types.Node, error) {
params, err := cli.Params(context.Background(), &types.QueryParamsRequest{})
if err != nil {
Expand Down
Loading

0 comments on commit b6d3b39

Please sign in to comment.