Skip to content

Commit

Permalink
Merge pull request #649 from anikitinDSR/rc-1.3.56
Browse files Browse the repository at this point in the history
Rc 1.3.56
  • Loading branch information
andkononykhin authored Apr 12, 2018
2 parents 768f548 + e06a28c commit 4c45a31
Show file tree
Hide file tree
Showing 90 changed files with 5,657 additions and 298 deletions.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -173,7 +173,7 @@ If you made changes in both indy-plenum and indy-node, you need to do the follow
- [Request Handling](https://github.com/hyperledger/indy-plenum/blob/master/docs/request_handling.md)
- [Catchup](https://github.com/hyperledger/indy-plenum/blob/master/docs/catchup.md)
- [Plugins](https://github.com/hyperledger/indy-plenum/blob/master/docs/plugins.md)
- Relationship between Entities and Transactions: [relationship diagram](docs/relationship diagram.png)
- Relationship between Entities and Transactions: [relationship diagram](docs/relationship-diagram.png)
- Supported transactions and their format: [transactions](docs/transactions.md)
- Supported requests (write, read) and their format: [requests](docs/requests.md)
- [Network roles and permissions](https://docs.google.com/spreadsheets/d/1TWXF7NtBjSOaUIBeIH77SyZnawfo91cJ_ns4TR-wsq4/edit#gid=0)
Expand Down
1 change: 1 addition & 0 deletions build-scripts/ubuntu-1604/build-indy-node.sh
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ fpm --input-type "python" \
--exclude "*.pyo" \
--depends at \
--depends iptables \
--depends libsodium18 \
--no-python-fix-dependencies \
--maintainer "Hyperledger <[email protected]>" \
--before-install "preinst_node" \
Expand Down
5 changes: 5 additions & 0 deletions build-scripts/ubuntu-1604/postinst_node
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
#!/bin/bash

# it should be fixed
ENABLE_STDOUT_LOGGING="enableStdOutLogging"
GENERAL_CONFIG_DIR="/etc/indy"
GENERAL_DATA_DIR="/var/lib/indy"
GENERAL_LOG_DIR="/var/log/indy"
Expand Down Expand Up @@ -33,6 +34,10 @@ mkdir -p $GENERAL_LOG_DIR
# create indy node config if does not exist
if [ ! -f $GENERAL_CONFIG_DIR/indy_config.py ]; then
cp $INSTALL_DIR/indy_node/general_config/indy_config.py $GENERAL_CONFIG_DIR
else
if [ -z "$(grep $ENABLE_STDOUT_LOGGING $GENERAL_CONFIG_DIR/indy_config.py)" ]; then
printf "\n%s\n%s\n" "# Disable stdout logging" "$ENABLE_STDOUT_LOGGING = False" >> $GENERAL_CONFIG_DIR/indy_config.py
fi
fi

chmod -R ug+rwx $GENERAL_CONFIG_DIR
Expand Down
8 changes: 7 additions & 1 deletion ci/ubuntu.dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,13 @@ ARG venv=venv
RUN apt-get update -y && apt-get install -y \
python3-nacl \
libindy-crypto=0.2.0 \
libindy
libindy=1.3.1~454 \
# rocksdb python wrapper
libbz2-dev \
zlib1g-dev \
liblz4-dev \
libsnappy-dev \
rocksdb=5.8.8

RUN indy_ci_add_user $uid $user $venv && \
indy_ci_charm_crypto $user $venv
Expand Down
65 changes: 49 additions & 16 deletions design/anoncreds.md
Original file line number Diff line number Diff line change
Expand Up @@ -232,8 +232,8 @@ reference to the CredDef, plus some revocation registry specific data.
```
{
"data": {
"id":"MMAD5g65TDQr1PPHHRoiGf3HHAD5g65TDQr1PPHHRoiGf2L5AD5g65TDQr1PPHHRoiGf1Degree1CLkey1CL_ACCUMreg1",
"type":"CL_ACCUM",
"id":"MMAD5g65TDQr1PPHHRoiGf:3:HHAD5g65TDQr1PPHHRoiGf2L5AD5g65TDQr1PPHHRoiGf1Degree1CLkey1:CL_ACCUM:reg1",
"revocDefType":"CL_ACCUM",
"tag": "reg1",
"credDefId":"HHAD5g65TDQr1PPHHRoiGf2L5AD5g65TDQr1PPHHRoiGf1Degree1CLkey1",
"value": {
Expand All @@ -253,8 +253,8 @@ reference to the CredDef, plus some revocation registry specific data.
....
}
```
* `id` (string): ID as a key in State Trie.
* `type` (enum string): Revocation Registry type (only `CL_ACCUM` is supported for now).
* `id` (string, ":" as a field separator): ID as a key in State Trie.
* `revocDefType` (enum string): Revocation Registry type (only `CL_ACCUM` is supported for now).

Revocation Registry is updated only during each issuance and revocation.
* `credDefId` (string): ID of the corresponding CredDef
Expand All @@ -281,6 +281,7 @@ That is rotation of keys is supported.

* key: `revocDefSubmitterDid | RevocRegMarker | credDefID | revocDefType | revocDefTag`
* value: aggregated txn `data` and `txnMetadata` (as in ledger)
* RevocRegMarker = "\04"



Expand All @@ -289,7 +290,7 @@ Gets a RevocRegDef by ID.
```
{
"data": {
"id":"MMAD5g65TDQr1PPHHRoiGf3HHAD5g65TDQr1PPHHRoiGf2L5AD5g65TDQr1PPHHRoiGf1Degree1CLkey1CL_ACCUMreg1",
"id":"MMAD5g65TDQr1PPHHRoiGf:3:HHAD5g65TDQr1PPHHRoiGf2L5AD5g65TDQr1PPHHRoiGf1Degree1CLkey1:CL_ACCUM:reg1",
},
...
}
Expand All @@ -302,7 +303,7 @@ Gets a list of RevocRegDefs according to the given filters.
"data": {
"submitterDid":"MMAD5g65TDQr1PPHHRoiGf",
"credDefId":"HHAD5g65TDQr1PPHHRoiGf2L5AD5g65TDQr1PPHHRoiGf1Degree1CLkey1", (optional)
"type":"CL_ACCUM", (optional)
"revocDefType":"CL_ACCUM", (optional)
"tag": "reg1", (optional)
},
...
Expand All @@ -322,8 +323,8 @@ So, it can be sent each time a new claim is issued/revoked.
```
{
"data": {
"revocRegDefId": "MMAD5g65TDQr1PPHHRoiGf3HHAD5g65TDQr1PPHHRoiGf2L5AD5g65TDQr1PPHHRoiGf1Degree1CLkey1CL_ACCUMreg1",
"type":"CL_ACCUM",
"revocRegDefId": "MMAD5g65TDQr1PPHHRoiGf:3:HHAD5g65TDQr1PPHHRoiGf2L5AD5g65TDQr1PPHHRoiGf1Degree1CLkey1:CL_ACCUM:reg1",
"revocDefType":"CL_ACCUM",
"value": {
"prevAccum":"<prev_accum_value>",
"accum":"<accum_value>",
Expand Down Expand Up @@ -363,8 +364,34 @@ This is needed to avoid dirty writes and updates of accumulator.
* value: aggregated txn `data` and `txnMetadata` (as in ledger);
contains aggregated accum_value, issued and revoked arrays.

* RevocRegEntryMarker = "\05"

<b>Hint</b>: We should consider using BitMask to store the current aggregated state of issued and revoked arrays
in the State Trie to reduce the required space.
in the State Trie to reduce the required space.

<b>Additional</b>: For `GET_REVOC_REG` and `GET_REVOC_REG_DELTA` transactions we should save `ACCUM` value
into different state record (state proof purposes):

* key: `revocDefSubmitterDid | RevocRegEntryAccumMarker | revocRegDefId`

* value: aggregated txn `data` and `txnMetadata` (as in ledger) with only accum value without issued and revoked arrays.

* Schema of "pruned" `data` from txn must be:
```
{
"data": {
"revocRegDefId": "MMAD5g65TDQr1PPHHRoiGf:3:HHAD5g65TDQr1PPHHRoiGf2L5AD5g65TDQr1PPHHRoiGf1Degree1CLkey1:CL_ACCUM:reg1",
"revocDefType":"CL_ACCUM",
"value": {
"accum":"<accum_value>",
}
},
....
}
```

* RevocRegEntryAccumMarker = "\06"


#### GET_REVOC_REG
Gets the accumulated state of the Revocation Registry by ID
Expand All @@ -375,7 +402,7 @@ Request:
```
{
"data": {
"revocRegDefId":"MMAD5g65TDQr1PPHHRoiGf3HHAD5g65TDQr1PPHHRoiGf2L5AD5g65TDQr1PPHHRoiGf1Degree1CLkey1CL_ACCUMreg1",
"revocRegDefId":"MMAD5g65TDQr1PPHHRoiGf:3:HHAD5g65TDQr1PPHHRoiGf2L5AD5g65TDQr1PPHHRoiGf1Degree1CLkey1:CL_ACCUM:reg1",
"timestamp": 20,
},
...
Expand All @@ -385,8 +412,8 @@ Reply:
```
{
"data": {
"revocRegDefId": "MMAD5g65TDQr1PPHHRoiGf3HHAD5g65TDQr1PPHHRoiGf2L5AD5g65TDQr1PPHHRoiGf1Degree1CLkey1CL_ACCUMreg1",
"type":"CL_ACCUM",
"revocRegDefId": "MMAD5g65TDQr1PPHHRoiGf:3:HHAD5g65TDQr1PPHHRoiGf2L5AD5g65TDQr1PPHHRoiGf1Degree1CLkey1:CL_ACCUM:reg1",
"revocDefType":"CL_ACCUM",
"value": {
"accum":"<accum_value>",
}
Expand All @@ -408,7 +435,7 @@ Request:
```
{
"data": {
"revocRegDefId": "MMAD5g65TDQr1PPHHRoiGf3HHAD5g65TDQr1PPHHRoiGf2L5AD5g65TDQr1PPHHRoiGf1Degree1CLkey1CL_ACCUMreg1",
"revocRegDefId": "MMAD5g65TDQr1PPHHRoiGf:3:HHAD5g65TDQr1PPHHRoiGf2L5AD5g65TDQr1PPHHRoiGf1Degree1CLkey1:CL_ACCUM:reg1",
"from": 20, (optional)
"to": 40
},
Expand All @@ -419,17 +446,23 @@ Reply:
```
{
"data": {
"revocRegId": "MMAD5g65TDQr1PPHHRoiGf3HHAD5g65TDQr1PPHHRoiGf2L5AD5g65TDQr1PPHHRoiGf1Degree1CLkey1CL_ACCUMreg1",
"type":"CL_ACCUM",
"revocRegId": "MMAD5g65TDQr1PPHHRoiGf:3:HHAD5g65TDQr1PPHHRoiGf2L5AD5g65TDQr1PPHHRoiGf1Degree1CLkey1:CL_ACCUM:reg1",
"revocDefType":"CL_ACCUM",
"stateProofFrom": <state proof for accum by timestamp 'from'> (if "from" parameter is presented in request)
"value": {
"accum":"<accum_value>",
"accum_to": "<accum_value from ledger for timestamp 'to'>",
"accum_from": "<accum_value from ledger for timestamp 'from'>" (if "from" parameter is presented in request)
"issued": [1, 45],
"revoked": [56, 78, 890],
}
},
....
}
```
Notes:
* accum_to and accum_from it's a transactions from ledger "as-is",
like reply for for GET_REVOC_REG query.

See next sections on how to get the state for the given timestamp.

## Timestamp Support in State
Expand Down
163 changes: 163 additions & 0 deletions design/validator_info.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,163 @@
# Extension of validator_info
The document contains list of possible extensions and modifications of validator_info command to simplify node state monitoring and debug process

* [Validator_info Description](#cur-description)
* [Modification - New Read Command](#new-command)
* [Extension - Additional Fields](#new-fields)

## Validator_info Description
Validator_info is a script that formats and prints data provided by node. The script should be run manually by
the user who has a read access to the \<node name\>_info.json file from the node's data directory.
This file is updated by node once a minute and contains following information:
```
{
"did": "Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv", # node's identidier
"response-version": "0.0.1", # 0.0.1 for now
"timestamp": 1519711338, # current time
"verkey": "33nHHYKnqmtGAVfZZGoP8hpeExeH45Fo8cKmd5mcnKYk7XgWNBxkkKJ", # node's verkey
"pool": { # current pool
"reachable": { # reachable nodes
"list": ["Node1", "Node2", "Node3", "Node4"],
"count": 4
},
"total-count": 4, # total count of nodes
"unreachable": { # unreachable nodes
"list": [],
"count": 0
}
},
"bindings": { # node's network configuration
"client": {
"port": 9702,
"protocol": "tcp"
},
"node": {
"port": 9701,
"protocol": "tcp"
}
},
"metrics": { # some numeric characteristics
"transaction-count": { # txn count by ledger
"pool": 4,
"ledger": 19,
"config": 0
},
"average-per-second": { # performance counters
"write-transactions": 0.013294133790137249,
"read-transactions": 0.0
},
"uptime": 300 # uptaime
},
"software": { # packets' versions
"indy-node": "1.3.319",
"sovrin": null
},
"alias": "Node1" # node's name
}
```

## Modification - New Read Command
Validator_info should be accessible as read command, available for TRUSTEE and STEWARDS only. New command should provide info from
all the connected nodes without need of consensus (similar to force=True flag in upgrade cmd). Command should allow
requesting all parameters or some subset of parameters.

For reference: [INDY-1184](https://jira.hyperledger.org/browse/INDY-1184)


## Extension - Additional Fields

* Host machine info
* Hardware
* HDD free space
* RAM available
* MBs used by node process
* Node's data folder size
* Software
* OS version
* pip freeze output
* all indy related packets versions
* Current datetime and timezone

* Configuration
* Config
* Content of main config
* Content of network config
* Content of user config
* Content of Genesis Transaction Files
* indy.env
* node_control.conf
* indy-node.service
* indy-node-control.service
* iptables related config

* Extraction from logs and ledgers
* grep exception from journalctl
* grep view change, catch up, shutdown, instance change from the last log file
* systemctl status indy-node
* systemctl status indy-node-control
* Uptime
* upgrade_log
* last N txns from pool ledger
* last N txns from config ledger
* last N txns from domain ledger
* pool ledger size
* config ledger size
* domain ledger size

* Pool info
* Global pool settings
* read-only mode
* Nodes' connect/disconnect events for the last N min
* Ping time to other nodes
* Current quorum values
* N value
* f value
* per action values
* Reachable nodes
* Unreachable nodes
* Blacklisted nodes
* List of Suspicious

* Protocol
* supported client versions
* supported protocol versions
* supported requests versions

* Node info
* name
* last N txns from the pool ledger that belong to the current node only
* current mode
* metrics
* view change status
* instance_change msgs
* waiting view_change_done msgs
* catchup status
* each ledger catchup status
* last ledger_status msgs
* waiting consistency_proof msgs
* number of replicas
* foreach replica
* name
* primary
* root hashes
* uncommitted txns count
* uncommitted root hashes
* watermarks
* last_ordered
* last_3pc key
* info about stashed txns
* total number of stashed txns
* number of stashed checkpoints
* min stashed PrePrepare
* min stashed Prepare
* min stashed Commit

For reference: [INDY-1175](https://jira.hyperledger.org/browse/INDY-1175)
9 changes: 9 additions & 0 deletions dev-setup/ubuntu/setup-dev-depend-ubuntu16.sh
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,15 @@ sudo apt-get install -y libsodium18
echo 'Installed libsodium'


echo 'Installing RocksDB...'
sudo apt-get install -y libbz2-dev \
zlib1g-dev \
liblz4-dev \
libsnappy-dev \
rocksdb=5.8.8
echo 'Installed RocksDB'


echo 'Installing Charm Crypto...'
sudo apt-get install -y python3-charm-crypto
echo 'Installed Charm Crypto'
Expand Down
Loading

0 comments on commit 4c45a31

Please sign in to comment.