The monitor verifies end-to-end functionality of the Hedera network and generates metrics from the results. It supports both publishing transactions to HAPI and subscribing to the mirror node API. Configuration is flexible and declarative, allowing one to express a mixture of transactions and their expected publish rates, and the tool will do its best to make it so. By default, the monitor is already set up with a basic scenario that creates a topic, submits a message every 10 seconds to it, and verifies the messages are received via the mirror node's gRPC and REST APIs.
This section covers the higher level properties to consider before diving into the specific publish and subscribe properties in the next sections. For a full list of configuration options see the config documentation.
First, make sure the monitor is configured to talk to the correct Hedera network by
setting hedera.mirror.monitor.network to
MAINNET, PREVIEWNET or TESTNET. If you are not using one of these public environments, the network can be set
to OTHER, and hedera.mirror.monitor.nodes and hedera.mirror.monitor.mirrorNode properties should be filled in.
Additionally, the operator information in hedera.mirror.monitor.operator is required and needs to be populated with a
valid payer account ID and its private key. Ensure this account has the necessary funds to publish transactions at the
rate you desire. The following is an example with a custom network configured:
hedera:
mirror:
monitor:
mirrorNode:
grpc:
host: 34.69.90.179
port: 5600
rest:
host: 35.239.255.55
port: 80
network: OTHER
nodes:
- accountId: 0.0.3
host: 34.94.106.61
- accountId: 0.0.4
host: 35.237.119.55
- accountId: 0.0.5
host: 35.245.27.193
- accountId: 0.0.6
host: 34.83.112.116
operator:
accountId: 0.0.1234
privateKey: 104...The monitor can be configured to publish multiple transaction types concurrently. It does this by specifying a list of scenarios that should actively publish. A unique scenario name should be used as a label for logs, metrics, and the REST API so that each scenario can be monitored separately. Each scenario has a target transactions per second (TPS) and uses a client-side rate limiter to achieve the desired rate. Additionally, a percentage of receipts or records can be requested for each transaction to verify transactions are reaching consensus.
The monitor can be used to publish at a very high TPS, with a single monitor able to achieve 10K TPS on the Hedera
network. To publish at higher rates, the hedera.mirror.monitor.publish.clients and
hedera.mirror.monitor.publish.responseThreads properties can be adjusted. With the default values of four clients and
40 response threads, the monitor can already achieve 10K TPS out of the box. The transaction publisher will round-robin
the list of clients to send transactions to the Hedera Network, and every transaction is sent to a randomly chosen node,
ensuring the load is distributed evenly across all nodes.
The type property specifies which transaction type to publish. It also affects which properties need to be
specified, with different transaction types requiring different properties to be set. See the
TransactionType
enum for a list of possible values for type. The properties can be seen as fields on the various
TransactionSupplier classes that
the TransactionType enum references. Most of these properties have a default and don't need to be explicitly
specified, but some are empty and may need to be populated.
For example, if you want to publish a topic message, you would open the TransactionType class,
find CONSENSUS_SUBMIT_MESSAGE, then open the
ConsensusSubmitMessageTransactionSupplier
class that it references. From there, you can see that fields maxTransactionFee, message, retry, and topicId are
available as properties. Only topicId doesn't have a default and will be required. Here's a YAML excerpt that
specifies some of those properties:
hedera:
mirror:
monitor:
publish:
scenarios:
pinger: # Scenario name
properties:
maxTransactionFee: 1000000
message: Hello world!
retry: true
topicId: 0.0.1000
recordPercent: 1.0
tps: 0.1
type: CONSENSUS_SUBMIT_MESSAGESome properties, such as transferTypes in
the CryptoTransferTransactionSupplier
, are Collections, which requires the YAML list syntax.
Scheduled transactions require unique configuration as they encompass two transactions during the entity creation. One (
outer) transaction for the Schedule entity create and one (inner/scheduled) transaction to eventually be executed. The
inner transaction must be signed by all required signatories as part of the ScheduleCreate or ScheduleSign scenario
transactions. Due to this the monitor by default will initially support only ScheduleCreate scenarios in which the
inner transaction is a CryptoCreate with receiverSignatureRequired set to true.
By default, all required signatures will be provided. However, this can be modified by
setting hedera.mirror.monitor.publish.scenarios.properties.signatoryCount to be a number greater than 0 but smaller
than hedera.mirror.monitor.publish.scenarios.properties.totalSignatoryCount
To execute a scheduled scenario set the hedera.mirror.monitor.publish.scenarios properties similar to the following
example.
hedera:
mirror:
monitor:
publish:
scenarios:
scheduledCryptoCreate:
properties:
nodeAccountId: 0.0.3
operatorAccountId: 0.0.1018
receiverSignatureRequired: true
signatoryCount: 4
totalSignatoryCount: 5
type: SCHEDULE_CREATENFT (Non-Fungible Token) transactions use the same TransactionSuppliers as Fungible Token transactions, but they
require that the serial number(s) to be operated upon be specified, as opposed to Fungible tokens where one simply sets
an amount to be operated on. Because most operations can only occur once for a given serial number (a serial number can
only be burned/wiped once, and it can only be transferred from Alice to Bob once without transferring it back), the
monitor requires a sufficient amount of serial numbers be minted prior to running any CRYPTO_TRANSFER
, TOKEN_BURN, or TOKEN_WIPE scenario involving the NFT. A TOKEN_MINT scenario can be set up to accomplish this. If
you are reusing an existing NFT, be sure to set the serialNumber property where applicable, as by default
the TransactionSupplier will start with serial number 1, which may have already been deleted or transferred elsewhere.
hedera:
mirror:
monitor:
publish:
scenarios:
nftMint:
properties:
type: NON_FUNGIBLE_UNIQUE
tokenId: ${nft.1}
amount: 1
type: TOKEN_MINT
tps: 105
nftTransfer:
properties:
recipientAccountId: ${account.1}
senderAccountId: ${account.2}
transferTypes:
- NFT
amount: 1
nftTokenId: ${nft.1}
tps: 100
type: CRYPTO_TRANSFERThe monitor can automatically create account, token, and topic entities on application startup using a special
expression syntax. This is useful to avoid boilerplate configuration and manual entity creation steps that vary per
environment. The syntax can currently only be used in hedera.mirror.monitor.publish.scenarios.properties
and hedera.mirror.monitor.subscribe.grpc.topicId.
The syntax takes the form of ${type.name} where type is one of account, schedule, token, or topic,
and name is a descriptive label. Based upon the entity type, it will create the appropriate entity on the network with
default values. The name label allows the same entity to be referenced in multiple places but only created once.
The following example uses the expression syntax to create the sender and recipient accounts as well as a token. These same entities are created once and reused in both the token associate and the token transfer transaction.
hedera:
mirror:
monitor:
publish:
scenarios:
htsAssociate:
limit: 1
properties:
accountId: ${account.them}
tokenId: ${token.foobar}
tps: 1
type: TOKEN_ASSOCIATE
htsTransfer:
properties:
recipientAccountId: ${account.them}
senderAccountId: ${account.me}
tokenId: ${token.foobar}
transferType: BOTH
tps: 1
type: CRYPTO_TRANSFERThe monitor can optionally subscribe to the mirror node gRPC and REST APIs simultaneously. Each subscription type can
have one or more scenarios. For the REST API, it can verify that a percentage of individual transactions have made it to
the mirror node by querying the /api/v1/transactions/{transactionId} REST endpoint. The exact percentage to verify is
controlled via the samplePercent property.
For gRPC, topicId is required and controls which topic should be registered for asynchronous notifications of topic
messages. Below is an example of both types:
hedera:
mirror:
monitor:
subscribe:
clients: 4
grpc:
hcs:
enabled: true
subscribers: 10
topicId: ${topic.ping}
rest:
transactionId:
enabled: false
samplePercent: 1.0For performance testing subscribers, the hedera.mirror.monitor.subscribe.clients property should be adjusted higher to
control the pool of client connections to the server. Since the communication is asynchronous, a number between 1-10
should suffice. Additionally, the
hedera.mirror.monitor.subscribe.grpc.subscribers property can be adjusted to increase the number of concurrent
subscribers for that scenario.
The monitor REST API provides a way to query the status of the scenarios currently publishing and subscribing to various
Hedera APIs. The monitor OpenAPI specification is available at /api/v1/docs/openapi
and /api/v1/docs/openapi.yaml. The Swagger UI is also available
at /api/v1/docs. This UI provides a form of documentation, and an interactive way to explore the monitor's REST API.
Lists all subscriber scenarios. If no scenarios can be found with the given input a 404 status code will be returned.
A protocol and status query parameter can be optionally supplied. The protocol query parameter can be
either GRPC or REST. The status flag can be one of the below:
COMPLETED: The scenario has completed normally due to reaching the configured duration or limitIDLE: The scenario has not completed but is not currently receiving any responsesRUNNING: The scenario is still actively receiving responses
GET /api/v1/subscriber
Example response:
[
{
"count": 74,
"status": "RUNNING",
"errors": {},
"elapsed": "PT12M14.906846353S",
"rate": 0.2,
"name": "HCS Subscribe",
"id": 1,
"protocol": "GRPC"
},
{
"count": 72,
"status": "RUNNING",
"errors": {
"TimeoutException": 1
},
"elapsed": "2d3h3s",
"rate": 0.1,
"name": "HCS REST",
"id": 1,
"protocol": "REST"
}
]Lists all subscriber scenarios with a given name. Also supports the aforementioned status query parameter. If no
scenarios can be found with the given input a 404 status code will be returned.
GET /api/v1/subscriber/{name}
Example response:
[
{
"count": 72,
"status": "RUNNING",
"errors": {
"TimeoutException": 1
},
"elapsed": "1h2m57s",
"rate": 0.1,
"name": "HCS REST",
"id": 1,
"protocol": "REST"
}
]Gets a subscriber scenario by its unique name and index. If no scenario can be found with the given input a 404 status code will be returned.
GET /api/v1/subscriber/{name}/{index}
Example response:
{
"count": 72,
"status": "RUNNING",
"errors": {
"TimeoutException": 1
},
"elapsed": "12m13s",
"rate": 0.1,
"name": "HCS REST",
"id": 1,
"protocol": "REST"
}The monitor includes a Grafana dashboard that
displays the various metrics collected by the application visually. The monitor uses Micrometer
to export the metrics to either Elasticsearch or Prometheus,
depending upon configuration. The Prometheus metrics can be scraped at http://localhost:8082/actuator/prometheus.
