oasis-node can report a number of metrics to Prometheus server. By default,
no metrics are collected and reported. There is one way to enable metrics
reporting:
- Pull mode listens on given address and waits for Prometheus to scrape the metrics.
To run oasis-node in pull mode set flag --metrics.mode pull and provide
the listen address with --metrics.address. For example
oasis-node --metrics.mode pull --metrics.address localhost:3000
Then, add the following segment to your prometheus.yml and restart
Prometheus:
- job_name : 'oasis-node'
scrape_interval: 5s
static_configs:
- targets: ['localhost:3000']oasis-node reports metrics starting with oasis_.
The following metrics are currently reported:
| Name | Type | Description | Labels | Package |
|---|---|---|---|---|
| oasis_abci_db_size | Gauge | Total size of the ABCI database (MiB). | consensus/cometbft/abci | |
| oasis_codec_size | Summary | CBOR codec message size (bytes). | call, module | common/cbor |
| oasis_consensus_proposed_blocks | Counter | Number of blocks proposed by the node. | backend | consensus/metrics |
| oasis_consensus_signed_blocks | Counter | Number of blocks signed by the node. | backend | consensus/metrics |
| oasis_finalized_rounds | Counter | Number of finalized rounds. | roothash | |
| oasis_grpc_client_calls | Counter | Number of gRPC calls. | call | common/grpc |
| oasis_grpc_client_latency | Summary | gRPC call latency (seconds). | call | common/grpc |
| oasis_grpc_client_stream_writes | Counter | Number of gRPC stream writes. | call | common/grpc |
| oasis_grpc_server_calls | Counter | Number of gRPC calls. | call | common/grpc |
| oasis_grpc_server_latency | Summary | gRPC call latency (seconds). | call | common/grpc |
| oasis_grpc_server_stream_writes | Counter | Number of gRPC stream writes. | call | common/grpc |
| oasis_node_cpu_stime_seconds | Gauge | CPU system time spent by worker as reported by /proc/<PID>/stat (seconds). | oasis-node/cmd/common/metrics | |
| oasis_node_cpu_utime_seconds | Gauge | CPU user time spent by worker as reported by /proc/<PID>/stat (seconds). | oasis-node/cmd/common/metrics | |
| oasis_node_disk_read_bytes | Gauge | Read data from block storage by the worker as reported by /proc/<PID>/io (bytes). | oasis-node/cmd/common/metrics | |
| oasis_node_disk_usage_bytes | Gauge | Size of datadir of the worker (bytes). | oasis-node/cmd/common/metrics | |
| oasis_node_disk_written_bytes | Gauge | Written data from block storage by the worker as reported by /proc/<PID>/io (bytes) | oasis-node/cmd/common/metrics | |
| oasis_node_mem_rss_anon_bytes | Gauge | Size of resident anonymous memory of worker as reported by /proc/<PID>/status (bytes). | oasis-node/cmd/common/metrics | |
| oasis_node_mem_rss_file_bytes | Gauge | Size of resident file mappings of worker as reported by /proc/<PID>/status (bytes) | oasis-node/cmd/common/metrics | |
| oasis_node_mem_rss_shmem_bytes | Gauge | Size of resident shared memory of worker. | oasis-node/cmd/common/metrics | |
| oasis_node_mem_vm_size_bytes | Gauge | Virtual memory size of worker (bytes). | oasis-node/cmd/common/metrics | |
| oasis_node_net_receive_bytes_total | Gauge | Received data for each network device as reported by /proc/net/dev (bytes). | device | oasis-node/cmd/common/metrics |
| oasis_node_net_receive_packets_total | Gauge | Received data for each network device as reported by /proc/net/dev (packets). | device | oasis-node/cmd/common/metrics |
| oasis_node_net_transmit_bytes_total | Gauge | Transmitted data for each network device as reported by /proc/net/dev (bytes). | device | oasis-node/cmd/common/metrics |
| oasis_node_net_transmit_packets_total | Gauge | Transmitted data for each network device as reported by /proc/net/dev (packets). | device | oasis-node/cmd/common/metrics |
| oasis_p2p_blocked_peers | Gauge | Number of blocked P2P peers. | p2p | |
| oasis_p2p_connections | Gauge | Number of P2P connections. | p2p | |
| oasis_p2p_peers | Gauge | Number of connected P2P peers. | p2p | |
| oasis_p2p_protocols | Gauge | Number of supported P2P protocols. | p2p | |
| oasis_p2p_topics | Gauge | Number of supported P2P topics. | p2p | |
| oasis_registry_entities | Gauge | Number of registry entities. | registry | |
| oasis_registry_nodes | Gauge | Number of registry nodes. | registry | |
| oasis_registry_runtimes | Gauge | Number of registry runtimes. | registry | |
| oasis_rhp_failures | Counter | Number of failed Runtime Host calls. | call | runtime/host/protocol |
| oasis_rhp_latency | Summary | Runtime Host call latency (seconds). | call | runtime/host/protocol |
| oasis_rhp_successes | Counter | Number of successful Runtime Host calls. | call | runtime/host/protocol |
| oasis_rhp_timeouts | Counter | Number of timed out Runtime Host calls. | runtime/host/protocol | |
| oasis_roothash_block_interval | Summary | Time between roothash blocks (seconds). | runtime | roothash |
| oasis_storage_failures | Counter | Number of storage failures. | call | storage/api |
| oasis_storage_latency | Summary | Storage call latency (seconds). | call | storage/api |
| oasis_storage_successes | Counter | Number of storage successes. | call | storage/api |
| oasis_storage_value_size | Summary | Storage call value size (bytes). | call | storage/api |
| oasis_tee_attestations_failed | Counter | Number of failed TEE attestations. | runtime, kind | runtime/host/sgx/common |
| oasis_tee_attestations_performed | Counter | Number of TEE attestations performed. | runtime, kind | runtime/host/sgx/common |
| oasis_tee_attestations_successful | Counter | Number of successful TEE attestations. | runtime, kind | runtime/host/sgx/common |
| oasis_txpool_accepted_transactions | Counter | Number of accepted transactions (passing check tx). | runtime | runtime/txpool |
| oasis_txpool_local_queue_size | Gauge | Size of the local transactions schedulable queue (number of entries). | runtime | runtime/txpool |
| oasis_txpool_pending_check_size | Gauge | Size of the pending to be checked queue (number of entries). | runtime | runtime/txpool |
| oasis_txpool_pending_schedule_size | Gauge | Size of the main schedulable queue (number of entries). | runtime | runtime/txpool |
| oasis_txpool_rejected_transactions | Counter | Number of rejected transactions (failing check tx). | runtime | runtime/txpool |
| oasis_txpool_rim_queue_size | Gauge | Size of the roothash incoming message transactions schedulable queue (number of entries). | runtime | runtime/txpool |
| oasis_up | Gauge | Is oasis-test-runner active for specific scenario. | oasis-node/cmd/common/metrics | |
| oasis_worker_aborted_batch_count | Counter | Number of aborted batches. | runtime | worker/compute/executor/committee |
| oasis_worker_batch_processing_time | Summary | Time it takes for a batch to finalize (seconds). | runtime | worker/compute/executor/committee |
| oasis_worker_batch_runtime_processing_time | Summary | Time it takes for a batch to be processed by the runtime (seconds). | runtime | worker/compute/executor/committee |
| oasis_worker_batch_size | Summary | Number of transactions in a batch. | runtime | worker/compute/executor/committee |
| oasis_worker_client_lb_healthy_instance_count | Gauge | Number of healthy instances in the load balancer. | runtime | runtime/host/loadbalance |
| oasis_worker_client_lb_requests | Counter | Number of requests processed by the given load balancer instance. | runtime, lb_instance | runtime/host/loadbalance |
| oasis_worker_epoch_number | Gauge | Current epoch number as seen by the worker. | runtime | worker/common/committee |
| oasis_worker_epoch_transition_count | Counter | Number of epoch transitions. | runtime | worker/common/committee |
| oasis_worker_execution_discrepancy_detected_count | Counter | Number of detected execute discrepancies. | runtime | worker/compute/executor/committee |
| oasis_worker_executor_committee_p2p_peers | Gauge | Number of executor committee P2P peers. | runtime | worker/common/committee |
| oasis_worker_executor_is_backup_worker | Gauge | 1 if worker is currently an executor backup worker, 0 otherwise. | runtime | worker/common/committee |
| oasis_worker_executor_is_worker | Gauge | 1 if worker is currently an executor worker, 0 otherwise. | runtime | worker/common/committee |
| oasis_worker_executor_liveness_live_ratio | Gauge | Ratio between live and total rounds. Reports 1 if node is not in committee. | runtime | worker/common/committee |
| oasis_worker_executor_liveness_live_rounds | Gauge | Number of live rounds in last epoch. | runtime | worker/common/committee |
| oasis_worker_executor_liveness_total_rounds | Gauge | Number of total rounds in last epoch. | runtime | worker/common/committee |
| oasis_worker_failed_round_count | Counter | Number of failed roothash rounds. | runtime | worker/common/committee |
| oasis_worker_keymanager_churp_committee_size | Gauge | Number of nodes in the committee | runtime, churp | worker/keymanager |
| oasis_worker_keymanager_churp_confirmed_applications_total | Gauge | Number of confirmed applications | runtime, churp | worker/keymanager |
| oasis_worker_keymanager_churp_enclave_rpc_failures_total | Counter | Number of failed enclave rpc calls. | runtime, churp, method | worker/keymanager |
| oasis_worker_keymanager_churp_enclave_rpc_latency_seconds | Summary | Latency of enclave rpc calls in seconds. | runtime, churp, method | worker/keymanager |
| oasis_worker_keymanager_churp_extra_shares_number | Gauge | Minimum number of extra shares. | runtime, churp | worker/keymanager |
| oasis_worker_keymanager_churp_handoff_interval | Gauge | Handoff interval in epochs | runtime, churp | worker/keymanager |
| oasis_worker_keymanager_churp_handoff_number | Counter | Epoch number of the last handoff | runtime, churp | worker/keymanager |
| oasis_worker_keymanager_churp_next_handoff_number | Counter | Epoch number of the next handoff | runtime, churp | worker/keymanager |
| oasis_worker_keymanager_churp_submitted_applications_total | Gauge | Number of submitted applications | runtime, churp | worker/keymanager |
| oasis_worker_keymanager_churp_threshold_number | Counter | Degree of the secret-sharing polynomial | runtime, churp | worker/keymanager |
| oasis_worker_keymanager_compute_runtime_count | Counter | Number of compute runtimes using the key manager. | runtime | worker/keymanager |
| oasis_worker_keymanager_consensus_ephemeral_secret_epoch_number | Gauge | Epoch number of the latest ephemeral secret. | runtime | worker/keymanager |
| oasis_worker_keymanager_consensus_master_secret_generation_number | Gauge | Generation number of the latest master secret. | runtime | worker/keymanager |
| oasis_worker_keymanager_consensus_master_secret_proposal_epoch_number | Gauge | Epoch number of the latest master secret proposal. | runtime | worker/keymanager |
| oasis_worker_keymanager_consensus_master_secret_proposal_generation_number | Gauge | Generation number of the latest master secret proposal. | runtime | worker/keymanager |
| oasis_worker_keymanager_consensus_master_secret_rotation_epoch_number | Gauge | Epoch number of the latest master secret rotation. | runtime | worker/keymanager |
| oasis_worker_keymanager_enclave_ephemeral_secret_epoch_number | Gauge | Epoch number of the latest ephemeral secret loaded into the enclave. | runtime | worker/keymanager |
| oasis_worker_keymanager_enclave_generated_ephemeral_secret_epoch_number | Gauge | Epoch number of the latest ephemeral secret generated by the enclave. | runtime | worker/keymanager |
| oasis_worker_keymanager_enclave_generated_master_secret_epoch_number | Gauge | Epoch number of the latest master secret generated by the enclave. | runtime | worker/keymanager |
| oasis_worker_keymanager_enclave_generated_master_secret_generation_number | Gauge | Generation number of the latest master secret generated by the enclave. | runtime | worker/keymanager |
| oasis_worker_keymanager_enclave_master_secret_generation_number | Gauge | Generation number of the latest master secret as seen by the enclave. | runtime | worker/keymanager |
| oasis_worker_keymanager_enclave_master_secret_proposal_epoch_number | Gauge | Epoch number of the latest master secret proposal loaded into the enclave. | runtime | worker/keymanager |
| oasis_worker_keymanager_enclave_master_secret_proposal_generation_number | Gauge | Generation number of the latest master secret proposal loaded into the enclave. | runtime | worker/keymanager |
| oasis_worker_keymanager_enclave_rpc_count | Counter | Number of remote Enclave RPC requests via P2P. | method | worker/keymanager/p2p |
| oasis_worker_keymanager_policy_update_count | Counter | Number of key manager policy updates. | runtime | worker/keymanager |
| oasis_worker_node_registered | Gauge | Is oasis node registered (binary). | worker/registration | |
| oasis_worker_node_registration_eligible | Gauge | Is oasis node eligible for registration (binary). | worker/registration | |
| oasis_worker_node_status_frozen | Gauge | Is oasis node frozen (binary). | worker/registration | |
| oasis_worker_node_status_runtime_faults | Gauge | Number of runtime faults. | runtime | worker/registration |
| oasis_worker_node_status_runtime_suspended | Gauge | Runtime node suspension status (binary). | runtime | worker/registration |
| oasis_worker_processed_block_count | Counter | Number of processed roothash blocks. | runtime | worker/common/committee |
| oasis_worker_processed_event_count | Counter | Number of processed roothash events. | runtime | worker/compute/executor/committee |
| oasis_worker_storage_commit_latency | Summary | Latency of storage commit calls (state + outputs) (seconds). | runtime | worker/compute/executor/committee |
| oasis_worker_storage_full_round | Gauge | The last round that was fully synced and finalized. | runtime | worker/storage/committee |
| oasis_worker_storage_pending_round | Gauge | The last round that is in-flight for syncing. | runtime | worker/storage/committee |
| oasis_worker_storage_round_sync_latency | Summary | Storage round sync latency (seconds). | runtime | worker/storage/committee |
| oasis_worker_storage_synced_round | Gauge | The last round that was synced but not yet finalized. | runtime | worker/storage/committee |
When oasis-node is configured to use CometBFT for BFT consensus, all
CometBFT metrics are also reported. Consult
CometBFT-core documentation for a list of reported by CometBFT.