✔️Monitor Node
This guide will teach you how to use Prometheus with Grafana to monitor your GLITCH node.
New chain of Glitch exposes data such as the height of the chain, the number of connected peers to your node, and more. To monitor this data, Prometheus collects metrics, and Grafana allows for displaying them on the dashboard.
A possible architecture could look like this:
Step 0: Install Prometheus and Grafana
Install Prometheus
Install Grafana
Step 1: Run your node
Substrate exposes an endpoint that serves metrics in the Prometheus exposition format available on port 9615. You can change the port --prometheus-port <PORT> and enable it to be accessed over an interface other than local host with --prometheus-external.
1# clear the dev database 2./glitch-node purge-chain --dev -y 3 4# start the template node in dev & tmp mode to experiment 5# optionally add the `--prometheus-port <PORT>` 6# or `--prometheus-external` flags 7./glitch-node --dev --tmp
Step 2: Retrieve the metrics
In a second terminal run:
1curl localhost:9615/metrics
Which should return a similar output to:
1# HELP substrate_authority_discovery_amount_external_addresses_last_published Number of external addresses published when authority discovery last published addresses. 2# TYPE substrate_authority_discovery_amount_external_addresses_last_published gauge 3substrate_authority_discovery_amount_external_addresses_last_published 1 4# HELP substrate_authority_discovery_authority_address_requests_pending Number of pending authority address requests. 5# TYPE substrate_authority_discovery_authority_address_requests_pending gauge 6substrate_authority_discovery_authority_address_requests_pending 0 7# HELP substrate_authority_discovery_authority_addresses_requested_total Number of times authority discovery has requested external addresses of a single authority. 8# TYPE substrate_authority_discovery_authority_addresses_requested_total counter 9substrate_authority_discovery_authority_addresses_requested_total 1432 10# HELP substrate_authority_discovery_dht_event_received Number of dht events received by authority discovery. 11# TYPE substrate_authority_discovery_dht_event_received counter 12substrate_authority_discovery_dht_event_received{name="value_found"} 1424 13substrate_authority_discovery_dht_event_received{name="value_not_found"} 8 14substrate_authority_discovery_dht_event_received{name="value_put_failed"} 127 15# HELP substrate_authority_discovery_handle_value_found_event_failure Number of times handling a dht value found event failed. 16# TYPE substrate_authority_discovery_handle_value_found_event_failure counter 17substrate_authority_discovery_handle_value_found_event_failure 0 18# HELP substrate_authority_discovery_known_authorities_count Number of authorities known by authority discovery. 19# TYPE substrate_authority_discovery_known_authorities_count gauge 20substrate_authority_discovery_known_authorities_count 2 21# HELP substrate_authority_discovery_times_published_total Number of times authority discovery has published external addresses. 22# TYPE substrate_authority_discovery_times_published_total counter 23substrate_authority_discovery_times_published_total 127 24# HELP substrate_block_height Block height info of the chain 25# TYPE substrate_block_height gauge 26substrate_block_height{status="best"} 99692 27substrate_block_height{status="finalized"} 99690 28substrate_block_height{status="sync_target"} 99692 29...
Here are some metrics that we collected:
Step 3: Configure Prometheus to scrape your Substrate node
In a prometheus.yml configuration file, configure Prometheus to scrape the exposed endpoint by adding it to the target array.
A scrape configuration containing exactly one endpoint to scrape: Here it's Prometheus itself.
1scrape_configs: 2 - job_name: "prometheus" 3 scrape_interval: 5s 4 static_configs: 5 - targets: ["localhost:9090"] 6 - job_name: 'substrate_node' 7 scrape_interval: 5s 8 static_configs: 9 - targets: ['localhost:9615']
Launch a Prometheus instance with the prometheus.yml config file.
1./prometheus --config.file prometheus.yml
Step 4: Visualizing Prometheus metrics with Grafana
Add new Dashboard to Grafana After installing Grafana, you can now access it by going to the
http://SERVER_IP_ADDRESS:3000/login. The default user and password is admin/admin.
And, find the +icon at the top-left corner to import the prometheus format, do step by step to complete the adding dashboard.
You can upload JSON file or put content JSON at Import via panel json
Then, the result will look like the image.
Last updated
