Metrics

Every node exposes a set of metrics on the port 10013 under the path /metrics. These metrics can be used to monitor the health of the node, and to diagnose issues.

Note

Apps expose metrics on the port 10013, under the path /metrics in the Prometheus format.

For a validator the following components expose metrics:

The validator app
The participant

For a Super Validator, in addition, we have the following components:

The SV app
The scan app

Scraping the metrics

We use OpenTelemetry to build the metrics and they are exposed in the Prometheus format, and can be scraped by a Prometheus server. For a reference of the existing metrics please check the metrics reference.

Histograms

We expose the histograms as exponential histograms which are converted to prometheus native histograms.

Note

Prometheus support must be enabled using the flag -enable-feature=native-histograms.

Native histograms are available only on the protobuf format so Prometheus will switch to the protobuf collection format.

You can switch back to regular histograms node by adding the following environment variable to a node: ADDITIONAL_CONFIG_DISABLE_NATIVE_HISTOGRAMS=”canton.monitoring.metrics.histograms=[]”

Enabling metrics

Configuring a helm deployment to enable metrics

To enable metrics in a helm deployment, set the metrics.enabled value to true (default false) in the helm values. This will create a ServiceMonitor kubernetes custom resource. For this to work it would require that the Prometheus operator is installed in the cluster.

Alternatively you can add prometheus scrape annotations to the charts that are configured to scrape port 10013.

Configuring a docker compose deployment to enable metrics

Note

Validator nodes only

When using docker compose for the deployment, the metrics are enabled by default. These can be accessed at http://validator.localhost/metrics for the validator app and at http://participant.localhost/metrics for the participant.

Grafana Dashboards

The release bundle (Download Bundle) contains a set of Grafana dashboards that are built based on the metrics above. These dashboards can be imported into a Grafana instance. The dashboards are built assuming a K8s deployment, and may need to be modified for other deployment types. The dashboards can be found under the grafana-dashboards folder in the release bundle.

Note

The dashboards are built using queries specific for Prometheus native histograms.