Prometheus metrics export
The metrics exporting feature enables you to monitor and analyze various metrics related to your workload. Below, we detail the key aspects of this feature:
Library Dependency💡
Our project utilizes the lightweight library github.com/VictoriaMetrics/metrics as an alternative to github.com/prometheus/client_golang. This library facilitates efficient handling and exporting of metrics for monitoring purposes.
Exporting Strategies💡
You have the flexibility to choose between two strategies for exporting metrics: pull or push.
Push Strategy:💡
With this strategy, the tool pushes metrics to the provided URL at the specified interval.
Configure the agent using the following flags:
- metrics_export_url: Specifies the URL where metrics will be pushed.
- metrics_export_interval_seconds: Defines the interval (in seconds) for pushing metrics.
Pull Strategy:💡
With this strategy, external app pull metrics from us and put them into Prometheus.
Configure the agent using the following flags: - metrics_export_url: Configure the agent to expose an HTTP server on the provided port.
Metrics Selection💡
The metrics available for monitoring encompass both system-level metrics and custom metrics related to your workloads. Here are some popular metrics from a Go application's perspective:
System Metrics:💡
go_goroutines
go_threads
go_memstats_alloc_bytes
go_memstats_heap_inuse_bytes
go_memstats_alloc_bytes_total
process_resident_memory_bytes
Custom Workload Metrics:💡
requests_total
requests_error
requests_duration_seconds
Labels for Querying💡
When querying custom workload metrics, you can utilize labels to specify job-related information:
{job="job_name_here", job_uuid="auto_generate_uuid", job_type="write|bulk_write|read|update..." agent="agent_name_here"}
Example query:💡
The job_uuid
label distinguishes between different job runs/attempts, allowing you to track and analyze performance across multiple executions of the same job. Additionally, all metrics are labeled with the name of the agent
, enabling you to differentiate metrics coming from different agents.
Additional Resources💡
Metrics have been extracted from VictoriaMetrics sources. For more in-depth information about VictoriaMetrics, you can refer to the following article: VictoriaMetrics: Creating the Best Remote Storage for Prometheus.
You can also refer to an excellent article discussing Prometheus metrics in Go, which can provide further insights (Prometheus Go Metrics)[https://povilasv.me/prometheus-go-metrics/].
Example Grafana Golang dashboard template.