Exporting OpenTelemetry Metrics to Prometheus
What is Prometheus?
Prometheus is an open-source monitoring system with flexible query language and real-time alerting. It records metrics in a time series database using a HTTP pull model.
What is OpenTelemetry?
OpenTelemetry is an open source and vendor-neutral API for distributed tracing (including logs and errors) and metrics. With OpenTelemetry, you can instrument your application once and then add or change OpenTelemetry-compatible backends without changing the instrumentation.
OpenTelemetry Collector
What is OpenTelemetry Collector?
OpenTelemetry Collector is a proxy between your application and a backend that receives telemetry data, transforms it, and then exports to backends that can store the data permanently.
Collector can also act as an agent that pulls telemetry data from monitored systems, for example, Redis or filesystem metrics.
Exporting metrics to Prometheus
You can configure OpenTelemetry Collector to export data to Prometheus using Prometheus exporter:
exporters:
prometheus:
endpoint: '1.2.3.4:1234'
namespace: test-space
const_labels:
label1: value1
'another label': spaced value
send_timestamps: true
metric_expiration: 180m
resource_to_telemetry_conversion:
enabled: true
OpenTelemetry Metrics
OpenTelemetry Go metrics
OpenTelemetry specifies how to collect, aggregate, and send metrics to backend platforms. Using OpenTelemetry instruments, you can create counter, gauge, and histogram metrics.
The simplest Counter instrument looks like this:
import "go.opentelemetry.io/otel/metric/instrument"
counter := Meter.SyncInt64().Counter(
"test.my_counter",
instrument.WithUnit("1"),
instrument.WithDescription("Just a test counter"),
)
// Increment the counter.
counter.Add(ctx, 1, attribute.String("foo", "bar"))
counter.Add(ctx, 10, attribute.String("hello", "world"))
Sending Go metrics to Prometheus
To send OpenTelemetry Go metric to Prometheus, you need to provide a Prometheus target:
import (
"go.opentelemetry.io/otel/exporters/prometheus"
"go.opentelemetry.io/otel/metric/global"
"go.opentelemetry.io/otel/sdk/export/metric/aggregation"
"go.opentelemetry.io/otel/sdk/metric/aggregator/histogram"
controller "go.opentelemetry.io/otel/sdk/metric/controller/basic"
processor "go.opentelemetry.io/otel/sdk/metric/processor/basic"
selector "go.opentelemetry.io/otel/sdk/metric/selector/simple"
)
func configureOpentelemetry() {
exporter := configureMetrics()
http.HandleFunc("/metrics", exporter.ServeHTTP)
fmt.Println("listenening on http://localhost:8088/metrics")
go func() {
_ = http.ListenAndServe(":8088", nil)
}()
}
func configureMetrics() *prometheus.Exporter {
config := prometheus.Config{}
ctrl := controller.New(
processor.NewFactory(
selector.NewWithHistogramDistribution(
histogram.WithExplicitBoundaries(config.DefaultHistogramBoundaries),
),
export.CumulativeExportKindSelector(),
processor.WithMemory(true),
),
)
exporter, err := prometheus.New(config, ctrl)
if err != nil {
panic(err)
}
global.SetMeterProvider(exporter.MeterProvider())
return exporter
}
And then configure Prometheus to scrape exposed metrics:
scrape_configs:
- job_name: 'opentelemetry'
static_configs:
- targets: ['localhost:8088']
You can find a runnable example at GitHub.