OpenTelemetry Collector

Introduction

OpenTelemetry Collectoropen in new window is a proxy between your application and a backend that receives telemetry data, transforms it, and then exports to backends that can store the data permanently. Collector can also act as an agent that pulls telemetry data from monitored systems, for example, Redis or filesystem metrics.

The most prominent OpenTelemetry Collector feature is the ability to operate on whole traces instead of individual spans. To achieve that OpenTelemetry Collector buffers the received spans in the RAM and groups them by a trace id. That is the key requirement to implement tail-based sampling.

OpenTelemetry Collector uses Apache 2.0 license which allows you to change the source code and install custom extensions. That comes at a cost of running and maintaining your own OpenTelemetry Collector servers.

Installation

OpenTelemetry Collector distributes pre-compiled binariesopen in new window for Linux, MacOS, and Windows.

Linux

To install otel-contrib-collector binary with the associated systemd service:

wget https://github.com/open-telemetry/opentelemetry-collector-contrib/releases/download/v0.40.0/otel-contrib-collector_0.40.0_amd64.deb
sudo dpkg -i otel-contrib-collector_0.40.0_amd64.deb

wget https://github.com/open-telemetry/opentelemetry-collector-contrib/releases/download/v0.40.0/otel-contrib-collector-0.40.0-1.x86_64.rpm
sudo rpm -ivh otel-contrib-collector-0.40.0-1.x86_64.rpm

You can check the status of the installed service with:

sudo systemctl status otel-contrib-collector

And then consult the logs if you find any issues:

sudo journalctl -u otel-contrib-collector -f

Compiling from sources

You can also compile OpenTelemetry Collector locally:

git clone https://github.com/open-telemetry/opentelemetry-collector-contrib.git
cd opentelemetry-collector-contrib
make install-tools
make otelcontribcol
./bin/otelcontribcol_linux_amd64 --config ./examples/local/otel-config.yaml

Configuration

By default, you can find the config file at /etc/otel-contrib-collector/config.yaml, for example:

# receivers configure how data gets into the Collector.
receivers:
  otlp:
    protocols:
      grpc:
      http:

# processors specify what happens with the received data.
processors:
  resourcedetection:
    detectors: [system]
  batch:
    send_batch_size: 10000
    timeout: 10s

# exporters configure how you send processed data to one or more backends.
exporters:
  otlp:
    endpoint: otlp.uptrace.dev:4317
    headers:
      uptrace-dsn: 'https://<key>@uptrace.dev/<project_id>'

# service pulls the configured receivers, processors, and exporters together into
# processing pipelines.
service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlp]

TIP

Don't repeat a common mistake by configuring a receiver or an exporter without adding it to the service processing pipeline.

You can use Collector Configurator to generate the config file or learn more about Collector using the official documentationopen in new window.

Host metrics

hostmetricsreceiveropen in new window is a Collector plugin that gathers various metrics about the host system, for example, CPU, RAM, disk metrics and more.

To start collecting host metrics, you need to install Collector on each system you want to monitor and add the following lines to Collector config:

receivers:
  hostmetrics:
    collection_interval: 10s
    scrapers:
      cpu:
      disk:
      filesystem:
      load:
      memory:
      network:
      paging:

For an example, see Collector Configurator.

Uptrace configuration

You can configure Collector to export data to Uptrace using the OTLP exporter:

exporters:
  otlp:
    endpoint: otlp.uptrace.dev:4317
    headers:
      # Copy your project DSN here
      uptrace-dsn: 'https://<key>@uptrace.dev/<project_id>'

# OpenTelemetry Collector

# Introduction

# Installation

# Linux

# Compiling from sources

# Configuration

# Host metrics

# Uptrace configuration