OpenTelemetry Collector
What is OpenTelemetry Collector?
OpenTelemetry Collector is a vendor-agnostic proxy/middleman between your application and a distributed tracing tool such Uptrace or Jaeger. Collector receives telemetry data, processes it, and then exports the data to tracing tools that can store it permanently.
OpenTelemetry Collector is written in Go and licensed under Apache 2.0 license which allows you to change the source code and install custom extensions. That comes at a cost of running and maintaining your own OpenTelemetry Collector instances.
When to use OpenTelemetry Collector?
Most of the time, sending telemetry data directly to a backend is the great way to get started with OpenTelemetry. But you may want to deploy a collector alongside your services to get batching, retries, sensitive data filtering, and more.
The most prominent OpenTelemetry Collector feature is the ability to operate on whole traces instead of individual spans. To achieve that, OpenTelemetry Collector buffers the received spans and groups them by a trace id. That is the key requirement to implement tail-based sampling.
OpenTelemetry Collector can also act as an agent that pulls telemetry data from monitored systems, for example, Redis or filesystem metrics.
Installation
OpenTelemetry Collector distributes pre-compiled binaries for Linux, MacOS, and Windows.
Linux
To install otelcol-contrib binary with the associated systemd service, run the following command replacing 0.49.0 with the desired version and amd64 with the desired architecture:
wget https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download/v0.49.0/otelcol-contrib_0.49.0_linux_amd64.deb
sudo dpkg -i otelcol-contrib_0.49.0_linux_amd64.deb
wget https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download/v0.49.0/otelcol_0.49.0_linux_amd64.rpm
sudo rpm -ivh otelcol_0.49.0_linux_amd64.rpm
You can check the status of the installed service with:
sudo systemctl status otelcol-contrib
And check the logs with:
sudo journalctl -u otelcol-contrib -f
You can edit the config at /etc/otelcol-contrib/config.yaml and restart OpenTelemetry Collector:
sudo systemctl restart otelcol-contrib
Compiling from sources
You can also compile OpenTelemetry Collector locally:
git clone https://github.com/open-telemetry/opentelemetry-collector-contrib.git
cd opentelemetry-collector-contrib
make install-tools
make otelcontribcol
./bin/otelcontribcol_linux_amd64 --config ./examples/local/otel-config.yaml
Configuration
By default, you can find the config file at /etc/otelcol-contrib/config.yaml, for example:
TIP
Don't repeat a common mistake by configuring a receiver or an exporter without adding it to the service processing pipeline. Such receivers and exporters are ignored.
# receivers configure how data gets into the Collector.
receivers:
otlp:
protocols:
grpc:
http:
# processors specify what happens with the received data.
processors:
resourcedetection:
detectors: [system]
batch:
send_batch_size: 10000
timeout: 10s
# exporters configure how to send processed data to one or more backends.
exporters:
otlp:
endpoint: otlp.uptrace.dev:4317
headers:
uptrace-dsn: 'https://<key>@uptrace.dev/<project_id>'
# service pulls the configured receivers, processors, and exporters together into
# processing pipelines. Unused receivers/processors/exporters are ignored.
service:
pipelines:
traces:
receivers: [otlp]
processors: [batch]
exporters: [otlp]
You can use Collector Configurator to generate the config file or learn more about Collector using the official documentation.
Host metrics
hostmetricsreceiver is a Collector plugin that gathers various metrics about the host system, for example, CPU, RAM, disk metrics and more.
To start collecting host metrics, you need to install Collector on each system you want to monitor and add the following lines to the Collector config:
receivers:
hostmetrics:
collection_interval: 10s
scrapers:
cpu:
disk:
filesystem:
load:
memory:
network:
paging:
If you are using unusual filesystems, you may want to configure the receiver more thoroughly, for example:
receivers:
hostmetrics:
collection_interval: 10s
scrapers:
cpu:
disk:
load:
filesystem:
include_fs_types:
match_type: strict
fs_types: [ext3, ext4]
memory:
network:
paging:
Also see Collector Configurator.
Uptrace configuration
You can configure OpenTelemetry Collector to export data to Uptrace using the OTLP exporter:
exporters:
otlp:
endpoint: otlp.uptrace.dev:4317
headers:
# Copy your project DSN here
uptrace-dsn: 'https://<key>@uptrace.dev/<project_id>'
