OpenTelemetry Host Metrics receiver
hostmetricsreceiver is an OpenTelemetry Collector plugin that gathers various metrics about the host system, for example, CPU, RAM, disk metrics and other system-level metrics.
By collecting and analyzing host metrics, you can gain insight into the performance and health of your host systems and identify potential problems or bottlenecks that could affect the overall performance of your applications and services.
What is OpenTelemetry Collector?
OpenTelemetry Collector is an agent that pulls telemetry data from systems you want to monitor and sends it to tracing tools using the OpenTelemetry protocol (OTLP).
OpenTelemetry Collector provides powerful data processing capabilities. It can aggregate, filter, transform, and enrich telemetry data as it flows through the system.
Host Metrics
To start collecting host metrics, you need to install Collector on each system you want to monitor and add the following lines to the Collector config:
processors:
resourcedetection:
detectors: [env, system]
cumulativetodelta:
receivers:
hostmetrics:
collection_interval: 10s
scrapers:
# CPU utilization metrics
cpu:
# Disk I/O metrics
disk:
# File System utilization metrics
filesystem:
# CPU load metrics
load:
# Memory utilization metrics
memory:
# Network interface I/O metrics & TCP connection metrics
network:
# Paging/Swap space utilization and I/O metrics
paging:
service:
pipelines:
metrics:
receivers: [otlp, hostmetrics]
processors: [cumulativetodelta, batch, resourcedetection]
exporters: [otlp/uptrace]
Filesystem metrics
If you are using unusual filesystems, you may want to configure filesystem receiver more thoroughly, for example, to scrape only supported filesystem types and avoid warnings:
receivers:
hostmetrics:
collection_interval: 10s
scrapers:
cpu:
disk:
load:
filesystem:
include_fs_types:
match_type: strict
fs_types: [ext3, ext4]
memory:
network:
paging:
Process metrics
To collect per process CPU, Memory, and Disk I/O metrics, you need to enable the respective scrapers:
receivers:
hostmetrics:
collection_interval: 10s
scrapers:
# Process count metrics
process:
# Per process CPU, Memory, and Disk I/O metrics
processes:
Those scrapers are disabled by default, because they require running OpenTelemetry Collector with elevated permissions in order to access information about other processes.
On Linux, you can achieve that by running otelcol-contrib under root user:
# /lib/systemd/system/otelcol-contrib.service
User=root
Group=root
Or using sudo to start the process:
# /lib/systemd/system/otelcol-contrib.service
ExecStart=sudo /usr/bin/otelcol-contrib $OTELCOL_OPTIONS
Resource detection
The Resource Detection Processor is responsible for detecting and setting resource information for telemetry data.
A resource represents a set of attributes that describe the entity producing the telemetry data. This can include information such as application name, version, environment, host name, and more
The resource detection processor can automatically discover resource attributes based on the environment in which it is running. For example, it can detect attributes from cloud providers, container orchestrators, or other runtime environments.
Here is how you can configure the processor in the OpenTelemetry Collector configuration file (otel-collector-config.yaml):
processors:
resourcedetection:
detectors: [env, system, ec2, gcp, azure]
timeout: 5s
# Override existing resource attributes if they already exist on incoming telemetry data.
override: true
service:
pipelines:
traces:
processors: [resourcedetection, batch]
metrics:
processors: [resourcedetection, batch]
By default, host.name is set to FQDN if possible, and a hostname provided by the OS is used as a fallback. This logic can be changed with the hostname_sources configuration, which is set to [dns, os] by default.
processors:
resourcedetection/system:
detectors: [system]
system:
hostname_sources: [os]
Container host metrics
On Linux, OpenTelemetry collects metrics from the Linux system directories. To collect metrics about the host system and not the container, you can mount the host filesystem when running the container:
# mount the entire filesystem
docker run -v /:/hostfs ...
# or mount only parts you need
docker run -v /proc:/hostfs/proc ...
Then configure root_path so the hostmetrics receiver knows where the root filesystem is:
receivers:
hostmetrics:
root_path: /hostfs
Exporting data to Uptrace
Uptrace is a OpenTelemetry APM that supports distributed tracing, metrics, and logs. You can use it to monitor applications and troubleshoot issues.
Uptrace comes with an intuitive query builder, rich dashboards, alerting rules with notifications, and integrations for most languages and frameworks.
Uptrace can process billions of spans and metrics on a single server and allows you to monitor your applications at 10x lower cost.
In just a few minutes, you can try Uptrace by visiting the cloud demo (no login required) or running it locally with Docker. The source code is available on GitHub.
To export host metrics data to Uptrace, follow the guide to configure otlp/uptrace exporter.
What's next?
OpenTelemetry Collector supports a variety of receivers that allow you to collect telemetry data from different sources. Receivers act as plugins within the Collector and handle the ingestion of data from various protocols and formats. Here are some of the commonly used receivers available in the OpenTelemetry Collector:
