OpenTelemetry Architecture
The primary goal of OpenTelemetry is to standardize the collection and exchange of telemetry data across different programming languages and frameworks. It allows developers to add instrumentation to their code that generates telemetry data such as traces, metrics, and logs.
What is OpenTelemetry?
OpenTelemetry is an open source observability framework hosted by Cloud Native Computing Foundation. It is a merger of OpenCensus and OpenTracing projects. See OpenTelemetry vs OpenTracing to learn how they compare.
OpenTelemetry aims to streamline the process of instrumenting applications for observability and provides a standardized approach to telemetry data collection.
OpenTelemetry promotes consistency across programming languages, frameworks, and environments, enabling developers to easily collect and analyze telemetry data for monitoring, debugging, and optimizing distributed systems.
Overview Diagram
OpenTelemetry architecture is designed to provide a standardized approach to collecting, transmitting, and processing telemetry data from applications and services. It consists of several key components that work together to enable observability in distributed systems.
Observability signals
OpenTelemetry provides a standardized way to capture observability signals:
- Metrics indicate that there is a problem.
- Traces tell you where the problem is.
- Logs help you find the root cause.
OpenTelemetry Metrics are quantitative measurements that help quantify the behavior of a system. They provide information about the current state or rate of some aspect of an application, such as CPU usage, memory consumption, or request latency. OpenTelemetry allows you to define and record custom metrics to monitor the performance and health of your application.
OpenTelemetry Traces provide a detailed record of a request's execution path as it flows through a distributed system. They capture the timing information of individual operations and their relationships, allowing you to understand the flow of requests, identify bottlenecks, and troubleshoot performance issues. With OpenTelemetry, you can instrument your code to generate distributed traces and correlate them across services.
OpenTelemetry Logs are textual records of events or messages that occur during the execution of an application. They help you understand application behavior, diagnose problems, and audit activities. OpenTelemetry provides a mechanism to capture structured logs from your application and enrich them with contextual information.
Instrumentation Libraries
OpenTelemetry provides libraries for different programming languages, which developers can use to instrument their applications and collect telemetry data.
OpenTelemetry instrumentations are plugins for popular frameworks and libraries that use OpenTelemetry API to record important operations, for example, HTTP requests, DB queries, logs, errors, and more.
OpenTelemetry SDK
OpenTelemetry SDK (Software Development Kit) is a collection of libraries and tools that enable developers to instrument their applications and collect telemetry data for monitoring purposes.
Otel SDK provides a standardized and extensible framework for integrating OpenTelemetry into various programming languages and environments.
Exporters
OpenTelemetry exporters are responsible for sending telemetry data to external systems or backends for storage, analysis, and visualization.
OpenTelemetry offers a range of exporters that support different protocols and formats for exporting telemetry data. Such exporters allow users to integrate OpenTelemetry seamlessly with their preferred monitoring and analysis tools.
OpenTelemetry Protocol
OpenTelemetry Protocol (OTLP) is an open source, vendor-neutral protocol for collecting, transmitting, and exporting telemetry data from software systems and applications.
OTLP defines the wire format and the structure of the data that is exchanged between the instrumented application and the backend system. It specifies the format for encoding telemetry data, including the schema for metrics, traces, and logs, and the rules for transporting that data across the network.
OTLP exporter allows to transmit the collected telemetry data to a backend for processing and analysis.
Context Propagation
Context propagation ensures that relevant contextual data, such as trace IDs, span IDs, and other metadata, is propagated consistently across different services and components of an application.
By propagating context, OpenTelemetry ensures that telemetry collected from different services and components remains correlated, even in distributed and microservices architectures. It enables end-to-end tracing, making it easier to understand request flows, performance bottlenecks, and system dependencies.
Resource
Resource attributes are key-value pairs that provide metadata about the monitored entity, such as a service, process, or container. They help identify the resource and provide additional information that can be used to filter and group telemetry data.
By including resource information in telemetry data, OpenTelemetry enables better analysis, visualization, and understanding of system behavior. It helps correlate and contextualize telemetry data from different sources and provides a more comprehensive view of the observed application or service.
OpenTelemetry Collector
OpenTelemetry Collector plays a critical role in the OpenTelemetry ecosystem by providing a flexible and scalable solution for collecting and processing telemetry data.
Otel Collector acts as a centralized intermediary, simplifying the complexity of data collection and enabling flexible integration with different backends and systems.
OpenTelemetry in Kubernetes
OpenTelemetry provides powerful observability capabilities that can be seamlessly integrated into Kubernetes environments, offering comprehensive monitoring and tracing for containerized applications and microservices. OpenTelemetry in Kubernetes enables you to instrument your applications and gather telemetry data across your entire cluster. It allows you to:
- Automatically inject OpenTelemetry instrumentation into your pods
- Collect metrics, traces, and logs from both applications and Kubernetes infrastructure
- Use the OpenTelemetry Collector to process and export data to your preferred backend, such as Uptrace
- Implement distributed tracing across microservices in your Kubernetes cluster
- Monitor the health and performance of your Kubernetes nodes, pods, and services
OpenTelemetry Operator
The OpenTelemetry Operator is a Kubernetes-native method for managing and configuring OpenTelemetry instrumentation in your cluster. It simplifies the deployment and management of OpenTelemetry components:
- Automating the injection of OpenTelemetry auto-instrumentation into your applications
- Managing the lifecycle of OpenTelemetry Collector instances
- Providing custom resource definitions (CRDs) for easy configuration of OpenTelemetry components
- Enabling dynamic updates to your OpenTelemetry setup without redeploying applications
- Facilitating the collection of Kubernetes-specific metadata to enrich your telemetry data
The OpenTelemetry Operator streamlines the process of implementing observability in Kubernetes, making it easier to gain insights into your containerized applications and infrastructure.
OpenTelemetry Backend
OpenTelemetry does not include a specific built-in backend or storage system to process and analyze the data. Instead, OpenTelemetry provides the flexibility to select and integrate with various backend systems based on your specific needs and preferences.
Backend systems receive the exported telemetry data from the instrumented applications or the OpenTelemetry Collector. These systems are responsible for storing, analyzing, and visualizing the telemetry data.
See OpenTelemetry backends for details.
Conclusion
OpenTelemetry architecture promotes flexibility, interoperability, and extensibility, enabling developers and operators to adopt observability practices that suit their specific requirements and environments.