OpenTelemetry Trace Context Propagation [Rust]

This guide covers Rust-specific implementation of context propagation. For a comprehensive overview of context propagation concepts, W3C TraceContext, propagators, and troubleshooting, see the OpenTelemetry Context Propagation guide.

Automatic propagation

OpenTelemetry Rust handles traceparent headers automatically when using specialized instrumentation libraries. Base HTTP client libraries (reqwest, hyper) and server libraries (axum, actix-web) require additional instrumentation crates for automatic header injection and extraction.

Auto-instrumentation

toml

[dependencies]
opentelemetry = "0.24"
opentelemetry_sdk = { version = "0.24", features = ["rt-tokio"] }
opentelemetry-otlp = "0.17"
tracing = "0.1"
tracing-opentelemetry = "0.25"
tracing-subscriber = "0.3"

rust

use opentelemetry::{global, KeyValue};
use opentelemetry_sdk::{trace, Resource};
use tracing_subscriber::{layer::SubscriberExt, util::SubscriberInitExt};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error + Send + Sync + 'static>> {
    // Initialize OpenTelemetry
    let tracer = opentelemetry_otlp::new_pipeline()
        .tracing()
        .with_exporter(
            opentelemetry_otlp::new_exporter()
                .tonic()
                .with_endpoint("https://api.uptrace.dev:4317")
        )
        .install_batch(opentelemetry_sdk::runtime::Tokio)?;

    // Create tracing layer with automatic propagation
    let telemetry = tracing_opentelemetry::layer().with_tracer(tracer);

    tracing_subscriber::registry()
        .with(telemetry)
        .with(tracing_subscriber::EnvFilter::from_default_env())
        .init();

    // HTTP clients and servers with instrumentation crates
    // automatically handle propagation when using tracing
    run_app().await?;

    global::shutdown_tracer_provider();
    Ok(())
}

Auto-instrumentation packages for HTTP clients (with tracing-reqwest, tracing-hyper) and web frameworks (with tower middleware) automatically inject W3C tracecontext headers to outgoing HTTP requests and extract them from incoming requests.

Manual propagation

When automatic instrumentation is not available, you can manually handle traceparent headers using OpenTelemetry's propagation API.

Extracting context

rust

use opentelemetry::{global, propagation::Extractor, Context};
use std::collections::HashMap;

fn handle_incoming_request(request_headers: &HashMap<String, String>) -> Result<(), Box<dyn std::error::Error>> {
    // Extract context from incoming request headers
    let parent_context = global::get_text_map_propagator(|propagator| {
        propagator.extract(&HeaderMapCarrier::new(request_headers))
    });

    // Create a new span with the extracted parent context
    let tracer = global::tracer("my-rust-service");

    let span = tracer
        .span_builder("handle-request")
        .with_kind(opentelemetry::trace::SpanKind::Server)
        .start_with_context(&tracer, &parent_context);

    let _guard = span.activate();

    // Your business logic here
    process_request(request_headers)?;

    Ok(())
}

// Header carrier implementation
struct HeaderMapCarrier<'a> {
    headers: &'a HashMap<String, String>,
}

impl<'a> HeaderMapCarrier<'a> {
    fn new(headers: &'a HashMap<String, String>) -> Self {
        Self { headers }
    }
}

impl<'a> Extractor for HeaderMapCarrier<'a> {
    fn get(&self, key: &str) -> Option<&str> {
        // Case-insensitive header lookup
        self.headers.iter()
            .find(|(k, _)| k.to_lowercase() == key.to_lowercase())
            .map(|(_, v)| v.as_str())
    }

    fn keys(&self) -> Vec<&str> {
        self.headers.keys().map(|k| k.as_str()).collect()
    }
}

fn process_request(headers: &HashMap<String, String>) -> Result<(), Box<dyn std::error::Error>> {
    println!("Processing request with {} headers", headers.len());
    Ok(())
}

Simple Axum Integration

rust

use axum::{
    extract::Request,
    http::{HeaderMap, StatusCode},
    middleware::{self, Next},
    response::Response,
    routing::get,
    Router,
};
use opentelemetry::{global, propagation::Extractor, trace::{SpanKind, Tracer}};

// Simple propagation middleware for Axum
async fn propagation_middleware(request: Request, next: Next) -> Result<Response, StatusCode> {
    let headers = request.headers();

    // Extract trace context from headers
    let parent_context = global::get_text_map_propagator(|propagator| {
        propagator.extract(&AxumHeaderExtractor::new(headers))
    });

    let tracer = global::tracer("axum-server");

    // Create span with extracted context
    let span = tracer
        .span_builder("http-request")
        .with_kind(SpanKind::Server)
        .start_with_context(&tracer, &parent_context);

    let _guard = span.activate();

    // Add basic attributes
    span.add_attribute("http.method", request.method().to_string());
    span.add_attribute("http.target", request.uri().path().to_string());

    let response = next.run(request).await;

    span.add_attribute("http.status_code", response.status().as_u16() as i64);

    Ok(response)
}

// Header extractor for Axum
struct AxumHeaderExtractor<'a> {
    headers: &'a HeaderMap,
}

impl<'a> AxumHeaderExtractor<'a> {
    fn new(headers: &'a HeaderMap) -> Self {
        Self { headers }
    }
}

impl<'a> Extractor for AxumHeaderExtractor<'a> {
    fn get(&self, key: &str) -> Option<&str> {
        self.headers.get(key)?.to_str().ok()
    }

    fn keys(&self) -> Vec<&str> {
        self.headers.keys().map(|k| k.as_str()).collect()
    }
}

// Route handler
async fn hello() -> &'static str {
    let tracer = global::tracer("handler");
    let span = tracer.span_builder("hello-handler").start(&tracer);
    let _guard = span.activate();
    "Hello, World!"
}

// Application setup
pub fn create_app() -> Router {
    Router::new()
        .route("/hello", get(hello))
        .layer(middleware::from_fn(propagation_middleware))
}

Injecting context

rust

use opentelemetry::{global, propagation::Injector, trace::Tracer};
use reqwest::Client;
use std::collections::HashMap;

async fn make_http_request(url: &str) -> Result<reqwest::Response, reqwest::Error> {
    let tracer = global::tracer("http-client");
    let span = tracer.span_builder("http-request").start(&tracer);
    let _guard = span.activate();

    // Create carrier for headers
    let mut headers = HashMap::new();

    // Inject current trace context into headers
    global::get_text_map_propagator(|propagator| {
        propagator.inject_context(&opentelemetry::Context::current(), &mut HeaderMapInjector::new(&mut headers));
    });

    span.add_attribute("http.url", url.to_string());
    span.add_attribute("http.method", "GET");

    // Make HTTP request with injected headers
    let client = Client::new();
    let mut request = client.get(url);

    for (key, value) in headers {
        request = request.header(&key, value);
    }

    let response = request.send().await?;
    span.add_attribute("http.status_code", response.status().as_u16() as i64);

    Ok(response)
}

// Header injector implementation
struct HeaderMapInjector<'a> {
    headers: &'a mut HashMap<String, String>,
}

impl<'a> HeaderMapInjector<'a> {
    fn new(headers: &'a mut HashMap<String, String>) -> Self {
        Self { headers }
    }
}

impl<'a> Injector for HeaderMapInjector<'a> {
    fn set(&mut self, key: &str, value: String) {
        self.headers.insert(key.to_string(), value);
    }
}

// Usage
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Initialize tracing...

    let response = make_http_request("https://api.example.com/users").await?;
    println!("Status: {}", response.status());

    Ok(())
}

Simple HTTP Client

rust

use opentelemetry::{global, propagation::Injector, trace::{SpanKind, Tracer}};
use reqwest::{Client, Response};
use std::collections::HashMap;

pub struct SimpleTracedClient {
    client: Client,
}

impl SimpleTracedClient {
    pub fn new() -> Self {
        Self {
            client: Client::new(),
        }
    }

    pub async fn get(&self, url: &str) -> Result<Response, reqwest::Error> {
        let tracer = global::tracer("http-client");
        let span = tracer
            .span_builder("HTTP GET")
            .with_kind(SpanKind::Client)
            .start(&tracer);

        let _guard = span.activate();

        span.add_attribute("http.method", "GET");
        span.add_attribute("http.url", url.to_string());

        // Inject trace context into request headers
        let mut headers_map = HashMap::new();
        global::get_text_map_propagator(|propagator| {
            propagator.inject_context(&opentelemetry::Context::current(), &mut HeaderMapInjector::new(&mut headers_map));
        });

        let mut request = self.client.get(url);
        for (key, value) in headers_map {
            if let (Ok(header_name), Ok(header_value)) = (
                key.parse::<reqwest::header::HeaderName>(),
                value.parse::<reqwest::header::HeaderValue>()
            ) {
                request = request.header(header_name, header_value);
            }
        }

        match request.send().await {
            Ok(response) => {
                span.add_attribute("http.status_code", response.status().as_u16() as i64);
                span.set_status(opentelemetry::trace::Status::Ok);
                Ok(response)
            }
            Err(e) => {
                span.set_status(opentelemetry::trace::Status::error(format!("Request failed: {}", e)));
                Err(e)
            }
        }
    }
}

// Usage
async fn example_usage() -> Result<(), Box<dyn std::error::Error>> {
    let client = SimpleTracedClient::new();
    let response = client.get("https://api.example.com/users").await?;
    println!("Response status: {}", response.status());
    Ok(())
}

Debugging propagation

Logging context

Log incoming traceparent headers and current span context for debugging:

rust

use opentelemetry::trace::TraceContextExt;
use tracing::{info, warn};

pub fn log_trace_context(request_headers: &HashMap<String, String>) {
    // Log incoming traceparent header
    match request_headers.get("traceparent") {
        Some(traceparent) => {
            info!("Incoming traceparent: {}", traceparent);
        }
        None => {
            info!("No traceparent header found");
        }
    }

    // Log current span context
    let current_context = opentelemetry::Context::current();
    let span = current_context.span();

    if span.span_context().is_valid() {
        let span_context = span.span_context();
        info!(
            "Current trace context - TraceId: {}, SpanId: {}, Sampled: {}",
            span_context.trace_id(),
            span_context.span_id(),
            span_context.trace_flags().is_sampled()
        );
    } else {
        warn!("No valid span context found");
    }
}

// Usage in middleware
async fn debug_middleware(request: Request, next: Next) -> Result<Response, StatusCode> {
    let headers: HashMap<String, String> = request
        .headers()
        .iter()
        .filter_map(|(k, v)| {
            v.to_str().ok().map(|v| (k.to_string(), v.to_string()))
        })
        .collect();

    log_trace_context(&headers);
    next.run(request).await
}

Validating format

Validate traceparent headers to ensure they follow the W3C specification:

rust

use regex::Regex;

pub struct TraceparentValidator {
    pattern: Regex,
}

impl TraceparentValidator {
    pub fn new() -> Self {
        // W3C traceparent format: 00-{trace-id}-{span-id}-{trace-flags}
        let pattern = Regex::new(r"^00-[0-9a-f]{32}-[0-9a-f]{16}-[0-9a-f]{2}$")
            .expect("Valid regex pattern");

        Self { pattern }
    }

    pub fn is_valid(&self, traceparent: &str) -> bool {
        self.pattern.is_match(traceparent)
    }

    pub fn parse(&self, traceparent: &str) -> Result<TraceparentInfo, String> {
        if !self.is_valid(traceparent) {
            return Err(format!("Invalid traceparent format: {}", traceparent));
        }

        let parts: Vec<&str> = traceparent.split('-').collect();

        Ok(TraceparentInfo {
            version: parts[0].to_string(),
            trace_id: parts[1].to_string(),
            span_id: parts[2].to_string(),
            flags: parts[3].to_string(),
            sampled: parts[3] == "01",
        })
    }
}

#[derive(Debug)]
pub struct TraceparentInfo {
    pub version: String,
    pub trace_id: String,
    pub span_id: String,
    pub flags: String,
    pub sampled: bool,
}

// Usage
fn validate_traceparent_example() {
    let validator = TraceparentValidator::new();
    let traceparent = "00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01";

    match validator.parse(traceparent) {
        Ok(info) => {
            println!("Valid traceparent: {:?}", info);
        }
        Err(e) => {
            println!("Invalid traceparent: {}", e);
        }
    }
}

Getting trace info

Access current trace context information:

rust

use opentelemetry::trace::TraceContextExt;
use serde_json::json;

pub fn get_trace_info() -> serde_json::Value {
    let current_context = opentelemetry::Context::current();
    let span = current_context.span();

    if !span.span_context().is_valid() {
        return json!({ "error": "No valid span context available" });
    }

    let span_context = span.span_context();

    json!({
        "trace_id": span_context.trace_id().to_string(),
        "span_id": span_context.span_id().to_string(),
        "sampled": span_context.trace_flags().is_sampled(),
        "remote": span_context.is_remote(),
        "trace_state": span_context.trace_state().header()
    })
}

// Axum handler to expose trace info
async fn trace_info_handler() -> axum::Json<serde_json::Value> {
    axum::Json(get_trace_info())
}

// Middleware to add trace info to response headers
async fn trace_info_middleware(request: Request, next: Next) -> Result<Response, StatusCode> {
    let mut response = next.run(request).await;

    // Add trace info to response headers for debugging
    let trace_info = get_trace_info();

    if trace_info.get("error").is_none() {
        let headers = response.headers_mut();

        if let Some(trace_id) = trace_info.get("trace_id").and_then(|v| v.as_str()) {
            if let Ok(header_value) = trace_id.parse() {
                headers.insert("X-Trace-Id", header_value);
            }
        }

        if let Some(span_id) = trace_info.get("span_id").and_then(|v| v.as_str()) {
            if let Ok(header_value) = span_id.parse() {
                headers.insert("X-Span-Id", header_value);
            }
        }
    }

    Ok(response)
}

Configuration

Environment variables

Configure propagation format using environment variables:

bash

# Default: W3C TraceContext and Baggage
export OTEL_PROPAGATORS="tracecontext,baggage"

# B3 format (requires opentelemetry-propagator-b3 crate)
export OTEL_PROPAGATORS="b3"

# Multiple formats
export OTEL_PROPAGATORS="tracecontext,b3,baggage"

Programmatic config

rust

use opentelemetry::{
    global,
    propagation::{composite::TextMapCompositePropagator, text_map_propagator::TextMapPropagator},
};
use opentelemetry_sdk::propagation::{TraceContextPropagator, BaggagePropagator};

fn configure_propagation() {
    // Create composite propagator with multiple formats
    let propagator = TextMapCompositePropagator::new(vec![
        // W3C TraceContext (default)
        Box::new(TraceContextPropagator::new()),
        // W3C Baggage
        Box::new(BaggagePropagator::new()),
    ]);

    // Set as global propagator
    global::set_text_map_propagator(propagator);
}

// Usage in application initialization
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error + Send + Sync + 'static>> {
    // Configure propagation before initializing tracing
    configure_propagation();

    // Initialize OpenTelemetry...
    init_tracing().await?;

    // Run application
    run_app().await?;

    Ok(())
}

Production considerations

Performance

Propagation has minimal performance impact in Rust:

Header extraction/injection is fast (< 1ms)
Context objects are lightweight
No network calls during propagation
Zero-cost abstractions when not recording

Async Context Propagation

Rust's async system automatically propagates context across .await points:

rust

async fn nested_async_operations() -> Result<(), Box<dyn std::error::Error>> {
    let tracer = global::tracer("async-service");
    let span = tracer.span_builder("parent-operation").start(&tracer);
    let _guard = span.activate();

    // Context is automatically available in nested async calls
    let result1 = async_database_call().await?;
    let result2 = async_http_call().await?;

    // Spawned tasks inherit the current context
    let handle = tokio::spawn(async move {
        background_processing().await
    });

    handle.await??;
    Ok((result1, result2))
}

async fn async_database_call() -> Result<String, Box<dyn std::error::Error>> {
    let tracer = global::tracer("database");
    let span = tracer.span_builder("db-query").start(&tracer);
    let _guard = span.activate();

    // Simulate database call
    tokio::time::sleep(tokio::time::Duration::from_millis(10)).await;
    Ok("Database result".to_string())
}

async fn async_http_call() -> Result<String, Box<dyn std::error::Error>> {
    let tracer = global::tracer("http-client");
    let span = tracer.span_builder("http-call").start(&tracer);
    let _guard = span.activate();

    // HTTP client will automatically inject trace context
    let client = reqwest::Client::new();
    let response = client.get("https://api.example.com/data").send().await?;
    Ok(response.text().await?)
}

async fn background_processing() -> Result<(), Box<dyn std::error::Error>> {
    let tracer = global::tracer("background");
    let span = tracer.span_builder("background-task").start(&tracer);
    let _guard = span.activate();

    tokio::time::sleep(tokio::time::Duration::from_millis(5)).await;
    Ok(())
}

Microservices Integration

Complete example for microservice communication:

rust

use axum::{
    extract::State,
    http::StatusCode,
    middleware,
    response::Json,
    routing::{get, post},
    Router,
};
use serde::{Deserialize, Serialize};
use std::sync::Arc;

#[derive(Clone)]
struct AppState {
    http_client: Arc<SimpleTracedClient>,
}

#[derive(Serialize, Deserialize)]
struct UserRequest {
    name: String,
    email: String,
}

#[derive(Serialize, Deserialize)]
struct UserResponse {
    id: u64,
    name: String,
    email: String,
}

// Service A: User service
async fn create_user(
    State(state): State<AppState>,
    Json(payload): Json<UserRequest>,
) -> Result<Json<UserResponse>, StatusCode> {
    let tracer = global::tracer("user-service");
    let span = tracer.span_builder("create-user").start(&tracer);
    let _guard = span.activate();

    span.add_attribute("user.name", payload.name.clone());
    span.add_attribute("user.email", payload.email.clone());

    // Call external service (context automatically propagated)
    let validation_result = state
        .http_client
        .get("http://validation-service:8080/validate")
        .await
        .map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;

    if !validation_result.status().is_success() {
        return Err(StatusCode::BAD_REQUEST);
    }

    // Create user (simulate database call)
    let user = UserResponse {
        id: 123,
        name: payload.name,
        email: payload.email,
    };

    span.set_status(opentelemetry::trace::Status::Ok);
    Ok(Json(user))
}

// Service B: Validation service
async fn validate_user(Json(payload): Json<UserRequest>) -> Result<StatusCode, StatusCode> {
    let tracer = global::tracer("validation-service");
    let span = tracer.span_builder("validate-user").start(&tracer);
    let _guard = span.activate();

    // Validation logic
    if payload.email.contains("@") && !payload.name.is_empty() {
        span.set_status(opentelemetry::trace::Status::Ok);
        Ok(StatusCode::OK)
    } else {
        span.set_status(opentelemetry::trace::Status::error("Validation failed"));
        Err(StatusCode::BAD_REQUEST)
    }
}

// Application setup with propagation
pub fn create_user_service() -> Router {
    let http_client = Arc::new(SimpleTracedClient::new());
    let state = AppState { http_client };

    Router::new()
        .route("/users", post(create_user))
        .layer(middleware::from_fn(propagation_middleware))
        .layer(middleware::from_fn(trace_info_middleware))
        .with_state(state)
}

pub fn create_validation_service() -> Router {
    Router::new()
        .route("/validate", post(validate_user))
        .layer(middleware::from_fn(propagation_middleware))
}

// Server startup
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error + Send + Sync + 'static>> {
    // Initialize tracing
    init_tracing().await?;

    // Choose service based on environment
    let app = match std::env::var("SERVICE_TYPE").as_deref() {
        Ok("validation") => create_validation_service(),
        _ => create_user_service(),
    };

    let listener = tokio::net::TcpListener::bind("0.0.0.0:8080").await?;
    println!("Server running on http://0.0.0.0:8080");

    axum::serve(listener, app).await?;

    Ok(())
}

async fn init_tracing() -> Result<(), Box<dyn std::error::Error + Send + Sync + 'static>> {
    // Configure propagation
    configure_propagation();

    // Initialize OpenTelemetry
    let tracer = opentelemetry_otlp::new_pipeline()
        .tracing()
        .with_exporter(
            opentelemetry_otlp::new_exporter()
                .tonic()
                .with_endpoint("https://api.uptrace.dev:4317")
        )
        .install_batch(opentelemetry_sdk::runtime::Tokio)?;

    let telemetry = tracing_opentelemetry::layer().with_tracer(tracer);

    tracing_subscriber::registry()
        .with(telemetry)
        .with(tracing_subscriber::EnvFilter::from_default_env())
        .init();

    Ok(())
}

OpenTelemetry APM

Uptrace is a OpenTelemetry APM that supports distributed tracing, metrics, and logs. You can use it to monitor applications and troubleshoot issues.

Uptrace Overview

Uptrace comes with an intuitive query builder, rich dashboards, alerting rules with notifications, and integrations for most languages and frameworks.

Uptrace can process billions of spans and metrics on a single server and allows you to monitor your applications at 10x lower cost.

In just a few minutes, you can try Uptrace by visiting the cloud demo (no login required) or running it locally with Docker. The source code is available on GitHub.

What's next?

Resource detectorsAdd resource attributes in Rust to describe service name, environment, and deployment metadata so Uptrace can group telemetry correctly.

SamplingTune OpenTelemetry sampling in Rust with parent-based, trace ID ratio, or tail sampling strategies to balance observability cost and fidelity in Uptrace.