OpenTelemetry Go net/http Instrumentation [otelhttp]

Learn how to instrument Go HTTP applications using otelhttp — OpenTelemetry's official HTTP instrumentation package.

otelhttp provides automatic distributed tracing and metrics for net/http servers and clients with minimal code changes.

Quick Setup

Minimal working example:

import (
    "fmt"
    "net/http"

    "go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
)

func main() {
    http.HandleFunc("/hello", func(w http.ResponseWriter, r *http.Request) {
        fmt.Fprintf(w, "Hello, World!")
    })

    handler := otelhttp.NewHandler(http.DefaultServeMux, "my-service")
    http.ListenAndServe(":8080", handler)
}

This single wrapper call automatically captures all incoming HTTP requests, traces request flow, records HTTP metrics, and exports telemetry data to your configured backend.

What is OpenTelemetry?

OpenTelemetry is an open-source observability framework that aims to standardize and simplify the collection, processing, and export of telemetry data from applications and systems.

OpenTelemetry supports multiple programming languages and platforms, making it suitable for a wide range of applications and environments.

OpenTelemetry enables developers to instrument their code and collect telemetry data, which can then be exported to various OpenTelemetry backends or observability platforms for analysis and visualization. The OpenTelemetry architecture provides a modular, vendor-neutral approach to observability.

Configuration can be managed through OpenTelemetry environment variables, providing a standardized way to configure exporters, resource attributes, and sampling behavior across environments.

net/http instrumentation

To install otelhttp instrumentation:

go get go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp

• Example
• Reference

otelhttp vs Manual Instrumentation

otelhttp provides significant advantages over manual HTTP span creation:

Using otelhttp eliminates boilerplate code while ensuring compliance with OpenTelemetry semantic conventions for HTTP instrumentation.

Handler options

The otelhttp handler supports several configuration options:

Instrumenting http.Server

You can instrument HTTP server by wrapping all your handlers:

handler := http.Handler(http.DefaultServeMux) // or use your router
handler = otelhttp.NewHandler(handler, "")

httpServer := &http.Server{
    Addr:         ":8888",
    ReadTimeout:  5 * time.Second,
    WriteTimeout: 10 * time.Second,
    IdleTimeout:  60 * time.Second,
    Handler:      handler,
}

err := httpServer.ListenAndServe()

Filtering requests

You can exclude some requests from being traced using otelhttp.WithFilter:

handler = otelhttp.NewHandler(handler, "", otelhttp.WithFilter(otelReqFilter))

func otelReqFilter(req *http.Request) bool {
    return req.URL.Path != "/ping"
}

Span name

You can customize span name formatting using otelhttp.WithSpanNameFormatter:

handler = otelhttp.NewHandler(handler, "", otelhttp.WithSpanNameFormatter(httpSpanName))

func spanName(operation string, req *http.Request) string {
    return operation
}

Route attribute

If you are instrumenting individual handlers (not all handlers at once), you can annotate handler spans with http.route attribute. This can be useful when you can't find an instrumentation for your router.

handler = otelhttp.WithRouteTag("/hello/:username", handler)

Advanced otelhttp Configuration

Configure otelhttp for production environments:

handler = otelhttp.NewHandler(handler, "my-service",
    otelhttp.WithMessageEvents(otelhttp.ReadEvents, otelhttp.WriteEvents),
    otelhttp.WithSpanOptions(trace.WithSpanKind(trace.SpanKindServer)),
    otelhttp.WithPublicEndpoint(), // For public APIs
)

Custom attributes for business logic:

handler = otelhttp.NewHandler(handler, "",
    otelhttp.WithSpanNameFormatter(func(operation string, r *http.Request) string {
        return fmt.Sprintf("%s %s", r.Method, r.URL.Path)
    }),
)

otelhttp Middleware Integration

Combine otelhttp with authentication middleware:

func authMiddleware(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        // Add user info to span
        span := trace.SpanFromContext(r.Context())
        span.SetAttributes(attribute.String("user.id", getUserID(r)))

        next.ServeHTTP(w, r)
    })
}

// Chain middleware
handler = authMiddleware(otelhttp.NewHandler(myHandler, ""))

Instrumenting http.Client

otelhttp provides a HTTP transport to instrument http.Client:

client := http.Client{
    Transport: otelhttp.NewTransport(http.DefaultTransport),
}

You can also use the following shortcuts to make HTTP requets:

• otelhttp.Get
• otelhttp.Post
• otelhttp.PostForm
• otelhttp.Head

import "go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"

resp, err := otelhttp.Get(ctx, "https://google.com/")

HTTP Metrics Collection

otelhttp automatically collects HTTP metrics following OpenTelemetry semantic conventions.

Server metrics:

Client metrics:

All metrics include attributes like http.request.method, http.response.status_code, and http.route.

Creating custom spans

In addition to automatic instrumentation, create custom spans to trace specific operations within your handlers:

import (
    "encoding/json"
    "net/http"

    "go.opentelemetry.io/otel"
    "go.opentelemetry.io/otel/attribute"
    "go.opentelemetry.io/otel/codes"
)

var tracer = otel.Tracer("my-service")

func userHandler(w http.ResponseWriter, r *http.Request) {
    ctx := r.Context()

    // Create a custom span for the database query
    ctx, span := tracer.Start(ctx, "fetch-user")
    defer span.End()

    userID := r.URL.Query().Get("id")
    span.SetAttributes(attribute.String("user.id", userID))

    // Replace with your actual database lookup
    user, err := fetchUserFromDB(ctx, userID)
    if err != nil {
        span.RecordError(err)
        span.SetStatus(codes.Error, err.Error())
        http.Error(w, "Internal server error", http.StatusInternalServerError)
        return
    }

    span.SetAttributes(attribute.Bool("user.found", user != nil))
    json.NewEncoder(w).Encode(user)
}

Recording errors

Record errors on spans to mark them as failed and capture error details:

import (
    "encoding/json"
    "net/http"

    "go.opentelemetry.io/otel/codes"
    "go.opentelemetry.io/otel/trace"
)

func orderHandler(w http.ResponseWriter, r *http.Request) {
    span := trace.SpanFromContext(r.Context())

    // Replace with your actual business logic
    order, err := processOrder(r.Context())
    if err != nil {
        span.RecordError(err)
        span.SetStatus(codes.Error, "order processing failed")
        http.Error(w, "Failed to process order", http.StatusInternalServerError)
        return
    }

    json.NewEncoder(w).Encode(order)
}

otelhttp Troubleshooting

Common otelhttp issues and solutions:

Missing HTTP spans - check handler wrapping:

// Wrong: forgot to wrap handler
http.HandleFunc("/api", apiHandler)

// Correct: wrap with otelhttp
http.Handle("/api", otelhttp.NewHandler(http.HandlerFunc(apiHandler), ""))

Client spans missing - ensure transport usage:

client := &http.Client{
    Transport: otelhttp.NewTransport(http.DefaultTransport), // Don't forget this
}

Missing traces in distributed requests:

// Ensure context propagation in HTTP clients
req := req.WithContext(ctx) // Pass parent context
resp, err := client.Do(req)

What is Uptrace?

Uptrace is a OpenTelemetry APM that supports distributed tracing, metrics, and logs. You can use it to monitor applications and troubleshoot issues.

Uptrace comes with an intuitive query builder, rich dashboards, alerting rules with notifications, and integrations for most languages and frameworks.

Uptrace can process billions of spans and metrics on a single server and allows you to monitor your applications at 10x lower cost.

In just a few minutes, you can try Uptrace by visiting the cloud demo (no login required) or running it locally with Docker. The source code is available on GitHub.

FAQ

What's the operation name parameter in NewHandler? The operation name helps identify different handlers in traces. Use descriptive names like "user-api" or "payment-service". When wrapping all handlers at once, use your service name. When wrapping individual handlers, use the handler's purpose.

Can otelhttp work with streaming responses? Yes, otelhttp supports streaming. The span remains open and closes when the response writer finishes. Message events (enabled via WithMessageEvents) track individual read/write operations during streaming.

How do I disable tracing for health checks? Use WithFilter to exclude specific endpoints. Return false for paths like /health or /ready to skip tracing those requests and reduce telemetry noise.

Does otelhttp work with popular Go routers? Yes, otelhttp works with any router that uses the standard http.Handler interface, including Chi, Gorilla Mux, and the standard library router. For framework-specific instrumentation with additional features, see Gin or Echo.

What's the performance overhead? Minimal — typically a few microseconds per request for span creation and context propagation. For high-traffic applications, configure sampling to control the volume of exported spans.

What's the difference between WithPublicEndpoint and default behavior? WithPublicEndpoint marks the handler as a public-facing entry point. When set, otelhttp creates a new trace root instead of continuing an incoming trace, linking to the parent via a Link. This is useful for public APIs where you don't want external callers to control your trace IDs.

What's next?

With net/http instrumentation configured, you have full visibility into your HTTP client and server operations. For higher-level web frameworks, explore Gin, Echo, or Gorilla Mux instrumentation guides.