Monitor Go net/http performance with OpenTelemetry

Learn how to monitor net/http performance using OpenTelemetry observability framework.

Package net/http provides HTTP client and server implementations. It allows you to make and serve HTTP requests in Golang.

What is OpenTelemetry?

OpenTelemetryopen in new window defines APIs and protocols for collecting telemetry data such as metrics, traces, and logs, and provides a variety of libraries, agents, and integrations for popular programming languages and technologies.

OpenTelemetry is an open and vendor-neutral solution that provides a unified approach to observability, making it easier for organizations to manage the complexity of their cloud-native infrastructure. It enables organizations to collect telemetry from their applications and send it to a variety of of distributed tracing toolsopen in new window.

net/http instrumentation

To install otelhttp instrumentation:

go get go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp

Instrumenting http.Server

You can instrument HTTP server by wrapping all your handlers:

handler := http.Handler(http.DefaultServeMux) // or use your router
handler = otelhttp.NewHandler(handler, "")

httpServer := &http.Server{
    Addr:         ":8888",
	ReadTimeout:  5 * time.Second,
	WriteTimeout: 10 * time.Second,
	IdleTimeout:  60 * time.Second,
	Handler:      handler,
}

err := httpServer.ListenAndServe()

Filtering requests

You can exclude some requests from being traced using otelhttp.WithFilteropen in new window:

handler = otelhttp.NewHandler(handler, "", otelhttp.WithFilter(otelReqFilter))

func otelReqFilter(req *http.Request) bool {
	return req.URL.Path != "/ping"
}

Span name

You can customize span name formatting using otelhttp.WithSpanNameFormatteropen in new window:

handler = otelhttp.NewHandler(handler, "", otelhttp.WithSpanNameFormatter(httpSpanName))

func spanName(operation string, req *http.Request) string {
	return operation
}

Route attribute

If you are instrumenting individual handlers (not all handlers at once), you can annotate handler spans with http.route attribute. This can be useful when you can't find an instrumentation for your router.

handler = otelhttp.WithRouteTag("/hello/:username", handler)

Instrumenting http.Client

otelhttp provides a HTTP transport to instrument http.Client:

client := http.Client{
	Transport: otelhttp.NewTransport(http.DefaultTransport),
}

You can also use the following shortcuts to make HTTP requets:

import "go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"

resp, err := otelhttp.Get(ctx, "https://google.com/")

What is Uptrace?

Uptrace is a DataDog competitoropen in new window that supports distributed tracing, metrics, and logs. You can use it to monitor applications and troubleshoot issues.

Uptrace Overview

Uptrace comes with an intuitive query builder, rich dashboards, alerting rules with notifications, and integrations for most languages and frameworks.

Uptrace can process billions of spans and metrics on a single server and allows you to monitor your applications at 10x lower cost.

In just a few minutes, you can try Uptrace by visiting the cloud demoopen in new window (no login required) or running it locally with Dockeropen in new window. The source code is available on GitHubopen in new window.

What's next?

Next, instrument more operations to get a more detailed picture. Try to prioritize network calls, disk operations, database queries, error and logs.

You can also create your own instrumentations using OpenTelemetry Go Tracing APIopen in new window.

Last Updated: