Monitor Go net/http performance with OpenTelemetry
Learn how to monitor net/http performance using OpenTelemetry observability framework.
Package net/http provides HTTP client and server implementations. It allows you to make and serve HTTP requests in Golang.
What is OpenTelemetry?
OpenTelemetry defines APIs and protocols for collecting telemetry data such as metrics, traces, and logs, and provides a variety of libraries, agents, and integrations for popular programming languages and technologies.
OpenTelemetry is an open and vendor-neutral solution that provides a unified approach to observability, making it easier for organizations to manage the complexity of their cloud-native infrastructure. It enables organizations to collect telemetry from their applications and send it to a variety of of distributed tracing tools.
net/http instrumentation
To install otelhttp instrumentation:
go get go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp
Instrumenting http.Server
You can instrument HTTP server by wrapping all your handlers:
handler := http.Handler(http.DefaultServeMux) // or use your router
handler = otelhttp.NewHandler(handler, "")
httpServer := &http.Server{
Addr: ":8888",
ReadTimeout: 5 * time.Second,
WriteTimeout: 10 * time.Second,
IdleTimeout: 60 * time.Second,
Handler: handler,
}
err := httpServer.ListenAndServe()
Filtering requests
You can exclude some requests from being traced using otelhttp.WithFilter:
handler = otelhttp.NewHandler(handler, "", otelhttp.WithFilter(otelReqFilter))
func otelReqFilter(req *http.Request) bool {
return req.URL.Path != "/ping"
}
Span name
You can customize span name formatting using otelhttp.WithSpanNameFormatter:
handler = otelhttp.NewHandler(handler, "", otelhttp.WithSpanNameFormatter(httpSpanName))
func spanName(operation string, req *http.Request) string {
return operation
}
Route attribute
If you are instrumenting individual handlers (not all handlers at once), you can annotate handler spans with http.route
attribute. This can be useful when you can't find an instrumentation for your router.
handler = otelhttp.WithRouteTag("/hello/:username", handler)
Instrumenting http.Client
otelhttp provides a HTTP transport to instrument http.Client
:
client := http.Client{
Transport: otelhttp.NewTransport(http.DefaultTransport),
}
You can also use the following shortcuts to make HTTP requets:
import "go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
resp, err := otelhttp.Get(ctx, "https://google.com/")
What is Uptrace?
Uptrace is a DataDog competitor that supports distributed tracing, metrics, and logs. You can use it to monitor applications and troubleshoot issues.
Uptrace comes with an intuitive query builder, rich dashboards, alerting rules with notifications, and integrations for most languages and frameworks.
Uptrace can process billions of spans and metrics on a single server and allows you to monitor your applications at 10x lower cost.
In just a few minutes, you can try Uptrace by visiting the cloud demo (no login required) or running it locally with Docker. The source code is available on GitHub.
What's next?
Next, instrument more operations to get a more detailed picture. Try to prioritize network calls, disk operations, database queries, error and logs.
You can also create your own instrumentations using OpenTelemetry Go Tracing API.