Collecting Golang Slog logs with OpenTelemetry
OpenTelemetry Slog is a handler for the new standard Golang structured logging library. The handler intercepts log messages and injects relevant OTel data into them. This data includes trace and span IDs that help correlate logs with specific requests or operations within a distributed system.
What is OpenTelemetry?
OpenTelemetry is an open source and vendor-neutral API for OpenTelemetry tracing, logs, and metrics.
OpenTelemetry provides a set of APIs, libraries, and instrumentation tools to instrument applications and collect data about their behavior and performance.
The goal of OpenTelemetry is to provide a standardized and vendor agnostic way to instrument applications for observability, for example, many open source tracing tools already support OpenTelemetry.
Otelslog
To install otelslog instrumentation:
go get github.com/remychantenay/slog-otel
To configure slog with Otel handler:
import slogotel "github.com/remychantenay/slog-otel"
slog.SetDefault(slog.New(slogotel.OtelHandler{
Next: slog.NewJSONHandler(os.Stdout, nil),
}))
Usage
Here is how you configure slog and OpenTelemetry:
package main
import (
"context"
"fmt"
"log/slog"
"os"
slogotel "github.com/remychantenay/slog-otel"
"github.com/uptrace/uptrace-go/uptrace"
"go.opentelemetry.io/otel"
"go.opentelemetry.io/otel/trace"
)
func main() {
ctx := context.Background()
uptrace.ConfigureOpentelemetry()
defer uptrace.Shutdown(ctx)
// Configure slog with Otel handler.
slog.SetDefault(slog.New(slogotel.OtelHandler{
Next: slog.NewJSONHandler(os.Stdout, nil),
}))
tracer := otel.Tracer("slog-example")
ctx, span := tracer.Start(ctx, "operation-name")
defer span.End()
slog.ErrorContext(ctx, "Hello world!", "locale", "en_US")
fmt.Println(uptrace.TraceURL(trace.SpanFromContext(ctx)))
}
By adding trace and span IDs to log messages, slog-otel makes it easier to trace the flow of a request through a complex system:
{
"time": "2024-03-29T11:29:48.721812065+02:00",
"level": "ERROR",
"msg": "Hello world!",
"locale": "en_US",
"trace_id": "17c13241232acba87526846a3eeba26e",
"span_id": "898b41b19299e7c1"
}
What is Uptrace?
Uptrace is a DataDog alternative that supports distributed tracing, metrics, and logs. You can use it to monitor applications and troubleshoot issues.
Uptrace comes with an intuitive query builder, rich dashboards, alerting rules with notifications, and integrations for most languages and frameworks.
Uptrace can process billions of spans and metrics on a single server and allows you to monitor your applications at 10x lower cost.
In just a few minutes, you can try Uptrace by visiting the cloud demo (no login required) or running it locally with Docker. The source code is available on GitHub.
What's next?
Next, instrument more operations to get a more detailed picture. Try to prioritize network calls, disk operations, database queries, error and logs.
You can also create your own instrumentations using OpenTelemetry Go Tracing API.
