OpenTelemetry Gin Monitoring [otelgin]

OpenTelemetry Gin instrumentation allows developers to monitor and diagnose issues with their Gin applications, providing valuable insights into application behavior in production.

Quick Setup

Minimal working example:

import (
    "github.com/gin-gonic/gin"
    "go.opentelemetry.io/contrib/instrumentation/github.com/gin-gonic/gin/otelgin"
)

func main() {
    router := gin.Default()
    router.Use(otelgin.Middleware("my-service"))

    router.GET("/", func(c *gin.Context) {
        c.JSON(200, gin.H{"message": "Hello World"})
    })
    router.Run(":8080")
}

This single middleware call automatically captures all incoming HTTP requests, traces request flow, and exports telemetry data to your configured backend.

What is Gin?

Gin is a high-performance web framework for building web applications and APIs in Go. It features an HTTP router that is up to 40x faster than other Go routers, making it one of the most popular choices for Go web development.

Gin's key features include:

• Fast HTTP router using radix tree for efficient routing
• Middleware support with a chain of handlers
• JSON validation and rendering
• Error management and recovery from panics
• Grouping routes for better organization

Gin's simplicity and performance make it a popular choice for building web applications and RESTful APIs in Go.

What is OpenTelemetry?

OpenTelemetry is an open-source observability framework that aims to standardize and simplify the collection, processing, and export of telemetry data from applications and systems.

OpenTelemetry supports multiple programming languages and platforms, making it suitable for a wide range of applications and environments.

OpenTelemetry enables developers to instrument their code and collect telemetry data, which can then be exported to various OpenTelemetry backends or observability platforms for analysis and visualization. The OpenTelemetry architecture provides a modular, vendor-neutral approach to observability.

Gin instrumentation

OpenTelemetry Gin instrumentation (otelgin) provides automatic tracing and metrics collection for your Gin applications. It captures HTTP request details, timing information, and error states without requiring manual instrumentation.

To install otelgin instrumentation:

go get go.opentelemetry.io/contrib/instrumentation/github.com/gin-gonic/gin/otelgin

• Example
• Reference

Usage

You can instrument Gin router by installing OpenTelemetry middleware:

import (
    "github.com/gin-gonic/gin"
    "go.opentelemetry.io/contrib/instrumentation/github.com/gin-gonic/gin/otelgin"
)

func main() {
    router := gin.Default()
    router.Use(otelgin.Middleware("my-service"))

    router.GET("/users/:id", getUser)
    router.Run(":8080")
}

Once instrumented, otelgin automatically creates spans for each incoming HTTP request, capturing method, route, status code, and timing information.

Middleware options

The otelgin middleware supports several configuration options:

Filtering health checks

Use the WithFilter option to exclude certain endpoints from tracing:

import (
    "net/http"

    "github.com/gin-gonic/gin"
    "go.opentelemetry.io/contrib/instrumentation/github.com/gin-gonic/gin/otelgin"
)

router := gin.Default()
router.Use(otelgin.Middleware("my-service",
    otelgin.WithFilter(func(r *http.Request) bool {
        // Return true to trace, false to skip
        return r.URL.Path != "/health" && r.URL.Path != "/ready"
    }),
))

You can also use WithGinFilter for access to Gin context:

router.Use(otelgin.Middleware("my-service",
    otelgin.WithGinFilter(func(c *gin.Context) bool {
        return c.FullPath() != "/health"
    }),
))

Custom tracer and meter providers

For more control over telemetry collection, you can specify custom providers:

import (
    "github.com/gin-gonic/gin"
    "go.opentelemetry.io/contrib/instrumentation/github.com/gin-gonic/gin/otelgin"
    "go.opentelemetry.io/otel"
)

router := gin.Default()
router.Use(otelgin.Middleware("my-service",
    otelgin.WithTracerProvider(otel.GetTracerProvider()),
    otelgin.WithMeterProvider(otel.GetMeterProvider()),
))

Custom span names

Customize how span names are generated:

import (
    "github.com/gin-gonic/gin"
    "go.opentelemetry.io/contrib/instrumentation/github.com/gin-gonic/gin/otelgin"
)

router := gin.Default()
router.Use(otelgin.Middleware("my-service",
    otelgin.WithSpanNameFormatter(func(c *gin.Context) string {
        return c.Request.Method + " " + c.FullPath()
    }),
))

Adding custom attributes

Extract custom attributes from requests to enhance your metrics:

import (
    "net/http"

    "github.com/gin-gonic/gin"
    "go.opentelemetry.io/contrib/instrumentation/github.com/gin-gonic/gin/otelgin"
    "go.opentelemetry.io/otel/attribute"
)

router := gin.Default()
router.Use(otelgin.Middleware("my-service",
    otelgin.WithMetricAttributeFn(func(r *http.Request) []attribute.KeyValue {
        return []attribute.KeyValue{
            attribute.String("http.client_ip", r.RemoteAddr),
            attribute.String("http.user_agent", r.UserAgent()),
        }
    }),
))

Instrumenting template rendering

To instrument template rendering, use the otelgin.HTML helper instead of c.HTML:

import (
    "net/http"

    "github.com/gin-gonic/gin"
    "go.opentelemetry.io/contrib/instrumentation/github.com/gin-gonic/gin/otelgin"
)

func exampleHandler(c *gin.Context) {
    otelgin.HTML(c, http.StatusOK, "index.tmpl", gin.H{
        "title": "Hello World",
    })
}

This creates a child span for the template rendering operation, helping you identify slow templates.

HTTP metrics

The otelgin middleware automatically collects the following HTTP server metrics:

These metrics follow the OpenTelemetry semantic conventions for HTTP and include attributes like method, route, and status code.

What is Uptrace?

Uptrace is an OpenTelemetry APM that supports distributed tracing, metrics, and logs. You can use it to monitor applications and troubleshoot issues.

Uptrace comes with an intuitive query builder, rich dashboards, alerting rules with notifications, and integrations for most languages and frameworks.

Uptrace can process billions of spans and metrics on a single server and allows you to monitor your applications at 10x lower cost.

In just a few minutes, you can try Uptrace by visiting the cloud demo (no login required) or running it locally with Docker. The source code is available on GitHub.

Creating Custom Spans

In addition to automatic instrumentation, you can create custom spans to trace specific operations within your handlers:

import (
    "github.com/gin-gonic/gin"
    "go.opentelemetry.io/otel"
    "go.opentelemetry.io/otel/attribute"
    "go.opentelemetry.io/otel/codes"
)

var tracer = otel.Tracer("my-service")

func getUserHandler(c *gin.Context) {
    ctx := c.Request.Context()

    // Create a custom span for database operation
    ctx, span := tracer.Start(ctx, "fetch-user-from-database")
    defer span.End()

    userID := c.Param("id")
    span.SetAttributes(attribute.String("user.id", userID))

    user, err := db.FindUser(ctx, userID)
    if err != nil {
        span.RecordError(err)
        span.SetStatus(codes.Error, err.Error())
        c.JSON(500, gin.H{"error": "Internal server error"})
        return
    }

    span.SetAttributes(attribute.Bool("user.found", user != nil))
    c.JSON(200, user)
}

Error Handling

OpenTelemetry Gin instrumentation automatically captures panics and errors. You can also manually record errors:

import (
    "github.com/gin-gonic/gin"
    "go.opentelemetry.io/otel/codes"
    "go.opentelemetry.io/otel/trace"
)

func errorHandler(c *gin.Context) {
    // Get the span from the request context
    span := trace.SpanFromContext(c.Request.Context())

    result, err := riskyOperation()
    if err != nil {
        // Record the error in the span
        span.RecordError(err)
        span.SetStatus(codes.Error, "operation failed")
        c.JSON(500, gin.H{"error": "Something went wrong"})
        return
    }

    c.JSON(200, result)
}

FAQ

What is otelgin? The otelgin package is the official OpenTelemetry instrumentation for Gin framework. It provides middleware that automatically creates spans for incoming HTTP requests, capturing method, route, status code, and timing information.

How do I filter health check endpoints? Use the WithFilter or WithGinFilter options to exclude specific endpoints. Return false for paths like /health or /ready to skip tracing those requests.

What metrics does otelgin collect? otelgin automatically collects http.server.request.duration, http.server.request.body.size, and http.server.response.body.size metrics following OpenTelemetry semantic conventions.

Does otelgin work with Gin's recovery middleware? Yes, otelgin works alongside gin.Recovery() middleware. Place the otelgin middleware first to ensure panics are captured in traces before recovery handles them.

How do I add custom attributes to all requests? Use the WithMetricAttributeFn or WithGinMetricAttributeFn options to extract custom attributes from requests and add them to all spans created by the middleware.

Can I use otelgin with Gin route groups? Yes, you can add the middleware to specific route groups instead of the entire router. This allows different tracing configurations for different parts of your API.

What's the performance impact of otelgin? The performance impact is minimal. The middleware adds a few microseconds per request for span creation and context propagation, which is negligible for most applications.

What's next?

With OpenTelemetry Gin instrumentation in place, you can monitor request latency, track error rates, and trace requests across your distributed systems.

Next steps to enhance your observability:

• Add database instrumentation with GORM or database/sql
• Correlate logs with traces using Logrus, Zap, or slog
• Create custom spans using the OpenTelemetry Go Tracing API
• Explore other Go frameworks like Echo or Beego
• Set up the OpenTelemetry Collector for production deployments