Monitor AWS Lambda Golang with OpenTelemetry

AWS Lambda is a serverless, event-driven compute service that lets you run code without provisioning or managing servers.

This article explains how you can monitor and optimize AWS Lambda performance using OpenTelemetry observability framework.

What is OpenTelemetry?

OpenTelemetryopen in new window is an open source and vendor-neutral API for distributed tracingopen in new window (including logs and errors) and metricsopen in new window.

Otel specifies how to collect and export telemetry data in a vendor agnostic way. With OpenTelemetry, you can instrumentopen in new window your application once and then add or change vendors without changing the instrumentation, for example, many open source tracing toolsopen in new window already support OpenTelemetry.

OpenTelemetry is available for most programming languages and provides interoperability across different languages and environments.

AWS Lambda containers

Lambda runs the code in isolated containers dynamically scaling the number of containers as needed. When there are no new requests, Lambda freezes idle containers.

When the container is frozen, all processes in the container will be frozen as well. If the process uses a timer to flush the buffered data, the timer won’t fire until the container thaws. The interval between the frozen and the thaw state is unpredictable, ranging from seconds to hours.

To mitigate that problem, the instrumentation has to flush data out before the Lambda function returns.

Installation

To install OpenTelemetry AWS Lambda instrumentation:

go get -u go.opentelemetry.io/contrib/instrumentation/github.com/aws/aws-lambda-go/otellambda

Usage

You can instrument your AWS Lambda function with OpenTelemetry like this:

import "go.opentelemetry.io/contrib/instrumentation/github.com/aws/aws-lambda-go/otellambda"

func main() {
	lambda.Start(otellambda.InstrumentHandler(HandleRequest))
}

To use otellambda with uptrace-goopen in new window, you need to configure otellambda.WithFlusher with the tracer provider used by Uptrace.

import (
	"go.opentelemetry.io/contrib/propagators/aws/xray"
	"go.opentelemetry.io/contrib/instrumentation/github.com/aws/aws-lambda-go/otellambda"
	"go.opentelemetry.io/contrib/instrumentation/github.com/aws/aws-lambda-go/otellambda/xrayconfig"
	"github.com/uptrace/uptrace-go/uptrace"
)

func main() {
	ctx := context.Background()

	uptrace.ConfigureOpentelemetry(
		// copy your project DSN here or use UPTRACE_DSN env var
		//uptrace.WithDSN("https://<token>@uptrace.dev/<project_id>"),

		uptrace.WithServiceName("myservice"),
		uptrace.WithServiceVersion("v1.0.0"),
	)
	defer uptrace.Shutdown(ctx)

	tp := uptrace.TracerProvider()
	lambda.Start(otellambda.InstrumentHandler(
		lambdaHandler(ctx),
		// Flush buffered spans to Uptrace.
		otellambda.WithFlusher(tp),
	))
}

See aws-lambdaopen in new window example for details.

OpenTelemetry Lambda

Alternatively, you can add opentelemetry-lambdaopen in new window as a Lambda layer, but it does not offer any advantages at the moment. See exampleopen in new window for details.

What is Uptrace?

Uptrace is an open source APMopen in new window for OpenTelemetry with an intuitive query builder, rich dashboards, automatic alerts, and integrations for most languages and frameworks.

You can get startedopen in new window with Uptrace by downloading a DEB/RPM package or a pre-compiled Go binary.

What's next?

Next, instrument more operations, for example, database queries, errors, and logs. You can also learn about OpenTelemetry Go Tracing API to create your own instrumentations.

Popular instrumentations:

Last Updated: