OpenTelemetry Node.js distro for Uptrace

This document explains how to configure OpenTelemetry JavaScript SDK for Node.js to export spans and metrics to Uptrace using OTLP/HTTP.

To learn about OpenTelemetry API, see OpenTelemetry JS Tracing APIopen in new window and OpenTelemetry JS Metrics APIopen in new window.

OpenTelemetry Node.js

To install @uptrace/node:

# npm
npm install @uptrace/node --save

# yarn
yarn add @uptrace/node --save

Configuration

You can configure Uptrace client using a DSN (Data Source Name) from the project settings page.

WARNING

You must call configureOpentelemetry as early as possible and before importing other packages, because OpenTelemetry SDK must patch libraries before you import them.

// The very first import must be Uptrace/OpenTelemetry.
const uptrace = require('@uptrace/node')
const otel = require('@opentelemetry/api')

// Start OpenTelemetry SDK and invoke instrumentations to patch the code.
uptrace.configureOpentelemetry({
  // copy your project DSN here or use UPTRACE_DSN env var
  //dsn: 'https://<token>@api.uptrace.dev/<project_id>',
  serviceName: 'myservice',
  serviceVersion: '1.0.0',
  deploymentEnvironment: 'production',
})

// Other imports. Express is monkey-patched at this point.
const express = require('express')

// Create a tracer.
const tracer = otel.trace.getTracer('app_or_package_name')

// Start the app.
const app = express()
app.listen(3000)

To avoid potential issues, it is strongly recommended to put the OpenTelemetry initialization code into a separate file called otel.js and use the --require flag to load the tracing code before the application code:

# JavaScript
node --require ./otel.js app.js

# TypeScript
ts-node --require ./otel.ts app.ts

If you are using AWS Lambda, you need to set the NODE_OPTIONS environment variable:

export NODE_OPTIONS="--require otel.js"

See express-pgopen in new window for a working example.

Automatic instrumentation

Whenever you load a module, OpenTelemetry automatically checks if there a matching instrumentation plugin and uses it to patch the original package.

You can also register instrumentations manually:

const uptrace = require('@uptrace/node')
const { HttpInstrumentation } = require('@opentelemetry/instrumentation-http')
const { PgInstrumentation } = require('@opentelemetry/instrumentation-pg')

uptrace.configureOpentelemetry({
  // Set dsn or UPTRACE_DSN env var.
  dsn: 'https://<token>@api.uptrace.dev/<project_id>',

  serviceName: 'myservice',
  serviceVersion: '1.0.0',

  instrumentations: [new PgInstrumentation(), new HttpInstrumentation()],
})

Configuration options

You can use the following options to configure Uptrace client.

Option	Description
`dsn`	A data source that is used to connect to uptrace.dev. For example, `https://<token>@uptrace.dev/<project_id>`.
`serviceName`	`service.name` resource attribute. For example, `myservice`.
`serviceVersion`	`service.version` resource attribute. For example, `1.0.0`.
`deploymentEnvironment`	`deployment.environment` resource attribute. For example, `production`.
`resourceAttributes`	Any other resource attributes.
`resource`	Resource contains attributes representing an entity that produces telemetry. Resource attributes are copied to all spans and events.

You can also use environment variables to configure the client:

Env var	Description
`UPTRACE_DSN`	A data source that is used to connect to uptrace.dev. For example, `https://<token>@uptrace.dev/<project_id>`.
`OTEL_RESOURCE_ATTRIBUTES`	Key-value pairs to be used as resource attributes. For example, `service.name=myservice,service.version=1.0.0`.
`OTEL_SERVICE_NAME=myservice`	Sets the value of the service.name resource attribute. Takes precedence over `OTEL_RESOURCE_ATTRIBUTES`.
`OTEL_PROPAGATORS`	Propagators to be used as a comma separated list. The default is `tracecontext,baggage`.

See OpenTelemetry documentationopen in new window for details.

Quickstart

Spend 5 minutes to install OpenTelemetry distro, generate your first trace, and click the link in your terminal to view the trace.

Step 0. Createopen in new window an Uptrace project to obtain a DSN (connection string), for example, https://<token>@uptrace.dev/<project_id>.
Step 1. Install @uptrace/nodeopen in new window:

# npm
npm install @uptrace/node --save

# yarn
yarn add @uptrace/node --save

Step 2. Copy the codeopen in new window to main.js replacing the <dsn>:

'use strict'

// The very first import must be Uptrace/OpenTelemetry.
const otel = require('@opentelemetry/api')
const uptrace = require('@uptrace/node')

// Start OpenTelemetry SDK and invoke instrumentations to patch the code.
uptrace.configureOpentelemetry({
  // Set dsn or UPTRACE_DSN env var.
  //dsn: 'https://<token>@api.uptrace.dev/<project_id>',
  serviceName: 'myservice',
  serviceVersion: '1.0.0',
})

// Create a tracer. Usually, tracer is a global variable.
const tracer = otel.trace.getTracer('app_or_package_name', '1.0.0')

// Create a root span (a trace) to measure some operation.
tracer.startActiveSpan('main-operation', (main) => {
  tracer.startActiveSpan('GET /posts/:id', (child1) => {
    child1.setAttribute('http.method', 'GET')
    child1.setAttribute('http.route', '/posts/:id')
    child1.setAttribute('http.url', 'http://localhost:8080/posts/123')
    child1.setAttribute('http.status_code', 200)
    child1.recordException(new Error('error1'))
    child1.end()
  })

  tracer.startActiveSpan('SELECT', (child2) => {
    child2.setAttribute('db.system', 'mysql')
    child2.setAttribute('db.statement', 'SELECT * FROM posts LIMIT 100')
    child2.end()
  })

  // End the span when the operation we are measuring is done.
  main.end()

  console.log(uptrace.traceUrl(main))
})

setTimeout(async () => {
  // Send buffered spans and free resources.
  await uptrace.shutdown()
})

Step 3. Run the exampleopen in new window to get a link for the generated trace:

node main.js
https://uptrace.dev/traces/<trace_id>

Step 4. Follow the link to view the trace:

Basic trace

Already using OTLP exporter?

If you are already using OTLP exporter, you can configure it to export data to Uptrace without changing any code.

To maximize performance and efficiency, consider the following recommendations when configuring the OTLP exporter.

Recommendation	Signals	Significance
Use `BatchSpanProcessor` to export multiple spans in a single request.	All	Essential
Enable `gzip` compression to compress the data before sending and reduce the traffic cost.	All	Essential
Prefer `delta` metrics temporality, because such metrics are smaller and Uptrace must convert `cumulative` metrics to `delta` anyway.	Metrics	Recommended
Use AWS X-Ray ID generator for OpenTelemetry.	Traces, Logs	Optional

To configure OpenTelemetry to send data to Uptrace, use the provided endpoint and pass the DSN via uptrace-dsn header:

# Only for OTLP/gRPC
export OTEL_EXPORTER_OTLP_ENDPOINT="otlp.uptrace.dev:4317"

# Only for OTLP/HTTP
export OTEL_EXPORTER_OTLP_ENDPOINT="https://otlp.uptrace.dev"

# Pass Uptrace DSN in gRPC/HTTP headers.
export OTEL_EXPORTER_OTLP_HEADERS="uptrace-dsn=https://<token>@api.uptrace.dev/<project_id>"

# Enable gzip compression.
export OTEL_EXPORTER_OTLP_COMPRESSION=gzip

# Enable exponential histograms.
export OTEL_EXPORTER_OTLP_METRICS_DEFAULT_HISTOGRAM_AGGREGATION=BASE2_EXPONENTIAL_BUCKET_HISTOGRAM

# Prefer delta temporality.
export OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE=DELTA

When configuring BatchSpanProcessor, use the following settings:

# Maximum allowed time to export data in milliseconds.
export OTEL_BSP_EXPORT_TIMEOUT=5000

# Maximum batch size.
# Using larger batch sizes can be problematic,
# because Uptrace rejects requests larger than 20MB.
export OTEL_BSP_MAX_EXPORT_BATCH_SIZE=10000

# Maximum queue size.
# Increase queue size if you have lots of RAM, for example,
# `10000 * number_of_gigabytes`.
export OTEL_BSP_MAX_QUEUE_SIZE=30000

# Max concurrent exports.
# Setting this to the number of available CPUs might be a good idea.
export OTEL_BSP_MAX_CONCURRENT_EXPORTS=2

Exporting traces

Hereopen in new window is how you can configure OTLP/gRPC exporter for Uptrace following the recommendations above:

'use strict'

const otel = require('@opentelemetry/api')
const { BatchSpanProcessor } = require('@opentelemetry/sdk-trace-base')
const { Resource } = require('@opentelemetry/resources')
const { NodeSDK } = require('@opentelemetry/sdk-node')
const { OTLPTraceExporter } = require('@opentelemetry/exporter-trace-otlp-http')
const { AWSXRayIdGenerator } = require('@opentelemetry/id-generator-aws-xray')

const dsn = process.env.UPTRACE_DSN
console.log('using dsn:', dsn)

const exporter = new OTLPTraceExporter({
  url: 'https://otlp.uptrace.dev/v1/traces',
  headers: { 'uptrace-dsn': dsn },
  compression: 'gzip',
})
const bsp = new BatchSpanProcessor(exporter, {
  maxExportBatchSize: 1000,
  maxQueueSize: 1000,
})

const sdk = new NodeSDK({
  spanProcessor: bsp,
  resource: new Resource({
    'service.name': 'myservice',
    'service.version': '1.0.0',
  }),
  idGenerator: new AWSXRayIdGenerator(),
})
sdk.start()

const tracer = otel.trace.getTracer('app_or_package_name', '1.0.0')

tracer.startActiveSpan('main', (main) => {
  main.end()
  console.log('trace id:', main.spanContext().traceId)
})

// Send buffered spans.
setTimeout(async () => {
  await sdk.shutdown()
}, 1000)

Node.js runtime metrics

To collect Node.js runtime metrics, install opentelemetry-node-metrics:

# npm
npm install opentelemetry-node-metrics --save

# yarm
yarn add opentelemetry-node-metrics --save

Then start collecting metrics like this:

const otel = require('@opentelemetry/api')
const uptrace = require('@uptrace/node')
const startNodeMetrics = require('opentelemetry-node-metrics')

uptrace.configureOpentelemetry({...})

// Must be called AFTER OpenTelemetry is configured.
const meterProvider = otel.metrics.getMeterProvider()
startNodeMetrics(meterProvider)

Troubleshooting

If OpenTelemetry is not working as expected, you can enable verbose logging to check for potential issues:

const { DiagConsoleLogger, DiagLogLevel, diag } = require('@opentelemetry/api')
diag.setLogger(new DiagConsoleLogger(), DiagLogLevel.DEBUG)

What's next?

Next, instrument more operations to get a more detailed picture. Try to prioritize network calls, disk operations, database queries, error and logs.

You can also create your own instrumentations using OpenTelemetry JS Tracing APIopen in new window.

OpenTelemetry Express.js

# OpenTelemetry Node.js distro for Uptrace

# OpenTelemetry Node.js

# Configuration

# Automatic instrumentation

# Configuration options

# Quickstart

# Already using OTLP exporter?

# Exporting traces

# Node.js runtime metrics

# Troubleshooting

# What's next?