OpenTelemetry Python distro for Uptrace

This guide shows you how to configure the OpenTelemetry Python SDK to export spans, logs, and metrics to Uptrace using the OTLP/HTTP protocol.

For detailed API documentation, see:

Overview

OpenTelemetry provides observability for your Python applications by collecting telemetry data (traces, metrics, and logs) and exporting it to monitoring systems like Uptrace.

Installation Options

Option 1: Using Uptrace Python Distribution (Recommended)

uptrace-python is a thin wrapper around opentelemetry-python that pre-configures the OpenTelemetry SDK for Uptrace. It provides convenience without adding new functionality.

Installation:

bash

pip install uptrace

Option 2: Direct OTLP Exporter

Skip the wrapper and use the OTLP exporter directly if you prefer more control over the configuration. See the Direct OTLP Configuration section below.

Quick Start Guide

Follow these steps to get your first trace running in 5 minutes:

Step 1: Create an Uptrace Project

Create an Uptrace project to obtain your DSN (Data Source Name)
Your DSN will look like: https://<secret>@api.uptrace.dev?grpc=4317

Step 2: Install the Package

bash

pip install uptrace

Step 3: Basic Configuration

Create a Python file with the following configuration:

python

#!/usr/bin/env python3

import uptrace
from opentelemetry import trace
from opentelemetry.trace import Status, StatusCode

# Configure OpenTelemetry with sensible defaults
uptrace.configure_opentelemetry(
    dsn="YOUR_UPTRACE_DSN_HERE",  # Replace with your actual DSN
    service_name="myservice",
    service_version="1.0.0",
    deployment_environment="production",
)

# Create a tracer instance
tracer = trace.get_tracer("app_or_package_name", "1.0.0")

Step 4: Create Your First Trace

python

# Create a root span to measure an operation
with tracer.start_as_current_span("main-operation") as main_span:
    # Create a child span for an HTTP request
    with tracer.start_as_current_span("GET /posts/:id", kind=trace.SpanKind.SERVER) as http_span:
        # Add HTTP-specific attributes
        http_span.set_attribute("http.method", "GET")
        http_span.set_attribute("http.route", "/posts/:id")
        http_span.set_attribute("http.url", "http://localhost:8080/posts/123")
        http_span.set_attribute("http.status_code", 200)

        # Handle errors within spans
        try:
            raise ValueError("Database connection failed")
        except Exception as exc:
            http_span.set_status(Status(StatusCode.ERROR, str(exc)))
            http_span.record_exception(exc)

    # Create a database operation span
    with tracer.start_as_current_span("database-query", kind=trace.SpanKind.CLIENT) as db_span:
        db_span.set_attributes({
            "db.system": "postgresql",
            "db.statement": "SELECT * FROM posts WHERE id = $1 LIMIT 100",
            "db.name": "blog_db"
        })

    # Print the trace URL for debugging
    print("View trace:", uptrace.trace_url(main_span))

# Send buffered spans and clean up resources
uptrace.shutdown()

Step 5: Run Your Application

bash

UPTRACE_DSN="your_dsn_here" python your_script.py

Expected output:

text

View trace: https://app.uptrace.dev/traces/<trace_id>

Step 6: View Your Trace

Click the generated link to view your trace in the Uptrace dashboard:

Configuration Options

Environment Variables

You can configure Uptrace using environment variables instead of hardcoding values:

bash

export UPTRACE_DSN="https://<secret>@api.uptrace.dev?grpc=4317"
export OTEL_SERVICE_NAME="myservice"
export OTEL_SERVICE_VERSION="1.0.0"
export OTEL_RESOURCE_ATTRIBUTES="deployment.environment=production"

Then use in your code:

python

import os
import uptrace

uptrace.configure_opentelemetry(
    # DSN will be read from UPTRACE_DSN environment variable
    service_name=os.getenv("OTEL_SERVICE_NAME", "myservice"),
    service_version=os.getenv("OTEL_SERVICE_VERSION", "1.0.0"),
    deployment_environment=os.getenv("DEPLOYMENT_ENVIRONMENT", "production"),
)

Resource Attributes

Resource attributes provide metadata about your service:

python

uptrace.configure_opentelemetry(
    dsn=dsn,
    service_name="user-service",
    service_version="2.1.0",
    deployment_environment="production",
    resource_attributes={
        "service.namespace": "backend",
        "service.instance.id": "instance-001",
        "cloud.provider": "aws",
        "cloud.region": "us-west-2"
    }
)

Sampling Configuration

For production environments with high traffic, configure sampling to reduce overhead:

python

import os

# Set sampling rate through environment variable
os.environ["OTEL_TRACES_SAMPLER"] = "traceidratio"
os.environ["OTEL_TRACES_SAMPLER_ARG"] = "0.1"  # Sample 10% of traces

uptrace.configure_opentelemetry(
    dsn=dsn,
    service_name="high-traffic-service",
)

Already Using OTLP Exporter?

If you already use the OTLP exporter, you can continue to use it with Uptrace by pointing it to the Uptrace OTLP endpoint.

Recommended Settings

For performance and reliability, we recommend:

Use BatchSpanProcessor and BatchLogProcessor for batching spans and logs, reducing the number of export requests.
Enable gzip compression to reduce bandwidth usage.
Prefer delta metrics temporality (Uptrace converts cumulative metrics automatically).
Use Protobuf encoding instead of JSON (Protobuf is more efficient and widely supported).
Use HTTP transport for simplicity and fewer configuration issues (unless you're already familiar with gRPC).
Optionally, use the AWS X-Ray ID generator to produce trace IDs compatible with AWS X-Ray.

Connecting to Uptrace

Choose an OTLP endpoint from the table below and pass your DSN via the uptrace-dsn header for authentication:

Transport	Endpoint	Port
gRPC	https://api.uptrace.dev:4317	4317
HTTPS	https://api.uptrace.dev	443

Note: Most OpenTelemetry SDKs support both transports. Use HTTP unless unless you're already familiar with gRPC.

Common Environment Variables

You can use environment variables to configure resource attributes and propagators::

Variable	Description
`OTEL_RESOURCE_ATTRIBUTES`	Comma-separated resource attributes, e.g., `service.name=myservice,service.version=1.0.0`.
`OTEL_SERVICE_NAME=myservice`	Sets the `service.name` attribute (overrides `OTEL_RESOURCE_ATTRIBUTES`).
`OTEL_PROPAGATORS`	Comma-separated list of context propagators (default: `tracecontext,baggage`).

Most language SDKs allow configuring the OTLP exporter entirely via environment variables:

shell

# Endpoint (choose HTTP or gRPC)
export OTEL_EXPORTER_OTLP_ENDPOINT="https://api.uptrace.dev"         # HTTP
#export OTEL_EXPORTER_OTLP_ENDPOINT="https://api.uptrace.dev:4317"   # gRPC

# Pass DSN for authentication
export OTEL_EXPORTER_OTLP_HEADERS="uptrace-dsn=<FIXME>"

# Performance optimizations
export OTEL_EXPORTER_OTLP_COMPRESSION=gzip
export OTEL_EXPORTER_OTLP_METRICS_DEFAULT_HISTOGRAM_AGGREGATION=BASE2_EXPONENTIAL_BUCKET_HISTOGRAM
export OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE=DELTA

Configure BatchSpanProcessor to balance throughput and payload size:

shell

export OTEL_BSP_EXPORT_TIMEOUT=10000         # Max export timeout (ms)
export OTEL_BSP_MAX_EXPORT_BATCH_SIZE=10000  # Avoid >32MB payloads
export OTEL_BSP_MAX_QUEUE_SIZE=30000         # Adjust for available memory
export OTEL_BSP_MAX_CONCURRENT_EXPORTS=2     # Parallel exports

Exporting Traces

python

#!/usr/bin/env python3

import os
import grpc
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.sdk.resources import Resource
from opentelemetry.semconv.resource import ResourceAttributes
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter

# Get DSN from environment variable
dsn = os.getenv("UPTRACE_DSN")
if not dsn:
    raise ValueError("UPTRACE_DSN environment variable is required")

print(f"Using DSN: {dsn}")

# Configure resource attributes
resource = Resource.create({
    ResourceAttributes.SERVICE_NAME: "myservice",
    ResourceAttributes.SERVICE_VERSION: "1.0.0",
})

# Create tracer provider
tracer_provider = TracerProvider(resource=resource)
trace.set_tracer_provider(tracer_provider)

# Configure OTLP exporter
exporter = OTLPSpanExporter(
    endpoint="https://api.uptrace.dev:4317",
    headers={"uptrace-dsn": dsn},
    timeout=30,
    compression=grpc.Compression.Gzip,
)

# Set up batch span processor
span_processor = BatchSpanProcessor(
    exporter,
    max_queue_size=1000,
    max_export_batch_size=1000,
)
tracer_provider.add_span_processor(span_processor)

# Create and use tracer
tracer = trace.get_tracer("app_or_package_name", "1.0.0")

with tracer.start_as_current_span("main") as span:
    trace_id = span.get_span_context().trace_id
    print(f"Trace ID: {trace_id:032x}")

# Ensure all spans are exported before shutdown
trace.get_tracer_provider().shutdown()

Exporting Metrics

python

#!/usr/bin/env python3

import os
import time
import grpc
from opentelemetry import metrics
from opentelemetry.sdk.metrics import MeterProvider
from opentelemetry.sdk.metrics.export import PeriodicExportingMetricReader
from opentelemetry.sdk.resources import Resource
from opentelemetry.semconv.resource import ResourceAttributes
from opentelemetry.exporter.otlp.proto.grpc.metric_exporter import OTLPMetricExporter

# Get DSN from environment variable
dsn = os.getenv("UPTRACE_DSN")
if not dsn:
    raise ValueError("UPTRACE_DSN environment variable is required")

print(f"Using DSN: {dsn}")

# Configure OTLP metric exporter
exporter = OTLPMetricExporter(
    endpoint="https://api.uptrace.dev:4317",
    headers={"uptrace-dsn": dsn},
    timeout=30,
    compression=grpc.Compression.Gzip,
)

# Set up periodic metric reader
reader = PeriodicExportingMetricReader(exporter, export_interval_millis=60000)

# Configure resource and meter provider
resource = Resource.create({
    ResourceAttributes.SERVICE_NAME: "myservice",
    ResourceAttributes.SERVICE_VERSION: "1.0.0",
})

provider = MeterProvider(metric_readers=[reader], resource=resource)
metrics.set_meter_provider(provider)

# Create and use meter
meter = metrics.get_meter("app_or_package_name", "1.0.0")
counter = meter.create_counter("requests_total", description="Total number of requests")

# Generate sample metrics
while True:
    counter.add(1, attributes={"status": "success"})
    time.sleep(1)

Exporting Logs

python

#!/usr/bin/env python3

import os
import logging
import grpc
from opentelemetry._logs import set_logger_provider
from opentelemetry.exporter.otlp.proto.grpc._log_exporter import OTLPLogExporter
from opentelemetry.sdk._logs import LoggerProvider, LoggingHandler
from opentelemetry.sdk._logs.export import BatchLogRecordProcessor
from opentelemetry.sdk.resources import Resource
from opentelemetry.semconv.resource import ResourceAttributes

# Get DSN from environment variable
dsn = os.getenv("UPTRACE_DSN")
if not dsn:
    raise ValueError("UPTRACE_DSN environment variable is required")

print(f"Using DSN: {dsn}")

# Configure resource attributes
resource = Resource.create({
    ResourceAttributes.SERVICE_NAME: "myservice",
    ResourceAttributes.SERVICE_VERSION: "1.0.0",
})

# Set up logger provider
logger_provider = LoggerProvider(resource=resource)
set_logger_provider(logger_provider)

# Configure OTLP log exporter
exporter = OTLPLogExporter(
    endpoint="https://api.uptrace.dev:4317",
    headers={"uptrace-dsn": dsn},
    timeout=30,
    compression=grpc.Compression.Gzip,
)

# Add log record processor
logger_provider.add_log_record_processor(BatchLogRecordProcessor(exporter))

# Set up logging handler
handler = LoggingHandler(level=logging.INFO, logger_provider=logger_provider)
logging.getLogger().addHandler(handler)

# Create and use logger
logger = logging.getLogger("myapp.area1")
logger.error("Houston, we have a problem.")

# Ensure all logs are exported before shutdown
logger_provider.shutdown()

Auto-Instrumentation

OpenTelemetry Python provides automatic instrumentation for popular libraries and frameworks. Enable it by installing the auto-instrumentation package:

bash

pip install opentelemetry-instrumentation

Supported Libraries

Auto-instrumentation is available for:

HTTP clients: requests, urllib3, httpx
Web frameworks: Flask, Django, FastAPI, Pyramid
Databases: psycopg2, SQLAlchemy, pymongo
Message queues: celery, redis

Enabling Auto-Instrumentation

Set the following environment variables:

bash

export OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED=true
export OTEL_TRACES_EXPORTER=otlp
export OTEL_METRICS_EXPORTER=otlp
export OTEL_LOGS_EXPORTER=otlp

Then run your application:

bash

UPTRACE_DSN="your_dsn_here" opentelemetry-instrument python your_app.py

Or configure it programmatically:

python

from opentelemetry.instrumentation.auto_instrumentation import sitecustomize

# Enable all available instrumentations
sitecustomize.instrument()

The BatchSpanProcessor spawns a background thread to export spans to OpenTelemetry backends. This doesn't work well with application servers like Gunicorn and uWSGI that use a pre-forking model. During the fork, the child process inherits locks from the parent process, causing deadlocks.

To resolve this issue, configure OpenTelemetry using post-fork hooks provided by your application server.

Gunicorn

Use the post_fork hook:

python

# gunicorn.conf.py
import uptrace

def post_fork(server, worker):
    uptrace.configure_opentelemetry(
        dsn="your_dsn_here",
        service_name="myservice",
        service_version="1.0.0",
    )

See the flask-gunicorn example for a complete implementation.

Gunicorn with uvicorn

For Gunicorn + uvicorn with async frameworks like FastAPI:

python

# gunicorn.conf.py
import uptrace

def post_fork(server, worker):
    uptrace.configure_opentelemetry(
        dsn="your_dsn_here",
        service_name="myservice",
        service_version="1.0.0",
    )

workers = 4
worker_class = "uvicorn.workers.UvicornWorker"

uWSGI

Use the postfork decorator:

python

from uwsgidecorators import postfork
import uptrace

@postfork
def init_tracing():
    uptrace.configure_opentelemetry(
        dsn="your_dsn_here",
        service_name="myservice",
        service_version="1.0.0",
    )

See the flask-uwsgi example for a complete implementation.

Troubleshooting

SSL Errors

If you encounter SSL errors like:

text

ssl_transport_security.cc:1468] Handshake failed with fatal error SSL_ERROR_SSL:
error:1000007d:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED

Try using different root certificates as a workaround:

bash

export GRPC_DEFAULT_SSL_ROOTS_FILE_PATH=/etc/ssl/certs/ca-certificates.crt

Common Issues

Missing DSN: Ensure your UPTRACE_DSN environment variable is set correctly
Network connectivity: Verify that your application can reach api.uptrace.dev:4317
Firewall restrictions: Check if your firewall allows outbound connections on port 4317
Resource limits: Ensure your application has sufficient memory and CPU resources

Debug Mode

Enable debug logging to troubleshoot issues:

python

import logging

# Enable debug logging
logging.basicConfig(level=logging.DEBUG)
logging.getLogger("opentelemetry").setLevel(logging.DEBUG)

uptrace.configure_opentelemetry(
    dsn=dsn,
    service_name="debug-service",
)

Health Checks

Implement health checks to monitor telemetry:

python

from opentelemetry import trace
from opentelemetry.sdk.trace import Span
import time

def check_telemetry_health():
    """Check if telemetry is working correctly"""
    tracer = trace.get_tracer("health-check")

    try:
        with tracer.start_as_current_span("health-check") as span:
            span.set_attribute("health.status", "ok")
            span.set_attribute("check.timestamp", time.time())

            # Check if span is recording
            if not span.is_recording():
                return False, "Span not recording"

            return True, "Telemetry is healthy"
    except Exception as e:
        return False, f"Telemetry check failed: {str(e)}"

# Use in your application
is_healthy, message = check_telemetry_health()
if not is_healthy:
    print(f"Telemetry health check failed: {message}")

What's Next?

Next, instrument more operations to get a more detailed picture. Try to prioritize network calls, disk operations, database queries, errors, and logs.

You can also create your own instrumentations using OpenTelemetry Python Tracing API.

Framework-Specific Guides

Sampling

Tracing API