OpenTelemetry Python distro for Uptrace
This guide shows you how to configure the OpenTelemetry Python SDK to export spans, logs, and metrics to Uptrace using the OTLP/HTTP protocol.
For detailed API documentation, see:
Overview
OpenTelemetry provides observability for your Python applications by collecting telemetry data (traces, metrics, and logs) and exporting it to monitoring systems like Uptrace.
Installation Options
Option 1: Using Uptrace Python Distribution (Recommended)
uptrace-python is a thin wrapper around opentelemetry-python that pre-configures the OpenTelemetry SDK for Uptrace. It provides convenience without adding new functionality.
Installation:
pip install uptrace
Option 2: Direct OTLP Exporter
Skip the wrapper and use the OTLP exporter directly if you prefer more control over the configuration. See the Direct OTLP Configuration section below.
Quick Start Guide
Follow these steps to get your first trace running in 5 minutes:
Step 1: Create an Uptrace Project
- Create an Uptrace project to obtain your DSN (Data Source Name)
- Your DSN will look like:
https://<secret>@api.uptrace.dev?grpc=4317
Step 2: Install the Package
pip install uptrace
Step 3: Basic Configuration
Create a Python file with the following configuration:
#!/usr/bin/env python3
import uptrace
from opentelemetry import trace
from opentelemetry.trace import Status, StatusCode
# Configure OpenTelemetry with sensible defaults
uptrace.configure_opentelemetry(
dsn="YOUR_UPTRACE_DSN_HERE", # Replace with your actual DSN
service_name="myservice",
service_version="1.0.0",
deployment_environment="production",
)
# Create a tracer instance
tracer = trace.get_tracer("app_or_package_name", "1.0.0")
Step 4: Create Your First Trace
# Create a root span to measure an operation
with tracer.start_as_current_span("main-operation") as main_span:
# Create a child span for an HTTP request
with tracer.start_as_current_span("GET /posts/:id", kind=trace.SpanKind.SERVER) as http_span:
# Add HTTP-specific attributes
http_span.set_attribute("http.method", "GET")
http_span.set_attribute("http.route", "/posts/:id")
http_span.set_attribute("http.url", "http://localhost:8080/posts/123")
http_span.set_attribute("http.status_code", 200)
# Handle errors within spans
try:
raise ValueError("Database connection failed")
except Exception as exc:
http_span.set_status(Status(StatusCode.ERROR, str(exc)))
http_span.record_exception(exc)
# Create a database operation span
with tracer.start_as_current_span("database-query", kind=trace.SpanKind.CLIENT) as db_span:
db_span.set_attributes({
"db.system": "postgresql",
"db.statement": "SELECT * FROM posts WHERE id = $1 LIMIT 100",
"db.name": "blog_db"
})
# Print the trace URL for debugging
print("View trace:", uptrace.trace_url(main_span))
# Send buffered spans and clean up resources
uptrace.shutdown()
Step 5: Run Your Application
UPTRACE_DSN="your_dsn_here" python your_script.py
Expected output:
View trace: https://app.uptrace.dev/traces/<trace_id>
Step 6: View Your Trace
Click the generated link to view your trace in the Uptrace dashboard:
Configuration Options
Environment Variables
You can configure Uptrace using environment variables instead of hardcoding values:
export UPTRACE_DSN="https://<secret>@api.uptrace.dev?grpc=4317"
export OTEL_SERVICE_NAME="myservice"
export OTEL_SERVICE_VERSION="1.0.0"
export OTEL_RESOURCE_ATTRIBUTES="deployment.environment=production"
Then use in your code:
import os
import uptrace
uptrace.configure_opentelemetry(
# DSN will be read from UPTRACE_DSN environment variable
service_name=os.getenv("OTEL_SERVICE_NAME", "myservice"),
service_version=os.getenv("OTEL_SERVICE_VERSION", "1.0.0"),
deployment_environment=os.getenv("DEPLOYMENT_ENVIRONMENT", "production"),
)
Resource Attributes
Resource attributes provide metadata about your service:
uptrace.configure_opentelemetry(
dsn=dsn,
service_name="user-service",
service_version="2.1.0",
deployment_environment="production",
resource_attributes={
"service.namespace": "backend",
"service.instance.id": "instance-001",
"cloud.provider": "aws",
"cloud.region": "us-west-2"
}
)
Sampling Configuration
For production environments with high traffic, configure sampling to reduce overhead:
import os
# Set sampling rate through environment variable
os.environ["OTEL_TRACES_SAMPLER"] = "traceidratio"
os.environ["OTEL_TRACES_SAMPLER_ARG"] = "0.1" # Sample 10% of traces
uptrace.configure_opentelemetry(
dsn=dsn,
service_name="high-traffic-service",
)
Already Using OTLP Exporter?
If you already use the OTLP exporter, you can continue to use it with Uptrace by pointing it to the Uptrace OTLP endpoint.
Recommended Settings
For performance and reliability, we recommend:
- Use
BatchSpanProcessor
andBatchLogProcessor
for batching spans and logs, reducing the number of export requests. - Enable
gzip
compression to reduce bandwidth usage. - Prefer
delta
metrics temporality (Uptrace converts cumulative metrics automatically). - Use Protobuf encoding instead of JSON (Protobuf is more efficient and widely supported).
- Use HTTP transport for simplicity and fewer configuration issues (unless you're already familiar with gRPC).
- Optionally, use the AWS X-Ray ID generator to produce trace IDs compatible with AWS X-Ray.
Connecting to Uptrace
Choose an OTLP endpoint from the table below and pass your DSN via the uptrace-dsn
header for authentication:
Transport | Endpoint | Port |
---|---|---|
gRPC | https://api.uptrace.dev:4317 | 4317 |
HTTPS | https://api.uptrace.dev | 443 |
Note: Most OpenTelemetry SDKs support both transports. Use HTTP unless unless you're already familiar with gRPC.
Common Environment Variables
You can use environment variables to configure resource attributes and propagators::
Variable | Description |
---|---|
OTEL_RESOURCE_ATTRIBUTES | Comma-separated resource attributes, e.g., service.name=myservice,service.version=1.0.0 . |
OTEL_SERVICE_NAME=myservice | Sets the service.name attribute (overrides OTEL_RESOURCE_ATTRIBUTES ). |
OTEL_PROPAGATORS | Comma-separated list of context propagators (default: tracecontext,baggage ). |
Most language SDKs allow configuring the OTLP exporter entirely via environment variables:
# Endpoint (choose HTTP or gRPC)
export OTEL_EXPORTER_OTLP_ENDPOINT="https://api.uptrace.dev" # HTTP
#export OTEL_EXPORTER_OTLP_ENDPOINT="https://api.uptrace.dev:4317" # gRPC
# Pass DSN for authentication
export OTEL_EXPORTER_OTLP_HEADERS="uptrace-dsn=<FIXME>"
# Performance optimizations
export OTEL_EXPORTER_OTLP_COMPRESSION=gzip
export OTEL_EXPORTER_OTLP_METRICS_DEFAULT_HISTOGRAM_AGGREGATION=BASE2_EXPONENTIAL_BUCKET_HISTOGRAM
export OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE=DELTA
Configure BatchSpanProcessor
to balance throughput and payload size:
export OTEL_BSP_EXPORT_TIMEOUT=10000 # Max export timeout (ms)
export OTEL_BSP_MAX_EXPORT_BATCH_SIZE=10000 # Avoid >32MB payloads
export OTEL_BSP_MAX_QUEUE_SIZE=30000 # Adjust for available memory
export OTEL_BSP_MAX_CONCURRENT_EXPORTS=2 # Parallel exports
Exporting Traces
#!/usr/bin/env python3
import os
import grpc
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.sdk.resources import Resource
from opentelemetry.semconv.resource import ResourceAttributes
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
# Get DSN from environment variable
dsn = os.getenv("UPTRACE_DSN")
if not dsn:
raise ValueError("UPTRACE_DSN environment variable is required")
print(f"Using DSN: {dsn}")
# Configure resource attributes
resource = Resource.create({
ResourceAttributes.SERVICE_NAME: "myservice",
ResourceAttributes.SERVICE_VERSION: "1.0.0",
})
# Create tracer provider
tracer_provider = TracerProvider(resource=resource)
trace.set_tracer_provider(tracer_provider)
# Configure OTLP exporter
exporter = OTLPSpanExporter(
endpoint="https://api.uptrace.dev:4317",
headers={"uptrace-dsn": dsn},
timeout=30,
compression=grpc.Compression.Gzip,
)
# Set up batch span processor
span_processor = BatchSpanProcessor(
exporter,
max_queue_size=1000,
max_export_batch_size=1000,
)
tracer_provider.add_span_processor(span_processor)
# Create and use tracer
tracer = trace.get_tracer("app_or_package_name", "1.0.0")
with tracer.start_as_current_span("main") as span:
trace_id = span.get_span_context().trace_id
print(f"Trace ID: {trace_id:032x}")
# Ensure all spans are exported before shutdown
trace.get_tracer_provider().shutdown()
Exporting Metrics
#!/usr/bin/env python3
import os
import time
import grpc
from opentelemetry import metrics
from opentelemetry.sdk.metrics import MeterProvider
from opentelemetry.sdk.metrics.export import PeriodicExportingMetricReader
from opentelemetry.sdk.resources import Resource
from opentelemetry.semconv.resource import ResourceAttributes
from opentelemetry.exporter.otlp.proto.grpc.metric_exporter import OTLPMetricExporter
# Get DSN from environment variable
dsn = os.getenv("UPTRACE_DSN")
if not dsn:
raise ValueError("UPTRACE_DSN environment variable is required")
print(f"Using DSN: {dsn}")
# Configure OTLP metric exporter
exporter = OTLPMetricExporter(
endpoint="https://api.uptrace.dev:4317",
headers={"uptrace-dsn": dsn},
timeout=30,
compression=grpc.Compression.Gzip,
)
# Set up periodic metric reader
reader = PeriodicExportingMetricReader(exporter, export_interval_millis=60000)
# Configure resource and meter provider
resource = Resource.create({
ResourceAttributes.SERVICE_NAME: "myservice",
ResourceAttributes.SERVICE_VERSION: "1.0.0",
})
provider = MeterProvider(metric_readers=[reader], resource=resource)
metrics.set_meter_provider(provider)
# Create and use meter
meter = metrics.get_meter("app_or_package_name", "1.0.0")
counter = meter.create_counter("requests_total", description="Total number of requests")
# Generate sample metrics
while True:
counter.add(1, attributes={"status": "success"})
time.sleep(1)
Exporting Logs
#!/usr/bin/env python3
import os
import logging
import grpc
from opentelemetry._logs import set_logger_provider
from opentelemetry.exporter.otlp.proto.grpc._log_exporter import OTLPLogExporter
from opentelemetry.sdk._logs import LoggerProvider, LoggingHandler
from opentelemetry.sdk._logs.export import BatchLogRecordProcessor
from opentelemetry.sdk.resources import Resource
from opentelemetry.semconv.resource import ResourceAttributes
# Get DSN from environment variable
dsn = os.getenv("UPTRACE_DSN")
if not dsn:
raise ValueError("UPTRACE_DSN environment variable is required")
print(f"Using DSN: {dsn}")
# Configure resource attributes
resource = Resource.create({
ResourceAttributes.SERVICE_NAME: "myservice",
ResourceAttributes.SERVICE_VERSION: "1.0.0",
})
# Set up logger provider
logger_provider = LoggerProvider(resource=resource)
set_logger_provider(logger_provider)
# Configure OTLP log exporter
exporter = OTLPLogExporter(
endpoint="https://api.uptrace.dev:4317",
headers={"uptrace-dsn": dsn},
timeout=30,
compression=grpc.Compression.Gzip,
)
# Add log record processor
logger_provider.add_log_record_processor(BatchLogRecordProcessor(exporter))
# Set up logging handler
handler = LoggingHandler(level=logging.INFO, logger_provider=logger_provider)
logging.getLogger().addHandler(handler)
# Create and use logger
logger = logging.getLogger("myapp.area1")
logger.error("Houston, we have a problem.")
# Ensure all logs are exported before shutdown
logger_provider.shutdown()
Auto-Instrumentation
OpenTelemetry Python provides automatic instrumentation for popular libraries and frameworks. Enable it by installing the auto-instrumentation package:
pip install opentelemetry-instrumentation
Supported Libraries
Auto-instrumentation is available for:
- HTTP clients: requests, urllib3, httpx
- Web frameworks: Flask, Django, FastAPI, Pyramid
- Databases: psycopg2, SQLAlchemy, pymongo
- Message queues: celery, redis
Enabling Auto-Instrumentation
Set the following environment variables:
export OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED=true
export OTEL_TRACES_EXPORTER=otlp
export OTEL_METRICS_EXPORTER=otlp
export OTEL_LOGS_EXPORTER=otlp
Then run your application:
UPTRACE_DSN="your_dsn_here" opentelemetry-instrument python your_app.py
Or configure it programmatically:
from opentelemetry.instrumentation.auto_instrumentation import sitecustomize
# Enable all available instrumentations
sitecustomize.instrument()
Application Servers
The BatchSpanProcessor
spawns a background thread to export spans to OpenTelemetry backends. This doesn't work well with application servers like Gunicorn and uWSGI that use a pre-forking model. During the fork, the child process inherits locks from the parent process, causing deadlocks.
To resolve this issue, configure OpenTelemetry using post-fork hooks provided by your application server.
Gunicorn
Use the post_fork hook:
# gunicorn.conf.py
import uptrace
def post_fork(server, worker):
uptrace.configure_opentelemetry(
dsn="your_dsn_here",
service_name="myservice",
service_version="1.0.0",
)
See the flask-gunicorn example for a complete implementation.
Gunicorn with uvicorn
For Gunicorn + uvicorn with async frameworks like FastAPI:
# gunicorn.conf.py
import uptrace
def post_fork(server, worker):
uptrace.configure_opentelemetry(
dsn="your_dsn_here",
service_name="myservice",
service_version="1.0.0",
)
workers = 4
worker_class = "uvicorn.workers.UvicornWorker"
uWSGI
Use the postfork decorator:
from uwsgidecorators import postfork
import uptrace
@postfork
def init_tracing():
uptrace.configure_opentelemetry(
dsn="your_dsn_here",
service_name="myservice",
service_version="1.0.0",
)
See the flask-uwsgi example for a complete implementation.
Troubleshooting
SSL Errors
If you encounter SSL errors like:
ssl_transport_security.cc:1468] Handshake failed with fatal error SSL_ERROR_SSL:
error:1000007d:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED
Try using different root certificates as a workaround:
export GRPC_DEFAULT_SSL_ROOTS_FILE_PATH=/etc/ssl/certs/ca-certificates.crt
Common Issues
- Missing DSN: Ensure your
UPTRACE_DSN
environment variable is set correctly - Network connectivity: Verify that your application can reach
api.uptrace.dev:4317
- Firewall restrictions: Check if your firewall allows outbound connections on port 4317
- Resource limits: Ensure your application has sufficient memory and CPU resources
Debug Mode
Enable debug logging to troubleshoot issues:
import logging
# Enable debug logging
logging.basicConfig(level=logging.DEBUG)
logging.getLogger("opentelemetry").setLevel(logging.DEBUG)
uptrace.configure_opentelemetry(
dsn=dsn,
service_name="debug-service",
)
Health Checks
Implement health checks to monitor telemetry:
from opentelemetry import trace
from opentelemetry.sdk.trace import Span
import time
def check_telemetry_health():
"""Check if telemetry is working correctly"""
tracer = trace.get_tracer("health-check")
try:
with tracer.start_as_current_span("health-check") as span:
span.set_attribute("health.status", "ok")
span.set_attribute("check.timestamp", time.time())
# Check if span is recording
if not span.is_recording():
return False, "Span not recording"
return True, "Telemetry is healthy"
except Exception as e:
return False, f"Telemetry check failed: {str(e)}"
# Use in your application
is_healthy, message = check_telemetry_health()
if not is_healthy:
print(f"Telemetry health check failed: {message}")
What's Next?
Next, instrument more operations to get a more detailed picture. Try to prioritize network calls, disk operations, database queries, errors, and logs.
You can also create your own instrumentations using OpenTelemetry Python Tracing API.