Django OpenTelemetry Instrumentation Guide

OpenTelemetry enables comprehensive monitoring of Django applications by automatically collecting telemetry data including request and response times, database query performance, and custom metrics. By integrating OpenTelemetry, you can capture distributed traces and metrics, then export this data to various observability backends for analysis and visualization.

What is Django?

Django is an open-source, high-level web framework written in Python. It follows the Model-View-Template (MVT) architectural pattern and provides a robust set of tools and features that simplify and accelerate web application development.

Django is designed to promote clean, maintainable code and follows the "Don't Repeat Yourself" (DRY) principle, emphasizing code reusability and reducing redundancy.

What is OpenTelemetry?

OpenTelemetry is an open-source observability framework that provides a standardized way to collect, process, and export telemetry data from applications and infrastructure. It combines metrics, logs, and distributed traces into a unified toolkit that helps developers understand how their systems are performing.

The framework solves vendor lock-in by providing consistent instrumentation across different technology stacks. Teams can instrument their applications once with OpenTelemetry and send that data to any compatible observability platform, making it easier to monitor complex, distributed systems without being tied to specific monitoring vendors.

Installation

To instrument a Django application with OpenTelemetry, install the required packages:

pip install opentelemetry-api opentelemetry-sdk opentelemetry-instrumentation-django

Additional Database-Specific Instrumentation

Depending on your database backend, you may also need to install additional instrumentation packages:

# For PostgreSQL
pip install opentelemetry-instrumentation-psycopg2

# For MySQL
pip install opentelemetry-instrumentation-dbapi

# For SQLite
pip install opentelemetry-instrumentation-sqlite3

Exporter Installation

To export telemetry data to observability backends, install an appropriate exporter:

# For OTLP (recommended)
pip install opentelemetry-exporter-otlp

# For console output (development/testing)
pip install opentelemetry-exporter-otlp-proto-http

OpenTelemetry SDK Configuration

Initialize OpenTelemetry in your Django application by adding the following configuration to your application's startup code. This should be placed in your manage.py, wsgi.py, or a dedicated configuration module:

from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import (
    BatchSpanProcessor,
    ConsoleSpanExporter,
)
from opentelemetry.sdk.resources import SERVICE_NAME, Resource

# Create resource with service information
resource = Resource(attributes={
    SERVICE_NAME: "your-django-app"
})

# Create and configure the tracer provider
provider = TracerProvider(resource=resource)
processor = BatchSpanProcessor(ConsoleSpanExporter())
provider.add_span_processor(processor)

# Set the global default tracer provider
trace.set_tracer_provider(provider)

# Create a tracer from the global tracer provider
tracer = trace.get_tracer("my.tracer.name")

Basic Usage

Manual Instrumentation

Django instrumentation relies on the DJANGO_SETTINGS_MODULE environment variable to locate your settings file. Since Django defines this variable in the manage.py file, you should instrument your Django application from that file:

# manage.py
import os
import sys
from opentelemetry.instrumentation.django import DjangoInstrumentor

def main():
    # Ensure DJANGO_SETTINGS_MODULE is set before instrumenting
    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "mysite.settings")

    # Initialize Django instrumentation
    DjangoInstrumentor().instrument()

    # Your existing Django management code here
    try:
        from django.core.management import execute_from_command_line
    except ImportError as exc:
        raise ImportError(
            "Couldn't import Django. Are you sure it's installed and "
            "available on your PYTHONPATH environment variable? Did you "
            "forget to activate a virtual environment?"
        ) from exc
    execute_from_command_line(sys.argv)

if __name__ == '__main__':
    main()

Auto-Instrumentation

For a simpler setup without code modifications, you can use OpenTelemetry's auto-instrumentation:

# First, install the auto-instrumentation packages
pip install opentelemetry-distro

# Bootstrap to install all relevant instrumentation packages
opentelemetry-bootstrap -a install

# Run your Django application with auto-instrumentation
opentelemetry-instrument python manage.py runserver --noreload

Note: Use the --noreload flag to prevent Django from running the initialization twice.

Creating Custom Spans

OpenTelemetry automatically traces incoming HTTP requests to your Django application. You can also create custom spans to trace specific parts of your code:

from opentelemetry import trace

tracer = trace.get_tracer(__name__)

def my_view(request):
    # Create a custom span for specific business logic
    with tracer.start_as_current_span("custom-business-logic"):
        # Your application code here
        result = perform_complex_operation()

        # Add attributes to the span for better observability
        span = trace.get_current_span()
        span.set_attribute("operation.result", str(result))
        span.set_attribute("user.id", request.user.id if request.user.is_authenticated else "anonymous")

    return HttpResponse("Hello, World!")

Deployment Configuration

uWSGI

When using Django with uWSGI, initialize OpenTelemetry using the post-fork hook to ensure proper instrumentation in worker processes. Add this to your application startup file:

from uwsgidecorators import postfork
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor, ConsoleSpanExporter
from opentelemetry.instrumentation.django import DjangoInstrumentor

@postfork
def init_tracing():
    # Initialize OpenTelemetry
    provider = TracerProvider()
    processor = BatchSpanProcessor(ConsoleSpanExporter())
    provider.add_span_processor(processor)
    trace.set_tracer_provider(provider)

    # Instrument Django
    DjangoInstrumentor().instrument()

Gunicorn

For Django with Gunicorn, use the post-fork hook in your gunicorn.conf.py:

from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor, ConsoleSpanExporter
from opentelemetry.instrumentation.django import DjangoInstrumentor

def post_fork(server, worker):
    server.log.info("Worker spawned (pid: %s)", worker.pid)

    # Initialize OpenTelemetry
    provider = TracerProvider()
    processor = BatchSpanProcessor(ConsoleSpanExporter())
    provider.add_span_processor(processor)
    trace.set_tracer_provider(provider)

    # Instrument Django
    DjangoInstrumentor().instrument()

Database Instrumentation

PostgreSQL

Configure Django to use PostgreSQL by updating your settings.py:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': 'mydatabase',
        'USER': 'mydatabaseuser',
        'PASSWORD': 'mypassword',
        'HOST': '127.0.0.1',
        'PORT': '5432',
    }
}

The PostgreSQL backend uses the psycopg2 library internally. Install and configure the instrumentation:

pip install opentelemetry-instrumentation-psycopg2

Then instrument the library in your initialization code:

from opentelemetry.instrumentation.psycopg2 import Psycopg2Instrumentor

Psycopg2Instrumentor().instrument()

For Gunicorn or uWSGI deployments, add this instrumentation to your post-fork hooks.

MySQL

Configure Django to use MySQL by updating your settings.py:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'mydatabase',
        'USER': 'mydatabaseuser',
        'PASSWORD': 'mypassword',
        'HOST': '127.0.0.1',
        'PORT': '3306',
    }
}

The MySQL backend uses the mysqlclient library, which implements the Python Database API. Install the DB API instrumentation:

pip install opentelemetry-instrumentation-dbapi

Then instrument the mysqlclient library:

import MySQLdb
from opentelemetry.instrumentation.dbapi import trace_integration

trace_integration(MySQLdb, "connect", "mysql")

For Gunicorn or uWSGI deployments, add this instrumentation to your post-fork hooks.

SQLite

Configure Django to use SQLite by updating your settings.py:

DATABASES = {
    "default": {
        "ENGINE": "django.db.backends.sqlite3",
        "NAME": BASE_DIR / "db.sqlite3",
    }
}

The SQLite backend uses Python's built-in sqlite3 library. Install the SQLite instrumentation:

pip install opentelemetry-instrumentation-sqlite3

Then instrument the sqlite3 library:

from opentelemetry.instrumentation.sqlite3 import SQLite3Instrumentor

SQLite3Instrumentor().instrument()

For Gunicorn or uWSGI deployments, add this instrumentation to your post-fork hooks.

Advanced Configuration

Environment Variables

OpenTelemetry supports configuration through environment variables. Common ones include:

# Service identification
export OTEL_SERVICE_NAME="my-django-app"
export OTEL_SERVICE_VERSION="1.0.0"

# Exporter configuration
export OTEL_EXPORTER_OTLP_ENDPOINT="http://localhost:4317"
export OTEL_EXPORTER_OTLP_PROTOCOL="grpc"

# Resource attributes
export OTEL_RESOURCE_ATTRIBUTES="service.name=my-django-app,service.version=1.0.0"

# Disable specific instrumentations if needed
export OTEL_PYTHON_DJANGO_INSTRUMENT=false

SQL Commentor

You can enable SQL commentor to enrich database queries with contextual information:

from opentelemetry.instrumentation.django import DjangoInstrumentor

DjangoInstrumentor().instrument(is_sql_commentor_enabled=True)

This will append contextual tags to your SQL queries, transforming SELECT * FROM users into SELECT * FROM users /*controller='users',action='index'*/.

Sampling Configuration

To control the volume of telemetry data, configure sampling in your tracer provider:

from opentelemetry.sdk.trace.sampling import TraceIdRatioBased

# Sample 10% of traces
sampler = TraceIdRatioBased(0.1)
provider = TracerProvider(sampler=sampler)

Custom Resource Attributes

Add custom resource attributes to provide additional context:

from opentelemetry.sdk.resources import Resource

resource = Resource.create({
    "service.name": "my-django-app",
    "service.version": "1.0.0",
    "deployment.environment": "production",
    "service.instance.id": "instance-123"
})

provider = TracerProvider(resource=resource)

Error Handling and Exception Tracking

OpenTelemetry automatically captures unhandled exceptions, but you can also manually record exceptions:

from opentelemetry import trace
from opentelemetry.trace import Status, StatusCode

def my_view(request):
    span = trace.get_current_span()
    try:
        # Your business logic here
        result = risky_operation()
        return JsonResponse({"result": result})
    except Exception as e:
        # Record the exception and set error status
        span.record_exception(e)
        span.set_status(Status(StatusCode.ERROR, str(e)))
        return JsonResponse({"error": "Something went wrong"}, status=500)

Performance Considerations

OpenTelemetry is designed to be efficient and lightweight, but it does introduce some overhead due to instrumentation and telemetry collection. The performance impact depends on several factors:

• Volume of telemetry data: More spans and attributes increase overhead
• Exporter configuration: Batch exporters are more efficient than real-time exporters
• Sampling rate: Lower sampling rates reduce overhead
• System resources: Available CPU and memory affect performance

To optimize performance:

1. Configure appropriate sampling rates to balance observability needs with performance
2. Use batch exporters instead of synchronous exporters when possible
3. Monitor resource usage and adjust configuration as needed
4. Profile your application to identify any performance bottlenecks introduced by instrumentation
5. Use environment variables to disable unnecessary instrumentations

Production Deployment Best Practices

Security Considerations

• Use secure endpoints (HTTPS) for exporting telemetry data
• Avoid logging sensitive information in span attributes
• Configure proper authentication for telemetry backends
• Review and sanitize custom attributes before adding them to spans

Resource Management

• Set appropriate memory limits for batch processors
• Configure timeouts for exporters to prevent blocking
• Monitor the telemetry pipeline's resource consumption
• Use compression when exporting large volumes of data

Monitoring the Monitor

• Set up health checks for your telemetry pipeline
• Monitor exporter success/failure rates
• Track the volume of telemetry data being generated
• Alert on telemetry pipeline failures

What is Uptrace?

Uptrace is a high-performance, open-source Application Performance Monitoring (APM) platform built for modern distributed systems. Whether you're debugging microservices, optimizing database queries, or ensuring SLA compliance, Uptrace provides unified observability without vendor lock-in.

Key Features

Uptrace offers all-in-one observability with a single interface for traces, metrics, and logs, eliminating the need for context switching between multiple tools. Built on OpenTelemetry standards for maximum compatibility, it provides:

• Blazing Fast Performance: Processes 10,000+ spans per second on a single core with advanced compression reducing 1KB spans to ~40 bytes
• Cost-Effective at Scale: AGPL open-source license with 95% storage savings vs. traditional solutions and no per-seat pricing or data ingestion limits
• Developer-First Experience: SQL-like query language for traces, PromQL-compatible metrics queries, and modern Vue.js UI with intuitive workflows

Architecture

Uptrace leverages ClickHouse for real-time analysis with sub-second queries on billions of spans and exceptional 10:1 compression ratios, while PostgreSQL handles metadata with ACID compliance and rich data types support.

Deployment Options

1. Cloud (Fastest): Uptrace Cloud requires no installation, maintenance, or scaling. Sign up for a free account and get 1TB of storage and 50,000 timeseries
2. Self-Hosted: Deploy using Docker Compose with full control over your data

uptrace-python OpenTelemetry Wrapper

uptrace-python is a thin wrapper over OpenTelemetry Python that configures OpenTelemetry SDK to export data to Uptrace. It does not add any new functionality and is provided only for your convenience.

Installation

pip install uptrace

Configuration for Django

Add the following configuration to your Django application's startup code (typically in manage.py or wsgi.py):

import uptrace
from opentelemetry import trace

# Configure OpenTelemetry with Uptrace
uptrace.configure_opentelemetry(
    # Set DSN or use UPTRACE_DSN environment variable
    dsn="<your-uptrace-dsn>",
    service_name="my-django-app",
    service_version="1.0.0",
    deployment_environment="production",
)

# Get a tracer
tracer = trace.get_tracer("my_django_app", "1.0.0")

Configuration Options

The following configuration options are supported:

Environment Variables

You can also use environment variables to configure the client:

# Data source name
export UPTRACE_DSN="https://<token>@uptrace.dev/<project_id>"

# Resource attributes
export OTEL_RESOURCE_ATTRIBUTES="service.name=my-django-app,service.version=1.0.0"

# Service name (takes precedence over OTEL_RESOURCE_ATTRIBUTES)
export OTEL_SERVICE_NAME="my-django-app"

# Propagators
export OTEL_PROPAGATORS="tracecontext,baggage"

Django Integration Example

Here's a complete example for integrating uptrace-python with Django:

# manage.py
import os
import sys
import uptrace
from opentelemetry.instrumentation.django import DjangoInstrumentor

def main():
    # Set Django settings module
    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "myproject.settings")

    # Configure Uptrace OpenTelemetry
    uptrace.configure_opentelemetry(
        # DSN from environment variable or directly
        # dsn="https://<token>@uptrace.dev/<project_id>",
        service_name="my-django-app",
        service_version="1.0.0",
        deployment_environment=os.environ.get("ENVIRONMENT", "development"),
    )

    # Instrument Django
    DjangoInstrumentor().instrument()

    try:
        from django.core.management import execute_from_command_line
    except ImportError as exc:
        raise ImportError(
            "Couldn't import Django. Are you sure it's installed and "
            "available on your PYTHONPATH environment variable? Did you "
            "forget to activate a virtual environment?"
        ) from exc
    execute_from_command_line(sys.argv)

if __name__ == '__main__':
    main()

Gunicorn Integration

With Gunicorn, use the post_fork hook in your gunicorn.conf.py:

import uptrace
from opentelemetry.instrumentation.django import DjangoInstrumentor

def post_fork(server, worker):
    uptrace.configure_opentelemetry(
        service_name="my-django-app",
        service_version="1.0.0",
        deployment_environment="production",
    )
    DjangoInstrumentor().instrument()

uWSGI Integration

With uWSGI, use the postfork decorator:

from uwsgidecorators import postfork
import uptrace
from opentelemetry.instrumentation.django import DjangoInstrumentor

@postfork
def init_tracing():
    uptrace.configure_opentelemetry(
        service_name="my-django-app",
        service_version="1.0.0",
        deployment_environment="production",
    )
    DjangoInstrumentor().instrument()

Viewing Traces

uptrace-python provides a convenient way to get trace URLs:

from opentelemetry import trace
import uptrace

def my_view(request):
    span = trace.get_current_span()

    # Your business logic here

    # Get the trace URL for debugging
    trace_url = uptrace.trace_url(span)
    print(f"View trace at: {trace_url}")

    return JsonResponse({"status": "success"})

Performance Recommendations

To maximize performance and efficiency when using Uptrace, consider the following recommendations:

1. Use BatchSpanProcessor: Essential for exporting multiple spans in a single request
2. Enable gzip compression: Essential to compress data before sending and reduce traffic costs
3. Prefer delta metrics temporality: Recommended because such metrics are smaller and Uptrace converts cumulative metrics to delta anyway
4. Use Protobuf over JSON: Recommended as a generally more efficient encoding
5. Prefer HTTP over gRPC: Recommended as a more mature transport with better compatibility

Troubleshooting SSL Errors

If you are getting SSL errors, try using different root certificates as a workaround:

export GRPC_DEFAULT_SSL_ROOTS_FILE_PATH=/etc/ssl/certs/ca-certificates.crt

Troubleshooting

Common Issues

1. DJANGO_SETTINGS_MODULE not found: Ensure the environment variable is set before calling DjangoInstrumentor().instrument()
2. Missing database instrumentation: Install the appropriate database-specific instrumentation packages
3. Worker process issues: Use post-fork hooks for uWSGI and Gunicorn deployments
4. High performance overhead: Adjust sampling rates and exporter configuration
5. Spans not appearing: Check that your exporter is configured correctly and the backend is reachable
6. Auto-reload conflicts: Use --noreload flag when running Django with runserver

Debug Mode

To enable debug logging for OpenTelemetry:

import logging

# Enable debug logging for OpenTelemetry
logging.getLogger("opentelemetry").setLevel(logging.DEBUG)
logging.basicConfig(level=logging.DEBUG)

Testing Instrumentation

You can test your instrumentation locally using the console exporter:

from opentelemetry.sdk.trace.export import ConsoleSpanExporter

# Use console exporter for testing
processor = BatchSpanProcessor(ConsoleSpanExporter())

Best Practices

1. Initialize instrumentation early: Set up OpenTelemetry before your Django application starts handling requests
2. Use meaningful span names: Create descriptive span names that help identify operations
3. Add relevant attributes: Include context-specific attributes to make traces more useful
4. Monitor performance impact: Regularly assess the overhead introduced by instrumentation
5. Configure appropriate sampling: Balance observability needs with performance requirements
6. Use semantic conventions: Follow OpenTelemetry semantic conventions for consistency
7. Implement proper error handling: Capture and record exceptions with appropriate context
8. Regular maintenance: Keep OpenTelemetry libraries updated and review configurations periodically

What's Next?

By integrating OpenTelemetry with Django, you gain valuable insights into your application's performance, behavior, and dependencies. You can monitor and troubleshoot issues, optimize performance, and ensure the reliability of your Django applications.

The telemetry data collected can help you:

• Identify performance bottlenecks
• Track request flows across services
• Monitor database query performance
• Understand user behavior patterns
• Detect and diagnose errors
• Optimize resource utilization
• Improve overall application reliability