Deploying Uptrace on Kubernetes

This comprehensive guide walks you through deploying Uptrace, an open-source APM and observability platform, on Kubernetes using Helm charts. Uptrace supports distributed tracing, metrics, and log management to help you monitor your applications effectively.

Prerequisites

Kubernetes Cluster

You'll need a running Kubernetes cluster. For local development and testing, you can create one using:

K3s - Lightweight Kubernetes distribution
Kind - Kubernetes in Docker
Minikube - Local Kubernetes cluster

Required Tools

kubectl - Kubernetes command-line tool
Helm - Kubernetes package manager

Understanding Helm

Helm is a package manager for Kubernetes, similar to apt or yum for Linux systems, but specifically designed for deploying and managing applications on Kubernetes clusters.

A Helm chart is a packaged Kubernetes application containing all necessary resources (YAML manifests) and configurations needed to deploy an application on Kubernetes.

The Uptrace Helm chart includes the Uptrace application and all required dependencies:

Redis (for caching)
PostgreSQL (for metadata storage)
ClickHouse (for observability data)
OpenTelemetry Collector (for metrics collection)

Initial Setup

Add Uptrace Helm Repository

First, add the Uptrace Helm repository to your local Helm installation:

shell

helm repo add uptrace https://charts.uptrace.dev --force-update

Create Monitoring Namespace

Create a dedicated namespace for Uptrace and its dependencies. This provides logical separation and makes resource management easier:

shell

kubectl create namespace monitoring

Note: You can delete all resources created in this guide by removing the namespace:

shell

kubectl delete namespace monitoring

Configuration Management

Understanding values.yaml Files

Helm uses values.yaml files to customize chart deployments without modifying the original templates. These files act as a central configuration point for defining variables that populate Helm template placeholders.

View Default Configuration

To see all available configuration options for the Uptrace chart:

shell

helm show values uptrace/uptrace --devel

Get Sample Configuration Files

For complete configuration examples, clone the Uptrace Helm charts repository:

shell

git clone https://github.com/uptrace/helm-charts.git
cd helm-charts
cat uptrace-values.yml

This repository contains sample *-values.yaml files for all components.

Dependency Setup

Redis Configuration

Uptrace uses Redis for in-memory caching. Since caching data is ephemeral, no persistent storage is required. A Redis instance with 32MB of free RAM is sufficient.

Using the Built-in Redis

The Uptrace Helm chart includes a built-in Redis server. Enable it in your uptrace-values.yaml:

yaml

redis:
  enabled: true

The chart will automatically configure Uptrace to connect to this Redis instance. You can customize it further:

yaml

redis:
  enabled: true
  password: '' # optional password
  persistence:
    enabled: true
    size: 2Gi

Connecting to Redis

To connect to your Redis instance for testing:

shell

kubectl port-forward service/uptrace-redis 6379:6379 -n monitoring
redis-cli

Using Existing Redis

If you have an existing Redis database, keep the built-in Redis disabled and configure Uptrace to use your instance:

yaml

redis:
  enabled: false

uptrace:
  config:
    redis_cache:
      addrs:
        alpha: 'your-redis-host:6379'

PostgreSQL Configuration

Uptrace uses PostgreSQL to store metadata including users, projects, and monitors. The metadata volume is typically small — a 1GB persistent volume should be sufficient.

Installing PostgreSQL

Install PostgreSQL using the CloudNativePG operator, which provides enterprise-grade PostgreSQL management:

shell

# Add the CloudNativePG repository
helm repo add cnpg https://cloudnative-pg.github.io/charts

# Install the PostgreSQL operator
helm upgrade --install cnpg \
  --namespace cnpg-system \
  --create-namespace \
  cnpg/cloudnative-pg

Verify the installation:

shell

kubectl get all -n cnpg-system

Using Existing PostgreSQL

If you have an existing PostgreSQL database, disable the bundled one and configure your connection details:

yaml

postgresql:
  enabled: false

uptrace:
  config:
    pg:
      addr: 'your-postgresql-host:5432'
      user: uptrace
      password: your-password
      database: uptrace

ClickHouse Configuration

ClickHouse stores all observability data including spans, logs, events, and metrics. Start with a pod having 4 CPUs, 1GB RAM, and 10GB disk space, then scale vertically as needed.

Installing ClickHouse

Install ClickHouse using the Altinity operator:

shell

kubectl apply -f https://raw.githubusercontent.com/Altinity/clickhouse-operator/master/deploy/operator/clickhouse-operator-install-bundle.yaml

Verify the operator is running:

shell

kubectl get pods -n kube-system | grep clickhouse-operator

Using Existing ClickHouse

For existing ClickHouse installations, disable the bundled one and configure your cluster details:

yaml

clickhouse:
  enabled: false

uptrace:
  config:
    ch_cluster:
      cluster: 'your-cluster-name'
      replicated: false
      distributed: false

      shards:
        - replicas:
            - addr: 'your-clickhouse-host:9000'
              database: uptrace
              user: uptrace
              password: your-password
              dial_timeout: 3s
              write_timeout: 5s
              max_retries: 3
              max_execution_time: 15s
              query_settings:
                session_timezone: UTC
                async_insert: 1
                query_cache_nondeterministic_function_handling: 'save'
                allow_suspicious_types_in_group_by: 1
                allow_suspicious_types_in_order_by: 1

Troubleshooting: If ClickHouse encounters issues in Kubernetes, refer to this troubleshooting guide.

OpenTelemetry Collector Setup

The chart uses the OpenTelemetry Operator to deploy collectors that gather pod metrics and other observability data.

Install cert-manager

OpenTelemetry Operator requires cert-manager for certificate management:

shell

helm repo add jetstack https://charts.jetstack.io --force-update
helm install cert-manager jetstack/cert-manager \
  --namespace cert-manager \
  --create-namespace \
  --version v1.17.2 \
  --set crds.enabled=true

Install OpenTelemetry Operator

shell

helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts --force-update
helm install otel-operator open-telemetry/opentelemetry-operator \
  --set "manager.collectorImage.repository=otel/opentelemetry-collector-k8s" \
  --set "manager.collectorImage.tag=0.123.0" \
  --set admissionWebhooks.certManager.enabled=false \
  --set admissionWebhooks.autoGenerateCert.enabled=true \
  --namespace opentelemetry \
  --create-namespace

Disable OpenTelemetry Collector (Optional)

If you prefer to manage OpenTelemetry Collector separately, disable it in your uptrace-values.yaml:

yaml

otelcol:
  enabled: false

otelcolDaemonset:
  enabled: false

Installing Uptrace

With all dependencies configured, install Uptrace using your custom values file:

shell

helm install uptrace uptrace/uptrace \
  -f uptrace-values.yaml \
  -n monitoring \
  --devel

Verify Installation

Check that all resources are running:

shell

kubectl get all -n monitoring

View Uptrace application logs:

shell

kubectl logs uptrace-0 -n monitoring

Database Connections

Connect to ClickHouse for troubleshooting:

shell

kubectl port-forward service/chi-uptrace1-uptrace1-0-0 9000:9000 -n monitoring
clickhouse-client

Accessing Uptrace

Local Access Setup

Uptrace will be available at http://uptrace.local with these default credentials:

Username: admin@uptrace.local
Password: admin

Add the domain to your /etc/hosts file:

text

127.0.0.1 uptrace.local

Platform-Specific Configuration

Minikube Setup

Enable the ingress controller:

shell

minikube addons enable ingress

Wait for ingress pods to be ready:

shell

kubectl get pods -n ingress-nginx

Get Minikube's IP address:

shell

minikube ip

Update your /etc/hosts file with the Minikube IP:

text

192.168.49.2 uptrace.local  # Replace with your minikube ip

AWS EKS Deployment

For external access on AWS EKS using the AWS Load Balancer Controller, add these annotations to your uptrace-values.yaml:

yaml

service:
  type: LoadBalancer
  port: 80
  loadBalancerSourceRanges:
    - '0.0.0.0/0' # Restrict this to your IP ranges for security
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-type: 'external'
    service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: 'ip'
    service.beta.kubernetes.io/aws-load-balancer-target-group-attributes: 'preserve_client_ip.enabled=true'
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: 'http'
    service.beta.kubernetes.io/aws-load-balancer-healthcheck-protocol: 'http'
    service.beta.kubernetes.io/aws-load-balancer-healthcheck-port: '80'
    service.beta.kubernetes.io/aws-load-balancer-healthcheck-path: '/'

Scaling and Maintenance

Horizontal Scaling

Scale Uptrace by increasing replicas in your uptrace-values.yaml:

yaml

uptrace:
  replicaCount: 3 # Increase based on your load requirements

Apply the changes:

shell

helm upgrade uptrace uptrace/uptrace \
  -f uptrace-values.yaml \
  -n monitoring \
  --devel

Upgrading Uptrace

Only upgrades to the next minor version are tested and supported, for example, upgrading from 1.1 to 1.2. Skipping minor versions (e.g., 1.1 to 1.3) is not supported — upgrade one minor version at a time.

Check Current Version

Before upgrading, verify the installed version and check what's available:

shell

# Current chart and app version
helm list -n monitoring

# Latest available version
helm repo update uptrace
helm search repo uptrace/uptrace --devel

Check the latest release notes on GitHub Releases.

Back Up Databases

Always create backups of both PostgreSQL and ClickHouse before upgrading.

PostgreSQL:

For a logical backup that works with the default chart, run pg_dump from a temporary PostgreSQL client pod. The command reads connection details from the CloudNativePG application secret created by the chart:

shell

kubectl run uptrace-pg-dump -n monitoring --rm -i --restart=Never \
  --image=postgres:17 \
  --env="PGHOST=$(kubectl get secret -n monitoring uptrace-postgresql-app -o jsonpath='{.data.host}' | base64 -d)" \
  --env="PGPORT=$(kubectl get secret -n monitoring uptrace-postgresql-app -o jsonpath='{.data.port}' | base64 -d)" \
  --env="PGUSER=$(kubectl get secret -n monitoring uptrace-postgresql-app -o jsonpath='{.data.username}' | base64 -d)" \
  --env="PGPASSWORD=$(kubectl get secret -n monitoring uptrace-postgresql-app -o jsonpath='{.data.password}' | base64 -d)" \
  --env="PGDATABASE=$(kubectl get secret -n monitoring uptrace-postgresql-app -o jsonpath='{.data.dbname}' | base64 -d)" \
  --command -- pg_dump > uptrace-pg-backup-$(date +%Y%m%d).sql

If you have configured CloudNativePG backups, you can also trigger an on-demand backup. The default chart creates the PostgreSQL cluster as uptrace-postgresql:

shell

kubectl apply -f - <<EOF
apiVersion: postgresql.cnpg.io/v1
kind: Backup
metadata:
  name: uptrace-pg-pre-upgrade
  namespace: monitoring
spec:
  cluster:
    name: uptrace-postgresql
EOF

ClickHouse:

The default chart runs the stock ClickHouse server image and does not install clickhouse-backup. Back up ClickHouse with your Kubernetes storage backup process, such as CSI VolumeSnapshot, or install and manage a ClickHouse backup tool as part of your deployment.

If you manage clickhouse-backup yourself and it is available in the ClickHouse pod:

shell

kubectl exec -n monitoring chi-uptrace1-uptrace1-0-0-0 -- \
  clickhouse-backup create uptrace-backup-$(date +%Y%m%d)

Run the Upgrade

Update the Helm repository and upgrade the release:

shell

helm repo update uptrace
helm upgrade uptrace uptrace/uptrace \
  -f uptrace-values.yaml \
  -n monitoring \
  --devel

Helm automatically validates the config, runs database migrations, and restarts Uptrace.

To install a specific chart version, use the --version flag:

shell

helm search repo uptrace/uptrace --devel --versions

helm upgrade uptrace uptrace/uptrace \
  -f uptrace-values.yaml \
  -n monitoring \
  --version <chart-version> \
  --devel

Verify the Upgrade

After the upgrade completes, confirm everything is running:

shell

# Check pod status and restarts
kubectl get pods -n monitoring

# View the Uptrace application logs for migration output
kubectl logs uptrace-0 -n monitoring --since=5m

# Verify the health endpoint
kubectl exec -n monitoring uptrace-0 -- wget -qO- http://localhost:80/api/v1/health

Rolling Back

Helm keeps a history of releases. If database migrations have not run, roll back directly:

shell

# List release history
helm history uptrace -n monitoring

# Roll back to the previous revision
helm rollback uptrace -n monitoring

If database migrations have already run, scale Uptrace down, remove the existing application-owned PostgreSQL objects, restore from your backup, and then roll back:

shell

# Stop Uptrace so it can't write during the restore
kubectl scale statefulset uptrace -n monitoring --replicas=0

# Recreate PostgreSQL from the backup
kubectl run uptrace-pg-restore -n monitoring --rm -i --restart=Never \
  --image=postgres:17 \
  --env="PGHOST=$(kubectl get secret -n monitoring uptrace-postgresql-app -o jsonpath='{.data.host}' | base64 -d)" \
  --env="PGPORT=$(kubectl get secret -n monitoring uptrace-postgresql-app -o jsonpath='{.data.port}' | base64 -d)" \
  --env="PGUSER=$(kubectl get secret -n monitoring uptrace-postgresql-app -o jsonpath='{.data.username}' | base64 -d)" \
  --env="PGPASSWORD=$(kubectl get secret -n monitoring uptrace-postgresql-app -o jsonpath='{.data.password}' | base64 -d)" \
  --env="PGDATABASE=$(kubectl get secret -n monitoring uptrace-postgresql-app -o jsonpath='{.data.dbname}' | base64 -d)" \
  --command -- sh -c \
  'psql -v ON_ERROR_STOP=1 --dbname="$PGDATABASE" -c "DROP OWNED BY CURRENT_USER CASCADE" && psql -v ON_ERROR_STOP=1 --dbname="$PGDATABASE"' \
  < uptrace-pg-backup-YYYYMMDD.sql

# Roll back to the previous release after PostgreSQL is restored
helm rollback uptrace -n monitoring

Restore ClickHouse using the same backup method you used before the upgrade. If you manage clickhouse-backup yourself:

shell

kubectl exec -n monitoring chi-uptrace1-uptrace1-0-0-0 -- \
  clickhouse-backup restore uptrace-backup-YYYYMMDD

Resource Management

Monitor Resource Usage

Keep an eye on resource consumption:

shell

kubectl top pods -n monitoring
kubectl top nodes

Adjust Resource Limits

Configure resource requests and limits in your uptrace-values.yaml:

yaml

uptrace:
  resources:
    requests:
      memory: '512Mi'
      cpu: '500m'
    limits:
      memory: '1Gi'
      cpu: '1000m'

Troubleshooting

Common Issues

Pods not starting: Check resource availability and node capacity
Database connection errors: Verify database credentials and network policies
Ingress not working: Ensure ingress controller is properly installed and configured
Performance issues: Monitor resource usage and scale components as needed

Useful Commands

View all resources in the monitoring namespace:

shell

kubectl get all -n monitoring -o wide

Describe problematic pods:

shell

kubectl describe pod <pod-name> -n monitoring

Check events for troubleshooting:

shell

kubectl get events -n monitoring --sort-by='.lastTimestamp'

Cleanup

Uninstall Uptrace

Remove the Uptrace release:

shell

helm uninstall uptrace -n monitoring

Complete Cleanup

Delete the entire monitoring namespace and all resources:

shell

kubectl delete namespace monitoring

Remove Operators (Optional)

If you installed operators specifically for this deployment:

shell

# Remove OpenTelemetry Operator
helm uninstall otel-operator -n opentelemetry
kubectl delete namespace opentelemetry

# Remove cert-manager
helm uninstall cert-manager -n cert-manager
kubectl delete namespace cert-manager

# Remove CloudNativePG Operator
helm uninstall cnpg -n cnpg-system
kubectl delete namespace cnpg-system

Alternative Deployment Methods

Kubernetes is one of several deployment options for Uptrace:

Docker - Quick deployment for development and small-scale production
DEB/RPM packages - Traditional server deployments
Ansible - Automated bare metal deployments

Choose the method that best fits your infrastructure and requirements.

Next Steps

Once Uptrace is running successfully:

Configure Applications: Set up your applications to send telemetry data to Uptrace
Create Dashboards: Build custom dashboards for your specific monitoring needs
Set Up Alerts: Configure alerting rules for critical metrics and traces
Explore Features: Discover advanced features like distributed tracing visualization and log correlation
Performance Tuning: Optimize configuration based on your data volume and query patterns

Ansible deploymentAutomate Uptrace deployments on bare metal with the official Ansible playbooks, inventories, and role based setup.

Docker deploymentQuick deployment of Uptrace using Docker Compose with pre-configured services including ClickHouse, PostgreSQL, and Redis.

Prerequisites

Understanding Helm

Initial Setup

Configuration Management

Dependency Setup

Installing Uptrace

Accessing Uptrace

Scaling and Maintenance

Troubleshooting

Cleanup

Alternative Deployment Methods

Next Steps