# Deploying Uptrace on Kubernetes

> Deploy Uptrace on Kubernetes via the Helm chart, provision dependencies, and manage namespaces plus upgrades with kubectl.

This comprehensive guide walks you through deploying Uptrace, an open-source APM and observability platform, on Kubernetes using Helm charts. Uptrace supports [distributed tracing](/opentelemetry/distributed-tracing), metrics, and log management to help you monitor your applications effectively.

## Prerequisites

### Kubernetes Cluster

You'll need a running Kubernetes cluster. For local development and testing, you can create one using:

- **K3s** - Lightweight Kubernetes distribution
- **Kind** - Kubernetes in Docker
- **Minikube** - Local Kubernetes cluster

### Required Tools

- **kubectl** - Kubernetes command-line tool
- **Helm** - Kubernetes package manager

## Understanding Helm

Helm is a package manager for Kubernetes, similar to `apt` or `yum` for Linux systems, but specifically designed for deploying and managing applications on Kubernetes clusters.

A **Helm chart** is a packaged Kubernetes application containing all necessary resources (YAML manifests) and configurations needed to deploy an application on Kubernetes.

The Uptrace Helm chart includes the Uptrace application and all required dependencies:

- Redis (for caching)
- PostgreSQL (for metadata storage)
- ClickHouse (for observability data)
- OpenTelemetry Collector (for metrics collection)

## Initial Setup

### Add Uptrace Helm Repository

First, add the Uptrace Helm repository to your local Helm installation:

```shell
helm repo add uptrace https://charts.uptrace.dev --force-update
```

### Create Monitoring Namespace

Create a dedicated namespace for Uptrace and its dependencies. This provides logical separation and makes resource management easier:

```shell
kubectl create namespace monitoring
```

**Note**: You can delete all resources created in this guide by removing the namespace:

```shell
kubectl delete namespace monitoring
```

## Configuration Management

### Understanding values.yaml Files

Helm uses `values.yaml` files to customize chart deployments without modifying the original templates. These files act as a central configuration point for defining variables that populate Helm template placeholders.

### View Default Configuration

To see all available configuration options for the Uptrace chart:

```shell
helm show values uptrace/uptrace --devel
```

### Get Sample Configuration Files

For complete configuration examples, clone the Uptrace Helm charts repository:

```shell
git clone https://github.com/uptrace/helm-charts.git
cd helm-charts
cat uptrace-values.yml
```

This repository contains sample `*-values.yaml` files for all components.

## Dependency Setup

### Redis Configuration

Uptrace uses Redis for in-memory caching. Since caching data is ephemeral, no persistent storage is required. A Redis instance with 32MB of free RAM is sufficient.

#### Using the Built-in Redis

The Uptrace Helm chart includes a built-in Redis server. Enable it in your `uptrace-values.yaml`:

```yaml
redis:
  enabled: true
```

The chart will automatically configure Uptrace to connect to this Redis instance. You can customize it further:

```yaml
redis:
  enabled: true
  password: '' # optional password
  persistence:
    enabled: true
    size: 2Gi
```

#### Connecting to Redis

To connect to your Redis instance for testing:

```shell
kubectl port-forward service/uptrace-redis 6379:6379 -n monitoring
redis-cli
```

#### Using Existing Redis

If you have an existing Redis database, keep the built-in Redis disabled and configure Uptrace to use your instance:

```yaml
redis:
  enabled: false

uptrace:
  config:
    redis_cache:
      addrs:
        alpha: 'your-redis-host:6379'
```

### PostgreSQL Configuration

Uptrace uses PostgreSQL to store metadata including users, projects, and monitors. The metadata volume is typically small — a 1GB persistent volume should be sufficient.

#### Installing PostgreSQL

Install PostgreSQL using the CloudNativePG operator, which provides enterprise-grade PostgreSQL management:

```shell
# Add the CloudNativePG repository
helm repo add cnpg https://cloudnative-pg.github.io/charts

# Install the PostgreSQL operator
helm upgrade --install cnpg \
  --namespace cnpg-system \
  --create-namespace \
  cnpg/cloudnative-pg
```

Verify the installation:

```shell
kubectl get all -n cnpg-system
```

#### Using Existing PostgreSQL

If you have an existing PostgreSQL database, disable the bundled one and configure your connection details:

```yaml
postgresql:
  enabled: false

uptrace:
  config:
    pg:
      addr: 'your-postgresql-host:5432'
      user: uptrace
      password: your-password
      database: uptrace
```

### ClickHouse Configuration

ClickHouse stores all observability data including spans, logs, events, and metrics. Start with a pod having 4 CPUs, 1GB RAM, and 10GB disk space, then scale vertically as needed.

#### Installing ClickHouse

Install ClickHouse using the Altinity operator:

```shell
kubectl apply -f https://raw.githubusercontent.com/Altinity/clickhouse-operator/master/deploy/operator/clickhouse-operator-install-bundle.yaml
```

Verify the operator is running:

```shell
kubectl get pods -n kube-system | grep clickhouse-operator
```

#### Using Existing ClickHouse

For existing ClickHouse installations, disable the bundled one and configure your cluster details:

```yaml
clickhouse:
  enabled: false

uptrace:
  config:
    ch_cluster:
      cluster: 'your-cluster-name'
      replicated: false
      distributed: false

      shards:
        - replicas:
            - addr: 'your-clickhouse-host:9000'
              database: uptrace
              user: uptrace
              password: your-password
              dial_timeout: 3s
              write_timeout: 5s
              max_retries: 3
              max_execution_time: 15s
              query_settings:
                session_timezone: UTC
                async_insert: 1
                query_cache_nondeterministic_function_handling: 'save'
                allow_suspicious_types_in_group_by: 1
                allow_suspicious_types_in_order_by: 1
```

**Troubleshooting**: If ClickHouse encounters issues in Kubernetes, refer to [this troubleshooting guide](https://altinity.com/blog/fixing-the-dreaded-clickhouse-crash-loop-on-kubernetes).

### OpenTelemetry Collector Setup

The chart uses the OpenTelemetry Operator to deploy collectors that gather pod metrics and other observability data.

#### Install cert-manager

OpenTelemetry Operator requires cert-manager for certificate management:

```shell
helm repo add jetstack https://charts.jetstack.io --force-update
helm install cert-manager jetstack/cert-manager \
  --namespace cert-manager \
  --create-namespace \
  --version v1.17.2 \
  --set crds.enabled=true
```

#### Install OpenTelemetry Operator

```shell
helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts --force-update
helm install otel-operator open-telemetry/opentelemetry-operator \
  --set "manager.collectorImage.repository=otel/opentelemetry-collector-k8s" \
  --set "manager.collectorImage.tag=0.123.0" \
  --set admissionWebhooks.certManager.enabled=false \
  --set admissionWebhooks.autoGenerateCert.enabled=true \
  --namespace opentelemetry \
  --create-namespace
```

#### Disable OpenTelemetry Collector (Optional)

If you prefer to manage OpenTelemetry Collector separately, disable it in your `uptrace-values.yaml`:

```yaml
otelcol:
  enabled: false

otelcolDaemonset:
  enabled: false
```

## Installing Uptrace

With all dependencies configured, install Uptrace using your custom values file:

```shell
helm install uptrace uptrace/uptrace \
  -f uptrace-values.yaml \
  -n monitoring \
  --devel
```

### Verify Installation

Check that all resources are running:

```shell
kubectl get all -n monitoring
```

View Uptrace application logs:

```shell
kubectl logs uptrace-0 -n monitoring
```

### Database Connections

Connect to ClickHouse for troubleshooting:

```shell
kubectl port-forward service/chi-uptrace1-uptrace1-0-0 9000:9000 -n monitoring
clickhouse-client
```

## Accessing Uptrace

### Local Access Setup

Uptrace will be available at [http://uptrace.local](http://uptrace.local) with these default credentials:

- **Username**: `admin@uptrace.local`
- **Password**: `admin`

Add the domain to your `/etc/hosts` file:

```text
127.0.0.1 uptrace.local
```

### Platform-Specific Configuration

#### Minikube Setup

Enable the ingress controller:

```shell
minikube addons enable ingress
```

Wait for ingress pods to be ready:

```shell
kubectl get pods -n ingress-nginx
```

Get Minikube's IP address:

```shell
minikube ip
```

Update your `/etc/hosts` file with the Minikube IP:

```text
192.168.49.2 uptrace.local  # Replace with your minikube ip
```

#### AWS EKS Deployment

For external access on AWS EKS using the AWS Load Balancer Controller, add these annotations to your `uptrace-values.yaml`:

```yaml
service:
  type: LoadBalancer
  port: 80
  loadBalancerSourceRanges:
    - '0.0.0.0/0' # Restrict this to your IP ranges for security
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-type: 'external'
    service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: 'ip'
    service.beta.kubernetes.io/aws-load-balancer-target-group-attributes: 'preserve_client_ip.enabled=true'
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: 'http'
    service.beta.kubernetes.io/aws-load-balancer-healthcheck-protocol: 'http'
    service.beta.kubernetes.io/aws-load-balancer-healthcheck-port: '80'
    service.beta.kubernetes.io/aws-load-balancer-healthcheck-path: '/'
```

## Scaling and Maintenance

### Horizontal Scaling

Scale Uptrace by increasing replicas in your `uptrace-values.yaml`:

```yaml
uptrace:
  replicaCount: 3 # Increase based on your load requirements
```

Apply the changes:

```shell
helm upgrade uptrace uptrace/uptrace \
  -f uptrace-values.yaml \
  -n monitoring \
  --devel
```

### Upgrading Uptrace

Only upgrades to the next minor version are tested and supported, for example, upgrading from 1.1 to 1.2. Skipping minor versions (e.g., 1.1 to 1.3) is not supported — upgrade one minor version at a time.

#### Check Current Version

Before upgrading, verify the installed version and check what's available:

```shell
# Current chart and app version
helm list -n monitoring

# Latest available version
helm repo update uptrace
helm search repo uptrace/uptrace --devel
```

Check the latest release notes on [GitHub Releases](https://github.com/uptrace/uptrace/releases).

#### Back Up Databases

Always create backups of both PostgreSQL and ClickHouse before upgrading.

**PostgreSQL:**

For a logical backup that works with the default chart, run `pg_dump` from a temporary PostgreSQL client pod. The command reads connection details from the CloudNativePG application secret created by the chart:

```shell
kubectl run uptrace-pg-dump -n monitoring --rm -i --restart=Never \
  --image=postgres:17 \
  --env="PGHOST=$(kubectl get secret -n monitoring uptrace-postgresql-app -o jsonpath='{.data.host}' | base64 -d)" \
  --env="PGPORT=$(kubectl get secret -n monitoring uptrace-postgresql-app -o jsonpath='{.data.port}' | base64 -d)" \
  --env="PGUSER=$(kubectl get secret -n monitoring uptrace-postgresql-app -o jsonpath='{.data.username}' | base64 -d)" \
  --env="PGPASSWORD=$(kubectl get secret -n monitoring uptrace-postgresql-app -o jsonpath='{.data.password}' | base64 -d)" \
  --env="PGDATABASE=$(kubectl get secret -n monitoring uptrace-postgresql-app -o jsonpath='{.data.dbname}' | base64 -d)" \
  --command -- pg_dump > uptrace-pg-backup-$(date +%Y%m%d).sql
```

If you have configured CloudNativePG backups, you can also trigger an on-demand backup. The default chart creates the PostgreSQL cluster as `uptrace-postgresql`:

```shell
kubectl apply -f - <<EOF
apiVersion: postgresql.cnpg.io/v1
kind: Backup
metadata:
  name: uptrace-pg-pre-upgrade
  namespace: monitoring
spec:
  cluster:
    name: uptrace-postgresql
EOF
```

**ClickHouse:**

The default chart runs the stock ClickHouse server image and does not install `clickhouse-backup`. Back up ClickHouse with your Kubernetes storage backup process, such as CSI `VolumeSnapshot`, or install and manage a ClickHouse backup tool as part of your deployment.

If you manage `clickhouse-backup` yourself and it is available in the ClickHouse pod:

```shell
kubectl exec -n monitoring chi-uptrace1-uptrace1-0-0-0 -- \
  clickhouse-backup create uptrace-backup-$(date +%Y%m%d)
```

#### Run the Upgrade

Update the Helm repository and upgrade the release:

```shell
helm repo update uptrace
helm upgrade uptrace uptrace/uptrace \
  -f uptrace-values.yaml \
  -n monitoring \
  --devel
```

Helm automatically validates the config, runs database migrations, and restarts Uptrace.

To install a specific chart version, use the `--version` flag:

```shell
helm search repo uptrace/uptrace --devel --versions

helm upgrade uptrace uptrace/uptrace \
  -f uptrace-values.yaml \
  -n monitoring \
  --version <chart-version> \
  --devel
```

#### Verify the Upgrade

After the upgrade completes, confirm everything is running:

```shell
# Check pod status and restarts
kubectl get pods -n monitoring

# View the Uptrace application logs for migration output
kubectl logs uptrace-0 -n monitoring --since=5m

# Verify the health endpoint
kubectl exec -n monitoring uptrace-0 -- wget -qO- http://localhost:80/api/v1/health
```

#### Rolling Back

Helm keeps a history of releases. If database migrations have not run, roll back directly:

```shell
# List release history
helm history uptrace -n monitoring

# Roll back to the previous revision
helm rollback uptrace -n monitoring
```

If database migrations have already run, scale Uptrace down, remove the existing application-owned PostgreSQL objects, restore from your backup, and then roll back:

```shell
# Stop Uptrace so it can't write during the restore
kubectl scale statefulset uptrace -n monitoring --replicas=0

# Recreate PostgreSQL from the backup
kubectl run uptrace-pg-restore -n monitoring --rm -i --restart=Never \
  --image=postgres:17 \
  --env="PGHOST=$(kubectl get secret -n monitoring uptrace-postgresql-app -o jsonpath='{.data.host}' | base64 -d)" \
  --env="PGPORT=$(kubectl get secret -n monitoring uptrace-postgresql-app -o jsonpath='{.data.port}' | base64 -d)" \
  --env="PGUSER=$(kubectl get secret -n monitoring uptrace-postgresql-app -o jsonpath='{.data.username}' | base64 -d)" \
  --env="PGPASSWORD=$(kubectl get secret -n monitoring uptrace-postgresql-app -o jsonpath='{.data.password}' | base64 -d)" \
  --env="PGDATABASE=$(kubectl get secret -n monitoring uptrace-postgresql-app -o jsonpath='{.data.dbname}' | base64 -d)" \
  --command -- sh -c \
  'psql -v ON_ERROR_STOP=1 --dbname="$PGDATABASE" -c "DROP OWNED BY CURRENT_USER CASCADE" && psql -v ON_ERROR_STOP=1 --dbname="$PGDATABASE"' \
  < uptrace-pg-backup-YYYYMMDD.sql

# Roll back to the previous release after PostgreSQL is restored
helm rollback uptrace -n monitoring
```

Restore ClickHouse using the same backup method you used before the upgrade. If you manage `clickhouse-backup` yourself:

```shell
kubectl exec -n monitoring chi-uptrace1-uptrace1-0-0-0 -- \
  clickhouse-backup restore uptrace-backup-YYYYMMDD
```

### Resource Management

#### Monitor Resource Usage

Keep an eye on resource consumption:

```shell
kubectl top pods -n monitoring
kubectl top nodes
```

#### Adjust Resource Limits

Configure resource requests and limits in your `uptrace-values.yaml`:

```yaml
uptrace:
  resources:
    requests:
      memory: '512Mi'
      cpu: '500m'
    limits:
      memory: '1Gi'
      cpu: '1000m'
```

## Troubleshooting

### Common Issues

1. **Pods not starting**: Check resource availability and node capacity
2. **Database connection errors**: Verify database credentials and network policies
3. **Ingress not working**: Ensure ingress controller is properly installed and configured
4. **Performance issues**: Monitor resource usage and scale components as needed

### Useful Commands

View all resources in the monitoring namespace:

```shell
kubectl get all -n monitoring -o wide
```

Describe problematic pods:

```shell
kubectl describe pod <pod-name> -n monitoring
```

Check events for troubleshooting:

```shell
kubectl get events -n monitoring --sort-by='.lastTimestamp'
```

## Cleanup

### Uninstall Uptrace

Remove the Uptrace release:

```shell
helm uninstall uptrace -n monitoring
```

### Complete Cleanup

Delete the entire monitoring namespace and all resources:

```shell
kubectl delete namespace monitoring
```

### Remove Operators (Optional)

If you installed operators specifically for this deployment:

```shell
# Remove OpenTelemetry Operator
helm uninstall otel-operator -n opentelemetry
kubectl delete namespace opentelemetry

# Remove cert-manager
helm uninstall cert-manager -n cert-manager
kubectl delete namespace cert-manager

# Remove CloudNativePG Operator
helm uninstall cnpg -n cnpg-system
kubectl delete namespace cnpg-system
```

## Alternative Deployment Methods

Kubernetes is one of several deployment options for Uptrace:

- [Docker](/get/hosted/docker) - Quick deployment for development and small-scale production
- [DEB/RPM packages](/get/hosted/install) - Traditional server deployments
- [Ansible](/get/hosted/ansible) - Automated bare metal deployments

Choose the method that best fits your infrastructure and requirements.

## Next Steps

Once Uptrace is running successfully:

1. **Configure Applications**: Set up your applications to send telemetry data to Uptrace
2. **Create Dashboards**: Build custom dashboards for your specific monitoring needs
3. **Set Up Alerts**: Configure alerting rules for critical metrics and traces
4. **Explore Features**: Discover advanced features like distributed tracing visualization and log correlation
5. **Performance Tuning**: Optimize configuration based on your data volume and query patterns