Notifications

Notification channels define where alerts are delivered and under what conditions. You can attach multiple channels to a single monitor and use condition expressions to route alerts selectively — for example, sending all alerts to a general channel but routing URGENT monitors to PagerDuty.

Channels

Requires the mailer option to be enabled in the Uptrace config file. Users must have a valid email address set in their profile. Self-hosted only.

Create an Incoming Webhook in your Slack workspace and paste the webhook URL into the channel form. Select the target channel or DM recipient in the Slack app configuration.

Create an Incoming Webhook integration in your Mattermost team and paste the webhook URL into the channel form.

Create a bot via @BotFather and copy the bot token. Add the bot to the target group or channel, then copy the chat ID from the group info.

Create an Incoming Webhook connector in your Teams channel and paste the webhook URL. Alerts are delivered as adaptive cards.

Create a service in PagerDuty with an "Events API v2" integration and copy the integration key. Uptrace creates and resolves incidents automatically.

Create an API integration in Opsgenie and copy the API key. Uptrace sends alerts with the monitor name as the title.

Enter your Prometheus AlertManager endpoint URL. Alerts are forwarded in the Alertmanager v2 format.

Create an Incoming Webhook in your Google Chat space and paste the webhook URL into the channel form.

Create an application in Pushover to get an API token. Provide the token and your user key in the channel form.

Generate an API key in your incident.io account settings and paste it into the channel form. Uptrace creates and resolves incidents automatically.

Uptrace sends a POST request with a JSON body to the configured URL on every alert state change. Use this to integrate with any custom system.

Configure your ServiceNow instance URL and credentials. Uptrace creates and resolves incidents through the ServiceNow REST API.

Create a channel via Alerting → Channels → New channel. Select the channel type, fill in the connection details, and optionally add a routing condition to filter which alerts it receives.

Channel conditions

When creating a channel, you can specify a condition expression to filter which alerts it receives. Conditions are written in the Expr language.

In addition to Expr's built-in functions, Uptrace provides:

Function	Returns
`monitorName() string`	The monitor name.
`alertName() string`	The alert name.
`alertType() string`	`"error"` or `"metric"`.
`attr(key string) string`	The value of an alert attribute.
`hasAttr(key string) bool`	`true` if the attribute exists on the alert.

Route only prod alerts to a Slack channel:

text

attr("deployment_environment") == "prod"

Route only prod hosts to PagerDuty:

text

attr("host_name") startsWith "prod-"

Route only URGENT monitors:

text

monitorName() contains "URGENT"

Route all metric alerts to one channel, all error alerts to another:

text

alertType() == "metric"

Notification frequency

On the first occurrence, Uptrace creates an alert and sends a notification immediately. For ongoing occurrences, Uptrace uses an adaptive interval that increases over time to reduce alert fatigue.

Metric monitors — interval starts at 15 minutes and doubles every 3 notifications. Maximum is 24 hours.

Notifications sent	Interval
1–3	15 minutes
4–6	30 minutes
7–9	1 hour
10–12	2 hours
...	doubles every 3
max	24 hours

Error monitors — interval starts at 1 hour and doubles every 2 notifications. Maximum is 1 week.

Notifications sent	Interval
1–2	1 hour
3–4	2 hours
5–6	4 hours
7–8	8 hours
...	doubles every 2
max	1 week

When an alert closes, Uptrace sends a recovery notification immediately. The total number of notifications is not limited — a metric monitor that never recovers will notify every 24 hours indefinitely.

Email notifications

Relevant for the self-hosted version only.

To receive email notifications in Uptrace Community, users must have correct email addresses configured and the mailer option must be enabled in the Uptrace config file.

Metric monitorsCreate metric monitors in Uptrace to alert on CPU, disk, network, span latency, error rates, and log volume using YAML-defined thresholds and rolling evaluation windows.

SearchingSearch spans and logs using natural syntax with word filters, AND OR pipes, quoted phrases, and scoped field operators.

Channels

Channel conditions

Notification frequency

Email notifications