Uptrace: Querying Spans
Introduction
Uptrace provides a powerful querying language that supports filters (where .status.code = "error), grouping (group by .group_id), and aggregates (p50(.duration)).
To write useful and performant queries, you need to pre-process raw data so it has a well-defined structure. You can achieve that by recording contextual information in span attributes and events. For logs, you can use structured logging.
Filters
Uptrace allows to filter spans and events by their attributes. Filters start with the keyword where, for example, where .name contains 'hello' or .count > 100. Uptrace automatically translates filters into SQL WHERE or HAVING, so you don't have to worry about that.
Uptrace supports the following span attribute types:
| Attribute type | Supported comparison operators |
|---|---|
string | =, like, contains, ~ (regexp), exists |
int64 and float64 | =, <, <=, >, >=, exists |
string array | contains, exists |
| Uptrace filter | Description |
|---|---|
where .status_code = "error" | Filter spans with error status code. Case-sensitive. |
where display.name like "hello%" | Filter span names that start with "hello". Case-insensitive. |
where display.name like "%hello" | Filter span names that end with "hello". Case-insensitive. |
where display.name contains "hello" | Filter span names that contain "hello". Case-insensitive. |
where display.name contains "foo|bar" | Same as .name contains "foo" OR .name contains "bar". |
where .duration > 1ms | Same as .duration > 1000. Uptrace supports μs, ms, and s units. |
where http.request_content_length > 1kb | Same as http.request_content_length > 1024. Uptrace supports kb, mb, gb, and tb units. |
where .event_count > 0 | Filter spans with events. |
where .event_error_count > 0 | Filter spans with error events. |
where .event_log_count > 0 | Filter spans with log events. |
where .is_event | Filter event spans, for example, exceptions or logs. |
where foo exists | Filter spans that have attribute foo. |
Grouping
Grouping expressions start with group by and work just like the corresponding SQL clause, for example, group by host.name groups spans by the attribute host.name and at the same time selects the host.name.
| Uptrace grouping | Note |
|---|---|
group by .group_id | Group similar spans together. |
group by .start_of_minute | Group spans by the minute they were created. Uptrace also supports grouping by hour, day, and week. |
group by host.name | Group spans by the host.name attribute. |
group by service.name, service.version | Group spans by the combination of service.name and service.version attributes. |
Aggregates
Aggregate functions perform a calculation on a set of values, and return a single value. They are often used together with grouping.
| Aggregate function | Example | Note |
|---|---|---|
any | any(.name) | Any (random) span name. |
avg | avg(.duration) | Average span duration. |
min, max | max(.duration) | Maximum span duration. |
p50, p75, p90, p99 | p50(.duration) | Span duration percentile. |
sum | sum(http.request_content_length) | Total number of processed bytes. |
top3, top10 | top3(code.function) | Top 3 most popular function names. |
uniq | uniq(http.client_ip) | Number of unique IP addresses. |
There is also a number of common pre-aggregated columns:
| Virtual column | Note |
|---|---|
.count | The equivalent of SQL count(*) that takes in account adjusted counts. |
.error_count | The number of spans with .status.code = 'error'. |
.error_rate | The result of .error_count / .count. |
Combining all together
You can write powerful queries combining filters, grouping, and aggregates together. For example, to select the number of unique visitors for each day excluding bots:
where http.user_agent.bot not exists | uniq(http.client_ip) | group by .start_of_day
