Span query language reference
This page lists every field, function, operator, and search operator the Uptrace span query language accepts. The language works on spans, logs, and events. To learn how to write queries, start with Querying spans; to select whole traces, see Querying traces.
A query is a list of clauses separated by |:
group by service_name | p50(_dur_ms) | where _status_code = 'error'
Built-in fields
A built-in field starts with an underscore, so it cannot collide with an attribute. An attribute name replaces dots with underscores, so service.name becomes service_name.
| Field | Type | Description |
|---|---|---|
_id | str | The record's own id. |
_parent_id | str | The id of the record's parent span. |
_trace_id | str | The id of the trace the record belongs to. |
_group_id | str | The id of the group the record was fingerprinted into. |
_grouping_rule_id | str | The id of the log grouping rule that assigned the hash. |
_type | str | The coarse category ingest classified the record into. |
_system | str | The categorization facet derived from the type, for example db:postgresql. |
_kind | str | The OpenTelemetry span kind, for example server or client. |
_name | str | The raw span name, which grouping uses. |
_event_name | str | The OpenTelemetry event name, for a span event. |
_display_name | str | The human-facing label ingest computed. |
_time | time | The time the record started, with microsecond accuracy. |
_dur_ms | float | The record's duration in milliseconds. |
_status_code | str | The record's status code: ok, error, or unset. |
_status_message | str | The record's status message. |
_error_count | int | The number of errors the record counts. |
_error_rate | float | The share of records that are errors, as a ratio of two counts. |
_dur_ms accepts two aliases: _duration and _dur. The parser normalizes both to _dur_ms, so the three spellings are the same field.
Data types
Uptrace discovers an attribute's type from the data. Each type accepts its own comparison operators:
| Type | Operators |
|---|---|
str | =, !=, in, like, not like, contains, ~ (regexp), exists |
int and float | =, !=, <, <=, >, >=, exists |
bool | =, != |
[]str | contains, exists |
time | <, <=, >, >= |
Annotate a type when discovery is not enough, which happens when one attribute holds different types across records:
foo::string | foo::int
Uptrace uses the annotation as a hint for reading columnar data. It does not convert the value.
String literals
"I'm a string\n" -- double quotes, with escape sequences
'I\'m a string\n' -- single quotes, with escape sequences
`^some-prefix-(\w+)$` -- backticks, no escape sequences, for a regexp
Units
Uptrace reads a unit from a name suffix, and then formats the column for you: bytes, seconds, milliseconds, utilization. When you cannot rename the attribute, alias the expression:
sum(heap_size) as heap_size_bytes
Clauses
| Clause | Example | Description |
|---|---|---|
select | select count(), p50(_dur_ms) | Names the result columns. The keyword is optional, so a bare expression is a selector too. |
where | where _status_code = 'error' | Filters records before grouping. |
group by | group by service_name, host_name | Groups records, and adds the columns to the result. |
having | having p50(_dur_ms) > 100ms | Filters the groups after aggregation. |
| search | error -timeout | Free-text search — see Search syntax. |
Rename a grouping column with as, and manipulate its value with a transform function:
group by lower(service_name) as service, upper(host_name) as host
group by extract(host_name, `^uptrace-prod-(\w+)$`) as host
Aggregate functions
| Function | Description |
|---|---|
count() | Number of matched records. Takes no arguments. |
countAll() | Same as count(): both count every record a pre-aggregated row weighs. |
countDistinct() | Number of distinct records. Takes no arguments. |
uniq(attr) | Number of distinct values of the attribute. |
sum(attr) | Sum of the values. |
avg(attr) | Average of the values. |
min(attr) | Smallest value. |
max(attr) | Largest value. |
median(attr) | Median of the values. |
p50(attr) | 50th percentile. Also p75, p90, p95, p99. |
quantile(level, attr) | Quantile at an arbitrary level, for example quantile(0.8, _dur_ms). |
any(attr) | An arbitrary value from the group. |
anyLast(attr) | The last value in the group. |
top3(attr) | The three most frequent values, as an array. Also top10. |
apdex(t1, t2) | Apdex score of span duration, with a satisfied threshold t1 and a tolerating threshold t2, for example apdex(500ms, 3s). |
merge(agg($alias.attr)) | Re-aggregates the per-trace partials of a cross-join expression — see Querying traces. |
count() counts records, and uniq(attr) counts distinct values of an attribute. To count affected users in each group of errors:
group by _group_id | uniq(enduser_id) | where _status_code = 'error'
Conditional aggregates
Add the If suffix to any aggregate to pass a condition as its last argument. The aggregate then reads only the records the condition matches:
countIf(_status_code = "error")
sumIf(http_request_content_length, _kind = "server")
p50If(_dur_ms, service_name = "api")
uniqIf(enduser_id, _status_code = "error")
The suffix works for every aggregate, and it requires at least one argument.
Virtual columns
A virtual column is a shortcut for a common expression:
| Column | The same as |
|---|---|
_error_rate | countIf(_status_code = "error") / count() |
Transform functions
A transform function returns a new value for each record. Use one in a grouping expression, a filter, or an aggregate's argument.
Strings
| Function | Description |
|---|---|
lower(attr) | Converts the value to lowercase. |
upper(attr) | Converts the value to uppercase. |
trimPrefix(attr, "prefix") | Removes the leading prefix. |
trimSuffix(attr, "suffix") | Removes the trailing suffix. |
extract(attr, pattern) | Extracts the part the regexp captures. |
replace(attr, substring, replacement) | Replaces every occurrence of the substring. |
replaceRegexp(attr, pattern, replacement) | Replaces every part that matches the regexp. |
group by replace(host_name, 'uptrace-prod-', '') as host
group by replaceRegexp(host, `^`, 'prefix ') as host
Rates
| Function | Description |
|---|---|
perMin(expr) | Divides the value by the number of minutes in the time interval. |
perSec(expr) | Divides the value by the number of seconds in the time interval. |
rate(expr) | Same as perSec. |
per_min and per_sec are accepted spellings of the first two.
perMin(count()) | perSec(count())
Type conversion
| Function | Description |
|---|---|
parseInt64(attr) | Parses the string as a 64-bit integer. |
parseFloat64(attr) | Parses the string as a 64-bit float. |
parseDateTime(attr) | Parses the string as a date with time. |
A value the function cannot parse becomes zero rather than an error.
Arrays
| Function | Description |
|---|---|
arrayJoin(attr) | Expands an array attribute into one row per element, so you can group by it. |
group by arrayJoin(db_sql_tables) as table | count()
Time bucketing
Each function truncates a time value to the start of its bucket, which makes it useful for grouping over time:
| Function | Rounds down to |
|---|---|
toStartOfSecond(_time) | The second. |
toStartOfMinute(_time) | The minute. |
toStartOfFiveMinutes(_time) | The 5-minute interval. |
toStartOfTenMinutes(_time) | The 10-minute interval. |
toStartOfFifteenMinutes(_time) | The 15-minute interval. |
toStartOfHour(_time) | The hour. |
toStartOfDay(_time) | The day. |
group by toStartOfDay(_time) as day | uniq(enduser_id) as visitors
JSON functions
When an attribute holds a JSON document, these functions read inside it. Two families are available, and both take the JSON string as the first argument.
The SQL/JSON path functions take one path:
| Function | Returns | Description |
|---|---|---|
JSON_EXISTS(attr, path) | bool | Whether the path exists. |
JSON_QUERY(attr, path) | str | The value at the path, as JSON. |
JSON_VALUE(attr, path) | str | The scalar value at the path. |
The ClickHouse functions take one or more path segments:
| Function | Returns | Description |
|---|---|---|
JSONHas(attr, path…) | bool | Whether the member exists. |
JSONLength(attr, path…) | int | Number of elements or members. |
JSONType(attr, path…) | str | The JSON type of the member. |
JSONExtractString(attr, path…) | str | The member as a string. |
JSONExtractInt(attr, path…) | int | The member as a signed integer. |
JSONExtractUInt(attr, path…) | int | The member as an unsigned integer. |
JSONExtractFloat(attr, path…) | float | The member as a float. |
JSONExtractBool(attr, path…) | bool | The member as a boolean. |
JSONExtractKeys(attr, path…) | str | The member's keys. |
JSONExtractRaw(attr, path…) | str | The member as raw JSON. |
JSONExtractArrayRaw(attr, path…) | str | The array's elements, each as raw JSON. |
group by JSONExtractString(http_request_body, 'user', 'plan') as plan | count()
where JSONHas(payload, 'error')
Search syntax
Search is the free-text clause of a query. Searching spans and logs documents it in full, with examples. The rules a reference needs:
- Uptrace splits the search string on whitespace into matchers, and every matcher must match.
- One matcher holds pipe-separated alternatives, so
select|updatematches either term. - Matching is case-insensitive substring matching, so
errmatcheserror. -termexcludes a term,attr:valuescopes the search to one attribute, and*:valuesearches the full-text index.- Quote a value with
"or'to keep spaces and punctuation. A backtick value matches literally, with no escape processing. - A scope must be a valid OpenTelemetry attribute key, or
*. Akey:prefix that is neither is read as part of the word. - An unscoped search looks at the attributes the query groups by. Search scope maps each grouping to the attributes it searches.
error "database connection" -timeout
*:hello *:error|warning -*:debug
Regexp search matchers (~pattern and ~scope:pattern) are not supported, and the parser rejects the ~ operator with regexp matchers are not supported. To match a pattern, use the ~ operator of a where filter instead, for example where log_message ~ 'group(.*)does not exist'.