Span query language reference

This page lists every field, function, operator, and search operator the Uptrace span query language accepts. The language works on spans, logs, and events. To learn how to write queries, start with Querying spans; to select whole traces, see Querying traces.

A query is a list of clauses separated by |:

group by service_name | p50(_dur_ms) | where _status_code = 'error'

Built-in fields

A built-in field starts with an underscore, so it cannot collide with an attribute. An attribute name replaces dots with underscores, so service.name becomes service_name.

Data types

Uptrace discovers an attribute's type from the data. Each type accepts its own comparison operators:

Annotate a type when discovery is not enough, which happens when one attribute holds different types across records:

foo::string | foo::int

Uptrace uses the annotation as a hint for reading columnar data. It does not convert the value.

String literals

"I'm a string\n"           -- double quotes, with escape sequences
'I\'m a string\n'          -- single quotes, with escape sequences
`^some-prefix-(\w+)$`      -- backticks, no escape sequences, for a regexp

Units

Uptrace reads a unit from a name suffix, and then formats the column for you: bytes, seconds, milliseconds, utilization. When you cannot rename the attribute, alias the expression:

sum(heap_size) as heap_size_bytes

Clauses

Rename a grouping column with as, and manipulate its value with a transform function:

group by lower(service_name) as service, upper(host_name) as host
group by extract(host_name, `^uptrace-prod-(\w+)$`) as host

Filters

A leading not negates a filter, a function call, or a group in parentheses. It binds tighter than and and or:

where not ipInRange(client_address, "10.0.0.0/8")
where not (http_route = "/api" or http_route = "/health")

A filter matches only a record that carries every attribute it reads, and a negation keeps that rule: !=, not in, and not (...) skip a record without the attribute. Add or attr not exists to keep those records too.

Comparing two attributes

A bare word after =, !=, <, <=, >, or >= is a second attribute, not a string value. Quote a string value:

where http_response_body_size > http_request_body_size
where client_address != server_address
where deployment_environment = "prod"

Both operands must have comparable types: two numbers, two strings, two booleans, or two times. The right side is one attribute, so move arithmetic to the left, as in where a - 2 * b > 0. When no record carries the right-side attribute, = or != on a string fails with a hint to quote the value.

Aggregate functions

count() counts records, and uniq(attr) counts distinct values of an attribute. To count affected users in each group of errors:

group by _group_id | uniq(enduser_id) | where _status_code = 'error'

Sampling weight

A stored record can stand for many records: a sampler that keeps one span in ten stores it with the weight 10, which is _count. The value aggregates weight each record by _count, so they describe the records the sampled ones stand for:

• sum is the weighted total, and avg is the weighted mean.
• stddev is the weighted population standard deviation.
• median, quantile, and p50…p99 are weighted quantiles.

count() already sums the weights. min, max, any, anyLast, uniq, argMax, argMin, and uniqValues do not weight, because a weight does not change their answer.

Conditional aggregates

Add the If suffix to any aggregate to pass a condition as its last argument. The aggregate then reads only the records the condition matches:

countIf(_status_code = "error")
sumIf(http_request_content_length, _kind = "server")
p50If(_dur_ms, service_name = "api")
uniqIf(enduser_id, _status_code = "error")

The suffix works for every aggregate, and it requires at least one argument.

Virtual columns

A virtual column is a shortcut for a common expression:

Transform functions

A transform function returns a new value for each record. Use one in a grouping expression, a filter, or an aggregate's argument.

Strings

group by replace(host_name, 'uptrace-prod-', '') as host
group by replaceRegexp(host, `^`, 'prefix ') as host

Rates

per_min and per_sec are accepted spellings of the first two.

perMin(count()) | perSec(count())

Numbers, IP addresses, and strings

group by bucket(_dur_ms, 100ms) as latency | count()
group by ipv4Subnet(client_address, 24) as subnet | count()
where ipInRange(client_address, "10.0.0.0/8")
where length(http_route) > 100

A missing attribute matches neither ipInRange nor not ipInRange, and a stored value that is not an address never fails the query.

Type conversion

A value the function cannot parse becomes zero rather than an error.

Arrays

group by arrayJoin(db_sql_tables) as table | count()

Time bucketing

Each function truncates a time value to the start of its bucket, which makes it useful for grouping over time:

group by toStartOfDay(_time) as day | uniq(enduser_id) as visitors

bucket also truncates a time, with any step of whole seconds, an optional offset, and an optional IANA timezone:

group by bucket(_time, 1d, "Europe/Berlin") as day | count()
where timeOfDay(_time, "Europe/Berlin") >= 8h and timeOfDay(_time, "Europe/Berlin") < 18h
where dayOfWeek(_time) in (6, 7)

JSON functions

When an attribute holds a JSON document, these functions read inside it. Two families are available, and both take the JSON string as the first argument.

The SQL/JSON path functions take one path:

The ClickHouse functions take one or more path segments:

group by JSONExtractString(http_request_body, 'user', 'plan') as plan | count()
where JSONHas(payload, 'error')

Search syntax

Search is the free-text clause of a query. Searching spans and logs documents it in full, with examples. The rules a reference needs:

• Uptrace splits the search string on whitespace into matchers, and every matcher must match.
• One matcher holds pipe-separated alternatives, so select|update matches either term.
• Matching is case-insensitive substring matching, so err matches error.
• -term excludes a term, attr:value scopes the search to one attribute, and *:value searches the full-text index.
• Quote a value with " or ' to keep spaces and punctuation. A backtick value matches literally, with no escape processing.
• A scope must be a valid OpenTelemetry attribute key, or *. A key: prefix that is neither is read as part of the word.
• An unscoped search looks at the attributes the query groups by. Search scope maps each grouping to the attributes it searches.

error "database connection" -timeout
*:hello *:error|warning -*:debug

See also

• Querying spans
• Querying traces
• Grouping and systems
• Searching spans and logs
• Metrics query language reference