Transformations

Uptrace allows to transform the ingested data using a YAML-based language. You can use it to rename/delete attribute keys, parse/change attribute values, sample/drop spans, etc.

Example

For example, to rename an attribute:

yaml

name: Rename service to service_name
scope: [spans, logs, events, datapoints]
type: rename_attr
old_key: service
new_key: service_name

You can use the YAML above to create an operation by following these steps:

In the navigation menu on the left, click "Project" -> "Transformations".
Click "New Operation" -> "New operation from YAML".
Paste the YAML and click "Create".

Scope

Each operation must have a scope field that limits the operation to a particular data type (aka signal). The scope is an array and must contain one of the following values:

spans
logs
events
datapoints

Conditions

You can further narrow down the telemetry data to be transformed using if expression:

yaml

name: Rename service to service_name
scope: [spans, logs, events, datapoints]
type: rename_attr
old_key: service
new_key: service_name
if: attr("deployment_environment") == "prod"

Uptrace uses Expr language for writing expressions. In addition to built-in functions provided by Expr, Uptrace also supports the following functions:

Function	Scope	Comment
`spanName()`	spans, logs, events	Returns the span name or an empty string.
`eventName()`	events	Returns the event name or an empty string.
`spanKind()`	spans	Returns the span kind.
`spanDuration()`	spans	Returns the span duration in nanoseconds.
`spanStatusCode()`	spans, logs, events	Returns the span status code.
`spanStatusMessage()`	spans, logs, events	Returns the span status message.
`metricName()`	datapoints	Returns the metric name.
`metricUnit()`	datapoints	Returns the metric unit.
`metricInstrument()`	datapoints	Returns the metric instrument.
`attr(key string)`	all	Returns the attribute value by key.
`hasAttr(key string)`	all	Returns `true` if the attribute exists.

Note that spanName() and eventName() return the original value as reported by OpenTelemetry, e.g. eventName() usually contains only a single word like message, log, etc.

Execution

Uptrace executes operations one by one after normalising attribute names, i.e. service.name is normalised to service_name so you should use the normalised name.

Execution will stop if the signal is dropped by a drop operation, but otherwise Uptrace will execute all operations regardless of errors.

To change the execution order, use the priority field.

yaml

name: Rename service to service_name
scope: [spans, logs, events, datapoints]
priority: 100
type: rename_attr
old_key: service
new_key: service_name

Uptrace executes transformations before it processes the data so you can't use certain attributes set by Uptrace, for example, the display_name attribute is not available and instead you should use spanName(), eventName(), logName(), and log_message.

For the same reason, you also can't use attributes extracted from structured logs and SQL queries, such as db_arg_* attributes.

Uptrace Attribute	Replacement
`display_name`	`spanName()`, `eventName()`, and `attr("log_message")`
`db_arg_*`	`attr("db_statement") contains "something"`
`http_response_status_class`	`http_response_status_code`
`user_agent_*`	`user_agent_original`

Operations

Rename attribute

To rename attr from service to service_name:

yaml

name: Rename service to service_name
scope: [spans, logs, events]
type: rename_attr
old_key: service
new_key: service_name

Delete attributes

To delete attrs:

yaml

name: Delete foo and bar
scope: [spans, logs, events, datapoints]
type: delete_attrs
keys:
  - foo
  - bar

To delete attrs using a regular expression:

yaml

name: Delete foo and bar
scope: [spans, logs, events, datapoints]
type: delete_attrs
regexp: ^(foo|bar)$

To delete attrs in the specific metric:

yaml

name: Delete foo and bar
scope: [datapoints]
type: delete_attrs
keys:
  - foo
  - bar
if: metricName() == "my_metric_name"

Keep attributes

To keep some attributes and delete others:

yaml

name: Keep foo and bar
scope: [spans, logs, events, datapoints]
type: keep_attrs
keys:
  - foo
  - bar

To keep attributes matching a regular expression:

yaml

name: Keep foo and bar
scope: [spans, logs, events, datapoints]
type: keep_attrs
regexp: ^(foo|bar)$

Drop

To drop logs that match if expression:

yaml

name: Drop hello logs
scope: [logs]
type: drop
if: attr("log_message") contains "hello"

To drop spans that match if expression:

yaml

name: Drop Redis evalsha
scope: [spans]
type: drop
if: spanName() == "evalsha" && spanStatusCode() != "error"

Sample

To sample 50% of logs that match if expression:

yaml

name: Sample 50% hello logs
scope: [logs]
type: sample
fraction: 0.5
if: attr("log_message") startsWith "hello"

Index Map

By default, Uptrace does not index attributes that contain maps. For example, the following attribute cannot be queried using the object.nested.foo path:

json

object = {
  "nested": {
    "foo": "bar",
    "hello": "world",
  }
}

To index such attributes, use the index_map operation:

yaml

name: Index the object attribute
scope: [spans]
type: index_map

# Map attributes to index.
maps: ['object']

# Map paths to include.
include_paths: ['object.nested.foo']
include_regexp: '^object.nested\.'

# Map paths to exclude.
exclude_paths: ['object.nested.hello']
exclude_regexp: 'secret'

If you don't specify any paths, the operation will index the entire map but will stop after 16 paths. This limit exists to keep the number of indexed attributes low and improve performance.

Script

Uptrace allows to write simple scripts that can modify telemetry data, for example:

yaml

name: Update status code if span has exception* attributes
scope: [spans]
type: script
if: hasAttr("exception_type") && hasAttr("exception_message")
then:
  - setSpanStatusCode("error")
  - setSpanStatusMessage(attr("exception_message"))
else:
  - setSpanStatusCode("ok")

You can use the following functions to change the telemetry data:

Function	Scope	Comment
`setSpanName(string)`	spans	Sets the span name.
`setLogName(string)`	logs	Sets the log name.
`setEventName(string)`	events	Sets the event name.
`setSpanKind(string)`	spans	Sets the span kind.
`setSpanDuration(nanoseconds)`	spans	Sets the span duration in nanoseconds.
`setSpanStatusCode(string)`	spans	Sets the span status code.
`setSpanStatusMessage(string)`	spans	Sets the span status message.
`setMetricName(string)`	datapoints	Sets the datapoint metric name.
`setMetricUnit(string)`	datapoints	Sets the datapoint metric unit.
`setAttr(key, value)`	all	Sets the attribute value.
`deleteAttr(key)`	all	Deletes the attribute.
`renameAttr(oldKey, newKey)`	all	Renames the attribute.
`parseInt(value)`	spans, logs, events	Parses the value as `int`.
`parseUint(value)`	spans, logs, events	Parses the value as `uint`.
`parseFloat(value)`	spans, logs, events	Parses the value as `float`.
`parseBool(value)`	spans, logs, events	Parses the value as `bool`.
`parseDuration(string)`	spans, logs, events	Parses the string value as `time.Duration`, e.g. `parseDuration("1s")`.
`parseBytes(string)`	spans, logs, events	Parses the string value as `int64` bytes , e.g. `parseBytes("1mb")`.
`replaceGlob(src, pattern, repl)`	all	Replaces `src` with `repl` if src matches glob `pattern`.
`replaceAllRegexp(src, pattern, repl)`	all	Replaces `src` with `repl` if src matches regexp `pattern`.
`extractAllRegexp(src, pattern, repl)`	all	Extracts `repl` if src matches regexp `pattern`.

To parse a string attribute value as a float:

yaml

name: Parse elapsed_ms
scope: [spans]
type: script
if: hasAttr("elapsed_ms")
then:
  - setAttr("elapsed_ms", parseFloat(attr("elapsed_ms")))

To reduce cardinality of an attribute:

yaml

name: Reduce http_target cardinality
scope: [spans]
type: script
if: metricName() startsWith "http_server_" && hasAttr("http_target")
then:
  - setAttr("http_target", replaceGlob(attr("http_target"), "/user/*/list/*", "/user/{userId}/list/{listId}"))

To replace all numbers in a string attribute using a regular expression:

yaml

name: Replace numbers in foo
scope: [spans]
type: script
if: hasAttr("foo")
then:
  - setAttr("foo", replaceAllRegexp(attr("foo"), "(\\d+)", "<number>"))

To extract all numbers from a string attribute using a regular expression:

text

name: Extract numbers from foo
scope: [spans]
type: script
if: hasAttr("foo")
then:
  - setAttr("foo", extractAllRegexp(attr("foo"), "(\\d+)", "$1"))

To convert a log into a span:

yaml

name: Convert logs with elapsed_ms to spans
scope: [logs]
type: script
if: hasAttr("elapsed_ms")
then:
  - setSpanName("operation-name")
  - setLogName("") # must zero the log name
  - setSpanDuration(parseFloat(attr("elapsed_ms")) * 1e6)
  - setSpanStatusCode("ok")

Expr Language Definition

DashboardsUptrace allows you to build custom dashboards from spans, events, logs, and metrics using a Prometheus-like query language.

Service graphService Graphs provide a visual representation of service interactions, dependencies, and performance metrics. Service graphs are built by analyzing span relationships and require specific span attributes.