Transformations

Uptrace allows to transform the ingested data using a YAML-based language. You can use it to rename/delete attribute keys, parse/change attribute values, sample/drop spans, etc.

Example

For example, to rename an attribute:

name: Rename service to service_name
scope: [spans, logs, events, datapoints]
type: rename_attr
old_key: service
new_key: service_name

You can use the YAML above to create an operation by following these steps:

In the navigation menu on the left, click "Project" -> "Transformations".
Click "New Operation" -> "New operation from YAML".
Paste the YAML and click "Create".

Scope

Each operation must have a scope field that limits the operation to a particular data type (aka signal). The scope is an array and must contain one of the following values:

spans
logs
events
datapoints

Conditions

You can further narrow down the telemetry data to be transformed using if expression:

name: Rename service to service_name
scope: [spans, logs, events, datapoints]
type: rename_attr
old_key: service
new_key: service_name
if: attr("deployment_environment") == "prod"

Uptrace uses Expropen in new window language for writing expressions. In addition to built-in functions provided by Expr, Uptrace also supports the following functions:

Function	Scope	Comment
`spanName()`	spans, logs, events	Returns the span name or empty string.
`logName()`	logs	Returns the event name or empty string.
`eventName()`	events	Returns the log name or empty string.
`spanKind()`	spans	Returns the span kind.
`spanDuration()`	spans	Returns the span duration in nanoseconds.
`spanStatusCode()`	spans, logs, events	Returns the span status code.
`spanStatusMessage()`	spans, logs, events	Returns the span status message.
`metricName()`	datapoints	Returns the metric name.
`metricUnit()`	datapoints	Returns the metric unit.
`metricInstrument()`	datapoints	Returns the metric instrument.
`attr(string)`	all	Returns the attribute value.
`hasAttr(string)`	all	Returns `true` if the attribute exists.

Execution

Uptrace executes operations one by one after normalising attribute names, i.e. service.name is normalised to service_name so you should use the normalised name.

Execution will stop if the signal is dropped by a drop operation, but otherwise Uptrace will execute all operations regardless of errors.

To change the execution order, use the priority field.

name: Rename service to service_name
scope: [spans, logs, events, datapoints]
priority: 100
type: rename_attr
old_key: service
new_key: service_name

Uptrace executes transformations before it processes the data so you can't use certain attributes set by Uptrace, for example, the display_name attribute is not available and instead you should use spanName(), eventName(), logName(), and log_message.

For the same reason, you also can't use attributes extracted from structured logs and SQL queries, such as db_arg_* attributes.

Uptrace Attribute	Replacement
`display_name`	`spanName()`, `eventName()`, `logName()`, and `log_message`
`db_arg_*`	`attr("db_statement") contains "something"`
`http_response_status_class`	`http_response_status_code`
`user_agent_*`	`user_agent_original`

Operations

Rename attribute

To rename attr from service to service_name:

name: Rename service to service_name
scope: [spans, logs, events]
type: rename_attr
old_key: service
new_key: service_name

Delete attributes

To delete attrs:

name: Delete foo and bar
scope: [spans, logs, events, datapoints]
type: delete_attrs
keys:
  - foo
  - bar

To delete attrs using a regular expression:

name: Delete foo and bar
scope: [spans, logs, events, datapoints]
type: delete_attrs
regexp: ^(foo|bar)$

To delete attrs in the specific metric:

name: Delete foo and bar
scope: [datapoints]
type: delete_attrs
keys:
  - foo
  - bar
if: metricName() == "my_metric_name"

Keep attributes

To keep some attributes and delete others:

name: Keep foo and bar
scope: [spans, logs, events, datapoints]
type: keep_attrs
keys:
  - foo
  - bar

To keep attributes matching a regular expression:

name: Keep foo and bar
scope: [spans, logs, events, datapoints]
type: keep_attrs
regexp: ^(foo|bar)$

Drop

To drop logs that match if expression:

name: Drop hello logs
scope: [logs]
type: drop
if: attr("log_message") contains "hello"

To drop spans that match if expression:

name: Drop Redis evalsha
scope: [spans]
type: drop
if: spanName() == "evalsha" && spanStatusCode() != "error"

Sample

To sample 50% of logs that match if expression:

name: Sample 50% hello logs
scope: [logs]
type: sample
fraction: 0.5
if: attr("log_message") startsWith "hello"

Script

Uptrace allows to write simple scripts that can modify telemetry data, for example:

name: Update status code if span has exception* attributes
scope: [spans]
type: script
if: hasAttr("exception_type") && hasAttr("exception_message")
then:
  - setSpanStatusCode("error")
  - setSpansStatusMessage(attr("exception_message"))
else:
  - setSpanStatusCode("ok")

You can use the following functions to change the telemetry data:

Function	Scope	Comment
`setSpanName(string)`	spans	Sets the span name.
`setLogName(string)`	logs	Sets the log name.
`setEventName(string)`	events	Sets the event name.
`setSpanKind(string)`	spans	Sets the span kind.
`setSpanDuration(nanoseconds)`	spans	Sets the span duration in nanoseconds.
`setSpanStatusCode(string)`	spans	Sets the span status code.
`setSpanStatusMessage(string)`	spans	Sets the span status message.
`setMetricName(string)`	datapoints	Sets the datapoint metric name.
`setMetricUnit(string)`	datapoints	Sets the datapoint metric unit.
`setAttr(key, value)`	all	Sets the attribute value.
`deleteAttr(key)`	all	Deletes the attribute.
`renameAttr(oldKey, newKey)`	all	Renames the attribute.
`parseInt(value)`	spans, logs, events	Parses the value as `int`.
`parseUint(value)`	spans, logs, events	Parses the value as `uint`.
`parseFloat(value)`	spans, logs, events	Parses the value as `float`.
`parseBool(value)`	spans, logs, events	Parses the value as `bool`.
`parseDuration(string)`	spans, logs, events	Parses the string value as `time.Duration`, e.g. `parseDuration("1s")`.
`parseBytes(string)`	spans, logs, events	Parses the string value as `int64` bytes , e.g. `parseBytes("1mb")`.
`replaceGlob(src, pattern, repl)`	all	Replaces `src` with `repl` if src matches glob `pattern`.
`replaceAllRegexp(src, pattern, repl)`	all	Replaces `src` with `repl` if src matches regexp `pattern`.
`extractAllRegexp(src, pattern, repl)`	all	Extracts `repl` if src matches regexp `pattern`.

To parse a string attribute value as a float:

name: Parse elapsed_ms
scope: [spans]
type: script
if: hasAttr("elapsed_ms")
then:
  - setAttr("elapsed_ms", parseFloat(attr("elapsed_ms")))

To reduce cardinality of an attribute:

name: Reduce http_target cardinality
scope: [spans]
type: script
if: metricName() startsWith "http_server_" && hasAttr("http_target")
then:
  - setAttr("http_target", replaceGlob(attr("http_target"), "/user/*/list/*", "/user/{userId}/list/{listId}"))

To replace all numbers in a string attribute using a regular expression:

name: Replace numbers in foo
scope: [spans]
type: script
if: hasAttr("foo")
then:
  - setAttr("foo", replaceAllRegexp(attr("foo"), "(\\d+)", "<number>"))

To extract all numbers from a string attribute using a regular expression:

name: Extract numbers from foo
scope: [spans]
type: script
if: hasAttr("foo")
then:
  - setAttr("foo", extractAllRegexp(attr("foo"), "(\\d+)", "$1"))

To convert a log into a span:

name: Convert logs with elapsed_ms to spans
scope: [logs]
type: script
if: hasAttr("elapsed_ms")
then:
  - setSpanName("operation-name")
  - setLogName("") # must zero the log name
  - setSpanDuration(parseFloat(attr("elapsed_ms")) * 1e6)
  - setSpanStatusCode("ok")

Expr Language Definitionopen in new window

# Transformations

# Example

# Scope

# Conditions

# Execution

# Operations

# Rename attribute

# Delete attributes

# Keep attributes

# Drop

# Sample

# Script

# Related links