Monitoring logs with Uptrace
TIP
To receive notifications about specific logs and errors, see Alerts and notifications.
Infrastructure logs
To monitor infrastructure logs, you can use Vector (Heroku, Fly) and FluentBit integrations.
If you are using AWS, you can also send CloudWatch Logs directly to Uptrace.
Application logs
To record application logs, you can use OpenTelemetry span events. You must set the event name to log
and use semantic attributes to record the context:
log.severity
to record the log severity. Must be one ofTRACE
,DEBUG
,INFO
,WARN
,ERROR
,FATAL
, andPANIC
.log.message
to record the message.code.function
to record the function name.code.filepath
to record the file path.code.lineno
to record the line number.
Go
For example, using Go programming language and OpenTelemetry events API:
span := trace.SpanFromContext(ctx)
span.AddEvent("log", trace.WithAttributes(
// Log severity and message.
attribute.String("log.severity", "ERROR"),
attribute.String("log.message", "request failed"),
// Optional.
attribute.String("code.function", "org.FetchUser"),
attribute.String("code.filepath", "org/user.go"),
attribute.Int("code.lineno", 123),
// Additional details.
attribute.String("foo", "hello world"),
))
You can also use instrumentations for popular logging libraries which allow recording logs using a more conventional API, for example, OpenTelemetry Zap and OpenTelemetry Logrus.
Python
OpenTelemetry Python comes with a handler for Python's logging
package so you can just use the standard logging API:
import logging
import uptrace
# Configure OpenTelemetry.
uptrace.configure_opentelemetry(...)
# Use logging API as usual.
logger = logging.getLogger(__name__)
logger.setLevel(logging.ERROR)
Grouping logs together
You can control how Uptrace groups logs together by providing grouping.fingerprint
attribute which can be a string or a number (hash/id):
log.severity = "info"
log.message = "unstructured log message 123 456 789"
grouping.fingerprint = "unstructured log message"
You can further customize grouping using grouping rules, but this is only available in the paid version.
Propagating trace context
When collecting third-party logs with Vector or FluentBit, trace context is not automatically propagated and logs can't be linked with spans.
To propagate context and associate a log record with a span, use the following attribute keys in the log message:
trace_id
for TraceId, hex-encoded.span_id
for SpanId, hex-encoded.trace_flags
for trace flags, formatted according to W3C traceflags format.
For example:
request failed trace_id=958180131ddde684c1dbda1aeacf51d3 span_id=0cf859e4f7510204