Trace Reporters
This documentation is for an unreleased version of Apache Flink. We recommend you use the latest stable version.

Trace Reporters #

Flink allows reporting traces to external systems. For more information about Flink’s tracing system go to the tracing system documentation.

Traces can be exposed to an external system by configuring one or several reporters in Flink configuration file. These reporters will be instantiated on each job and task manager when they are started.

Below is a list of parameters that are generally applicable to all reporters. All properties are configured by setting traces.reporter.<reporter_name>.<property> in the configuration. Reporters may additionally offer implementation-specific parameters, which are documented in the respective reporter’s section.

Key Default Type Description
traces.reporter.<name>.factory.class
 (none) String The reporter factory class to use for the reporter named <name>.
traces.reporter.<name>.scope.delimiter
 "." String The delimiter used to assemble the metric identifier for the reporter named <name>.
traces.reporter.<name>.scope.variables.excludes
 "." String The set of variables that should be excluded for the reporter named <name>. Only applicable to tag-based reporters.
traces.reporter.<name>.scope.variables.additional
 Map The map of additional variables that should be included for the reporter named <name>.
traces.reporter.<name>.<parameter>
 (none) String Configures the parameter <parameter> for the reporter named <name>.
traces.reporter.<name>.filter.includes
 "*:*:*" List<String> The spans that should be included for the reporter named <name>. Filters are specified as a list, with each filter following this format:
<scope>[:<name>[,<name>]]
A span matches a filter if the scope pattern and at least one of the name patterns match.
  • scope: Filters based on the logical scope.
    Specified as a pattern where * matches any sequence of characters and . separates scope components.

    For example:
    "jobmanager.job" matches any job-related spans on the JobManager,
    "*.job" matches all job-related spans and
    "*.job.*" matches all spans below the job-level (i.e., task/operator spans etc.).

  • name: Filters based on the span name.
    Specified as a comma-separate list of patterns where * matches any sequence of characters.

    For example, "*Records*,*Bytes*" matches any span where the name contains "Records" or "Bytes".
traces.reporter.<name>.filter.excludes
 List<String> The spans that should be excluded for the reporter named <name>. The format is identical to filter.includes

All reporter configurations must contain the factory.class property.

Example reporter configuration that specifies multiple reporters:

traces.reporters: otel,my_other_otel

traces.reporter.otel.factory.class: org.apache.flink.traces.otel.OpenTelemetryTraceReporterFactory
traces.reporter.otel.exporter.endpoint: http://127.0.0.1:1337
traces.reporter.otel.scope.variables.additional: region:eu-west-1,environment:local,flink_runtime:1.17.1

traces.reporter.my_other_otel.factory.class: org.apache.flink.common.metrics.OpenTelemetryTraceReporterFactory
traces.reporter.my_other_otel.exporter.endpoint: http://196.168.0.1:31337

Important: The jar containing the reporter must be accessible when Flink is started. Reporters are loaded as plugins. All reporters documented on this page are available by default.

You can write your own Reporter by implementing the org.apache.flink.traces.reporter.TraceReporter and org.apache.flink.traces.reporter.TraceReporterFactory interfaces. Be careful that all the method must not block for a significant amount of time, and any reporter needing more time should instead run the operation asynchronously.

Reporters #

The following sections list the supported reporters.

OpenTelemetry #

(org.apache.flink.traces.otel.OpenTelemetryTraceReporterFactory) #

OpenTelemetryTraceReporterFactory currently supports only gRPC.

Parameters:

Key Default Type Description
exporter.endpoint
 (none) String Endpoint for the OpenTelemetry Reporters.
exporter.timeout
 (none) String Timeout for OpenTelemetry Reporters, as Duration string. Example: 10s for 10 seconds
service.name
 (none) String service.name passed to OpenTelemetry Reporters.
service.version
 (none) String service.version passed to OpenTelemetry Reporters.

Example configuration:

traces.reporter.otel.factory.class: org.apache.flink.metrics.otel.OpenTelemetryTraceReporterFactory
traces.reporter.otel.exporter.endpoint: http://127.0.0.1:1337

Slf4j #

(org.apache.flink.traces.slf4j.Slf4jTraceReporter) #

Example configuration:

traces.reporter.slf4j.factory.class: org.apache.flink.traces.slf4j.Slf4jTraceReporterFactory

Back to top