- Version 15.x
- Version 14.x
- Version 13.x
- Version 12.x
- Older Versions
- Available for:
Teleport leverages OpenTelemetry to generate traces and export them to any OpenTelemetry Protocol (OTLP) capable exporter. In the event that your telemetry backend doesn't support receiving OTLP traces, you may be able to leverage the OpenTelemetry Collector to proxy traces from OTLP to a format that your telemetry backend accepts.
In order to enable tracing for a
teleport instance, add the following section to that instance's configuration file (
For a detailed description of these configuration fields, see the configuration reference page.
tracing_service: enabled: yes exporter_url: grpc://collector.example.com:4317 sampling_rate_per_million: 1000000
Please note that tracing was added in the following versions: 8.3.18, 9.3.15, and 10.1.0. Enabling the above configuration on a version prior to the versions with tracing support will result in a failure to parse the config file.
It is important to choose the sampling rate wisely. Sampling at a rate of 100% could have a negative impact on the
performance of your cluster. Teleport honors the sampling rate included in any incoming requests, which means
that even when the
tracing_service is enabled and the sampling rate is 0, if Teleport receives a request that has a span which is
sampled, then Teleport will sample and export all spans that are generated in response to that request.
exporter_url setting indicates where Teleport should send spans to. Supported schemes are
file:// (if no scheme is provided, then
grpc:// is used).
file://, the url must be a path to a directory that Teleport has write permissions for. Spans will be saved to files within
the provided directory, each file containing one proto encoded span per line. Files are rotated after exceeding 100MB, in order to
override the default limit add
?limit=<desired_file_size_in_bytes> to the
By default the connection to the exporter is insecure, to support TLS add the following to the
# Optional path to CA certificates are used to validate the exporter. ca_certs: - /var/lib/teleport/exporter_ca.pem # Optional path tp TLS certificates are used to enable mTLS for the exporter https_keypairs: - key_file: /var/lib/teleport/exporter_key.pem cert_file: /var/lib/teleport/exporter_cert.pem
teleport.yaml, start your
teleport instance to apply the new configuration.
To capture traces from
tsh simply add the
--trace flag to your command. All traces generated by
tsh --trace will be
proxied to the
exporter_url defined for the Auth Service of the cluster the command is being run on.
tsh --trace ssh root@myservertsh --trace ls
Exporting traces from
tsh to a different exporter than the one defined in the Auth Service config
is also possible via the
--trace-exporter flag. A URL must be provided that adheres to the same
format as the
exporter_url of the
tsh --trace --trace-exporter=grpc://collector.example.com:4317 ssh root@myservertsh --trace --trace-exporter=file:///var/lib/teleport/traces ls