Fork me on GitHub

Teleport

Audit Events and Records

Improve

Teleport logs cluster activity by emitting various events into its audit log. There are two components of the audit log:

  1. Cluster Events: Teleport logs events like successful user logins along with metadata like remote IP address, time, and the session ID.
  2. Recorded Sessions: Every SSH, desktop, or Kubernetes shell session is recorded and can be replayed later. By default, the recording is done by Teleport Nodes, but can be configured to be done by the proxy.
  1. Cluster Events: Teleport logs events like successful user logins along with metadata like remote IP address, time, and the session ID.
  2. Recorded Sessions: Every SSH, desktop, or Kubernetes shell session is recorded and can be replayed later. Teleport Cloud manages the storage of session recording data.

You can use Enhanced Session Recording with BPF to get even more comprehensive audit logs with advanced security.

Events

Teleport supports multiple storage backends for storing audit events. The dir backend uses the local filesystem of an Auth Service host. When this backend is used, events are written to the filesystem in JSON format. The dir backend rotates the event file approximately once every 24 hours, but never deletes captured events.

For High Availability configurations, users can refer to our DynamoDB or Firestore chapters for information on how to configure the SSH events and recorded sessions to be stored on network storage. When these backends are in use, audit events will eventually expire and be removed from the log. The default retention period is 1 year, but this can be overridden using the audit_retention_period configuration parameter.

It is even possible to store audit logs in multiple places at the same time. For more information on how to configure the audit log, refer to the storage section of the example configuration file in the Teleport Configuration Reference.

Let's examine the Teleport audit log using the dir backend. The event log is stored in Teleport's data dir under the log directory. This is usually /var/lib/teleport/log. Each day is represented as a file:

ls -l /var/lib/teleport/log/

total 104

-rw-r----- 1 root root 31638 Jan 22 20:00 2017-01-23.00:00:00.log

-rw-r----- 1 root root 91256 Jan 31 21:00 2017-02-01.00:00:00.log

-rw-r----- 1 root root 15815 Feb 32 22:54 2017-02-03.00:00:00.log

Teleport Cloud manages the storage of audit logs for you. You can access your audit logs via the Teleport Web UI by clicking:

Activity > Audit Log

Audit logs use JSON format. They are human readable but can also be programmatically parsed. Each line represents an event and has the following format:

{
    // Event type. See below for the list of all possible event types.
    "event": "session.start",
    // A unique ID for the event log. Useful for  deduplication.
    "uid": "59cf8d1b-7b36-4894-8e90-9d9713b6b9ef",
    // Teleport user name
    "user": "ekontsevoy",
    // OS login
    "login": "root",
    // Server namespace. This field is reserved for future use.
    "namespace": "default",
    // Unique server ID
    "server_id": "f84f7386-5e22-45ff-8f7d-b8079742e63f",
    // Server Labels
    "server_labels": {
      "datacenter": "us-east-1",
      "label-b": "x"
    }
    // Session ID. Can be used to replay the session.
    "sid": "8d3895b6-e9dd-11e6-94de-40167e68e931",
    // Address of the SSH node
    "addr.local": "10.5.l.15:3022",
    // Address of the connecting client (user)
    "addr.remote": "73.223.221.14:42146",
    // Terminal size
    "size": "80:25",
    // Timestamp
    "time": "2017-02-03T06:54:05Z"
}

Event types

The possible event types are:

Event TypeDescription
authAuthentication attempt. Adds the following fields: {"success": "false", "error": "access denied"}
session.startStarted an interactive shell session.
session.endAn interactive shell session has ended.
session.joinA new user has joined the existing interactive shell session.
session.leaveA user has left the session.
session.diskA list of files opened during the session. Requires Enhanced Session Recording.
session.networkA list of network connections made during the session. Requires Enhanced Session Recording.
session.commandA list of commands ran during the session. Requires Enhanced Session Recording.
session.recording.accessA session recording has been accessed.
execRemote command has been executed via SSH, like tsh ssh [email protected] ls /. The following fields will be logged: {"command": "ls /", "exitCode": 0, "exitError": ""}
scpRemote file copy has been executed. The following fields will be logged: {"path": "/path/to/file.txt", "len": 32344, "action": "read" }
resizeTerminal has been resized.
user.loginA user logged into web UI or via tsh. The following fields will be logged: {"user": "[email protected]", "method": "local"} .
app.session.startA user accessed an application
app.session.chunkA record of activity during an app session

Recorded sessions

In addition to logging start and end events, Teleport can also record the entire session. For SSH or Kubernetes sessions this captures the entire stream of bytes from the PTY. For desktop sessions the recording includes the contents of the screen.

Teleport can store the recorded sessions in an AWS S3 bucket or in a local filesystem (including NFS).

The recorded sessions are stored as raw bytes in the sessions directory under log. Each session is a protobuf-encoded stream of binary data.

You can replay recorded sessions using the tsh play command or the Web UI.

For example, replay a session via CLI:

tsh play 4c146ec8-eab6-11e6-b1b3-40167e68e931

Print the session events in JSON to stdout:

tsh play 4c146ec8-eab6-11e6-b1b3-40167e68e931 --format=json

Teleport Cloud automatically stores recorded sessions.

You can replay recorded sessions using the tsh play command or the Web UI.

For example, replay a session via CLI:

tsh play 4c146ec8-eab6-11e6-b1b3-40167e68e931

Print the session events in JSON to stdout:

tsh play 4c146ec8-eab6-11e6-b1b3-40167e68e931 --format=json

Modes

Available only for SSH sessions and when Teleport is configured with auth_service.session_recording: node.

Modes define how Teleport deals with recording failures, such as a full disk error. They are configured per-service at the role level, where the strictest value takes precedence. The available modes are:

ModeAfter a recording failure
Best effort (best_effort)Disables recording without terminating the session.
Strict (strict)Immediately terminates the session.

If the user role doesn’t specify a recording mode, best_effort will be used. Here is an example of a role configured to use strict mode for SSH sessions:

kind: role
version: v5
metadata:
  name: ssh-strict
spec:
  options:
    record_session:
      ssh: strict