This document outlines the Teleport Authentication Service and Certificate Management. It explains how Users and Nodes are identified and granted access to Nodes and Services.
Teleport Auth handles both authentication and authorization. These topics are related but different, and they are often discussed jointly as "Auth".
Authentication is proving an identity. "I say I am Bob, and I really am Bob. See look I have Bob's purple hat." The job of an Authentication system is to define the criteria by which users must prove their identity. Is having a purple hat enough to show that a person is Bob? Maybe, maybe not. To identify users and nodes to Teleport Auth, we require them to present a cryptographically-signed certificate issued by the Teleport Auth Certificate Authority.
Authorization is proving access to something: "Bob has a purple hat, but also a debit card and the correct PIN code. Bob can access a bank account with the number 814000001344. Can Bob get $20 out of the ATM?" The ATM's Authentication system would validate Bob's PIN Code, while the Authorization system would use a stored mapping from Bob to account #814000001344 to decide whether Bob could withdraw cash. Authorization defines and determines permissions that users have within a system, such as access to cash within a banking system, or data in a filesystem. Before users are granted access to nodes, the Auth Service checks their identity against a stored mapping in a database.
One can think of an SSH certificate as a "permit" issued and time-stamped by a trusted authority. In this case, the authority is the Auth Server's Certificate Authority. A certificate contains four important pieces of data:
- List of principals (identities) this certificate belongs to.
- Signature of the certificate authority who issued it.
- The expiration date, also known as "time-to-live" or simply TTL.
- Additional data, such as the node role, is stored as a certificate extension.
Teleport uses SSH certificates to authenticate nodes and users within a cluster.
Two CAs are used inside the Auth Server because nodes and users each need their own certificates.
- The Node CA issues certificates which identify a node (i.e. host, server, computer). These certificates are used to add new nodes to a cluster and identify connections coming from the node.
- The User CA issues certificates which identify a User. These certificates are used to authenticate users when they try to connect to a cluster node.
Node Certificates identify a node within a cluster and establish the permissions of the node to access other Teleport services. The presence of a signed certificate on a node makes it a cluster member.
- To join a cluster for the first time, a node must present a "join token" to the auth server. The token can be static (configured via config file) or a dynamic, single-use token generated by
tctl nodes add.
When using dynamic tokens, their default time to live (TTL) is 30
minutes, but it can be reduced (not increased) via
tctl nodes add --ttl flag.
- When a new node joins the cluster, the auth server generates a new public/private keypair for the node and signs its certificate. This node certificate contains the node's role(s) (
node) as a certificate extension (opaque signed string).
All nodes in a cluster can connect to the Auth Server's API
implemented as an HTTP REST service running over the SSH
tunnel. This API connection is authenticated with the node certificate and the
encoded role is checked to enforce access control. For example, a client
connection using a certificate with only the
node role won't be able to add
and delete users. This client connection would only be authorized to get auth
servers registered in the cluster.
The Auth Server uses its User CA to issue user certificates. User certificates
are stored on a user's machine in the
~/.tsh/keys/example.com directory or also
by the system's SSH agent if it is running.
- To get permission to join a cluster for the first time a user must provide their username, password, and 2nd-factor token. Users can log in with
tsh loginor via the Web UI. The Auth server checks the username and password against its identity storage and checks the 2nd-factor token.
- If the correct credentials were offered, the Auth Server will generate a signed certificate and return it to the client. For users, certificates are stored in
~/.tshby default. If the client uses the Web UI the signed certificate is associated with a secure WebSocket session.
In addition to a user's identity, user certificates also contain user roles and SSH options, like "permit-agent-forwarding" . This additional data is stored as a certificate extension and is protected by the CA signature.
When a client requests access to a node cluster, the Auth Server first checks that a certificate exists and hasn't expired. If it has expired, the client must re-authenticate with their username, password, and 2nd factor. If the certificate is still valid, the Auth Server validates the certificate's signature. The client is then granted access to the cluster. From here, the Proxy Server establishes a connection between client and node.
By default, all user certificates have an expiration date, also known as the *time to live *(TTL). This TTL can be configured by a Teleport administrator. However, the node certificates issued by an Auth Server are valid indefinitely by default.
Teleport supports certificate rotation, i.e. the process of invalidating all
previously-issued certificates for nodes and users regardless of their TTL.
Certificate rotation is triggered by
rotate. When this command is invoked by a Teleport
administrator on one of a cluster's Auth Servers, the following happens:
- A new certificate authority (CA) key is generated.
- The old CA will be considered valid alongside the new CA for a while. This period is called a grace period.
- During the grace period, all previously issued certificates will be considered valid, assuming their TTL isn't expired.
- After the grace period is over, the certificates issued by the old CA are no longer accepted.
This process is repeated twice, once for the node CA and once for the user CA.
Take a look at the Certificate Rotation Guide to learn how to do certificate rotation in practice.
Clients can also connect to the auth API through the Teleport proxy to use a limited subset of the API to discover the member nodes of the cluster.
The Auth service maintains state using a database of users, credentials,
certificates, and audit logs. The default storage location is
/var/lib/teleport or an admin-configured storage destination.
There are three types of data stored by the auth server:
- Cluster State The auth server stores its own keys in a cluster state storage. All of cluster dynamic configuration is stored there as well, including:
- Node membership information and online/offline status for each node.
- List of active sessions.
- List of locally stored users
- RBAC configuration (roles and permissions).
- Other dynamic configuration.
- Audit Log When users log into a Teleport cluster, execute remote commands, and log out, that activity is recorded in the audit log. See Audit Log for more details. More on this in the Audit Log section.
- Recorded Sessions When Teleport users launch remote shells via
tsh sshcommand, their interactive sessions are recorded and stored by the auth server. Each recorded session is a file that is saved in /var/lib/teleport by default but can also be saved in external storage, like an AWS S3 bucket.
The Teleport auth server keeps the audit log of SSH-related events that take place on any node within a Teleport cluster. Each node in a cluster emits audit events and submits them to the auth server. The events recorded include:
- Successful user logins
- Node IP addresses, Application, Database and Kubernetes FQDN
- Session time
- Session IDs
Because all SSH events like
session_start are by default reported by the Teleport node service, they will not be logged if you are using OpenSSH
sshd daemon on your nodes. Recording proxy mode
Only an SSH server can report what's happening to the Teleport auth server.
The audit log is a JSON file that is by default stored on the auth server's
/var/lib/teleport/log. The format of the file is documented
in the Admin Manual.
Teleport users are encouraged to export the events into external, long term storage.
If multiple Teleport auth servers are used
to service the same cluster (High Availability mode) a network file system must be used for
/var/lib/teleport/log to allow them to combine all audit events into the
same audit log. Learn how to deploy Teleport in High Availability Mode..
Different types of cluster data can be configured with different storage back-ends as shown in the table below:
|Data Type||Supported Back-ends||Notes|
|Cluster state||Multi-server (High Availability) configuration is only supported using |
|Audit Log Events||If |
The reason Teleport designers split the audit log events and the recorded sessions into different back-ends is because of the nature of the data. A recorded session is a compressed binary stream (blob) while the event is a well-defined JSON structure.
dir works well enough for both in small deployments, but large clusters require specialized data stores: S3 is perfect for uploading session blobs, while DynamoDB or
etcd are better suited to store the cluster state.
The combination of DynamoDB + S3 is especially popular among AWS users because it allows them to run Teleport clusters completely devoid of local state.
For High Availability in production, a Teleport cluster can be serviced by multiple auth servers running in sync. Check High Availability configuration in the Admin Guide.