Teleport SSH Nodes
The Teleport Node service is optional. You can use it to replace OpenSSH on your infrastructure. Here is why we recommend Teleport Node service instead of OpenSSH:
- The node service supports BPF recording of all syscalls, network calls and files accessed during SSH session.
- It can record terminal sessions.
- It provides automatic registration, certificate and certificate authority rotation,
- It can provision OS user and update sudoers files according to teleport roles.
- You can connect nodes to proxies with outbound persistent tunnels, for your IoT lab or remote infrastructure.
Just like with OpenSSH, the
node service provides SSH access to every node with any clients supporting client SSH certificates:
- Teleport CLI client:
- Teleport Proxy UI accessed via a web browser.
- Ansible and other SSH compatible clients.
A node candidate becomes a Teleport Node when it joins a cluster and authenticates itself to receive cluster certificate.
All cluster Nodes keep the Auth Server updated on their status with periodic ping messages. They report their IP addresses and the values of their assigned labels. Clients can access the list of all Nodes in their cluster via the Auth Server API or CLI.
Nodes can register with Auth servers directly, or use proxies to establish the connection to auth servers. The latter is helpful if you have multiple proxies and nodes all over the world.
Node's identity is represented by SSH host certificate it receives after registering withing the cluster:
This certificate contains information about the node including:
- The host ID, a generated UUID unique to a node
- A nodename, which defaults to
hostnameof the node, but can be configured.
- The cluster_name, which defaults to the
hostnameof the auth server, but can be configured
- The node role (i.e.
node,proxy) encoded as a certificate extension
- The cert Expiry time
A Teleport Cluster is a set of one or more machines whose certificates are signed by the same certificate authority (CA) operating in the Auth Server. A certificate is issued to a node when it joins the cluster for the first time.
Once a Node gets a signed certificate from the Node CA, the Node is considered a member of the cluster, even if that cluster has only one node.
Nodes support two modes - a standard mode and a reverse tunnel mode.
In standard mode, nodes act like OpenSSH servers that only accept client SSH certificates. Users can connect to nodes through Proxy as a jump-host or directly:
In reverse tunnel mode, nodes establish reverse tunnels back to proxy. Nodes do not bind to any interface on the host, making sure the connection is only possible through proxies:
You can mix both modes in the same cluster, depending on your use case. For example, you can have several IOT devices joining the cluster via reverse tunnel and a large fleet of servers in the internal network using standard mode.
Cluster state is stored in a central storage location configured by the Auth Server. Each node (or proxy) is stateless and holds no secrets such as keys or passwords.
The cluster state includes:
- Node membership information and online/offline status for each node.
- List of active sessions.
- List of locally stored users.
- RBAC configuration (roles and permissions).
- Dynamic configuration.
By default, nodes submit SSH session traffic to the Auth server
for storage. These recorded sessions can be replayed later via
command or in a web browser.
Some Teleport users assume that audit and session recording happen by default on the Teleport proxy server. This is not the case in default configuration because a proxy cannot see the encrypted traffic, it is encrypted end-to-end, i.e. from an SSH client to an SSH server/node, see the diagram below:
In this mode, the proxy terminates (decrypts) the SSH connection using the certificate supplied by the client via SSH agent forwarding and then establishes its own SSH connection to the final destination server, effectively becoming an authorized "man in the middle". This allows the proxy server to forward SSH session data to the auth server to be recorded, as shown below:
The recording proxy mode, although less secure, was added to allow Teleport
users to enable session recording for OpenSSH's servers running
sshd, which is
helpful when gradually transitioning large server fleets to Teleport.
We consider the "recording proxy mode" to be less secure for two reasons:
- It grants additional privileges to the Teleport proxy. In the default mode, the proxy stores no secrets and cannot "see" the decrypted data. This makes a proxy less critical to the security of the overall cluster. But if an attacker gains physical access to a proxy Node running in the "recording" mode, they will be able to see the decrypted traffic and client keys stored in the proxy's process memory.
- Recording proxy mode requires SSH Agent Forwarding. Agent Forwarding is required because without it, a proxy will not be able to establish the 2nd connection to the destination Node.
However, there are advantages of proxy-based session recording too. When sessions are recorded at the Nodes, a root user can add iptables rules to prevent sessions logs from reaching the Auth Service. With sessions recorded at the proxy, users with root privileges on Nodes have no way of disabling the audit.
See the reference to learn how to turn on the recording proxy mode. Note that the recording mode is configured on the Auth Service.