Production Guide

This guide provides more in-depth details for running Teleport in Production.

Prerequisites

• Read the Architecture Overview
• Read through the Installation Guide to see the available packages and binaries available.
• Read the CLI Docs for teleport

Designing your cluster

Before installing anything there are a few things you should think about.

• Where will you host Teleport?
  • On-premises
  • Cloud VMs such as AWS EC2 or GCE
  • An existing Kubernetes Cluster
• What does your existing network configuration look like?
  • Are you able to administer the network firewall rules yourself or do you need to work with a network admin?
  • Are these nodes accessible to the public Internet or behind NAT?
• Which users, (Roles or ClusterRoles on k8s) are set up on the existing system?
  • Can you add new users or Roles yourself or do you need to work with a system administrator?

Firewall configuration

Teleport services listen on several ports. This table shows the default port numbers.

Installation

We have a detailed installation guide which shows how to install all available binaries or install from source. Reference that guide to learn the best way to install Teleport for your system, then return to finish your production install.

Filesystem layout

By default, a Teleport node has the following files present. The location of all of them is configurable.

Running Teleport in production

Systemd unit file

In production, we recommend starting teleport daemon via an init system like systemd. If systemd and unit files are new to you, check out this helpful guide. Here's an example systemd unit file for the Teleport Proxy, Node, and Auth Service.

There are a couple of important things to notice about this file:

1. The start command in the unit file specifies --config as a file and there are very few flags passed to the teleport binary. Most of the configuration for Teleport should be done in the configuration file.
2. The ExecReload command allows admins to run systemctl reload teleport. This will attempt to perform a graceful restart of Teleport but it only works if network-based backend storage like DynamoDB or etc 3.3 is configured. Graceful Restarts will fork a new process to handle new incoming requests and leave the old daemon process running until existing clients disconnect.

Start the Teleport service

You can start Teleport via systemd unit by enabling the .service file with the systemctl tool.

# sudo cp teleport/examples/systemd/teleport.service /etc/systemd/system
cd /etc/systemd/system

# Use your text editor of choice to create the .service file
# Here we use vim
vi teleport.service

# Use the file linked above as is or customize as needed
# save the file
systemctl enable teleport
systemctl start teleport

# Show the status of the unit
systemctl status teleport

# Follow the tail of service logs
journalctl -fu teleport

# If you modify teleport.service later you will need to
# reload the systemctl daemon and reload teleport
# to apply your changes
systemctl daemon-reload
systemctl reload teleport

You can also perform restarts or upgrades by sending kill signals to a Teleport daemon manually.

Adding Nodes to the cluster

We've written a dedicated guide on Adding Nodes to your Cluster which shows how to generate or set join tokens and use them to add nodes.

Security considerations

SSL/TLS for Teleport Proxy

TLS stands for Transport Layer Security (TLS), and its now-deprecated predecessor, Secure Sockets Layer (SSL). Teleport requires TLS authentication to ensure that communication between nodes, clients and web proxy remains secure and comes from a trusted source.

During our Getting Started Guide guide we skip over setting up TLS so that you can quickly try Teleport. Obtaining a TLS certificate is easy and is free with thanks to Let's Encrypt.

If you use certbot, you get this list of files provided:

README
cert.pem
chain.pem
fullchain.pem
privkey.pem

The files that are needed for Teleport are these:

# proxy service
# ...
https_keypairs:
- key_file: /var/lib/teleport/webproxy_key.pem
  cert_file: /var/lib/teleport/webproxy_cert.pem

If you already have a certificate these should be uploaded to the Teleport Proxy and can be set via https_key_file and https_cert_file. Make sure any certificate files uploaded contain a full certificate chain, complete with any intermediate certificates required.

# This section configures the 'proxy service'
proxy_service:
    # Turns 'proxy' role on. Default is 'yes'
    enabled: yes

    # The DNS names the proxy HTTPS endpoint as accessible by cluster users.
    # Defaults to the proxy's hostname if not specified. If running multiple
    # proxies behind a load balancer, this name must point to the load balancer
    # (see public_addr section below)
    public_addr: proxy.example.com:3080

    # TLS certificate for the HTTPS connection. Configuring these properly is
    # critical for Teleport security.
    https_keypairs:
    - key_file: /var/lib/teleport/webproxy_key.pem
      cert_file: /var/lib/teleport/webproxy_cert.pem

When setting up on Teleport on AWS or GCP, we recommend leveraging their certificate managers.

• ACM on AWS
• Google-managed SSL certificates on GCP

When setting up Teleport with a Cloud Provider, it can be common to terminate TLS at the load balancer, then use an autoscaling group for the proxy nodes. When setting up the proxy nodes start Teleport with:

teleport start --insecure --roles=proxy --config=/etc/teleport.yaml

See Teleport Proxy High Availability for more info.

CA pinning

Teleport nodes use the HTTPS protocol to offer the join tokens to the auth server. In a zero-trust environment, you must assume that an attacker can hijack the IP address of the auth server.

To prevent this from happening, you need to supply every new node with an additional bit of information about the auth server. This technique is called CA pinning. It works by asking the auth server to produce a CA pin, which is a hashed value of its private key, i.e. it cannot be forged by an attacker.

To get the current CA pin run the following code on the auth server:

tctl status

# Output
Cluster  staging.example.com
User CA  never updated
Host CA  never updated
CA pin   sha256:7e12c17c20d9cb504bbcb3f0236be3f446861f1396dcbb44425fe28ec1c108f1

The CA pin at the bottom needs to be passed to the new nodes when they're starting for the first time, i.e. when they join a cluster:

Via CLI:

sudo teleport start \
   --roles=node \
   --token=/path/to/token.file \
   --ca-pin=sha256:7e12c17c20d9cb504bbcb3f0236be3f446861f1396dcbb44425fe28ec1c108f1 \
   --auth-server=10.12.0.6:3025

or via /etc/teleport.yaml on a node:

teleport:
  auth_token: "1ac590d36493acdaa2387bc1c492db1a"
  ca_pin: "sha256:7e12c17c20d9cb504bbcb3f0236be3f446861f1396dcbb44425fe28ec1c108f1"
  auth_servers:
    - "10.12.0.6:3025"

Secure data storage

By default, the teleport daemon uses the local directory /var/lib/teleport to store its data. This applies to any role or service including auth, node, or proxy. While an auth node hosts the most sensitive data you will want to prevent unauthorized access to this directory. Make sure that regular/non-admin users do not have access to this folder, particularly on the auth server. Change the ownership of the directory with chown

sudo teleport start

If you are logged in as root you may want to create a new OS-level user first. On Linux, create a new user called <username> with the following commands:

adduser <username>
su <username>