Fork me on GitHub

Teleport

Kubernetes Access from a Standalone Teleport Cluster

Improve

While you can register a Kubernetes cluster with Teleport by running the Teleport Kubernetes Service on that cluster, you can also run the Teleport Kubernetes Service on a Linux host outside the cluster. This is useful if you want to decouple your Teleport deployment from the Kubernetes clusters you want to manage access to.

In this setup, the Teleport Kubernetes Service uses a kubeconfig file to authenticate to the API server of your chosen Kubernetes cluster.

When running Teleport in production, we recommend that you follow the practices below to avoid security incidents. These practices may differ from the examples used in this guide, which are intended for demo environments:

  • Avoid using sudo in production environments unless it's necessary.
  • Create new, non-root, users and use test instances for experimenting with Teleport.
  • Run Teleport's services as a non-root user unless required. Only the SSH Service requires root access. Note that you will need root permissions (or the CAP_NET_BIND_SERVICE capability) to make Teleport listen on a port numbered < 1024 (e.g. 443).
  • Follow the "Principle of Least Privilege" (PoLP). Don't give users permissive roles when giving them more restrictive roles will do instead. For example, assign users the built-in access,editor roles.
  • When joining a Teleport resource service (e.g., the Database Service or Application Service) to a cluster, save the invitation token to a file. Otherwise, the token will be visible when examining the teleport command that started the agent, e.g., via the history command on a compromised system.

Prerequisites

  • A running Teleport cluster. For details on how to set this up, see one of our Getting Started guides.

  • The tctl admin tool and tsh client tool version >= 11.0.3.

    tctl version

    Teleport v11.0.3 go1.19

    tsh version

    Teleport v11.0.3 go1.19

    See Installation for details.

  • A running Teleport cluster. For details on how to set this up, see our Enterprise Getting Started guide.

  • The tctl admin tool and tsh client tool version >= 11.0.3, which you can download by visiting the customer portal.

    tctl version

    Teleport v11.0.3 go1.19

    tsh version

    Teleport v11.0.3 go1.19

  • A Teleport Cloud account. If you do not have one, visit the sign up page to begin your free trial.

  • The tctl admin tool and tsh client tool version >= 10.3.8. To download these tools, visit the Downloads page.

    tctl version

    Teleport v10.3.8 go1.19

    tsh version

    Teleport v10.3.8 go1.19

  • A Kubernetes cluster you would like to access.
  • A Linux host deployed on your own infrastructure to run the Teleport Kubernetes Service. This can run outside of your Kubernetes cluster.
  • The kubectl command line tool installed on your workstation.

To connect to Teleport, log in to your cluster using tsh, then use tctl remotely:

tsh login --proxy=teleport.example.com [email protected]
tctl status

Cluster teleport.example.com

Version 11.0.3

CA pin sha256:abdc1245efgh5678abdc1245efgh5678abdc1245efgh5678abdc1245efgh5678

You can run subsequent tctl commands in this guide on your local machine.

For full privileges, you can also run tctl commands on your Auth Service host.

To connect to Teleport, log in to your cluster using tsh, then use tctl remotely:

tsh login --proxy=myinstance.teleport.sh [email protected]
tctl status

Cluster myinstance.teleport.sh

Version 10.3.8

CA pin sha256:sha-hash-here

You must run subsequent tctl commands in this guide on your local machine.

Step 1/4. Generate a kubeconfig file

The Teleport Kubernetes Service uses a kubeconfig file to authenticate to your Kubernetes cluster. In this section, we will generate a kubeconfig file so we can configure the Teleport Kubernetes Service to use it later in this guide.

Ensure your context is correct

First, configure your local kubectl command to point at the Kubernetes cluster you want to register. You can use the following command to verify that the correct cluster is selected:

kubectl config get-contexts

Use this command to switch to the cluster assigned to CONTEXT_NAME:

e.g., my-context

CONTEXT_NAME=
kubectl config use-context ${CONTEXT_NAME?}

Run the script

On your workstation, download Teleport's get-kubeconfig.sh script, which you will use to generate the kubeconfig file:

curl -OL \https://raw.githubusercontent.com/gravitational/teleport/v11.0.3/examples/k8s-auth/get-kubeconfig.sh

get-kubeconfig.sh creates a service account for the Teleport Kubernetes Service that can get Kubernetes pods as well as impersonate users, groups, and other service accounts. The Teleport Kubernetes Service uses this service account to manage access to resources in your Kubernetes cluster. The script also ensures that there is a Kubernetes Secret in your cluster to store service account credentials.

get-kubeconfig.sh also creates a namespace called teleport for the resources it deploys, though you can choose a different name by assigning the TELEPORT_NAMESPACE environment variable in the shell where you run the script.

After creating resources, get-kubeconfig.sh writes a new kubeconfig file called kubeconfig in the directory where you run the script.

Run the get-kubeconfig.sh script:

bash get-kubeconfig.sh

The script is successful if you see this message:

Done!

Move the kubeconfig file to the host you are using to run the Teleport Kubernetes Service. We will assume that the kubeconfig file exists at /var/lib/teleport/kubeconfig.

You can connect multiple Kubernetes clusters to Teleport from one kubeconfig file if it contains multiple entries. Use merge-kubeconfigs.sh to combine multiple kubeconfig files generated by get-kubeconfig.sh.

Step 2/4. Set up the Teleport Kubernetes Service

In this step, you will install the Teleport Kubernetes Service and configure it to use the kubeconfig file you generated to access a Kubernetes cluster.

Get a join token

Establish trust between your Teleport cluster and your new Kubernetes Service instance by creating a join token:

tctl nodes add --roles=kube

The invite token: abcd123-insecure-do-not-use-this

This token will expire in 30 minutes.

Run this on the new node to join the cluster:

> teleport start \

--roles=kube \

--token=abcd123-insecure-do-not-use-this \

--ca-pin=sha256:abdc1245efgh5678abdc1245efgh5678abdc1245efgh5678abdc1245efgh5678 \

--auth-server=192.0.2.255:3025

Please note:

- This invitation token will expire in 30 minutes

- 192.0.2.255:3025 must be reachable from the new node

On the host where you are running the Teleport Kubernetes Service, create a file called /tmp/token that consists only of your token:

echo
| sudo tee /tmp/token

Install the Teleport Kubernetes Service

Run the following commands on the host where you will install the Teleport Kubernetes Service:

Teleport Edition

Download Teleport's PGP public key

sudo curl https://apt.releases.teleport.dev/gpg \ -o /usr/share/keyrings/teleport-archive-keyring.asc

Source variables about OS version

source /etc/os-release

Add the Teleport APT repository for v11. You'll need to update this

file for each major release of Teleport.

Note: if using a fork of Debian or Ubuntu you may need to use '$ID_LIKE'

and the codename your distro was forked from instead of '$ID' and '$VERSION_CODENAME'.

Supported versions are listed here: https://github.com/gravitational/teleport/blob/master/build.assets/tooling/cmd/build-os-package-repos/runners.go#L42-L67

echo "deb [signed-by=/usr/share/keyrings/teleport-archive-keyring.asc] \ https://apt.releases.teleport.dev/${ID?} ${VERSION_CODENAME?} stable/v11" \ | sudo tee /etc/apt/sources.list.d/teleport.list > /dev/null

sudo apt-get update
sudo apt-get install teleport

Source variables about OS version

source /etc/os-release

Add the Teleport YUM repository for v11. You'll need to update this

file for each major release of Teleport.

Note: if using a fork of RHEL/CentOS or Amazon Linux you may need to use '$ID_LIKE'

and the codename your distro was forked from instead of '$ID'

Supported versions are listed here: https://github.com/gravitational/teleport/blob/master/build.assets/tooling/cmd/build-os-package-repos/runners.go#L133-L153

sudo yum-config-manager --add-repo $(rpm --eval "https://yum.releases.teleport.dev/$ID/$VERSION_ID/Teleport/%{_arch}/stable/v11/teleport.repo")
sudo yum install teleport

Tip: Add /usr/local/bin to path used by sudo (so 'sudo tctl users add' will work as per the docs)

echo "Defaults secure_path = /sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin" > /etc/sudoers.d/secure_path

Optional: Using DNF on newer distributions

$ sudo dnf config-manager --add-repo https://rpm.releases.teleport.dev/teleport.repo

$ sudo dnf install teleport

curl https://get.gravitational.com/teleport-v11.0.3-linux-amd64-bin.tar.gz.sha256

<checksum> <filename>

curl -O https://get.gravitational.com/teleport-v11.0.3-linux-amd64-bin.tar.gz
shasum -a 256 teleport-v11.0.3-linux-amd64-bin.tar.gz

Verify that the checksums match

tar -xzf teleport-v11.0.3-linux-amd64-bin.tar.gz
cd teleport
sudo ./install
curl https://get.gravitational.com/teleport-v11.0.3-linux-arm-bin.tar.gz.sha256

<checksum> <filename>

curl -O https://get.gravitational.com/teleport-v11.0.3-linux-arm-bin.tar.gz
shasum -a 256 teleport-v11.0.3-linux-arm-bin.tar.gz

Verify that the checksums match

tar -xzf teleport-v11.0.3-linux-arm-bin.tar.gz
cd teleport
sudo ./install
curl https://get.gravitational.com/teleport-v11.0.3-linux-arm64-bin.tar.gz.sha256

<checksum> <filename>

curl -O https://get.gravitational.com/teleport-v11.0.3-linux-arm64-bin.tar.gz
shasum -a 256 teleport-v11.0.3-linux-arm64-bin.tar.gz

Verify that the checksums match

tar -xzf teleport-v11.0.3-linux-arm64-bin.tar.gz
cd teleport
sudo ./install

Using this APT repo may result in breaking upgrades upon "apt upgrade" as all major versions will be

published under the same component. We recommend following the instructions in the

"Debian/Ubuntu (DEB)" tab instead.

Download Teleport's PGP public key

sudo curl https://deb.releases.teleport.dev/teleport-pubkey.asc \ -o /usr/share/keyrings/teleport-archive-keyring.asc

Add the Teleport APT repository

echo "deb [signed-by=/usr/share/keyrings/teleport-archive-keyring.asc] https://deb.releases.teleport.dev/ stable main" \ | sudo tee /etc/apt/sources.list.d/teleport.list > /dev/null

sudo apt-get update
sudo apt-get install teleport
sudo yum-config-manager --add-repo https://rpm.releases.teleport.dev/teleport.repo
sudo yum install teleport

Optional: Using DNF on newer distributions

$ sudo dnf config-manager --add-repo https://rpm.releases.teleport.dev/teleport.repo

$ sudo dnf install teleport

On the host where you will run your Teleport service, install a version of Teleport that is compatible with the Teleport Cloud version, v10.3.8:

curl https://get.gravitational.com/teleport-v10.3.8-linux-amd64-bin.tar.gz.sha256

<checksum> <filename>

curl -O https://get.gravitational.com/teleport-v10.3.8-linux-amd64-bin.tar.gz
shasum -a 256 teleport-v10.3.8-linux-amd64-bin.tar.gz

Verify that the checksums match

tar -xzf teleport-v10.3.8-linux-amd64-bin.tar.gz
cd teleport
sudo ./install

Before installing a teleport binary with a version besides v10.3.8, read our compatibility rules to ensure that the binary is compatible with Teleport Cloud.

When running multiple teleport binaries within a cluster, the following rules apply:

  • Patch and minor versions are always compatible, for example, any 8.0.1 component will work with any 8.0.3 component and any 8.1.0 component will work with any 8.3.0 component.
  • Servers support clients that are 1 major version behind, but do not support clients that are on a newer major version. For example, an 8.x.x Proxy Service is compatible with 7.x.x resource services and 7.x.x tsh, but we don't guarantee that a 9.x.x resource service will work with an 8.x.x Proxy Service. This also means you must not attempt to upgrade from 6.x.x straight to 8.x.x. You must upgrade to 7.x.x first.
  • Proxy Services and resource services do not support Auth Services that are on an older major version, and will fail to connect to older Auth Services by default. This behavior can be overridden by passing --skip-version-check when starting Proxy Services and resource services.

Configure the Teleport Kubernetes Service

On the host where you will run the Teleport Kubernetes Service, create a file at /etc/teleport.yaml with the following content:

version: v3
teleport:
  join_params:
    token_name: "/tmp/token"
    method: token
  proxy_server: "teleport.example.com:3080"
auth_service:
  enabled: off
proxy_service:
  enabled: off
ssh_service:
  enabled: off
kubernetes_service:
  enabled: "yes"
  kubeconfig_file: "/var/lib/teleport/kubeconfig"
  labels:
    "region": "us-east1"

Edit /etc/teleport.yaml to replace teleport.example.com:3080 with the host and port of your Teleport Proxy Service or Teleport Cloud tenant, e.g., mytenant.teleport.sh:443.

Warning

When using kubeconfig_file, Amazon EKS users may need to replace illegal characters in the context names. Supported characters are alphanumeric characters, ., _, and -. EKS typically includes : and @ in their kubeconfig files, which are not allowed in Teleport.

Start the Teleport Kubernetes Service

On the host where you will run the Teleport Kubernetes Service, execute the following command, depending on whether you installed Teleport using a package manager or via a TAR archive:

sudo systemctl start teleport
sudo teleport install systemd --output=/etc/systemd/system/teleport.service;
sudo systemctl enable teleport;
sudo systemctl start teleport;

Step 3/4. Grant access to your Teleport user

Enable your Teleport user to access resources in your Kubernetes cluster so you can connect to the cluster later in this guide.

Kubernetes authentication

To authenticate to a Kubernetes cluster via Teleport, your Teleport roles must allow access as at least one Kubernetes user or group. Ensure that you have a Teleport role that grants access to the cluster you plan to interact with.

Run the following command to get the Kubernetes user for your current context:

kubectl config view \-o jsonpath="{.contexts[?(@.name==\"$(kubectl config current-context)\")].context.user}"

Create a file called kube-access.yaml with the following content, replacing USER with the output of the command above.

kind: role
metadata:
  name: kube-access
version: v5
spec:
  allow:
    kubernetes_labels:
      '*': '*'
    kubernetes_groups:
    - viewers
    kubernetes_users:
    - USER
  deny: {}

Retrieve your Teleport user:

e.g., myuser

TELEPORT_USER=
tctl get user/${TELEPORT_USER?} > user.yaml

Add kube-access to your Teleport user's list of roles:

   roles:
+  - kube-access

Apply your changes:

tctl create -f kube-access.yaml
tctl create -f user.yaml

Now that Teleport RBAC is configured, you can authenticate to your Kubernetes cluster via Teleport. To interact with your Kubernetes cluster, you will need to configure authorization within Kubernetes.

Kubernetes authorization

To configure authorization within your Kubernetes cluster, you need to create Kubernetes RoleBindings or ClusterRoleBindings that grant permissions to the subjects listed in kubernetes_users and kubernetes_groups.

For example, you can grant some limited read-only permissions to the viewers group used in the kube-access role defined above:

Create a file called viewers-bind.yaml with the following contents:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: viewers-crb
subjects:
- kind: Group
  # Bind the group "viewers", corresponding to the kubernetes_groups we assigned our "kube-access" role above
  name: viewers
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  # "view" is a default ClusterRole that grants read-only access to resources
  # See: https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles
  name: view
  apiGroup: rbac.authorization.k8s.io

Apply the ClusterRoleBinding with kubectl:

kubectl apply -f viewers-bind.yaml

Log out of Teleport and log in again.

Step 4/4. Access your Kubernetes cluster

After Teleport starts with the above config, you should be able to see all new clusters:

tsh kube ls

Kube Cluster Name Labels Selected

--------------------------------------- ------ --------

my-cluster region=us-east-1

To access your cluster, run the following command, replacing my-cluster with the name of the cluster you would like to access:

tsh kube login my-cluster

Logged into kubernetes cluster "my-cluster". Try 'kubectl version' to test the connection.