Setting Up Teleport Access Controls for Kubernetes

The Teleport Kubernetes Service is a proxy that sits between Kubernetes users and one or more Kubernetes clusters.

When a user authenticates to Teleport, they receive a kubeconfig that lets them send requests to their authorized Kubernetes clusters via the Teleport Kubernetes Service. The Kubernetes Service can then inspect, modify, or disallow these requests depending on the privileges you have assigned to the Teleport user via their roles.

In this guide, we will use a local Kubernetes cluster to show you how to configure Teleport's role-based access control (RBAC) system to manage access to Kubernetes clusers, groups, users, and resources.

Prerequisites

tsh login --proxy=teleport.example.com [email protected]
tctl status
Cluster  teleport.example.com
Version  12.1.1
CA pin   sha256:abdc1245efgh5678abdc1245efgh5678abdc1245efgh5678abdc1245efgh5678

tsh login --proxy=myinstance.teleport.sh [email protected]
tctl status
Cluster  myinstance.teleport.sh
Version  12.1.2
CA pin   sha256:sha-hash-here

To run the local demo environment, ensure that you have the following tools installed on your workstation:

Step 1/3. Prepare Kubernetes resources

Start minikube

Start minikube with the Docker driver:

minikube start --driver=docker

This command should start a local Kubernetes cluster and set your context to minikube. To verify this, run the following command:

kubectl config current-context
minikube

Deploy demo pods

On your workstation, create a manifest file called pods.yaml with the following content:

apiVersion: v1
kind: Namespace
metadata:
  name: development
  labels:
    name: development
---
apiVersion: v1
kind: Namespace
metadata:
  name: production
  labels:
    name: production
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: webapp
  namespace: development
spec:
  selector:
    matchLabels:
      app: nginx-webapp
  template:
    metadata:
      labels:
        app: nginx-webapp
    spec:
      containers:
        - name: nginx
          image: nginx:1.23
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: webapp
  namespace: production
spec:
  selector:
    matchLabels:
      app: nginx-webapp
  template:
    metadata:
      labels:
        app: nginx-webapp
    spec:
      containers:
        - name: nginx
          image: nginx:1.23
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: loadbalancer
  namespace: development
spec:
  selector:
    matchLabels:
      app: nginx-loadbalancer
  template:
    metadata:
      labels:
        app: nginx-loadbalancer
    spec:
      containers:
        - name: nginx
          image: nginx:1.23
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: loadbalancer
  namespace: production
spec:
  selector:
    matchLabels:
      app: nginx-loadbalancer
  template:
    metadata:
      labels:
        app: nginx-loadbalancer
    spec:
      containers:
        - name: nginx
          image: nginx:1.23

This manifest creates two namespaces, development and production, and deploys two nginx pods into each one: webapp and loadbalancer. Apply the new resources:

kubectl apply -f pods.yaml

Ensure that the resources are deployed:

kubectl -n development get pods
kubectl -n production get pods

You should see both the loadbalancer and webapp pods in each namespace.

Install Kubernetes RBAC resources

Now that we have deployed our webapp and loadbalancer pods in our development and production namespaces, we will create a Kubernetes role that can view all pods in all namespaces. Later in this guide, we will define a Teleport role that further restricts the access Teleport users can have to resources in your cluster.

Create a manifest file called k8s-rbac.yaml with the following content:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: pod-viewer
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "watch", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: pod-viewer
subjects:
- kind: Group
  name: developers
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: pod-viewer
  apiGroup: rbac.authorization.k8s.io

Apply your changes:

kubectl apply -f k8s-rbac.yaml

Install the Teleport Kubernetes Service

Now that you have some workloads running on Kubernetes and RBAC resources to manage access to them, install the Teleport Kubernetes Service in your demo cluster so you can get more control into the resources that Kubernetes users can access.

To allow Helm to install charts that are hosted in the Teleport Helm repository, use helm repo add:

helm repo add teleport https://charts.releases.teleport.dev

To update the cache of charts from the remote repository, run helm repo update:

helm repo update

Request a token that the Kubernetes Service will use to join your Teleport cluster:

tctl nodes add --roles=kube --ttl=1h --format=json

Copy this token so you can use it when running the Teleport Kubernetes Service.

Install the Teleport Kubernetes Service in your cluster, assigning proxy-address to the host and port of your Teleport Proxy Service (e.g., mytenant.teleport.sh:443) and token to the token you requested earlier:

helm install teleport-agent teleport/teleport-kube-agent \  --set kubeClusterName=minikube \  --set proxyAddr=proxy-address \  --set authToken=token \  --set labels.region=local --set labels.platform=minikube \  --create-namespace \  --namespace=teleport \  --version 12.1.1

This tctl nodes add command supplies the soon-to-be-added Kubernetes Service instance with two labels: region:local and platform:minikube. We will use these to configure access controls for the cluster later in this guide.

Verify that the teleport pod is running in your cluster:

kubectl -n teleport get pods

You can check that the Teleport Kubernetes Service registered itself with your Teleport cluster by executing the following command:

tctl get kube_servers

The output should resemble the following:

kind: kube_server
metadata:
  expires: "2023-01-24T16:20:00.571214635Z"
  id: 0000000000000000000
  name: minikube
spec:
  cluster:
    kind: kube_cluster
    metadata:
      labels:
        platform: minikube
        region: local
      name: minikube
    spec:
      aws: {}
      azure: {}
      gcp: {}
    version: v3
  host_id: 00000000-0000-0000-0000-000000000000
  hostname: remote.kube.proxy.teleport.cluster.local
  rotation:
    current_id: ""
    last_rotated: "0001-01-01T00:00:00Z"
    schedule:
      standby: "0001-01-01T00:00:00Z"
      update_clients: "0001-01-01T00:00:00Z"
      update_servers: "0001-01-01T00:00:00Z"
    started: "0001-01-01T00:00:00Z"
  version: 12.1.1
version: v3

Step 2/3. Define a Teleport role

The Teleport Kubernetes Service determines how to proxy a Teleport user's requests to a Kubernetes API server by looking up the user's roles. Based on this information, the Kubernetes Service accepts or denies the request.

For valid requests, the Kubernetes Service rewrites the request headers to impersonate the Teleport user's desired Kubernetes user and groups, and forwards the request to the API server.

In this section, we will define a Teleport role that:

• Authenticates the user to a Kubernetes cluster as a member of the developers group. In the previous section, we authorized members of this group to view pods in all namespaces.
• Enables the user to access webapp pods in the production namespace and all pods in the development namespace.
• Denies the user access to all other pods.

Define a role

Create a file called kube-access.yaml with the following content:

kind: role
metadata:
  name: kube-access
version: v6
spec:
  allow:
    kubernetes_labels:
      'region': '*'
      'platform': 'minikube'
    kubernetes_resources:
      - kind: pod
        namespace: "production"
        name: "^webapp-[a-z0-9-]+$"
      - kind: pod
        namespace: "development"
        name: "*"
    kubernetes_groups:
    - developers
    kubernetes_users:
    - minikube
  deny: {}

In this role, we have defined the following allow rules:

• kubernetes_labels: Allows access to Kubernetes clusters in all regions, but only with the platform:minikube label.
• kubernetes_resources: Allows access to pods in the webapp deployment in the production namespace and all pods in the development namespace. Note the use of a regular expression (beginning ^ and ending in $) to match pod names that Kubernetes generates automatically.
• kubernetes_groups: Authenticates the user to your Kubernetes cluster as a member of the Kubernetes group developers, which we associated with the pod-viewer Kubernetes Role earlier in this guide.
• kubernetes_users: Authenticates the user to your Kubernetes cluster as the default minikube user.

Create the role

Once you have finished configuring the kube-access role, create it using the following command:

tctl create kube-access.yaml

Assign the kube-access role to your Teleport user by running the following commands, depending on whether you authenticate as a local Teleport user or via the github, saml, or oidc authentication connectors:

tctl get users/$(tsh status -f json | jq -r '.active.username') > out.yaml

  roles:
   - access
   - auditor
   - editor
+  - kube-access

tctl create -f out.yaml

tctl get github/github --with-secrets > github.yaml

  teams_to_roles:
    - organization: octocats
      team: admins
      roles:
        - access
+       - kube-access

tctl create -f github.yaml

tctl get --with-secrets saml/mysaml > saml.yaml

  attributes_to_roles:
    - name: "groups"
      value: "my-group"
      roles:
        - access
+       - kube-access

tctl create -f saml.yaml

tctl get oidc/myoidc --with-secrets > oidc.yaml

  claims_to_roles:
    - name: "groups"
      value: "my-group"
      roles:
        - access
+       - kube-access

tctl create -f saml.yaml

Log out of your Teleport cluster and log in again to assume the new role.

Step 3/3. Access resources

At this point, you have configured the Teleport Kubernetes Service to give your Teleport user access to the webapp pod in the production namespace. In this step, we will authenticate to your Kubernetes cluster via Teleport and test our new access controls.

List the Kubernetes clusters you can access via Teleport:

tsh kube ls

You should see the minikube cluster you registered earlier:

Kube Cluster Name Labels                         Selected
----------------- ------------------------------ --------
minikube          platform=minikube region=local

To access your Kubernetes cluster via Teleport, authenticate to it and update your kubeconfig:

tsh kube login minikube

When listing pods in all namespaces, the Teleport Kubernetes Service will filter the pods it retrieves to show only those that your Teleport user can access. Run the following command:

kubectl get pods --all-namespaces

The output will show the webapp pod in the production namespace and both the webapp and loadbalancer pods in the development namespace:

NAMESPACE     NAME                           READY   STATUS    RESTARTS   AGE
development   loadbalancer-000000000-00000   1/1     Running   0          36m
development   webapp-0000000000-00000        1/1     Running   0          36m
production    webapp-0000000000-00000        1/1     Running   0          36m

You can access information about the webapp pod in the production namespace:

kubectl -n production get pods/webapp-0000000000-00000 -o json

Also note that the kube-access role we created earlier mapped your Teleport user to the developers Kubernetes group, which has permissions only to view pods:

kubectl auth can-i create pods
no

By configuring Teleport roles and Kubernetes RBAC resources, you can fine-tune the access that users in your organization have to your Kubernetes-based infrastructure.

When you authenticated to your minikube cluster via tsh kube login, Teleport generated a kubeconfig that connects to your cluster via Teleport:

kubectl config current-context
teleport.example.com-minikube

If you want to regain full control of your minikube cluster, you can use the default minikube context instead:

kubectl config use-context minikube

Next steps

For more detailed information on how Teleport RBAC for Kubernetes works, consult the Kubernetes Access Controls Guide. You can leave your minikube cluster running so you can try out different Teleport and Kubernetes RBAC configurations.

Now that you know how to configure Teleport's RBAC system to control access to Kubernetes clusters, learn how to set up Resource Access Requests for just-in-time access and Access Request plugins so you can manage access with your communication workflow of choice.