
The Teleport Kubernetes Service is a proxy that sits between Kubernetes users and one or more Kubernetes clusters.
When a user authenticates to Teleport, they receive a kubeconfig that lets them send requests to their authorized Kubernetes clusters via the Teleport Kubernetes Service. The Kubernetes Service can then inspect, modify, or disallow these requests depending on the privileges you have assigned to the Teleport user via their roles.
In this guide, we will use a local Kubernetes cluster to show you how to configure Teleport's role-based access control (RBAC) system to manage access to Kubernetes clusers, groups, users, and resources.
Prerequisites
-
A running Teleport cluster. For details on how to set this up, see one of our Getting Started guides.
-
The
tctl
admin tool andtsh
client tool version >= 12.1.1.tctl versionTeleport v12.1.1 go1.19
tsh versionTeleport v12.1.1 go1.19
See Installation for details.
-
A running Teleport Enterprise cluster. For details on how to set this up, see our Enterprise Getting Started guide.
-
The Enterprise
tctl
admin tool andtsh
client tool version >= 12.1.1, which you can download by visiting the customer portal.tctl versionTeleport Enterprise v12.1.1 go1.19
tsh versionTeleport v12.1.1 go1.19
Please use the latest version of Teleport Enterprise documentation.
To connect to Teleport, log in to your cluster using tsh
, then use tctl
remotely:
tsh login --proxy=teleport.example.com [email protected]tctl statusCluster teleport.example.com
Version 12.1.1
CA pin sha256:abdc1245efgh5678abdc1245efgh5678abdc1245efgh5678abdc1245efgh5678
You can run subsequent tctl
commands in this guide on your local machine.
For full privileges, you can also run tctl
commands on your Auth Service host.
To connect to Teleport, log in to your cluster using tsh
, then use tctl
remotely:
tsh login --proxy=myinstance.teleport.sh [email protected]tctl statusCluster myinstance.teleport.sh
Version 12.1.2
CA pin sha256:sha-hash-here
You must run subsequent tctl
commands in this guide on your local machine.
To run the local demo environment, ensure that you have the following tools installed on your workstation:
Tool | Purpose | Installation link |
---|---|---|
minikube | Local Kubernetes deployment tool | Install minikube |
Helm | Kubernetes package manager | Install Helm |
kubectl | Kubernetes admin CLI | Install kubectl |
Docker | Required minikube driver | Get Started With Docker |
Step 1/3. Prepare Kubernetes resources
Start minikube
Start minikube with the Docker driver:
minikube start --driver=docker
This command should start a local Kubernetes cluster and set your context to
minikube
. To verify this, run the following command:
kubectl config current-contextminikube
Deploy demo pods
On your workstation, create a manifest file called pods.yaml
with the
following content:
apiVersion: v1
kind: Namespace
metadata:
name: development
labels:
name: development
---
apiVersion: v1
kind: Namespace
metadata:
name: production
labels:
name: production
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: webapp
namespace: development
spec:
selector:
matchLabels:
app: nginx-webapp
template:
metadata:
labels:
app: nginx-webapp
spec:
containers:
- name: nginx
image: nginx:1.23
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: webapp
namespace: production
spec:
selector:
matchLabels:
app: nginx-webapp
template:
metadata:
labels:
app: nginx-webapp
spec:
containers:
- name: nginx
image: nginx:1.23
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: loadbalancer
namespace: development
spec:
selector:
matchLabels:
app: nginx-loadbalancer
template:
metadata:
labels:
app: nginx-loadbalancer
spec:
containers:
- name: nginx
image: nginx:1.23
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: loadbalancer
namespace: production
spec:
selector:
matchLabels:
app: nginx-loadbalancer
template:
metadata:
labels:
app: nginx-loadbalancer
spec:
containers:
- name: nginx
image: nginx:1.23
This manifest creates two namespaces, development
and production
, and
deploys two nginx
pods into each one: webapp
and loadbalancer
. Apply the
new resources:
kubectl apply -f pods.yaml
Ensure that the resources are deployed:
kubectl -n development get podskubectl -n production get pods
You should see both the loadbalancer
and webapp
pods in each namespace.
Install Kubernetes RBAC resources
Now that we have deployed our webapp
and loadbalancer
pods in our
development
and production
namespaces, we will create a Kubernetes role that
can view all pods in all namespaces. Later in this guide, we will define a
Teleport role that further restricts the access Teleport users can have to
resources in your cluster.
Create a manifest file called k8s-rbac.yaml
with the following content:
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: pod-viewer
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "watch", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: pod-viewer
subjects:
- kind: Group
name: developers
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: ClusterRole
name: pod-viewer
apiGroup: rbac.authorization.k8s.io
Apply your changes:
kubectl apply -f k8s-rbac.yaml
Install the Teleport Kubernetes Service
Now that you have some workloads running on Kubernetes and RBAC resources to manage access to them, install the Teleport Kubernetes Service in your demo cluster so you can get more control into the resources that Kubernetes users can access.
To allow Helm to install charts that are hosted in the Teleport Helm repository, use helm repo add
:
helm repo add teleport https://charts.releases.teleport.dev
To update the cache of charts from the remote repository, run helm repo update
:
helm repo update
Request a token that the Kubernetes Service will use to join your Teleport cluster:
tctl nodes add --roles=kube --ttl=1h --format=json
Copy this token so you can use it when running the Teleport Kubernetes Service.
Install the Teleport Kubernetes Service in your cluster, assigning proxy-address to the host and port of your Teleport Proxy Service
(e.g., mytenant.teleport.sh:443
) and token to the token you
requested earlier:
helm install teleport-agent teleport/teleport-kube-agent \ --set kubeClusterName=minikube \ --set proxyAddr=proxy-address \ --set authToken=token \ --set labels.region=local --set labels.platform=minikube \ --create-namespace \ --namespace=teleport \ --version 12.1.1
This tctl nodes add
command supplies the soon-to-be-added Kubernetes Service
instance with two labels: region:local
and platform:minikube
. We will use
these to configure access controls for the cluster later in this guide.
Verify that the teleport
pod is running in your cluster:
kubectl -n teleport get pods
You can check that the Teleport Kubernetes Service registered itself with your Teleport cluster by executing the following command:
tctl get kube_servers
The output should resemble the following:
kind: kube_server
metadata:
expires: "2023-01-24T16:20:00.571214635Z"
id: 0000000000000000000
name: minikube
spec:
cluster:
kind: kube_cluster
metadata:
labels:
platform: minikube
region: local
name: minikube
spec:
aws: {}
azure: {}
gcp: {}
version: v3
host_id: 00000000-0000-0000-0000-000000000000
hostname: remote.kube.proxy.teleport.cluster.local
rotation:
current_id: ""
last_rotated: "0001-01-01T00:00:00Z"
schedule:
standby: "0001-01-01T00:00:00Z"
update_clients: "0001-01-01T00:00:00Z"
update_servers: "0001-01-01T00:00:00Z"
started: "0001-01-01T00:00:00Z"
version: 12.1.1
version: v3
Step 2/3. Define a Teleport role
The Teleport Kubernetes Service determines how to proxy a Teleport user's requests to a Kubernetes API server by looking up the user's roles. Based on this information, the Kubernetes Service accepts or denies the request.
For valid requests, the Kubernetes Service rewrites the request headers to impersonate the Teleport user's desired Kubernetes user and groups, and forwards the request to the API server.
In this section, we will define a Teleport role that:
- Authenticates the user to a Kubernetes cluster as a member of the
developers
group. In the previous section, we authorized members of this group to view pods in all namespaces. - Enables the user to access
webapp
pods in theproduction
namespace and all pods in thedevelopment
namespace. - Denies the user access to all other pods.
Define a role
Create a file called kube-access.yaml
with the following content:
kind: role
metadata:
name: kube-access
version: v6
spec:
allow:
kubernetes_labels:
'region': '*'
'platform': 'minikube'
kubernetes_resources:
- kind: pod
namespace: "production"
name: "^webapp-[a-z0-9-]+$"
- kind: pod
namespace: "development"
name: "*"
kubernetes_groups:
- developers
kubernetes_users:
- minikube
deny: {}
In this role, we have defined the following allow
rules:
kubernetes_labels
: Allows access to Kubernetes clusters in all regions, but only with theplatform:minikube
label.kubernetes_resources
: Allows access to pods in thewebapp
deployment in theproduction
namespace and all pods in thedevelopment
namespace. Note the use of a regular expression (beginning^
and ending in$
) to match pod names that Kubernetes generates automatically.kubernetes_groups
: Authenticates the user to your Kubernetes cluster as a member of the Kubernetes groupdevelopers
, which we associated with thepod-viewer
KubernetesRole
earlier in this guide.kubernetes_users
: Authenticates the user to your Kubernetes cluster as the defaultminikube
user.
Create the role
Once you have finished configuring the kube-access
role, create it using the
following command:
tctl create kube-access.yaml
Assign the kube-access
role to your Teleport user by running the following
commands, depending on whether you authenticate as a local Teleport user or via
the github
, saml
, or oidc
authentication connectors:
Retrieve your local user's configuration resource:
tctl get users/$(tsh status -f json | jq -r '.active.username') > out.yaml
Edit out.yaml
, adding kube-access
to the list of existing roles:
roles:
- access
- auditor
- editor
+ - kube-access
Apply your changes:
tctl create -f out.yaml
Retrieve your github
configuration resource:
tctl get github/github --with-secrets > github.yaml
Edit github.yaml
, adding kube-access
to the
teams_to_roles
section. The team you will map to this role will depend on how
you have designed your organization's RBAC, but it should be the smallest team
possible within your organization. This team must also include your user.
Here is an example:
teams_to_roles:
- organization: octocats
team: admins
roles:
- access
+ - kube-access
Apply your changes:
tctl create -f github.yaml
Note the --with-secrets
flag in the tctl get
command. This adds the value of
spec.signing_key_pair.private_key
to saml.yaml
. This is a sensitive value,
so take precautions when creating this file and remove it after updating the resource.
Retrieve your saml
configuration resource:
tctl get --with-secrets saml/mysaml > saml.yaml
Edit saml.yaml
, adding kube-access
to the
attributes_to_roles
section. The attribute you will map to this role will
depend on how you have designed your organization's RBAC, but it should be the
smallest group possible within your organization. This group must also include
your user.
Here is an example:
attributes_to_roles:
- name: "groups"
value: "my-group"
roles:
- access
+ - kube-access
Apply your changes:
tctl create -f saml.yaml
Note the --with-secrets
flag in the tctl get
command. This adds the value of
spec.signing_key_pair.private_key
to saml.yaml
. This is a sensitive value,
so take precautions when creating this file and remove it after updating the resource.
Retrieve your oidc
configuration resource:
tctl get oidc/myoidc --with-secrets > oidc.yaml
Edit oidc.yaml
, adding kube-access
to the
claims_to_roles
section. The claim you will map to this role will depend on
how you have designed your organization's RBAC, but it should be the smallest
group possible within your organization. This group must also include your
user.
Here is an example:
claims_to_roles:
- name: "groups"
value: "my-group"
roles:
- access
+ - kube-access
Apply your changes:
tctl create -f saml.yaml
Note the --with-secrets
flag in the tctl get
command. This adds the value of
spec.signing_key_pair.private_key
to saml.yaml
. This is a sensitive value,
so take precautions when creating this file and remove it after updating the resource.
Log out of your Teleport cluster and log in again to assume the new role.
Step 3/3. Access resources
At this point, you have configured the Teleport Kubernetes Service to give your
Teleport user access to the webapp
pod in the production
namespace. In this
step, we will authenticate to your Kubernetes cluster via Teleport and test our
new access controls.
List the Kubernetes clusters you can access via Teleport:
tsh kube ls
You should see the minikube
cluster you registered earlier:
Kube Cluster Name Labels Selected
----------------- ------------------------------ --------
minikube platform=minikube region=local
To access your Kubernetes cluster via Teleport, authenticate to it and update your kubeconfig:
tsh kube login minikube
When listing pods in all namespaces, the Teleport Kubernetes Service will filter the pods it retrieves to show only those that your Teleport user can access. Run the following command:
kubectl get pods --all-namespaces
The output will show the webapp
pod in the production
namespace and both the
webapp
and loadbalancer
pods in the development
namespace:
NAMESPACE NAME READY STATUS RESTARTS AGE
development loadbalancer-000000000-00000 1/1 Running 0 36m
development webapp-0000000000-00000 1/1 Running 0 36m
production webapp-0000000000-00000 1/1 Running 0 36m
You can access information about the webapp
pod in the production
namespace:
kubectl -n production get pods/webapp-0000000000-00000 -o json
Also note that the kube-access
role we created earlier mapped your Teleport
user to the developers
Kubernetes group, which has permissions only to view
pods:
kubectl auth can-i create podsno
By configuring Teleport roles and Kubernetes RBAC resources, you can fine-tune the access that users in your organization have to your Kubernetes-based infrastructure.
When you authenticated to your minikube
cluster via tsh kube login
, Teleport
generated a kubeconfig that connects to your cluster via Teleport:
kubectl config current-contextteleport.example.com-minikube
If you want to regain full control of your minikube
cluster, you can use the
default minikube
context instead:
kubectl config use-context minikube
Next steps
For more detailed information on how Teleport RBAC for Kubernetes works, consult
the Kubernetes Access Controls Guide. You can leave your
minikube
cluster running so you can try out different Teleport and Kubernetes
RBAC configurations.
Now that you know how to configure Teleport's RBAC system to control access to Kubernetes clusters, learn how to set up Resource Access Requests for just-in-time access and Access Request plugins so you can manage access with your communication workflow of choice.