Authenticating AWS EKS Kubernetes Clusters with Okta SSO

In this article on securing access to Kubernetes with Okta, we will explore how Teleport brings an additional layer of security to clusters based on Elastic Kubernetes Service (EKS) managed service from Amazon Web Services (AWS).

Before proceeding further, ensure you have configured the Teleport authentication server to work with Okta SSO. Refer to the guide and the documentation for the setup.

This guide will use a configured Okta account. The configuration will use the domain cloudnativelabs.in while the Teleport proxy and authentication server are associated with the proxy.teleport-demo.in domain.

Step 1 - Provisioning an Amazon EKS cluster

This tutorial will launch an EKS cluster with three Ubuntu nodes and the most recent version of Kubernetes. You can replace the parameters such as the region and node type. Download the CLI, eksctl, and configure AWS CLI to follow along.

touch eks-config
export KUBECONFIG=$PWD/eks-config

eksctl create cluster \
	--name cluster1 \
	--region ap-south-1 \
	--version 1.23 \
	--nodegroup-name ng-workers \
	--node-type t3.medium \
	--nodes 3 \
	--nodes-min 3 \
	--nodes-max 6 \
	--node-ami-family Ubuntu2004 \
	--set-kubeconfig-context=true

When the provisioning is complete, the kubeconfig contents are written to eks-config file in the current directory. You can list the nodes and verify the access to the cluster.

kubectl get nodes
NAME                                            STATUS   ROLES    AGE   VERSION
ip-192-168-23-11.ap-south-1.compute.internal    Ready    <none>   31m   v1.23.15
ip-192-168-52-137.ap-south-1.compute.internal   Ready    <none>   32m   v1.23.15
ip-192-168-87-12.ap-south-1.compute.internal    Ready    <none>   32m   v1.23.15

Step 2 - Registering Amazon EKS Kubernetes cluster with Teleport

Similar to other configurations, such as servers, Teleport expects an agent to run within the target cluster. This agent can be installed through a Helm chart by pointing it to the Teleport proxy server endpoint.

Before proceeding further, we need to get the token responsible for validating the agent. Run the below command to create the token based on the Kubernetes role. Make sure you log in to Teleport as a user with roles editor and access.

tsh --proxy=proxy.teleport-demo.in login --auth=local --user=tele-admin
Enter password for Teleport user tele-admin:
Enter your OTP token:
> Profile URL:        https://proxy.teleport-demo.in:443
  Logged in as:       tele-admin
  Cluster:            proxy.teleport-demo.in
  Roles:              access, editor
  Logins:             root, janakiramm, -teleport-internal-join
  Kubernetes:         enabled
  Valid until:        2023-03-12 00:33:56 +0530 IST [valid for 12h0m0s]
  Extensions:         client-ip, permit-agent-forwarding, permit-port-forwarding, permit-pty, private-key-policy

TOKEN=$(tctl nodes add --roles=kube --ttl=10000h --format=json | jq -r '.[0]') 

The next step is to add Teleport’s repo and update Helm, which provides us access to the Helm chart.

helm repo add teleport https://charts.releases.teleport.dev
helm repo update

The below environment variables contain key parameters needed by the Helm chart.

PROXY_ADDR=proxy.teleport-demo.in:443
CLUSTER=eks-cluster

helm install teleport-agent teleport/teleport-kube-agent \
  --set kubeClusterName=${CLUSTER?} \
  --set proxyAddr=${PROXY_ADDR?} \
  --set authToken=${TOKEN?} \
  --create-namespace \
  --namespace=teleport-agent \
  --version 12.1.0 

Wait for a few minutes, and check if the agent pod is up and running within the teleport-agent namespace.

kubectl get pods -n teleport-agent
NAME               READY   STATUS    RESTARTS   AGE
teleport-agent-0   1/1     Running   0          47s

You can also verify the logs with the below command:

kubectl logs teleport-agent-0 -n teleport-agent

Step 3 - Configuring Okta as SSO provider for Amazon EKS Kubernetes Cluster

Kubernetes uses an identity embedded within the kubeconfig file to access the cluster. We need to add that identity to Teleport’s kubernetes_users role for the user to assume the role.

First, let’s get the current user from the kubeconfig file.

kubectl config view -o jsonpath="{.contexts[?(@.name==\"$(kubectl config current-context)\")].context.user}"

By default, EKS creates a user based on the format IAM_USER@CLUSTER_NAME.REGION.eksctl.io. Let’s tell Teleport that this user will assume the role of kubernetes_users by creating an RBAC definition in a file by name kube-access.yaml and applying it.

kind: role
metadata:
  name: kube-access
version: v5
spec:
  allow:
    kubernetes_labels:
      '*': '*'
    kubernetes_groups:
    - viewers
    kubernetes_users:
    - [email protected]
  deny: {}

tctl create -f kube-access.yaml
role 'kube-access' has been created

After this, we must update the ODIC connector created in the previous tutorial. This step ensures that users belonging to specific groups within the Okta directory can gain access to the cluster.

Let’s update the Okta connector definition to add auditor and kube-access role to the okta-admin group.

Notice how the okta-admin group configured in Okta is mapped to the Teleport roles.

Overwrite the connector configuration by applying it again.

tctl create -f okta-sso.yaml
authentication connector 'okta' has been created

Finally, we need to create a cluster-wide role binding in Kubernetes to bind the groups defined in Teleport to the local roles. Since the Teleport RBAC definition only has viewers under the kubernetes_groups section, let’s bind that role with the view cluster role in Kubernetes.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: viewers-crb
subjects:
- kind: Group
  name: viewers
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: view
  apiGroup: rbac.authorization.k8s.io

kubectl apply -f viewers-bind.yaml

This step essentially closes the loop by mapping Teleport roles with Kubernetes cluster roles.

Step 4 - Accessing EKS clusters through Okta Identity

It’s time to test the configuration by signing into Teleport as an Okta user and then using the tsh CLI to list the registered clusters.

tsh --proxy=proxy.teleport-demo.in login --auth=okta

This opens up the browser window for entering the Okta credentials. Once you log in, the CLI confirms your identity.

tsh --proxy=proxy.teleport-demo.in login --auth=okta --browser=none
> Profile URL:        https://proxy.teleport-demo.in:443
  Logged in as:       [email protected]
  Cluster:            proxy.teleport-demo.in
  Roles:              auditor, editor, kube-access
  Logins:             -teleport-nologin-98e7073f-959e-4443-9867-8084a20d18c3, -teleport-internal-join
  Kubernetes:         enabled
  Kubernetes users:   janakiramm@cluster1.ap-south-1.eksctl.io
  Kubernetes groups:  viewers
  Valid until:        2023-03-12 00:56:48 +0530 IST [valid for 12h0m0s]
  Extensions:         permit-agent-forwarding, permit-port-forwarding, permit-pty, private-key-policy

Notice that the current user is [email protected]. Now, let’s list the registered Kubernetes clusters and log onto EKS cluster which is running in the AWS cloud.

tsh kube ls
Kube Cluster Name Labels Selected
----------------- ------ --------
eks-cluster

Logged into Kubernetes cluster "eks-cluster". Try 'kubectl version' to test the connection.

At this point, the Teleport client has added another context to the original kubeconfig file. You can verify it by running the following command:

kubectl config view

Notice the current context pointing to proxy.teleport-demo.in-cluster1. You can continue to use the standard Kubernetes client CLI, kubectl, to access the cluster through Teleport’s proxy server transparently.

Let’s try to list all the pods running in the kube-system namespace.

kubectl get pods -n kube-system
NAME                       READY   STATUS    RESTARTS   AGE
aws-node-8qtcw             1/1     Running   0          61m
aws-node-97296             1/1     Running   0          61m
aws-node-qj9xs             1/1     Running   0          60m
coredns-5b88d66b9c-2mwnx   1/1     Running   0          71m
coredns-5b88d66b9c-ctjpj   1/1     Running   0          71m
kube-proxy-8rvpl           1/1     Running   0          61m
kube-proxy-mtkwq           1/1     Running   0          61m
kube-proxy-xtstc           1/1     Running   0          60m

Since the Kubernetes role has view permission, we can list the pods. To verify the policy, let’s try to create a namespace.

kubectl create ns test
Error from server (Forbidden): namespaces is forbidden: User "default" cannot create resource "namespaces" in API group "" at the cluster scope

This results in errors due to the lack of permission to create resources.

Since the action violates the access policy defined by the RBAC, it is also visible in Teleport audit logs.

The details clearly explain that a POST request has been made to the Kubernetes API server attempting to create the namespace, which was declined.

Pod Level RBAC

Teleport role also supports per-pod level RBAC, Allowing access to pods in specific production namespace and all pods in the development namespace.

We can define fine-grained RBAC policies that map the users from the Okta groups to Teleport users to Kubernetes roles. This gives ultimate control to cluster administrators and DevOps teams to allow or restrict access to Kubernetes resources.

In this tutorial, we have learned how to leverage Okta as an SSO provider for Teleport to define access policies for the Amazon EKS cluster.

Try Teleport with a 14 days trial.