Skip to main content
Version: 14.x

Register a Kubernetes Cluster using IAM Joining

In this guide, we will show you how to register a Kubernetes cluster with Teleport by using the agent's IAM identity to automatically join the Teleport cluster.

You can register multiple Kubernetes clusters with Teleport by deploying the Teleport Kubernetes Service on each cluster you want to register without having to distribute a joining secret to the Kubernetes cluster.

Once the Kubernetes cluster is registered for the first time, the agent will store its Teleport identity in a Kubernetes secret. The agent will use this identity to automatically join the cluster on subsequent restarts.

Support for joining a cluster with the Proxy Service behind a layer 7 load balancer or reverse proxy is available in Teleport 13.0+.

Prerequisites​

  • A running Teleport cluster version 14.3.33 or above. If you want to get started with Teleport, sign up for a free trial or set up a demo environment.

  • The tctl admin tool and tsh client tool.

    Visit Installation for instructions on downloading tctl and tsh.

  • A Kubernetes cluster version >= v1.17.0
  • An existing IAM OpenID Connect (OIDC) provider for your cluster
  • Helm >= 3.4.2
  • AWS CLI >= 2.10.3 or 1.27.81

Verify that Helm and Kubernetes are installed and up to date.

$ helm version
# version.BuildInfo{Version:"v3.4.2"}

$ kubectl version
# Client Version: version.Info{Major:"1", Minor:"17+"}
# Server Version: version.Info{Major:"1", Minor:"17+"}
  • To check that you can connect to your Teleport cluster, sign in with tsh login, then verify that you can run tctl commands using your current credentials. tctl is supported on macOS and Linux machines. For example: 
    $ tsh login --proxy=teleport.example.com [email protected]
    $ tctl status
    # Cluster  teleport.example.com
    # Version  14.3.33
    # CA pin   sha256:abdc1245efgh5678abdc1245efgh5678abdc1245efgh5678abdc1245efgh5678
    If you can connect to the cluster and run the tctl status command, you can use your current credentials to run subsequent tctl commands from your workstation. If you host your own Teleport cluster, you can also run tctl commands on the computer that hosts the Teleport Auth Service for full permissions.

Step 1/3. Create a Kubernetes service account with an IAM identity​

Teleport supports a mode where agents running in AWS can join the cluster using the IAM identity they are running as. It allows you to register Kubernetes clusters running in AWS without having to distribute a joining secret to the Kubernetes cluster.

To securely join the cluster without relying on the EKS node's Identity, a Teleport agent must run as a separate Kubernetes service account with an attached IAM role. Relying on the node's identity is not recommended as it can be easily compromised since every pod running on the node has access to the node's identity if IAM Roles for Service Accounts (IRSA) is not configured.

For IRSA to work correctly, it requires the Kubernetes cluster to have an IAM OpenID Connect that maps IAM roles to Kubernetes service accounts.

The Kubernetes service account must have access to the sts:GetCallerIdentity API but does not require any other permissions.

To create the IAM policy, run the following command:

$ cat >iam-policy.json <<EOF
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "sts:GetCallerIdentity",
            "Resource": "*"
        }
    ]
}
EOF

Then create the IAM policy:

$ aws iam create-policy --policy-name kube-iam-policy --policy-document file://iam-policy.json
{
    "Policy": {
        "PolicyName": "kube-iam-policy",
        "PolicyId": "ANPAW2Y2Q2Y2Y2Y2Y2Y2Y",
        "Arn": "arn:aws:iam::aws:policy/kube-iam-policy",
        "Path": "/",
        "DefaultVersionId": "v1",
        "AttachmentCount": 0,
        "PermissionsBoundaryUsageCount": 0,
        "IsAttachable": true,
        "Description": "",
        "CreateDate": "2021-03-18T15:12:00+00:00",
        "UpdateDate": "2021-03-18T15:12:00+00:00"
    }
}

Now we need to create the Kubernetes service account and map it to the IAM role. There are two ways of doing this. You can use eksctl if your cluster was provisioned using it or you can use the AWS CLI method.

eksctl supports automatic creation of new IAM roles and mapping it into the Kubernetes Service Account in the target namespace.

$ eksctl create iamserviceaccount \
    --name teleport-kube-agent-sa \
    --namespace teleport-agent \
    --cluster kube-cluster \
    --region aws-region \
    --attach-policy-arn arn:aws:iam::aws:policy/kube-iam-policy \
    --role-name kube-iam-role \
    --approve

The referenced parameters are:

  • teleport-kube-agent-sa is the name of the Kubernetes service account.
  • teleport-agent is the namespace where the Teleport Kubernetes Service is running.
  • aws-region is the AWS region where the cluster is running.
  • kube-iam-policy is the name of the IAM policy created in the previous step.
  • kube-cluster is the name of the Kubernetes cluster.
  • kube-iam-role is the name of the IAM role to create.

Once the command completes, you should see a new IAM role created in your AWS account and a new Kubernetes service account created in the target namespace.

Step 2/3. Create the AWS joining token​

Create a dynamic token which will allow agents from your AWS account to join your Teleport cluster using the roles defined.

Under the hood, Kubernetes Service instances will prove that they are running in your AWS account by sending a signed Identity Document which matches an allow rule configured in your AWS joining token.

Create the following token.yaml with an allow rule specifying your AWS account and the AWS ARN the agents will be running as.

$ cat >token.yaml <<EOF
kind: token
version: v2
metadata:
  # the token name is not a secret because instances must prove that they are
  # running in your AWS account to use this token
  name: kube-iam-token
spec:
  # use the minimal set of roles required
  roles: [Kube]
  # set the join method allowed for this token
  join_method: iam
  allow:
  # aws_arn is optional and allows you to restrict the IAM role of joining Agents
  # to a specific IAM role
  - aws_account: "$account_id"
    aws_arn: "arn:aws:sts::$account_id:assumed-role/kube-iam-role/*"
EOF

Run tctl create token.yaml to create the token.

Step 3/3. Deploy the Teleport Kubernetes Service​

Set up the Teleport Helm repository.

Allow Helm to install charts that are hosted in the Teleport Helm repository:

$ helm repo add teleport https://charts.releases.teleport.dev

Update the cache of charts from the remote repository so you can upgrade to all available releases:

$ helm repo update

Switch kubectl to the Kubernetes cluster and run:

# Deploy a Kubernetes agent. It dials back to the Teleport cluster tele.example.com.
$ CLUSTER=iam-cluster
$ PROXY=tele.example.com:443
# Install the Teleport Kubernetes agent. Does not create a service account and uses the existing
#   service account. See serviceAccount.create and serviceAccount.name parameters.
$ helm install teleport-agent teleport/teleport-kube-agent \
  --set kubeClusterName=${CLUSTER?} \
  --set roles="kube\,app\,discovery" \
  --set proxyAddr=${PROXY?} \
  --set joinParams.method=iam \
  --set joinParams.tokenName=kube-iam-token \
  --set serviceAccount.create=false \
  --set serviceAccount.name=teleport-kube-agent-sa \
  --create-namespace \
  --namespace=teleport-agent \
  --version 14.3.33

Make sure that the Teleport agent pod is running. You should see one Teleport agent pod pod with a single ready container:

$ kubectl -n teleport-agent get pods
NAME               READY   STATUS    RESTARTS   AGE
teleport-agent-0   1/1     Running   0          32s

List connected clusters using tsh kube ls and switch between them using tsh kube login:

$ tsh kube ls

# Kube Cluster Name Selected
# ----------------- --------
# iam-cluster

# kubeconfig now points to the iam-cluster cluster
$ tsh kube login iam-cluster
# Logged into Kubernetes cluster "iam-cluster". Try 'kubectl version' to test the connection.

# kubectl command executed on `iam-cluster` but is routed through the `tele.example.com` cluster.
$ kubectl get pods

If the agent pod is healthy and ready but you cannot see your Kubernetes cluster, it is likely related to RBAC permissions associated with your roles. On the other hand, if you can see your Kubernetes cluster but unable to see any pods, it's likely that your Teleport role does not allow access to pods in the Kubernetes cluster. For both cases, please refer to the section below.

Not seeing Kubernetes clusters?

Kubernetes authentication​

To authenticate to a Kubernetes cluster via Teleport, your Teleport roles must allow access as at least one Kubernetes user or group. Ensure that you have a Teleport role that grants access to the cluster you plan to interact with.

Run the following command to get the Kubernetes user for your current context:

$ kubectl config view \
-o jsonpath="{.contexts[?(@.name==\"$(kubectl config current-context)\")].context.user}"

Create a file called kube-access.yaml with the following content, replacing USER with the output of the command above.

kind: role
metadata:
  name: kube-access
version: v6
spec:
  allow:
    kubernetes_labels:
      '*': '*'
    kubernetes_resources:
      - kind: 'pod'
        namespace: '*'
        name: '*'
    kubernetes_groups:
    - viewers
    kubernetes_users:
    # Replace USER with the Kubernetes user for your current context.
    - USER
  deny: {}

Apply your changes:

$ tctl create -f kube-access.yaml

Assign the kube-access role to your Teleport user by running the appropriate commands for your authentication provider:

  1. Retrieve your local user's configuration resource:

    $ tctl get users/$(tsh status -f json | jq -r '.active.username') > out.yaml

  2. Edit out.yaml, adding kube-access to the list of existing roles:

      roles:
       - access
       - auditor
       - editor
    +  - kube-access

  3. Apply your changes:

    $ tctl create -f out.yaml

  4. Sign out of the Teleport cluster and sign in again to assume the new role.

Now that Teleport RBAC is configured, you can authenticate to your Kubernetes cluster via Teleport. To interact with your Kubernetes cluster, you will need to configure authorization within Kubernetes.

Kubernetes authorization​

To configure authorization within your Kubernetes cluster, you need to create Kubernetes RoleBindings or ClusterRoleBindings that grant permissions to the subjects listed in kubernetes_users and kubernetes_groups.

For example, you can grant some limited read-only permissions to the viewers group used in the kube-access role defined above:

Create a file called viewers-bind.yaml with the following contents:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: viewers-crb
subjects:
- kind: Group
  # Bind the group "viewers", corresponding to the kubernetes_groups we assigned our "kube-access" role above
  name: viewers
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  # "view" is a default ClusterRole that grants read-only access to resources
  # See: https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles
  name: view
  apiGroup: rbac.authorization.k8s.io

Apply the ClusterRoleBinding with kubectl:

$ kubectl apply -f viewers-bind.yaml

Log out of Teleport and log in again.

Next steps​

To see all of the options you can set in the values file for the teleport-kube-agent Helm chart, consult our reference guide.