Configuring Kubernetes OIDC Joining With IaC

Report an issue with this page

In this guide you will see how to configure Teleport to allow Kubernetes Agents to join without secret tokens. Teleport supports three ways to dynamically create resources from code:

• The Teleport Kubernetes Operator, which allows you to manage Teleport resources from Kubernetes
• The Teleport Terraform Provider, which allows you to manage Teleport resources via Terraform
• The tctl CLI, which allows you to manage Teleport resources from your local computer or your CI environment

How it works

This guide relies on the OIDC variant of the Kubernetes join method. Most Kubernetes clusters use an OpenID Connect (OIDC) provider to sign their Service Account tokens.

In this guide you will configure Teleport to trust the Kubernetes OIDC provider so the agent running in the Kubernetes cluster can use its Kubernetes-issued token to join the Teleport cluster.

Prerequisites

To follow this guide, you must have:

• A Teleport cluster running v18.1.5 or above
• A Kubernetes cluster you want to enroll, whose service account tokens are signed by a publicly reachable OIDC provider.

note

Cloud-managed Kubernetes clusters either have public OIDC provider by default, or this can be configured when creating the cluster:

• EKS and GKE clusters should all have a publicly reachable OIDC provider.
• this is optional on AKS clusters
• OIDC Discovery is optional on OKE cluster

The cluster does not have to be public, only its OIDC provider has to.

tctl

• A running Teleport cluster. If you do not have one, read Getting Started.

• The tctl and tsh clients.

  Installing tctl and tsh clients

  1. Determine the version of your Teleport cluster. The tctl and tsh clients must be at most one major version behind your Teleport cluster version. Send a GET request to the Proxy Service at /v1/webapi/find and use a JSON query tool to obtain your cluster version. Replace teleport.example.com:443 with the web address of your Teleport Proxy Service:

     TELEPORT_DOMAIN=teleport.example.com:443
     TELEPORT_VERSION="$(curl -s https://$TELEPORT_DOMAIN/v1/webapi/find | jq -r '.server_version')"

  2. Follow the instructions for your platform to install tctl and tsh clients:

     Mac

     Download the signed macOS .pkg installer for Teleport, which includes the tctl and tsh clients:

     curl -O https://cdn.teleport.dev/teleport-${TELEPORT_VERSION?}.pkg

     In Finder double-click the pkg file to begin installation.

     danger

     Using Homebrew to install Teleport is not supported. The Teleport package in Homebrew is not maintained by Teleport and we can't guarantee its reliability or security.

     Windows - Powershell

     curl.exe -O https://cdn.teleport.dev/teleport-v${TELEPORT_VERSION?}-windows-amd64-bin.zip
     Unzip the archive and move the `tctl` and `tsh` clients to your %PATH%
     NOTE: Do not place the `tctl` and `tsh` clients in the System32 directory, as this can cause issues when using WinSCP.
     Use %SystemRoot% (C:\Windows) or %USERPROFILE% (C:\Users\<username>) instead.

     Linux

     All of the Teleport binaries in Linux installations include the tctl and tsh clients. For more options (including RPM/DEB packages and downloads for i386/ARM/ARM64) see our installation page.

     curl -O https://cdn.teleport.dev/teleport-v${TELEPORT_VERSION?}-linux-amd64-bin.tar.gz
     tar -xzf teleport-v${TELEPORT_VERSION?}-linux-amd64-bin.tar.gz
     cd teleport
     sudo ./install
     Teleport binaries have been copied to /usr/local/bin

Kubernetes Operator

A running operator by following either:

• the guide to enable the operator in the teleport-cluster Helm chart.
• the guide to setup a standalone operator.

You must also set the namespace in which you deployed the operator as this is the namespace where you will deploy the CustomResources:

for operators deployed with the `teleport-cluster` Helm chart
export OPERATOR_NAMESPACE="teleport-cluster"

for standalone operators
export OPERATOR_NAMESPACE="teleport-iac"

Terraform

A functional Teleport Terraform provider by following the Terraform provider guide.

Step 1/4. Detect the OIDC provider URL

In this step we will connect to the cluster and recover the OIDC provider URL. If you already have the cluster's OIDC provider URL (e.g. as an output from the cluster provisioning), you can skip to step 2.

Validate that you have access:

kubectl cluster-info

Kubernetes control plane is running at https://kube.example.com:443CoreDNS is running at https://kube.example.com:443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxyMetrics-server is running at https://kube.example.com:443/api/v1/namespaces/kube-system/services/https:metrics-server:/proxy
To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

Retrieve the OIDC issuer URL in the issuer field of the OIDC configuration endpoint:

kubectl get --raw=/.well-known/openid-configuration

{"issuer":"https://oidc.example.com/path/to/issuer/","jwks_uri":"https://kube.example.com:443/openid/v1/jwks","response_types_supported":["id_token"],"subject_types_supported":["public"],"id_token_signing_alg_values_supported":["RS256"]}

In the example above, it is https://oidc.example.com/path/to/issuer/.

important

The issuer URL must be copied exactly, including the presence or absence of trailing / in the URL.

Validate that the OIDC provider is publicly reachable by running the following command and making sure it returns JSON:

curl https://oidc.example.com/path/to/issuer/.well-known/openid-configuration

{  "issuer": "https://oidc.example.com/path/to/issuer/",  "jwks_uri": "https://oidc.example.com/path/to/issuer/openid/v1/jwks",  "response_types_supported": [    "id_token"  ],  "subject_types_supported": [    "public"  ],  "id_token_signing_alg_values_supported": [    "RS256"  ]}

Step 2/4. Create the Join Token

In this step we will configure Teleport to trust tokens signed by the Kubernetes OIDC provider. This will allow agents running in this Kubernetes cluster to join the Teleport cluster.

note

We recommend naming the token after the Kubernetes cluster it's trusting, this will allow you to enroll other clusters without facing token name conflicts.

Write the manifest

When creating the token, you will need to specify the future agent namespace (teleport) and agent release name (teleport-agent). This will be used by Teleport to check if the agent can join.

tctl

Create the file token.yaml with the following content:

kind: token
version: v2
metadata:
  name: kube-cluster-name-oidc
spec:
  roles: ['Kube']
  join_method: "kubernetes"
  kubernetes:
    type: oidc
    oidc:
      issuer: "https://oidc.example.com/path/to/issuer/"
    allow:
      - service_account: "teleport:teleport-agent"

Kubernetes Operator

Create the file token.yaml with the following content:

apiVersion: resources.teleport.dev/v1
kind: TeleportProvisionToken
metadata:
  name: kube-cluster-name-oidc
spec:
  roles: ['Kube']
  join_method: "kubernetes"
  kubernetes:
    type: oidc
    oidc:
      issuer: "https://oidc.example.com/path/to/issuer/"
    allow:
      - service_account: "teleport:teleport-agent"

note

Kubernetes validates all custom resource names to follow RFC 1123, which includes specifications for hostnames. This requires the metadata.name field of Teleport resources controlled by the operator to consist of lowercase alphanumeric characters, - or ., and to start and end with an alphanumeric character.

Terraform

Create the file token.tf with the following content:

resource "teleport_token" "kube-cluster-name" {
  version = "v2"
  metadata = {
    name = "kube-cluster-name-oidc"
  }

  spec = {
    roles = ["Kube"]
    join_method = "kubernetes"
    kubernetes = {
      type = "oidc"
      oidc = {
        issuer = "https://oidc.example.com/path/to/issuer/"
      }
      allow = [
        {
          service_account = "teleport:teleport-agent"
        }
      ]
    }
  }
}

Apply the manifest

tctl

tctl create -f token.yaml
token 'kube-cluster-name-oidc' has been created

Kubernetes Operator

kubectl apply -n "$OPERATOR_NAMESPACE" -f token.yaml

teleportprovisiontokenv2.resources.teleport.dev/kube-cluster-name-oidc created

List the created Kubernetes resources:

kubectl get teleportprovisiontokenv2 -n "$OPERATOR_NAMESPACE"

NAME                                  AGEkube-cluster-name-oidc   10m

Terraform

terraform plan
[...]Plan: 1 to add, 0 to change, 0 to destroy.
terraform apply
teleport_provision_token.kube-cluster-name-oidc: Creating...teleport_provision_token.kube-cluster-name-oidc: Creation complete after 0s [id=kube-cluster-name-oidc]
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Step 3/4. Configure the agent

In this step we will write the teleport-kube-agent Helm chart configuration so that the agent uses the previously created token to join.

You must have the following information:

• The Teleport cluster address: teleport.example.com:443
• The Teleport cluster name: teleport.example.com
• The name of the kubernetes cluster you are connecting: kube-cluster-name

Write the following values.yaml YAML manifest.

roles: "kube"
proxyAddr: teleport.example.com:443
# If you use Teleport Community Edition or AGPL, set to false
enterprise: true
updater:
  enabled: true
joinParams:
  method: "kubernetes"
  tokenName: kube-cluster-name-oidc
kubeClusterName: kube-cluster-name
teleportClusterName: teleport.example.com

Step 4/4. Deploy the agent

In this step we will deploy a release of the teleport-kube-agent chart using the values we wrote previously.

Configure Helm to fetch Teleport charts from the Teleport Helm repository:

helm repo add teleport https://charts.releases.teleport.dev

Refresh the local Helm cache by fetching the latest charts:

helm repo update

Finally deploy the Teleport Kubernetes Agent using the values we previously set in the provision token:

• The namespace in which you will deploy the agent: teleport
• The name of the Teleport Agent release: teleport-agent

helm upgrade --install teleport-agent teleport/teleport-kube-agent \  --namespace teleport --create-namespace \  --version 19.0.0-dev \  --values values.yaml

Validate that the agent successfully joined the cluster by checking its pod readiness (this can take a couple of minutes):

kubectl get pods -n teleport

NAME                                     READY   STATUS    RESTARTS   AGEteleport-agent-0                         1/1     Running   0          9steleport-agent-updater-6b8b74996-jz4qg   1/1     Running   0          15s

Next steps

Look at the teleport-kube-agent Helm chart reference for the list of supported values.