Fork me on GitHub

Teleport

Teleport Kubernetes Operator (Preview)

Improve
Preview

Teleport Kubernetes Operator is currently in Preview mode.

The Teleport Kubernetes Operator provides a way for Kubernetes users to manage some Teleport resources through Kubernetes, following the Operator Pattern.

The Teleport Kubernetes Operator is deployed alongside its custom resource definitions. Once deployed, users can use a Kubernetes client like kubectl or their existing CI/CD Kubernetes pipelines to create Teleport custom resources. The Teleport Kubernetes Operator watches for those resources and does API calls to Teleport to reach the desired state.

Currently supported Teleport resources are:

  • users
  • roles

This guide covers how to:

  • Install Teleport Operator on Kubernetes.
  • Manage Teleport users and roles using Kubernetes.

Prerequisites

  • A running Teleport cluster. For details on how to set this up, see one of our Getting Started guides.

  • The tctl admin tool and tsh client tool version >= 10.3.1.

    tctl version

    Teleport v10.3.1 go1.18

    tsh version

    Teleport v10.3.1 go1.18

    See Installation for details.

  • A running Teleport cluster. For details on how to set this up, see our Enterprise Getting Started guide.

  • The tctl admin tool and tsh client tool version >= 10.3.1, which you can download by visiting the customer portal.

    tctl version

    Teleport v10.3.1 go1.18

    tsh version

    Teleport v10.3.1 go1.18

  • A Teleport Cloud account. If you do not have one, visit the sign up page to begin your free trial.

  • The tctl admin tool and tsh client tool version >= 10.2.2. To download these tools, visit the Downloads page.

    tctl version

    Teleport v10.2.2 go1.18

    tsh version

    Teleport v10.2.2 go1.18

  • Kubernetes cluster (with or without teleport-cluster Helm chart already deployed);
  • Helm
  • kubectl

Validate Kubernetes connectivity by running the following command:

kubectl cluster-info

Kubernetes control plane is running at https://127.0.0.1:6443

CoreDNS is running at https://127.0.0.1:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

Metrics-server is running at https://127.0.0.1:6443/api/v1/namespaces/kube-system/services/https:metrics-server:https/proxy

Tip

Users wanting to experiment locally with the operator can use minikube to start a local Kubernetes cluster:

minikube start

Step 1/3. Install teleport-cluster Helm chart with the operator

Add the Teleport Helm Chart repo:

helm repo add teleport https://charts.releases.teleport.dev
helm repo update

Install the Helm chart for the Teleport Cluster with operator.enabled=true.

helm install teleport-cluster teleport/teleport-cluster \ --create-namespace --namespace teleport-cluster \ --set clusterName=teleport-cluster.teleport-cluster.svc.cluster.local \ --set operator.enabled=true \ --set teleportVersionOverride=10.3.1

This command installs the required Kubernetes CRDs and deploys the Teleport Kubernetes Operator next to the Teleport cluster. All resources (except CRDs, which are cluster-scoped) are created in the teleport-cluster namespace.

All subsequent commands will take place in the teleport-cluster namespace, so let's use it by default:

kubectl config set-context --current --namespace teleport-cluster

Step 2/3. Manage Teleport users and roles using kubectl

Create a manifest called teleport-resources.yaml that describes two custom resources: a TeleportUser and a TeleportRole:

apiVersion: resources.teleport.dev/v5
kind: TeleportRole
metadata:
  name: myrole
spec:
  allow:
    rules:
      - resources: ['user', 'role']
        verbs: ['list','create','read','update','delete']
---
apiVersion: resources.teleport.dev/v2
kind: TeleportUser
metadata:
  name: myuser
spec:
  roles: ['myrole']

Apply the manifests to the Kubernetes cluster:

kubectl apply -f teleport-resources.yaml

List the created Kubernetes resources:

kubectl get teleportroles

NAME AGE

myrole 10m

kubectl get teleportusers

NAME AGE

myuser 10m

Check the user myuser has been created in Teleport and has been granted the role myrole:

PROXY_POD=$(kubectl get po -l app=teleport-cluster -o jsonpath='{.items[0].metadata.name}')
kubectl exec -it "$PROXY_POD" -c teleport -- tctl users ls

User Roles

----------------------------- -----------------------------

bot-teleport-operator-sidecar bot-teleport-operator-sidecar

myuser myrole

At this point the Teleport Kubernetes Operator is functional and Teleport users and roles can be managed from Kubernetes.

Step 3/3. Explore the Teleport CRDs

Available fields can be browsed with kubectl explain in a cluster with Teleport CRDs installed. For example the command:

kubectl explain teleportroles.spec

Returns the following fields:

KIND:     TeleportRole
VERSION:  resources.teleport.dev/v5

RESOURCE: spec <Object>

DESCRIPTION:
    Role resource definition v5 from Teleport

FIELDS:
   allow	<Object>
     Allow is the set of conditions evaluated to grant access.

   deny	<Object>
     Deny is the set of conditions evaluated to deny access. Deny takes priority
     over allow.

   options	<Object>
     Options is for OpenSSH options like agent forwarding.

Troubleshooting

Resource is not reconciled in Teleport

The Teleport Operator watches for new resources or changes in Kubernetes. When a change happens, it triggers the reconciliation loop. This loop is in charge of validating the resource, checking if it already exists in Teleport and making calls to the Teleport API to create/update/delete the resource. The reconciliation loop also adds a status field on the Kubernetes resource.

If an error happens and the reconciliation loop is not sucessful, an item in status.conditions will describe what went wrong. This allows users to diagnose errors by inspecting Kubernetes resources with kubectl:

kubectl get teleportusers myuser -o yaml

For example, if a user has been granted a nonexistent role the status will look like:

apiVersion: resources.teleport.dev/v2
kind: TeleportUser
# [...]
status:
  conditions:
  - lastTransitionTime: "2022-07-25T16:15:52Z"
    message: Teleport resource has the Kubernetes origin label.
    reason: OriginLabelMatching
    status: "True"
    type: TeleportResourceOwned
  - lastTransitionTime: "2022-07-25T17:08:58Z"
    message: 'Teleport returned the error: role my-non-existing-role is not found'
    reason: TeleportError
    status: "False"
    type: SuccessfullyReconciled

Here SuccessfullyReconciled is False and the error is role my-non-existing-role is not found.

If the status is not present or does not give sufficient information to solve the issue, check the operator logs:

$ kubectl logs "$PROXY_POD" -c operator
Note

In case of multi-replica deployments, only one operator instance is running the reconciliation loop. This operator is called the leader and is the only one producing reconciliation logs. The other operator instances are waiting with the following log:

leaderelection.go:248] attempting to acquire leader lease teleport/431e83f4.teleport.dev...

To diagnose reconciliation issues, you will have to inspect all pods to find the one reconciling the resources.

Next Steps

Helm Chart parameters are documented in the teleport-cluster Helm chart reference.

See the Helm Deployment guides detailing specific setups like running teleport on AWS or GCP.

Check out access controls documentation