Deploying Machine ID on Kubernetes
This guide shows you how to deploy the Machine ID daemon tbot
, on a Kubernetes
cluster.
In the setup we demonstrate in this guide, tbot
runs as a Kubernetes
deployment. It writes output credentials to a Kubernetes secret, which can then
be mounted in the pods that need to use the credentials. While tbot
can also
run as a sidecar within the same pod as the service that needs to use the
credentials it generates, we recommend running tbot
as a standalone deployment
due the limited support Kubernetes has for sidecars.
In this guide, we demonstrate the kubernetes
join method, in which tbot
proves its identity to the Teleport Auth Service by presenting a JSON web token
(JWT) signed by the Kubernetes API server. This JWT contains identifies the
service account, the pod and the namespace in which tbot
is running. The
Teleport Auth Service checks the signature of the JWT against the Kubernetes
cluster's public signing key.
Using another join method
When deploying tbot
to a Teleport cluster, it is generally recommended to use
the kubernetes
join method. This will work with most Kubernetes clusters.
The guide that follows will demonstrate configuring this join method.
However, when using certain cloud Kubernetes services, it is possible to use the
join method associated with that platform rather than the kubernetes
join
method. This may be beneficial if you wish to manage the joining of tbot
within the Kubernetes clusters and on standard VMs on the same platform with
a single join token. These services are:
- Google Kubernetes Engine: Where
GCP Workload Identity
is configured for the cluster, it is possible to use the
gcp
join method. See the GCP Platform Guide for further information. - AWS Elastic Kubernetes Service: Where
IAM Roles for Service Accounts (IRSA)
is configured for the cluster, it is possible to use the
iam
join method. See the AWS Platform Guide for further information.
Prerequisites
-
A running Teleport cluster version 15.4.22 or above. If you want to get started with Teleport, sign up for a free trial or set up a demo environment.
-
The
tctl
admin tool andtsh
client tool.On Teleport Enterprise, you must use the Enterprise version of
tctl
, which you can download from your Teleport account workspace. Otherwise, visit Installation for instructions on downloadingtctl
andtsh
for Teleport Community Edition.
- To check that you can connect to your Teleport cluster, sign in with
tsh login
, then verify that you can runtctl
commands using your current credentials.tctl
is supported on macOS and Linux machines. For example:If you can connect to the cluster and run the$ tsh login --proxy=teleport.example.com [email protected]
$ tctl status
# Cluster teleport.example.com
# Version 15.4.22
# CA pin sha256:abdc1245efgh5678abdc1245efgh5678abdc1245efgh5678abdc1245efgh5678tctl status
command, you can use your current credentials to run subsequenttctl
commands from your workstation. If you host your own Teleport cluster, you can also runtctl
commands on the computer that hosts the Teleport Auth Service for full permissions. - A Kubernetes cluster with support for Token Request Projection (which graduated to a generally available feature in Kubernetes 1.20).
kubectl
authenticated with the ability to create resources in the cluster you wish to deploytbot
into.
The examples in this guide will install a tbot
deployment in the default
Namespace of the Kubernetes cluster. Adjust references to default
to the
Namespace you wish to use.
Step 1/5. Prepare Kubernetes RBAC
In order to prepare the Kubernetes cluster for Machine ID, several Kubernetes RBAC resources must be created.
A ServiceAccount will be created and later assigned to the Pod that will run
tbot
. This creates a static identity that we can allow access to join the
Teleport Cluster and also provides an identity to which we can assign Kubernetes
privileges.
A Role granting the ability to read and write to secrets in the Namespace will
be created and then assigned to the ServiceAccount using a RoleBinding. This
will allow the tbot
Pod to read and write credentials to a Secret.
Create a file called k8s-rbac.yaml
:
# This ServiceAccount will be used to give the `tbot` pods a discrete identity
# which can be validated by the Teleport Auth Server.
apiVersion: v1
kind: ServiceAccount
metadata:
name: tbot
namespace: default
---
# This role grants the ability to manage secrets within the namespace - this is
# necessary for the `kubernetes_secret` destination to work correctly.
#
# You may wish to add the `resourceNames` field to the role to further restrict
# this access in sensitive environments.
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: secrets-admin
namespace: default
rules:
- apiGroups: [""]
resources: ["secrets"]
verbs: ["*"]
---
# Bind the role to the service account created for tbot.
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: tbot-secrets-admin
namespace: default
subjects:
- kind: ServiceAccount
name: tbot
roleRef:
kind: Role
name: secrets-admin
apiGroup: rbac.authorization.k8s.io
Apply this file to your Kubernetes cluster:
$ kubectl apply -f ./k8s-rbac.yaml
Step 2/5. Create a Bot
Next, you need to create a Bot. A Bot is a Teleport identity for a machine or group of machines. Like users, bots have a set of roles and traits which define what they can access.
Create bot.yaml
:
kind: bot
version: v1
metadata:
# name is a unique identifier for the Bot in the cluster.
name: example
spec:
# roles is a list of roles to grant to the Bot. Don't worry if you don't know
# what roles you need to specify here, the Access Guides will walk you through
# creating and assigning roles to the already created Bot.
roles: []
Make sure you replace example
with a unique, descriptive name for your Bot.
Use tctl
to apply this file:
$ tctl create bot.yaml
Step 3/5. Create a join token
Next, a join token needs to be configured. This will be used by tbot
to join
the cluster. As the kubernetes
join method will be used, the public key of the
Kubernetes cluster must first be determined. The public key used to sign JWTs
is exposed on the "JWKS" endpoint of the Kubernetes API server. This public key
can then be used by the Teleport Auth to verify that the Service Account JWT
presented by tbot
is signed legitimately by the Kubernetes cluster.
Run the following commands to determine the JWKS formatted public key:
$ kubectl proxy -p 8080
$ curl http://localhost:8080/openid/v1/jwks
{"keys":[--snip--]}%
Create bot-token.yaml
, ensuring you insert the value from the JWKS endpoint
in spec.kubernetes.static_jwks.jwks
:
kind: token
version: v2
metadata:
# name will be specified in the `tbot` to use this token
name: example-bot
spec:
roles: [Bot]
# bot_name should match the name of the bot created earlier in this guide.
bot_name: example
join_method: kubernetes
kubernetes:
# static_jwks configures the Auth Server to validate the JWT presented by
# `tbot` using the public key from a statically configured JWKS.
type: static_jwks
static_jwks:
jwks: |
# Place the data returned by the curl command here
{"keys":[--snip--]}
# allow specifies the rules by which the Auth Server determines if `tbot`
# should be allowed to join.
allow:
- service_account: "default:tbot" # service_account
Use tctl
to apply this file:
$ tctl create -f bot-token.yaml
Step 4/5. Create a tbot
deployment
First, a ConfigMap will be created to contain the configuration file for tbot
.
This will then be mounted into the Pod.
Create k8s-deployment-config.yaml
, replacing the value of token
with the
name of the token you created earlier and the value of proxy_server
with the
address of your Teleport Proxy Service:
apiVersion: v1
kind: ConfigMap
metadata:
name: tbot-config
namespace: default
data:
tbot.yaml: |
version: v2
onboarding:
join_method: kubernetes
# ensure token is set to the name of the join token you created earlier
token: bot-kubernetes
storage:
# a memory destination is used for the bots own state since the kubernetes
# join method does not require persistence.
type: memory
# ensure this is configured to the address of your Teleport Proxy Service.
proxy_server: example.teleport.sh:443
# outputs will be filled in during the completion of an access guide.
outputs: []
Apply this file to your Kubernetes cluster:
$ kubectl apply -f k8s-deployment-config.yaml
With the ConfigMap created, you can now create the tbot
deployment itself.
Create k8s-deployment.yaml
:
apiVersion: apps/v1
kind: Deployment
metadata:
name: tbot
namespace: default
spec:
replicas: 1
strategy:
type: Recreate
selector:
matchLabels:
app.kubernetes.io/name: tbot
template:
metadata:
labels:
app.kubernetes.io/name: tbot
spec:
containers:
- name: tbot
image: public.ecr.aws/gravitational/tbot-distroless:15.4.22
args:
- start
- -c
- /config/tbot.yaml
env:
# POD_NAMESPACE is required for the kubernetes_secret` destination
# type to work correctly.
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
# KUBERNETES_TOKEN_PATH specifies the path to the service account
# JWT to use for joining.
# This path is based on the configuration of the volume and
# volumeMount.
- name: KUBERNETES_TOKEN_PATH
value: /var/run/secrets/tokens/join-sa-token
# TELEPORT_ANONYMOUS_TELEMETRY enables the submission of anonymous
# usage telemetry. This helps us shape the future development of
# `tbot`. You can disable this by omitting this.
- name: TELEPORT_ANONYMOUS_TELEMETRY
value: "1"
volumeMounts:
- mountPath: /config
name: config
- mountPath: /var/run/secrets/tokens
name: join-sa-token
serviceAccountName: tbot
volumes:
- name: config
configMap:
name: tbot-config
- name: join-sa-token
projected:
sources:
- serviceAccountToken:
path: join-sa-token
# 600 seconds is the minimum that Kubernetes supports. We
# recommend this value is used.
expirationSeconds: 600
# `example.teleport.sh` must be replaced with the name of
# your Teleport cluster.
audience: example.teleport.sh
Replace example.teleport.sh
with the name of your Teleport cluster - this is
not necessarily the public address (the port should not be included).
This is an example manifest, consider modifying it to fit within the conventions of deployments to your clusters (e.g customizing labels).
The default tbot-distroless
image does not contain the FIPS-compliant
binaries. If you operate in an environment where FIPS compliance is required,
please use the tbot-fips-distroless
image instead.
Apply this file to your Kubernetes cluster:
$ kubectl apply -f ./k8s-deployment.yaml
Use kubectl
to verify that the deployment is healthy:
$ kubectl describe deployment/tbot
$ kubectl logs deployment/tbot
With this complete, tbot
is now successfully deployed to your cluster.
However, it is not yet producing any useful output.
Step 5/5. Configure outputs
Follow one of the access guides to configure an output that meets your access needs.
In order to adjust the access guides to work well with Kubernetes, use the Kubernetes Secret destination type. This will write the generated artifacts to a specified Kubernetes Secret, for example:
outputs:
- type: identity
destination:
type: kubernetes_secret
name: identity-output
The output can then be consumed by mounting this secret within another pod:
apiVersion: v1
kind: Pod
metadata:
name: tsh
namespace: default
spec:
containers:
- name: tsh
image: public.ecr.aws/gravitational/teleport-distroless:15.4.22
command:
- tsh
args:
- -i
- /identity-output/identity
- --proxy
- example.teleport.sh:443
- ls
volumeMounts:
- name: identity-output
mountPath: /identity-output
volumes:
- name: identity-output
secret:
secretName: identity-output
Next steps
- Follow the access guides to finish configuring
tbot
for your environment. - Read the configuration reference to explore all the available configuration options.
- More information about
TELEPORT_ANONYMOUS_TELEMETRY
.