Teleport Workload Identity with SPIFFE: Achieving Zero Trust in Modern Infrastructure
May 23
Virtual
Register Today
Teleport logoTry For Free
Fork me on GitHub

Teleport

Join Methods and Token Reference

This guide explains the core concepts behind the Teleport joining process, references all support join methods and classifies them based on their security properties. This guide does not explain step-by-step how to join an instance with each join method, but links to the relevant How-To guides when possible.

Prerequisite

You must be familiar with the Teleport Core concepts before reading this page.

Definitions

Joining

Joining a Teleport cluster is the act of establishing trust between a new Teleport instance and all the existing instances already part of the Teleport cluster. At the end of the joining process, the Auth Service signs certificates for the joining instance. Those certificates represent the trust that was established. With them, the newly-joined instance can interact with the other Teleport instances.

To request its certificates, an instance must prove its identity to the Auth Service. Teleport offers multiple ways for a joining instance to prove its authenticity, they are called the join methods.

The joining process only happens when a Teleport service doesn't have valid certificates. Once the token is exchanged for certificates, those certificates are used on all subsequent attempts to connect. In most cases, this happens during the first startup.

Join methods

A join method is a way for the Auth Service to validate that an instance requesting to join the Teleport cluster is legitimate. Some join methods are universal while others rely on the joining instance context. For example cloud-provider join-methods (such as iam, gcp or azure) or CI-provider (such as github, gitlab, circleci) are more flexible and provide better security guarantees but require the joining instance to run in a specific cloud-provider.

Different join methods may provide different security guarantees. e.g. some join methods allow the joining instance to request renewable certificates while other will require the instance to join again to renew its certificate.

The join method and its parameters are specified in the token resource.

Token

A Token is a Teleport resource that specifies which join method can be used in which context. For example, a token can allow SSH services to join with the iam join method if they are in the AWS account 333333333333 and can assume the role teleport-instance-role:

kind: token
version: v2
metadata:
  name: my-iam-token
spec:
  roles: [Node]
  join_method: iam
  allow:
  - aws_account: "333333333333"
    aws_arn: "arn:aws:sts::333333333333:assumed-role/teleport-instance-role/i-*"
Important

The token name may, or may not be sensitive depending on the join method. Secret-based join methods rely on the token name to be secret. In such cases the token name must be protected as knowing the token name in enough for an instance to join the cluster.

Classification of join methods

Secret vs delegated

Secret-based join methods

Secret-based join methods are universal: Teleport service can use a secret-based join method regardless of the platform/cloud provider it runs on. The joining instance sends the secret and the Auth Service validates that it matches the one it knows. Those joining methods are inherently prone to secret exfiltration and the delegated join methods should be preferred when available. If you have to use a secret-based join method, it is recommended to use short-lived tokens (valid only 1 hour for example) to reduce the risk of the token leaking.

Secret-based join methods are:

Warning

Teleport supports static tokens for backward compatibility, their use should be avoided.

Delegated join methods

Delegated join methods rely on the context of the joining instance and a third party to establish trust. The third party can be a cloud provider, a CI platform or the container runtime. Those methods cannot be used for every instance (e.g. joining an SSH agent from a Raspberry Pi is not possible) but should be preferred when possible.

Delegated join methods might also offer more granularity. For example, cloud-provider based join methods can allow instances to join based on their Availability Zone, service account, or cloud account ID.

Delegated join methods are:

Renewable vs non-renewable

Depending on the join method used, the Auth Service might issue renewable or non-renewable certificates.

When the certificate is about to expire, instances with renewable certificates can request a new one without having to use a token again. Typically, secret-based join methods provide renewable certificates because the secret token is sensitive and typically short-lived. With a single join, the instance can stay part of the cluster indefinitely.

Renewable join-methods are:

Nodes with non-renewable certificates must join again in order to get a new certificate before expiry. The instance will have to prove again that it is legitimate. The non-renewable join methods guarantee that an attacker stealing the instance certificates will not be able to maintain access to the Teleport cluster. Those join methods can be considered more secure and more appropriate for temporary workloads such as CI/CD pipelines or containerized environments.

Non-renewable join methods are:

Token resource reference

The token resource has the following common fields for all join methods:

# token.yaml
kind: token
version: v2
metadata:
  name: my-token-name
spec:
  # System roles describe what services the joining Teleport instance can run.
  # Those roles are written on the instance certificate. If you want to change
  # them (e.g. add Application access to an SSH Node), you need to:
  # - edit the token to update the roles (e.g. add "App")
  # - un-register the Teleport instance
  # - modify its configuration to enable the new service (here "app_service.enabled")
  # - have the instance join again 
  #
  # You should use the minimal set of system roles required.
  # Common roles are:
  # - "Node" for SSH Service
  # - "Proxy" for Proxy Service
  # - "Kube" for Kubernetes Service
  # - "App" for Application Service
  # - "Db" for Database Service
  # - "WindowsDesktop" for Windows Desktop Service
  # - "Discovery" for Discovery Service
  # - "Bot" for MachineID (when set, "spec.bot_name" must be set in the token)
  roles: 
    - Node
    - App
  join_method: gcp
  # Only set bot name when the token is used for MachineID.
  # When set, the token must have the "Bot" role as well.
  bot_name: my-bot
  # SuggestedLabels is a set of labels that resources should set when using this
  # token to enroll themselves in the cluster.
  # Currently, only node-join scripts create a configuration according to the suggestion.
  suggested_labels:
    teams: ["sales-eng", "eng", "qa"]
    application: ["demo-product"]
  # SuggestedAgentMatcherLabels is a set of labels to be used by discovery agents to match on resources.
  # When an agent uses this token, the agent should monitor resources that match those labels.
  # For databases, this means adding the labels to `db_service.resources.labels`.
  # Currently, only node-join scripts create a configuration according to the suggestion.
  suggested_agent_matcher_labels:
    teams: ["sales-eng"]

Join methods

Static tokens

Danger

This join method is inherently less secure because long-lived tokens can be stolen and reused. Relying on it significantly reduces the security benefits of using Teleport. Its usage is strongly discouraged. You should use ephemeral tokens instead.

Static tokens are tokens defined in the Auth Service configuration (teleport.yaml). The token name must be kept secret as knowing it allows to join instances to the Teleport cluster.

auth_service:
    enabled: yes
    # Pre-defined tokens for adding new instances to a cluster. Each token specifies
    # the role a new node will be allowed to assume. The more secure way to
    # add instances is to use `tctl nodes add --ttl` command to generate auto-expiring
    # tokens.
    #
    # We recommend to use tools like `pwgen` to generate sufficiently random
    # tokens of 32+ byte length.
    tokens:
        - "proxy,node:xxxxx"
        - "auth:yyyy"
        - "discovery,app,db:zzzzz"

Ephemeral tokens

Ephemeral tokens are secret tokens created dynamically via the CLI or Teleport API. They are time-bound and are typically created just before joining an instance to the Teleport cluster.

They can be created by the CLI (a strong random value is picked when not specified, default TTL is 30 minutes):

$ tctl tokens add --type discovery,app --ttl 15m

Or as Teleport resources:

kind: token
version: v2
metadata:
  expires: "2023-11-24T21:45:40.104524Z"
  name: abcd123-insecure-do-not-use-this
spec:
  join_method: token
  roles:
    - Discovery
    - App

When a MachineID bot uses an ephemeral join token, the token is deleted.

AWS IAM role: iam

The IAM join method is available to any Teleport process running anywhere with access to IAM credentials, such as an EC2 instance with an attached IAM role. No specific permissions or IAM policy is required: an IAM role with no attached policies is sufficient. No IAM credentials are required on the Teleport Auth Service.

Support for joining a cluster with the Proxy Service behind a layer 7 load balancer or reverse proxy is available in Teleport 13.0+.

This is the recommended method to join workload running on AWS.

# token.yaml
kind: token
version: v2
metadata:
  # the token name is not a secret because instances must prove that they are
  # running in your AWS account to use this token
  name: iam-token
spec:
  # use the minimal set of roles required (e.g. Node, App, Kube, DB, WindowsDesktop)
  roles: [Node]
  
  # set the join method allowed for this token
  join_method: iam
  
  allow:
    # specify the AWS account which Teleport processes may join from
    - aws_account: "111111111111"
    # multiple allow rules are supported
    - aws_account: "222222222222"
    # aws_arn is optional and allows you to restrict the IAM role of joining
    # Teleport processes
    - aws_account: "333333333333"
      aws_arn: "arn:aws:sts::333333333333:assumed-role/teleport-node-role/i-*"

AWS EC2 identity document: ec2

The EC2 join method is available to any Teleport process running on an EC2 instance. Only one Teleport process per EC2 instance may use the EC2 join method.

IAM credentials with ec2:DescribeInstances permissions are required on your Teleport Auth Service. No IAM credentials are required on the Teleport processes joining the cluster.

The EC2 join method is not available in Teleport Enterprise Cloud and Teleport Team. Teleport Enterprise Cloud and Team customers can use the IAM join method or ephemeral secret tokens.

# token.yaml
kind: token
version: v2
metadata:
  # the token name is not a secret because instances must prove that they are
  # running in your AWS account to use this token
  name: ec2-token
spec:
  # use the minimal set of roles required (e.g. Node, App, Kube, DB, WindowsDesktop)
  roles: [Node]

  # set the join method allowed for this token
  join_method: ec2

  # aws_iid_ttl is the amount of time after the EC2 instance is launched during
  # which it should be allowed to join the cluster. Use a short TTL to decrease
  # the risk of stolen EC2 Instance Identity Documents being used to join your
  # cluster.
  #
  # When launching your first Teleport process using the EC2 join method, you may need to
  # temporarily configure a higher `aws_iid_ttl` value so that you have time
  # to get Teleport set up and configured. This feature works best once Teleport
  # is configured in an EC2 AMI to start automatically on launch.
  aws_iid_ttl: 5m

  allow:
  - aws_account: "111111111111" # your AWS account ID
    aws_regions: # use the minimal set of AWS regions required
    - us-west-1
    - us-west-2

Azure managed identity: azure

The Azure join method is available in Teleport 12.1+. It is available to any Teleport process running in an Azure Virtual Machine. Support for joining a cluster with the Proxy Service behind a layer 7 load balancer or reverse proxy is available in Teleport 13.0+.

# token.yaml
kind: token
version: v2
metadata:
  # the token name is not a secret because instances must prove that they are
  # running in your Azure subscription to use this token
  name: azure-token
spec:
  # use the minimal set of roles required
  roles: [Node]
  join_method: azure
  azure:
    allow:
      # specify the Azure subscription which Teleport processes may join from
      - subscription: 11111111-1111-1111-1111-111111111111
      # multiple allow rules are supported
      - subscription: 22222222-2222-2222-2222-222222222222
      # resource_groups is optional and allows you to restrict the resource group of
      # joining Teleport processes
      - subscription: 33333333-3333-3333-3333-333333333333
        resource_groups: ["group1", "group2"]

GCP service account: gcp

The GCP join method is available to any Teleport process running on a GCP VM. The VM must have a service account assigned to it (the default service account is fine). No IAM roles are required on the Teleport process joining the cluster.

# token.yaml
kind: token
version: v2
metadata:
  # the token name is not a secret because instances must prove that they are
  # running in your GCP project to use this token
  name: gcp-token
spec:
  # use the minimal set of roles required (e.g. Node, Proxy, App, Kube, DB, WindowsDesktop)
  roles: [Node]

  # set the join method allowed for this token
  join_method: gcp

  gcp:
    allow:
      # The GCP project ID(s) that VMs can join from.
      - project_ids: ["example-project-id"]
        # (Optional) The locations that VMs can join from. Note: both regions and
        # zones are accepted.
        locations: ["us-west1", "us-west2-a"]
        # (Optional) The email addresses of service accounts that VMs can join
        # with.
        service_accounts: ["[email protected]"]

GitHub Actions: github

Teleport supports secure joining on both GitHub-hosted and self-hosted GitHub Actions runners as well as GitHub Enterprise Server. This join method is typically used with Machine ID to access Teleport-protected resources in GitHub Actions pipelines.

kind: token
version: v2
metadata:
  # name identifies the token. When configuring a bot or node to join using this
  # token, this name should be specified.
  name: github-token
spec:
  # For Machine ID and GitHub joining, roles will always be "Bot" and
  # join_method will always be "github".
  roles: [Bot]
  join_method: github

  # bot_name specifies the name of the bot that this token will grant access to
  # when it is used.
  bot_name: github-demo

  github:
    # enterprise_server_host allows joining from GitHub Actions workflows in a
    # GitHub Enterprise Server instance. For normal situations, where you are
    # using github.com, this option should be omitted. If you are using GHES,
    # this value should be configured to the hostname of your GHES instance.
    enterprise_server_host: ghes.example.com

    # enterprise_slug allows the slug of a GitHub Enterprise organisation to be
    # included in the expected issuer of the OIDC tokens. This is for
    # compatibility with the include_enterprise_slug option in GHE.
    #
    # This field should be set to the slug of your Github Enterprise organization if this is enabled. If
    # this is not enabled, then this field must be left empty. This field cannot
    # be specified if `enterprise_server_host` is specified.
    #
    # See https://docs.github.com/en/enterprise-cloud@latest/actions/deployment/security-hardening-your-deployments/about-security-hardening-with-openid-connect#customizing-the-issuer-value-for-an-enterprise
    # for more information about customized issuer values.
    enterprise_slug: slug

    # allow is an array of rule configurations for what GitHub Actions workflows
    # should be allowed to join. All options configured within one allow entry
    # must be satisfied for the GitHub Actions run to be allowed to join. Where
    # multiple allow entries are specified, any run which satisfies all of the
    # options within a single entry will be allowed to join.
    #
    # An allow entry must include at least one of:
    # - repository
    # - repository_owner
    # - sub
    allow:
      - # repository is a fully qualified (e.g. including the owner) name of a
        # GitHub repository.
        repository: gravitational/teleport
        # repository_owner is the name of an organization or user that a
        # repository belongs to.
        repository_owner: gravitational
        # workflow is the exact name of a workflow as configured in the GitHub 
        # Action workflow YAML file.
        workflow: my-workflow
        # environment is the environment associated with the GitHub Actions run.
        # If no environment is configured for the GitHub Actions run, this will
        # be empty.
        environment: production
        # actor is the GitHub username that caused the GitHub Actions run,
        # whether by committing or by directly despatching the workflow.
        actor: octocat
        # ref is the git ref that triggered the action run.
        ref: ref/heads/main
        # ref_type is the type of the git ref that triggered the action run.
        ref_type: branch
        # sub is a concatenated string of various attributes of the workflow 
        # run. GitHub explains the format of this string at:
        # https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/about-security-hardening-with-openid-connect#example-subject-claims
        sub: repo:gravitational/example-repo:environment:production

CircleCI: circleci

This join method is typically used with Machine ID to access Teleport-protected resources in Circle CI pipelines.

kind: token
version: v2
metadata:
  name: example-bot
spec:
  roles: [Bot]
  join_method: circleci
  bot_name: example
  circleci:
    organization_id: $ORGANIZATION_ID
    # allow specifies the rules by which the Auth Server determines if `tbot`
    # should be allowed to join.
    allow:
      - # CircleCI context id. See the CircleCI MachineID guide to learn
        # how to create a context and recover its ID.
        context_id: 00000000-0000-0000-0000-000000000000
        # CircleCI projectID.
        project_id: 1234

GitLab: gitlab

Teleport supports secure joining on both cloud-hosted and self-hosted GitLab instances. The minimum supported GitLab version is 15.7.

This join method is typically used with MachineID to access Teleport-protected resources in Gitlab CI pipelines.

kind: token
version: v2
metadata:
  # name identifies the token. When configuring a bot or node to join using this
  # token, this name should be specified.
  name: gitlab-demo
spec:
  # The Bot role indicates that this token grants access to a bot user, rather
  # than allowing a node to join.
  roles: [Bot]
  # join_method for GitLab joining will always be "gitlab".
  join_method: gitlab

  # bot_name specifies the name of the bot that this token will grant access to
  # when it is used.
  bot_name: gitlab-demo

  gitlab:
    # domain should be the domain of your GitLab instance. If you are using
    # GitLab's cloud hosted offering, omit this field entirely.
    domain: gitlab.example.com
    # allow is an array of rule configurations for what GitLab CI jobs should be
    # allowed to join. All options configured within one allow entry
    # must be satisfied for the GitLab CI run to be allowed to join. Where
    # multiple allow entries are specified, any job which satisfies all of the
    # options within a single entry will be allowed to join.
    #
    # An allow entry must include at least one of:
    # - project_path
    # - namespace_path
    # - sub
    # This ensures that GitLab CI runs in other GitLab user's projects are not
    # able to access your Teleport cluster.
    allow:
      # project_path restricts joins to jobs that originate within the
      # specified project.
      #
      # This field supports glob-style matching:
      # - Use '*' to match zero or more characters.
      # - Use '?' to match any single character.
      - project_path: my-user/my-project
        # namespace_path restricts joins to any run within project that exists
        # within the specified namespace. A namespace will either be a username
        # or the name of a group.
        #
        # This field supports glob-style matching:
        # - Use '*' to match zero or more characters.
        # - Use '?' to match any single character.
        namespace_path: my-user
        # pipeline_source restricts joins to jobs triggered by certain criteria,
        # e.g triggered through the web interface.
        pipeline_source: web
        # environment restricts joins to jobs that are associated with the
        # specified environment
        environment: production
        # ref_type restricts joins to jobs that were triggered by a specific
        # type of git reference. Either `branch` or `tag`.
        ref_type: branch
        # ref restricts joins to jobs that were triggered by a specific git
        # reference. Combine this with `ref_type` to create allow rules that
        # can only be triggered by a specific branch or tag.
        #
        # This field supports glob-style matching:
        # - Use '*' to match zero or more characters.
        # - Use '?' to match any single character.
        ref: main
        # sub is a single string that concatenates the project_path, ref_type
        # and ref. This can be used to restrict joins using a single string,
        # whilst also describing a specific project and git ref.
        #
        # It is better to use the individual fields, as it is easy to mis-format
        # the sub string.
        #
        # This field supports glob-style matching:
        # - Use '*' to match zero or more characters.
        # - Use '?' to match any single character.
        sub: project_path:my-user/my-project:ref_type:branch:ref:main
        # user_login restricts joins to jobs that were triggered by a specific
        # username.
        user_login: octocat
        # user_email restricts joins to jobs that were triggered by a specific
        # user with the given email
        user_email: [email protected]
        # ref_protected if set to true restricts joins to jobs running against a
        # protected ref.
        # If omitted, the protection status of the ref is not checked.
        ref_protected: true
        # environment_protected if set to true restricts joins to jobs running
        # against a protected ref.
        # If omitted, the protection status of the ref is not checked.
        environment_protected: true
        # ci_config_sha restricts joins to jobs that are using a specific
        # commit of CI configuration.
        ci_config_sha: ffffffffffffffffffffffffffffffffffffffff
        # ci_config_ref_uri restricts joins to jobs that are using a specific
        # CI configuration source.
        ci_config_ref_uri: gitlab.example.com/my-group/my-project//.gitlab-ci.yml@refs/heads/main
        # deployment_tier restricts joins to jobs that are deploying to a
        # specific deployment_tier.
        deployment_tier: production
        # project_visibility restricts joins to jobs that are running against a
        # project with a specific visibility configuration.
        project_visibility: public

Kubernetes: kubernetes

The Kubernetes join methods exists in two variants:

Kubernetes In-cluster

Kubernetes in-cluster joining is available for any Teleport process running in the same Kubernetes cluster than the Auth Service. It uses the Kubernetes ServiceAccount tokens to validate the pod identity. The method relies on the Kubernetes TokenReview API which is typically only reachable from within the Kubernetes cluster. Because of this limitation, this join method is only available for self-hosted Teleport clusters in Kubernetes.

This method should be preferred when available as tokens are revoked as soon as the pod enters the Terminated state.

The Kubernetes in-cluster join method is available in self-hosted versions of Teleport 12+.

# token.yaml
kind: token
version: v2
metadata:
  # The token name is not a secret as the Kubernetes join method relies on the
  # Kubernetes signature to establish trust and not on the join token name.
  name: kubernetes-token
  # set a long expiry time, the default for tokens is only 30 minutes
  expires: "2050-01-01T00:00:00Z"
spec:
  # Use the minimal set of system roles required.
  roles: [App]

  # set the join method allowed for this token
  join_method: kubernetes
  
  kubernetes:
    # If type is not specified, it defaults to in_cluster
    type: in_cluster
    allow:
      # Service account names follow the format "namespace:serviceaccountname".
      - service_account: "teleport-agent:teleport-app-service"

Kubernetes JWKS

Kubernetes JWKS joining is available for any Teleport process running in Kubernetes. The Auth Service does not have to run in Kubernetes so this method can be used with any Teleport cluster, including Teleport Cloud. This join method works by exporting the public Kubernetes signing keys and using them to validate Kubernetes SA token signatures. The signature validation can be performed by an Auth Service without access to the Kubernetes.

The Kubernetes JWKS join method is available in Teleport 14+.

kind: token
version: v2
metadata:
  name: example
spec:
  roles: [App]
  join_method: kubernetes
  kubernetes:
    # static_jwks configures the Auth Server to validate the JWT presented by
    # `tbot` using the public key from a statically configured JWKS.
    type: static_jwks
    static_jwks:
      jwks: |
        # Place the kubernetes JWKS here (`kubectl get --raw /openid/v1/jwks`)
        {"keys":[--snip--]}
    # allow specifies the rules by which the Auth Server determines if the node
    # should be allowed to join.
    allow:
      - service_account: "namespace:serviceaccount"
Warning

After rotating the Kubernetes CA, you must update the Kubernetes JWKS tokens to contain the new Kubernetes singing keys (update the spec.kubernetes.static_jwks.jwks field).