Skip to main content
Version: 18.x

Terraform EKS Auto-Discovery Configuration

Report an IssueView as Markdown

This guide shows you how to use Terraform to configure Teleport and AWS to automatically enroll EKS clusters in your Teleport cluster.

How it works​

Teleport cluster auto-discovery involves two components:

  1. The Teleport Discovery Service that watches for new clusters or changes to previously discovered clusters. It dynamically registers each discovered cluster as a kube_cluster resource in your Teleport cluster. It does not need connectivity to the clusters it discovers.
  2. The Teleport Kubernetes Service that monitors the dynamic kube_cluster resources registered by the Discovery Service. It proxies communications between users and the cluster.

The teleport-discovery-aws Terraform module creates the AWS IAM resources and the Teleport integration, discovery_config, and provision token that configure the Discovery Service to enroll EKS clusters. When the Discovery Service has the required IAM permissions, it also provisions an EKS Access Entry for each discovered cluster so the Kubernetes Service can forward traffic to it.

Prerequisites​

  • A running Teleport cluster. If you want to get started with Teleport, sign up for a free trial or set up a demo environment.

  • The tctl and tsh clients.

    Installing tctl and tsh clients

    1. Determine the version of your Teleport cluster. The tctl and tsh clients must be at most one major version behind your Teleport cluster version. Send a GET request to the Proxy Service at /v1/webapi/find and use a JSON query tool to obtain your cluster version. Replace teleport.example.com:443 with the web address of your Teleport Proxy Service:

      TELEPORT_DOMAIN=teleport.example.com:443
      TELEPORT_VERSION="$(curl -s https://$TELEPORT_DOMAIN/v1/webapi/find | jq -r '.server_version')"

    2. Follow the instructions for your platform to install tctl and tsh clients:

      Download the signed macOS .pkg installer for Teleport, which includes the tctl and tsh clients:

      curl -O https://cdn.teleport.dev/teleport-${TELEPORT_VERSION?}.pkg

      In Finder double-click the pkg file to begin installation.

      danger

      Using Homebrew to install Teleport is not supported. The Teleport package in Homebrew is not maintained by Teleport and we can't guarantee its reliability or security.

  • An AWS account with IAM permissions to create roles, policies, and an OIDC provider. See the next step for the full permissions Terraform needs.

  • Terraform v1.0.0+.

    terraform version
    Terraform v1.0.0

  • A host to run the Teleport Discovery and Kubernetes services. Teleport Cloud runs the Discovery Service for you, but you still need to run your own Kubernetes Service. See the manual guide for service setup.

  • To check that you can connect to your Teleport cluster, sign in with tsh login, then verify that you can run tctl commands using your current credentials.

    For example, run the following command, assigning teleport.example.com to the domain name of the Teleport Proxy Service in your cluster and [email protected] to your Teleport username:

    tsh login --proxy=teleport.example.com --user=[email protected]
    tctl status
    Cluster  teleport.example.com
    Version  18.7.6
    CA pin   sha256:abdc1245efgh5678abdc1245efgh5678abdc1245efgh5678abdc1245efgh5678

    If you can connect to the cluster and run the tctl status command, you can use your current credentials to run subsequent tctl commands from your workstation. If you host your own Teleport cluster, you can also run tctl commands on the computer that hosts the Teleport Auth Service for full permissions.

Use the teleport-discovery-aws Terraform module​

Step 1/5. Configure AWS Terraform provider​

Configure the AWS Terraform provider and AWS IAM permissions for Terraform to manage AWS resources.

AWS IAM permissions required for AWS Terraform provider

The AWS Terraform provider will need the following AWS IAM permissions to manage AWS resources created by the teleport-discovery-aws module:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "TerraformIdentity",
      "Effect": "Allow",
      "Action": "sts:GetCallerIdentity",
      "Resource": "*"
    },
    {
      "Sid": "ManageIamRole",
      "Effect": "Allow",
      "Action": [
        "iam:CreateRole",
        "iam:DeleteRole",
        "iam:GetRole",
        "iam:ListInstanceProfilesForRole",
        "iam:ListRolePolicies",
        "iam:ListRoles",
        "iam:ListRoleTags",
        "iam:TagRole",
        "iam:UntagRole",
        "iam:UpdateAssumeRolePolicy"
      ],
      "Resource": "*"
    },
    {
      "Sid": "ManageIamPolicy",
      "Effect": "Allow",
      "Action": [
        "iam:CreatePolicy",
        "iam:CreatePolicyVersion",
        "iam:DeletePolicy",
        "iam:DeletePolicyVersion",
        "iam:GetPolicy",
        "iam:GetPolicyVersion",
        "iam:ListPolicies",
        "iam:ListPolicyTags",
        "iam:ListPolicyVersions",
        "iam:TagPolicy",
        "iam:UntagPolicy"
      ],
      "Resource": "*"
    },
    {
      "Sid": "ManageRolePolicyAttachments",
      "Effect": "Allow",
      "Action": [
        "iam:AttachRolePolicy",
        "iam:DetachRolePolicy",
        "iam:ListAttachedRolePolicies"
      ],
      "Resource": "*"
    },
    {
      "Sid": "ManageOidcProvider",
      "Effect": "Allow",
      "Action": [
        "iam:CreateOpenIDConnectProvider",
        "iam:DeleteOpenIDConnectProvider",
        "iam:GetOpenIDConnectProvider",
        "iam:ListOpenIDConnectProviders",
        "iam:TagOpenIDConnectProvider",
        "iam:UntagOpenIDConnectProvider",
        "iam:UpdateOpenIDConnectProviderThumbprint"
      ],
      "Resource": "*"
    }
  ]
}

Step 2/5. Configure Teleport Terraform provider​

For a local quick start, log in with tsh and run tctl terraform env in the same shell you use for terraform commands:

tsh login
eval "$(tctl terraform env)"

See the Local Demo guide for more detail on local setup. If you are running Terraform in CI, Spacelift, or another remote environment, refer to Using the Teleport Terraform Provider for the appropriate guide.

Step 3/5. Configure the Terraform module inputs​

Add the teleport-discovery-aws module to your Terraform configuration.

module "aws_discovery" {
  source = "terraform.releases.teleport.dev/teleport/discovery/aws"
  version = "~> 18.0"

  # Required inputs:
  # Assign example.teleport.sh:443 to your Teleport cluster's proxy public address in host:port form.
  teleport_proxy_public_addr = "example.teleport.sh:443"
  # teleport_discovery_group_name must match the discovery group name in your Discovery Service config file.
  # Teleport Cloud clusters run the Discovery Service in the group name "cloud-discovery-group".
  # Do not modify this input unless you intend to run your own Discovery Service.
  teleport_discovery_group_name = "cloud-discovery-group"

  aws_matchers = [
    {
      types   = ["eks"]
      # EKS discovery supports "*" to discover clusters across all enabled regions.
      regions = ["*"]
      # tags filter which EKS clusters are enrolled. The wildcard matches all tags.
      tags    = { "*" : ["*"] }
      # Also enroll HTTP applications running inside discovered clusters.
      kube_app_discovery = true
    }
  ]

  # Optional inputs:
  # Apply the additional AWS tag "origin=example" to all AWS resources created by this module
  apply_aws_tags = { origin = "example" }
  # Apply the additional Teleport label "origin=example" to all Teleport resources created by this module
  apply_teleport_resource_labels = { origin = "example" }
}

Add a Terraform output for the module so that Terraform will display its outputs:

output "aws_discovery" {
  value = module.aws_discovery
}

See the teleport-discovery-aws reference for a complete description of the module inputs and outputs.

Granting access to discovered EKS clusters​

The IAM role created by this module includes permissions for the Discovery Service to create and update EKS Access Entries. When the Discovery Service finds a new EKS cluster, it automatically provisions an Access Entry so the Teleport Kubernetes Service can forward traffic.

By default, the Discovery Service grants access to its own IAM role. If your Kubernetes Service runs with a different IAM role, add setup_access_for_arn to the EKS matcher with that role's ARN:

aws_matchers = [
  {
    types                = ["eks"]
    regions              = ["*"]
    tags                 = { "*" : ["*"] }
    setup_access_for_arn = "arn:aws:iam::123456789012:role/teleport-kube-service"
  }
]

Step 4/5. Apply the Terraform module​

terraform init
terraform apply

Terraform should plan to create the following resources:

  • AWS IAM role for Teleport Discovery Service to assume
  • AWS IAM policy that grants the AWS permissions necessary for Teleport to discover resources in AWS
  • AWS IAM policy attachment to attach the IAM policy to the Discovery Service IAM role
  • AWS OIDC Provider for Teleport Discovery Service to assume an IAM role using OIDC
  • Teleport discovery_config cluster resource that configures Teleport for AWS resource discovery
  • Teleport integration cluster resource for AWS OIDC
  • Teleport token cluster resource that allows Teleport nodes to use AWS IAM credentials to join the cluster

Review the Terraform plan and confirm the plan actions.

After Terraform finishes applying the plan, it should display the module outputs:

aws_discovery = {
  "aws_oidc_provider_arn" = "arn:aws:iam::123456789012:oidc-provider/example.teleport.sh"
  "teleport_discovery_config_name" = "discovery-aws-account-123456789012"
  "teleport_discovery_service_iam_policy_arn" = "arn:aws:iam::123456789012:policy/teleport-discovery-<timestamp>"
  "teleport_discovery_service_iam_role_arn" = "arn:aws:iam::123456789012:role/teleport-discovery-<timestamp>"
  "teleport_integration_name" = "discovery-aws-account-123456789012"
  "teleport_provision_token_name" = "discovery-aws-account-123456789012"
}

The AWS resources should have the following tags:

  • origin=example
  • teleport.dev/cluster=<cluster-name>
  • teleport.dev/integration=discovery-aws-account-<account-id>
  • teleport.dev/iac-tool=terraform

The Teleport resources should have the following labels:

  • origin=example
  • teleport.dev/iac-tool=terraform

Step 5/5. Check discovery status​

After applying the Terraform module, the Teleport Discovery Service should start to discover EKS clusters in your AWS account and enroll them in your Teleport cluster as kube_cluster resources.

note

It may take a few minutes for EKS clusters to be discovered and enrolled.

Navigate to the Teleport Web UI and select Zero Trust Access > Integrations. By default, the integration created by the teleport-discovery-aws Terraform module is named discovery-aws-account-<aws-account-id>.

Click on the integration for your AWS account to review the discovery status.

The integration page provides an overview of how many EKS clusters have been discovered and any issues encountered during the discovery process.

Updating module configuration​

The module inputs can be changed and re-applied to adjust the AWS discovery integration.

For example, if you had previously configured specific AWS regions, you can switch to the wildcard to discover EKS clusters in all enabled regions:

-regions = ["us-west-1"]
+regions = ["*"]

Apply Terraform again:

terraform apply

Review the Terraform plan before confirming the changes.

After Terraform finishes applying its plan, the Discovery Service will pick up the change to the dynamic discovery_config and begin to enroll EKS clusters in the newly-configured regions.

Troubleshooting​

Discovery Service troubleshooting​

First, check if any Kubernetes clusters have been discovered. To do this, you can use the tctl get kube_cluster command and check if the expected Kubernetes clusters have already been registered with your Teleport cluster.

If some Kubernetes clusters do not appear in the list, check if the Discovery Service selector labels match the missing Kubernetes cluster tags or look into the Discovery Service logs for permission errors.

Check that the Discovery Service is running with credentials for the correct AWS account. It can discover resources in another AWS account, but it must be configured to assume a role in the other AWS account if that's the case.

Check if there is more than one Discovery Services running:

tctl inventory status --connected

If you are running multiple Discovery Services, you must ensure that each service is configured with the same discovery_group value if they are watching the same cloud Kubernetes clusters or a different value if they are watching different cloud Kubernetes clusters. If this is not configured correctly, a typical symptom is kube_cluster resources being intermittently deleted from your Teleport cluster's registry.

Kubernetes Service troubleshooting​

If the tctl get kube_cluster command returns the discovered clusters, but the tctl kube ls command does not include them, check that you have set the kubernetes_service.resources section correctly.

kubernetes_service:
  enabled: true
  resources:
  - labels:
      "env": "prod"

If the section is correctly configured, but clusters still do not appear or return authentication errors, please check if permissions have been correctly configured in your target cluster or that you have the correct permissions to list Kubernetes clusters in Teleport.

Next steps​