Kubernetes Clusters Discovery

Kubernetes Clusters Discovery allows Kubernetes clusters hosted on cloud providers to be discovered and enrolled automatically.

While discovering a new Kubernetes cluster, Teleport does not install any component on the cluster. Instead, it requires direct access to the cluster's API and minimal access permissions.

Supported clouds

• AWS: Discovery for AWS EKS clusters.
• Azure: Discovery for Azure AKS clusters.
• Google Cloud: Discovery for Google Kubernetes Engine clusters.

How Kubernetes Clusters Discovery works

Kubernetes Clusters Discovery consists of two steps:

Polling cloud APIs

The Teleport Discovery Service is responsible for scanning the configured cloud providers and identifying if any Kubernetes clusters match a set of filtering labels. When the process identifies a new Kubernetes cluster, it creates a dynamic resource within Teleport. This resource includes information imported from the cloud provider such as:

• Name: Cluster name
• Labels
  • Cluster tags.
  • Cluster location.
  • Identification of which cloud account the cluster belongs to — AWS Account ID / Azure Subscription ID.

tip

You can import the cluster under a different name into Teleport's registry. To achieve this, you must attach the following tag to the resources — EKS, AKS, GKE — in your cloud provider:

• key: TeleportKubernetesName
• value: desired name

The Discovery Service will check if the cluster includes the tag and use its value as the resource name in Teleport.

You should use this feature whenever there are clusters in different regions/cloud providers with the same name to prevent them from colliding in Teleport.

In addition to detecting new Kubernetes clusters, the Discovery Service also removes — from Teleport's registry — the Kubernetes clusters that have been deleted or whose tags no longer meet the filtering labels.

warning

Discovery Service exposes a configuration parameter - discovery_service.discovery_group - that allows you to group discovered resources into different sets. This parameter is used to prevent Discovery Agents watching different sets of cloud resources from colliding against each other and deleting resources created by another services.

When running multiple Discovery Services, you must ensure that each service is configured with the same discovery_group value if they are watching the same cloud resources or a different value if they are watching different cloud resources.

It is possible to run a mix of configurations in the same Teleport cluster meaning that some Discovery Services can be configured to watch the same cloud resources while others watch different resources. As an example, a 4-agent high availability configuration analyzing data from two different cloud accounts would run with the following configuration.

• 2 Discovery Services configured with discovery_group: "prod" polling data from Production account.
• 2 Discovery Services configured with discovery_group: "staging" polling data from Staging account.

The following snippet describes the different configuration options for the Discovery Service and their default values.

# This section configures the Discovery Service
discovery_service:
    enabled: "yes"
    # discovery_group is used to group discovered resources into different
    # sets. This is useful when you have multiple Teleport Discovery services
    # running in the same cluster but polling different cloud providers or cloud
    # accounts. It prevents discovered services from colliding in Teleport when
    # managing discovered resources.
    discovery_group: "prod"
    aws:
       # AWS resource types. Valid options are:
       # eks - discovers and registers AWS EKS clusters
       # ec2 - discovers and registers AWS EC2 Machines
     - types: ["eks"]
       # AWS regions to search for resources from
       regions: ["us-east-1", "us-west-1"]
       # AWS resource tags to match when registering resources
       # Optional section: Defaults to "*":"*"
       tags:
         "env": "prod"
       # AWS role to assume when discovering resources in the AWS Account.
       # This value is an optional AWS role ARN to assume when polling EKS clusters
       assume_role_arn: arn:aws:iam::123456789012:role/iam-discovery-role
       # External ID is an optional value that should be set when accessing
       # your AWS account from a third-party service (delegated access).
       external_id: "example-external-id"
    # Matchers for discovering Azure-hosted resources.
    azure:
      # Azure resource types. Valid options are:
      # 'aks' - discovers and registers Azure AKS Kubernetes Clusters.
    - types: ["aks"]
      # Azure regions to search for resources from. Valid options are:
      # '*' - discovers resources in all regions (default).
      # Any valid Azure region name. List all valid regions using the Azure "az" cli: `az account list-locations -o table`
      regions: ["*"]
      # Azure subscription IDs to search resources from. Valid options are:
      # '*' - discovers resources in all subscriptions (default).
      # Any subscription_id: `az account subscription list -o table`
      subscriptions: ["*"]
      # Azure resource groups to search resources from. Valid options are:
      # '*' - discovers resources in all resource groups within configured subscription(s) (default).
      # Any resource_groups: `az group list -o table`
      resource_groups: ["*"]
      # Azure resource tag filters used to match resources.
      # Optional section: Defaults to "*":"*"
      tags:
        "env": "prod"
    # Matchers for discovering GCP-hosted resources.
    gcp:
      # GCP resource types. Valid options are:
      # 'gke' - discovers and registers GCP GKE Kubernetes Clusters.
    - types: ["gke"]
      # GCP location to search for resources from. Valid options are:
      # '*' - discovers resources in all locations (default).
      # Any valid GCP region or zone name.
      locations: ["*"]
      # GCP project ID
      project_ids: ["myproject"]
      # GCP resource tag filters used to match resources.
      # Optional section: Defaults to "*":"*"
      tags:
        "*" : "*"

Forwarding requests to the Kubernetes Cluster

The Teleport Kubernetes Service is responsible for monitoring the dynamic resources created or updated by the Discovery Service and forwarding requests to the Kubernetes clusters they represent. To work correctly, it requires direct access to the target Kubernetes clusters and permissions to forward requests.

To turn on dynamic resource monitoring in the Kubernetes Service, you must configure the kubernetes_service.resources section as shown in the following snippet:

## This section configures the Kubernetes Service
kubernetes_service:
    enabled: "yes"
    # Matchers for dynamic Kubernetes cluster resources created with the "tctl create" command or by Kubernetes auto-discovery.
    resources:
    - labels:
        "*": "*" # can be configured to limit the clusters to watched by this service.

Both services — Discovery and Kubernetes — can be configured in the same Teleport process or separate processes.