Fork me on GitHub

Teleport

Configure Teleport to Automatically Enroll EC2 instances (Preview)

Improve

The Teleport Discovery Service can connect to Amazon EC2 and automatically discover and enroll EC2 instances matching configured labels. It will then execute an install script on these discovered instances using AWS Systems Manager that will install Teleport, start it and join the cluster.

Prerequisites

  • A running Teleport cluster. For details on how to set this up, see one of our Getting Started guides.

  • The tctl admin tool and tsh client tool version >= 11.0.3.

    tctl version

    Teleport v11.0.3 go1.19

    tsh version

    Teleport v11.0.3 go1.19

    See Installation for details.

  • A running Teleport cluster. For details on how to set this up, see our Enterprise Getting Started guide.

  • The tctl admin tool and tsh client tool version >= 11.0.3, which you can download by visiting the customer portal.

    tctl version

    Teleport v11.0.3 go1.19

    tsh version

    Teleport v11.0.3 go1.19

  • A Teleport Cloud account. If you do not have one, visit the sign up page to begin your free trial.

  • The tctl admin tool and tsh client tool version >= 10.3.8. To download these tools, visit the Downloads page.

    tctl version

    Teleport v10.3.8 go1.19

    tsh version

    Teleport v10.3.8 go1.19

  • AWS account with EC2 instances and permissions to create and attach IAM policies as well as permission to create Systems Manager documents.
  • EC2 instances running Ubuntu/Debian/RHEL/Amazon Linux 2 if making use of the default Teleport install script. (For other Linux distributions, you can install Teleport manually.)

To connect to Teleport, log in to your cluster using tsh, then use tctl remotely:

tsh login --proxy=teleport.example.com [email protected]
tctl status

Cluster teleport.example.com

Version 11.0.3

CA pin sha256:abdc1245efgh5678abdc1245efgh5678abdc1245efgh5678abdc1245efgh5678

You can run subsequent tctl commands in this guide on your local machine.

For full privileges, you can also run tctl commands on your Auth Service host.

To connect to Teleport, log in to your cluster using tsh, then use tctl remotely:

tsh login --proxy=myinstance.teleport.sh [email protected]
tctl status

Cluster myinstance.teleport.sh

Version 10.3.8

CA pin sha256:sha-hash-here

You must run subsequent tctl commands in this guide on your local machine.

Step 1/5. Create an EC2 invite token

When discovering EC2 instances, Teleport makes use of IAM invite tokens for authenticating joining Nodes.

Create a file called token.yaml:

# token.yaml
kind: token
version: v2
metadata:
  # the token name is not a secret because instances must prove that they are
  # running in your AWS account to use this token
  name: aws-discovery-iam-token
  # set a long expiry time, as the default for tokens is only 30 minutes
  expires: "3000-01-01T00:00:00Z"
spec:
  # use the minimal set of roles required
  roles: [Node]

  # set the join method allowed for this token
  join_method: iam

  allow:
  # specify the AWS account which Nodes may join from
  - aws_account: "123456789"

Add the token to the Teleport cluster with:

tctl create -f token.yaml

Step 2/5. Create an AWS Systems Manager Document

Teleport makes use of AWS Systems Manager documents to execute commands necessary to install and start Teleport on discovered EC2 instances.

A document with the following content should be created, named TeleportDiscoveryInstaller:

# document.yaml
---
schemaVersion: '2.2'
description: aws:runShellScript
parameters:
  token:
    type: String
    description: "(Required) The Teleport invite token to use when joining the cluster."
  scriptName:
    type: String
    description: "(Required) The Teleport installer script to use when joining the cluster."
mainSteps:
- action: aws:downloadContent
  name: downloadContent
  inputs:
    sourceType: "HTTP"
    destinationPath: "/tmp/installTeleport.sh"
    sourceInfo:
      url: "https://teleport.example.com/webapi/scripts/installer/{{ scriptName }}"
- action: aws:runShellScript
  name: runShellScript
  inputs:
    timeoutSeconds: '300'
    runCommand:
      - /bin/sh /tmp/installTeleport.sh "{{ token }}"

Here, teleport.example.com should be substituted with the public proxy URL of the Teleport cluster, e.g. mytenant.teleport.sh for a Teleport Cloud tenant.

The document may be created by executing the following AWS command:

aws ssm create-document \ --content file://document.yaml \ --name "TeleportDiscoveryInstaller" \ --document-type "Command" \ --document-format YAML

When Teleport executes the Systems Manager document it will provide the token and scriptName parameters with defaults.

  • {{ scriptName }} will by default evaluate to default-installer
  • {{ token }} will by default evaluate to aws-discovery-iam-token

The document TeleportDiscoveryInstaller may be named differently however the discovery_service section must be modified to include the ssm section. For example:

discovery_service:
  aws:
    - types: ["ec2"] 
      ssm:
        document_name: "OtherTeleportDiscoveryDocument"

Step 3/5. Set up AWS IAM credentials

In order for Teleport to send commands to EC2 instances, the instance performing discovery requires an IAM policy with permissions to execute Systems Manager commands and retrieve EC2 instances:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ssm:SendCommand",
                "ec2:DescribeInstances",
                "ssm:GetCommandInvocation"
            ],
            "Resource": "*"
        }
    ]
}

All EC2 instances that are to be added to the Teleport cluster by the Discovery Service must include the AmazonSSMManagedInstanceCore IAM policy in order to receive commands from the Discovery Service.

This policy includes the following permissions:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ssm:DescribeAssociation",
                "ssm:GetDeployablePatchSnapshotForInstance",
                "ssm:GetDocument",
                "ssm:DescribeDocument",
                "ssm:GetManifest",
                "ssm:GetParameter",
                "ssm:GetParameters",
                "ssm:ListAssociations",
                "ssm:ListInstanceAssociations",
                "ssm:PutInventory",
                "ssm:PutComplianceItems",
                "ssm:PutConfigurePackageResult",
                "ssm:UpdateAssociationStatus",
                "ssm:UpdateInstanceAssociationStatus",
                "ssm:UpdateInstanceInformation"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "ssmmessages:CreateControlChannel",
                "ssmmessages:CreateDataChannel",
                "ssmmessages:OpenControlChannel",
                "ssmmessages:OpenDataChannel"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "ec2messages:AcknowledgeMessage",
                "ec2messages:DeleteMessage",
                "ec2messages:FailMessage",
                "ec2messages:GetEndpoint",
                "ec2messages:GetMessages",
                "ec2messages:SendReply"
            ],
            "Resource": "*"
        }
    ]
}

Step 4/5. Configure Teleport to discover EC2 instances

The Teleport Discovery Service requires a valid auth token to connect to the cluster. Generate one by running the following command against your Teleport Auth Service and save it in /tmp/token on the Node that will run the Discovery Service:

tctl tokens add --type=discovery

In order to enable EC2 instance discovery the discovery_service.aws section must include at least one entry:

version: v2
teleport:
  join_params:
    token_name: "/tmp/token"
    method: token
  auth_servers:
  - "teleport.example.com:3080"
auth_service:
  enabled: off
proxy_service:
  enabled: off
ssh_service:
  enabled: off
discovery_service:
  enabled: "yes"
  aws:
   - types: ["ec2"]
     regions: ["us-east-1","us-west-1"]
     tags:
       "env": "prod" # Match EC2 instances where tag:env=prod

Once Teleport is configured for discovery, it can be started and EC2 instances matching the tags specified in the AWS section will begin to be added to the Teleport cluster automatically.

Step 5/5. [Optional] Customize the default installer script

To customize the default installer script, execute the following command on your workstation:

tctl get installer/default-installer > teleport-default-installer.yaml

The resulting teleport-default-installer.yaml can the be edited to change what gets executed when enrolling discovered EC2 instances.

After making the desired changes to the default installer, the resource can be updated by executing:

tctl create teleport-default-installer.yaml

Multiple installer resources can exist and be specified in the install/script_name section:

discovery_service:
  aws:
    - types: ["ec2"] 
      tags:
       - "env": "prod"
      install: # optional section when default-installer is used.
        script_name: "default-installer" 
    - types: ["ec2"] 
      tags:
       - "env": "devel"
      install:
        script_name: "devel-installer"		

The installer resource has the following templating options:

  • {{ .MajorVersion }}: the major version of Teleport to use when installing from the repository.
  • {{ .PublicProxyAddr }}: the public address of the Teleport Proxy Service to connect to.

These can be used as follows:

kind: installer
metadata:
  name: default-installer
spec:
  script: |
    echo {{ .PublicProxyAddr }}
    echo Teleport-{{ .MajorVersion }}
version: v1

Which, when retrieved by the Systems Manager document will evaluate to a script with the following contents:

echo teleport.example.com
echo Teleport-11.0.3

The default installer will take the following actions:

  • Add an official Teleport repository to supported Linux distributions.
  • Install Teleport via apt or yum.
  • Generate the Teleport config file and write it to /etc/teleport.yaml.
  • Enable and start the Teleport service.

Troubleshooting

If Installs are showing failed or instances are failing to appear check the Command history in AWS System Manager -> Node Management -> Run Command. Select the instance-id of the Target to review Errors.

Next Steps

Documentation on IAM invite tokens can be found in our Joining Nodes via AWS IAM Role guide.

Information on IAM best practices on EC2 instances managed by Systems Manager can be found for IAM best practices on EC2 instances managed by Systems Manager can be found in the AWS Cloud Operations & Migrations Blog.

Full documentation on EC2 discovery configuration can be found through the config file reference documentation.

The complete default installer can be found with the Teleport source.