Scaling Privileged Access for Modern Infrastructure: Real-World Insights
Apr 25
Virtual
Register Today
Teleport logoTry For Free
Fork me on GitHub

Teleport

Database Access Getting Started Guide

Teleport can provide secure access to Aurora via the Teleport Database Service. This allows for fine-grained access control through Teleport's RBAC.

In this guide, you will:

  1. Configure an AWS Aurora database with IAM authentication.
  2. Join the Aurora database to your Teleport cluster.
  3. Connect to the Aurora database via the Teleport Database Service.

Prerequisites

  • A running Teleport cluster. If you want to get started with Teleport, sign up for a free trial or set up a demo environment.

  • The tctl admin tool and tsh client tool version >= 15.2.2.

    On Teleport Enterprise, you must use the Enterprise version of tctl, which you can download from your Teleport account workspace. Otherwise, visit Installation for instructions on downloading tctl and tsh for Teleport Community Edition.

  • An AWS account with a PostgreSQL AWS Aurora database and permissions to create and attach IAM policies.
  • A host, e.g., an EC2 instance, where you will run the Teleport Database Service.
  • To check that you can connect to your Teleport cluster, sign in with tsh login, then verify that you can run tctl commands using your current credentials. tctl is supported on macOS and Linux machines. For example:
    tsh login --proxy=teleport.example.com --user=[email protected]
    tctl status

    Cluster teleport.example.com

    Version 15.2.2

    CA pin sha256:abdc1245efgh5678abdc1245efgh5678abdc1245efgh5678abdc1245efgh5678

    If you can connect to the cluster and run the tctl status command, you can use your current credentials to run subsequent tctl commands from your workstation. If you host your own Teleport cluster, you can also run tctl commands on the computer that hosts the Teleport Auth Service for full permissions.

Step 1/5. Set up Aurora

In order to allow Teleport connections to an Aurora instance, the instance needs to support IAM authentication.

If you don't have a database provisioned yet, create an instance of an Aurora PostgreSQL in the RDS control panel. Make sure to choose the "Standard create" database creation method and enable "Password and IAM database authentication" in the Database Authentication dialog.

For existing Aurora instances, the status of IAM authentication is displayed on the Configuration tab and can be enabled by modifying the database instance.

Next, create the following IAM policy and attach it to the AWS user or service account. The Teleport Database Service will need to use the credentials of this AWS user or service account in order to use this policy.

{
   "Version": "2012-10-17",
   "Statement": [
      {
         "Effect": "Allow",
         "Action": [
             "rds-db:connect"
         ],
         "Resource": [
             "arn:aws:rds-db:<region>:<account-id>:dbuser:<resource-id>/*"
         ]
      }
   ]
}

This policy allows any database account to connect to the Aurora instance specified with resource ID using IAM auth.

Resource ID

The database resource ID is shown on the Configuration tab of a particular database instance in the RDS control panel, under "Resource id". For regular RDS database it starts with db- prefix. For Aurora, use the database cluster resource ID (cluster-), not the individual instance ID.

Finally, connect to the database and create a database account with IAM auth support (or update an existing one). Once connected, execute the following SQL statements to create a new database account and allow IAM auth for it:

CREATE USER alice;
GRANT rds_iam TO alice;

For more information about connecting to the PostgreSQL instance directly, see the AWS documentation.

See the Automatic User Provisioning guide for how to configure Teleport to create accounts for your PostgreSQL users on demand.

Step 2/5. Configure the Teleport Database Service

The Database Service requires a valid auth token to connect to the cluster. Generate one by running the following command against your Teleport Auth Service and save it in /tmp/token on the node that will run the Database Service:

tctl tokens add --type=db

For users with a lot of infrastructure in AWS, or who might create or recreate many instances, consider alternative methods for joining new EC2 instances running Teleport:

Install Teleport on the host where you will run the Teleport Database Service:

Install Teleport on your Linux server:

  1. Assign edition to one of the following, depending on your Teleport edition:

    EditionValue
    Teleport Enterprise Cloudcloud
    Teleport Enterprise (Self-Hosted)enterprise
    Teleport Community Editionoss
  2. Get the version of Teleport to install. If you have automatic agent updates enabled in your cluster, query the latest Teleport version that is compatible with the updater:

    TELEPORT_DOMAIN=example.teleport.com
    TELEPORT_VERSION="$(curl https://$TELEPORT_DOMAIN/v1/webapi/automaticupgrades/channel/default/version | sed 's/v//')"

    Otherwise, get the version of your Teleport cluster:

    TELEPORT_DOMAIN=example.teleport.com
    TELEPORT_VERSION="$(curl https://$TELEPORT_DOMAIN/v1/webapi/ping | jq -r '.server_version')"
  3. Install Teleport on your Linux server:

    curl https://goteleport.com/static/install.sh | bash -s ${TELEPORT_VERSION} edition

    The installation script detects the package manager on your Linux server and uses it to install Teleport binaries. To customize your installation, learn about the Teleport package repositories in the installation guide.

On the node where you will run the Teleport Database Service, configure Teleport and point it to your Aurora database instance. Make sure to update the database endpoint and region appropriately. The --proxy flag must point to the address of your Teleport Proxy Service.

sudo teleport db configure create \ --token=/tmp/token \ --name=aurora \ --proxy=teleport.example.com:443 \ --protocol=postgres \ --uri=postgres-aurora-instance-1.abcdefghijklm.us-west-1.rds.amazonaws.com:5432 \ --aws-region=us-west-1 \ --output file:///etc/teleport.yaml

On the node where you will run the Teleport Database Service, configure Teleport and point it to your Aurora database instance. Make sure to update the database endpoint and region appropriately. The --proxy flag must point to the address of your Teleport Cloud tenant.

sudo teleport db configure create \ --token=/tmp/token \ --name=aurora \ --proxy=mytenant.teleport.sh:443 \ --protocol=postgres \ --uri=postgres-aurora-instance-1.abcdefghijklm.us-west-1.rds.amazonaws.com:5432 \ --aws-region=us-west-1 \ --output file:///etc/teleport.yaml
AWS Credentials

The node that connects to the database should have AWS credentials configured with the policy from step 1.

Step 3/5. Start the Database Service

Start the Teleport Database Service in your environment:

Configure the Database Service to start automatically when the host boots up by creating a systemd service for it. The instructions depend on how you installed the Database Service.

On the host where you will run the Database Service, enable and start Teleport:

sudo systemctl enable teleport
sudo systemctl start teleport

On the host where you will run the Database Service, create a systemd service configuration for Teleport, enable the Teleport service, and start Teleport:

sudo teleport install systemd -o /etc/systemd/system/teleport.service
sudo systemctl enable teleport
sudo systemctl start teleport

You can check the status of the Database Service with systemctl status teleport and view its logs with journalctl -fu teleport.

Step 4/5. Create a user and role

Create the role that will allow a user to connect to any database using any database account:

tctl create <<EOFkind: roleversion: v3metadata: name: dbspec: allow: db_labels: '*': '*' db_names: - '*' db_users: - '*'EOF

Create the Teleport user assigned the db role we've just created:

tctl users add --roles=access,db alice
tctl users add --roles=access,requester,db alice

Step 5/5. Connect

Now that Aurora is configured with IAM authentication, Teleport is running, and the local user is created, we're ready to connect to the database.

Log in to Teleport with the user we've just created.

tsh login --proxy=teleport.example.com --user=alice
tsh login --proxy=mytenant.teleport.sh --user=alice

Now we can inspect available databases:

tsh db ls

Finally, connect to the database:

tsh db connect --db-user=alice --db-name postgres aurora

Auditing

You can view database session activity in the audit log. After a session is uploaded, you can play back the audit data with the tsh play command.

Database session ID will be in a UUID format (ex: 307b49d6-56c7-4d20-8cf0-5bc5348a7101) See the audit log to get a database session ID with a key of sid.

Example:

tsh play --format json database.session
    {
        "cluster_name": "teleport.example.com",
        "code": "TDB02I",
        "db_name": "example",
        "db_origin": "dynamic",
        "db_protocol": "postgres",
        "db_query": "select * from sample;",
        "db_roles": [
            "access"
        ],
        "db_service": "example",
        "db_type": "rds",
        "db_uri": "databases-1.us-east-1.rds.amazonaws.com:5432",
        "db_user": "alice",
        "ei": 2,
        "event": "db.session.query",
        "sid": "307b49d6-56c7-4d20-8cf0-5bc5348a7101",
        "success": true,
        "time": "2023-10-06T10:58:32.88Z",
        "uid": "a649d925-9dac-44cc-bd04-4387c295580f",
        "user": "alice"
    }

The audit log is viewable in Activity under Management in the Web UI for users with permission to the event resources. Database sessions do not appear in the session recordings page.

Troubleshooting

Certificate error

If your tsh db connect error includes the following text, you likely have an RDS database created before July 28, 2020, which presents an X.509 certificate that is incompatible with Teleport:

x509: certificate relies on legacy Common Name field, use SANs instead

AWS provides instructions to rotate your SSL/TLS certificate.

No credential providers error

If you see the error NoCredentialProviders: no valid providers in chain in Database Service logs then Teleport is not detecting the required credentials to connect via AWS IAM permissions. Check whether the credentials or security role has been applied in the machine running the Teleport Database Service.

When running on EKS, this error may occur if the Teleport Database Service cannot access IMDSv2 when the PUT requests hop limit on the worker node instance is set to 1. You can use the following commands to check the hop limit:

aws ec2 describe-instances --instance-ids <node-instance-id> | grep HttpPutResponseHopLimit
"HttpPutResponseHopLimit": 1,

See IMDSv2 support for EKS and EKS best practices for more details.

Timeout errors

The Teleport Database Service needs connectivity to your database endpoints. That may require enabling inbound traffic on the database from the Database Service on the same VPC or routing rules from another VPC. Using the nc program you can verify connections to databases:

nc -zv postgres-instance-1.sadas.us-east-1.rds.amazonaws.com 5432

Connection to postgres-instance-1.sadas.us-east-1.rds.amazonaws.com (172.31.24.172) 5432 port [tcp/postgresql] succeeded!

Not authorized to perform sts:AssumeRole

The Database Service assumes an IAM role in one of following situations:

  • An IAM role is used as db_user when accessing AWS services that require IAM roles as database users, such as DynamoDB, Keyspaces, Opensearch, and Redshift Serverless.
  • The assume_role_arn field is specified for the database resources or dynamic resource matchers.

When both of the above conditions are true for a database connection, the Database Service performs a role chaining by assuming the IAM role specified assume_role_arn first then using that IAM role to assume the IAM role for db_user.

You may encounter the following error if the trust relationship is not configured properly between the IAM roles:

AccessDenied: User: arn:aws:sts::111111111111:assumed-role/database-service-role/i-* is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::111111111111:role/database-user-role

To allow IAM Role role1 to assume IAM Role role2, the following is generally required:

role1 or its AWS account should be set as Principal in role2's trust policy.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::111111111111:role/role1"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::111111111111:root"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::123456789012:role/role1"
      },
      "Action": "sts:AssumeRole",
      "Condition": {
        "StringEquals": {
          "sts:ExternalId": "example-external-id"
        }
      }
    }
  ]
}

role1 requires sts:AssumeRole permissions, for example:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": "sts:AssumeRole",
            "Effect": "Allow",
            "Resource": "arn:aws:iam::111111111111:role/role2"
        }
    ]
}

Note that this policy can be omitted when role1 and role2 are in the same AWS account and role1's full ARN is configured as Principal in role2's trust policy.

role1 also requires sts:AssumeRole permissions in its boundary policy, for example:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": "sts:AssumeRole",
            "Effect": "Allow",
            "Resource": "*"
        }
    ]
}

Note that this is only required when a boundary policy is attached to role1.

You can test the trust relationship by running this AWS CLI command as role1:

aws sts assume-role --role-arn arn:aws:iam::111111111111:role/role2 --role-session-name test-trust-relationship

Learn more on how to use trust policies with IAM roles.

Next Steps

For the next steps, dive deeper into the topics relevant to your Database Access use-case, for example: