Scaling Privileged Access for Modern Infrastructure: Real-World Insights
Apr 25
Virtual
Register Today
Teleport logoTry For Free
Fork me on GitHub

Teleport

Database Access with Redis Cluster

If you want to configure Redis Standalone, please read Database Access with Redis.

Teleport can provide secure access to Redis cluster via the Teleport Database Service. This allows for fine-grained access control through Teleport's RBAC.

In this guide, you will:

  1. Configure an Redis cluster database with mutual TLS authentication.
  2. Join the Redis cluster database to your Teleport cluster.
  3. Connect to the Redis cluster database via the Teleport Database Service.

Prerequisites

  • A running Teleport cluster. If you want to get started with Teleport, sign up for a free trial or set up a demo environment.

  • The tctl admin tool and tsh client tool version >= 15.2.2.

    On Teleport Enterprise, you must use the Enterprise version of tctl, which you can download from your Teleport account workspace. Otherwise, visit Installation for instructions on downloading tctl and tsh for Teleport Community Edition.

  • Redis version 6.0 or newer.

    Note

    RESP3 (REdis Serialization Protocol) is currently not supported.

  • redis-cli version 6.2 or newer installed and added to your system's PATH environment variable.

  • A host where you will run the Teleport Database Service.

  • A certificate authority to issue server certificates for nodes in your Redis Cluster.

    Distributed databases like Redis Cluster use mTLS for node-to-node communication. Teleport requires that you have your own CA to issue certificates for node-to-node mTLS communication.

    Teleport uses a split-CA architecture for database access. The Teleport db CA issues server certificates and the db_client CA issues client certificates.

    Databases are configured to trust the Teleport db_client CA for client authentication, but not the db CA. Additionally, Teleport only issues ephemeral db_client CA certificates.

    When a Redis Cluster node connects to another Redis Cluster node, it must present a certificate that the other node trusts for client authentication. Since Teleport does not issue long-lived db_client certificates, the node needs to have a long-lived certificate issued by another CA that its peer node trusts.

    The split db and db_client CA architecture was introduced as a security fix in Teleport versions: 13.4.17, 14.3.7, and 15.

    See Database CA Migrations for more information.

  • To check that you can connect to your Teleport cluster, sign in with tsh login, then verify that you can run tctl commands using your current credentials. tctl is supported on macOS and Linux machines.

    For example:

    tsh login --proxy=teleport.example.com --user=[email protected]
    tctl status

    Cluster teleport.example.com

    Version 15.2.2

    CA pin sha256:abdc1245efgh5678abdc1245efgh5678abdc1245efgh5678abdc1245efgh5678

    If you can connect to the cluster and run the tctl status command, you can use your current credentials to run subsequent tctl commands from your workstation. If you host your own Teleport cluster, you can also run tctl commands on the computer that hosts the Teleport Auth Service for full permissions.

Step 1/6. Set up the Teleport Database Service

The Database Service requires a valid auth token to connect to the cluster. Generate one by running the following command against your Teleport Auth Service and save it in /tmp/token on the node that will run the Database Service:

tctl tokens add --type=db

Install and configure Teleport where you will run the Teleport Database Service:

Install Teleport on your Linux server:

  1. Assign edition to one of the following, depending on your Teleport edition:

    EditionValue
    Teleport Enterprise Cloudcloud
    Teleport Enterprise (Self-Hosted)enterprise
    Teleport Community Editionoss
  2. Get the version of Teleport to install. If you have automatic agent updates enabled in your cluster, query the latest Teleport version that is compatible with the updater:

    TELEPORT_DOMAIN=example.teleport.com
    TELEPORT_VERSION="$(curl https://$TELEPORT_DOMAIN/v1/webapi/automaticupgrades/channel/default/version | sed 's/v//')"

    Otherwise, get the version of your Teleport cluster:

    TELEPORT_DOMAIN=example.teleport.com
    TELEPORT_VERSION="$(curl https://$TELEPORT_DOMAIN/v1/webapi/ping | jq -r '.server_version')"
  3. Install Teleport on your Linux server:

    curl https://goteleport.com/static/install.sh | bash -s ${TELEPORT_VERSION} edition

    The installation script detects the package manager on your Linux server and uses it to install Teleport binaries. To customize your installation, learn about the Teleport package repositories in the installation guide.

On the host where you will run the Teleport Database Service, start Teleport with the appropriate configuration.

Note that a single Teleport process can run multiple different services, for example multiple Database Service agents as well as the SSH Service or Application Service. The step below will overwrite an existing configuration file, so if you're running multiple services add --output=stdout to print the config in your terminal, and manually adjust /etc/teleport.yaml.

Generate a configuration file at /etc/teleport.yaml for the Database Service:

sudo teleport db configure create \ -o file \ --token=/tmp/token \ --proxy=teleport.example.com:443 \ --name=example-redis \ --protocol=redis \ --uri=rediss://redis.example.com:6379?mode=cluster \ --labels=env=dev
sudo teleport db configure create \ -o file \ --token=/tmp/token \ --proxy=mytenant.teleport.sh:443 \ --name=example-redis \ --protocol=redis \ --uri=rediss://redis.example.com:6379?mode=cluster \ --labels=env=dev

Configure the Teleport Database Service to start automatically when the host boots up by creating a systemd service for it. The instructions depend on how you installed the Teleport Database Service.

On the host where you will run the Teleport Database Service, enable and start Teleport:

sudo systemctl enable teleport
sudo systemctl start teleport

On the host where you will run the Teleport Database Service, create a systemd service configuration for Teleport, enable the Teleport service, and start Teleport:

sudo teleport install systemd -o /etc/systemd/system/teleport.service
sudo systemctl enable teleport
sudo systemctl start teleport

You can check the status of the Teleport Database Service with systemctl status teleport and view its logs with journalctl -fu teleport.

Teleport provides Helm charts for installing the Teleport Database Service in Kubernetes Clusters.

Set up the Teleport Helm repository.

Allow Helm to install charts that are hosted in the Teleport Helm repository:

helm repo add teleport https://charts.releases.teleport.dev

Update the cache of charts from the remote repository so you can upgrade to all available releases:

helm repo update

Install the Teleport Kube Agent into your Kubernetes Cluster with the Teleport Database Service configuration.

JOIN_TOKEN=$(cat /tmp/token)
helm install teleport-kube-agent teleport/teleport-kube-agent \ --create-namespace \ --namespace teleport-agent \ --set roles=db \ --set proxyAddr=teleport.example.com:443 \ --set authToken=${JOIN_TOKEN?} \ --set "databases[0].name=example-redis" \ --set "databases[0].uri=rediss://redis.example.com:6379?mode=cluster" \ --set "databases[0].protocol=redis" \ --set "labels.env=dev" \ --version 15.2.2

Install the Teleport Kube Agent into your Kubernetes Cluster with the Teleport Database Service configuration.

JOIN_TOKEN=$(cat /tmp/token)
helm install teleport-kube-agent teleport/teleport-kube-agent \ --create-namespace \ --namespace teleport-agent \ --set roles=db \ --set proxyAddr=mytenant.teleport.sh:443 \ --set authToken=${JOIN_TOKEN?} \ --set "databases[0].name=example-redis" \ --set "databases[0].uri=rediss://redis.example.com:6379?mode=cluster" \ --set "databases[0].protocol=redis" \ --set "labels.env=dev" \ --version 15.2.2

Make sure that the Teleport agent pod is running. You should see one teleport-kube-agent pod with a single ready container:

kubectl -n teleport-agent get pods
NAME READY STATUS RESTARTS AGEteleport-kube-agent-0 1/1 Running 0 32s
Tip

A single Teleport process can run multiple services, for example multiple Database Service instances as well as other services such the SSH Service or Application Service.

Step 2/6. Create a Teleport user

Tip

To modify an existing user to provide access to the Database Service, see Database Access Access Controls

Create a local Teleport user with the built-in access role:

tctl users add \ --roles=access \ --db-users=\* \ --db-names=\* \ alice

Create a local Teleport user with the built-in access and requester roles:

tctl users add \ --roles=access,requester \ --db-users=\* \ --db-names=\* \ alice
FlagDescription
--rolesList of roles to assign to the user. The builtin access role allows them to connect to any database server registered with Teleport.
--db-usersList of database usernames the user will be allowed to use when connecting to the databases. A wildcard allows any user.
--db-namesList of logical databases (aka schemas) the user will be allowed to connect to within a database server. A wildcard allows any database.
Warning

Database names are only enforced for PostgreSQL and MongoDB databases.

For more detailed information about database access controls and how to restrict access see RBAC documentation.

Step 3/6. Create Redis users

Each Redis user must be protected by a strong password. We recommend using OpenSSL to generate passwords:

openssl rand -hex 32
Note

If you have access to Redis you can also generate a password by using the below command from the Redis console:

ACL GENPASS

Create a users.acl file, which defines users for your Redis deployment, passwords required to log in as a given user, and sets of ACL rules. Redis allows you to provide passwords in plaintext or an SHA256 hash. We strongly recommend using an SHA256 hash instead of plaintext passwords.

You can use the command below to generate an SHA256 hash from a password.

echo -n STRONG_GENERATED_PASSWORD | sha256sum
user alice on #57639ed88a85996453555f22f5aa4147b4c9614056585d931e5d976f610651e9 allcommands allkeys
user default off

For more ACL examples refer to the Redis documentation.

Warning

It's very important to either disable or protect with a password the default user. Otherwise, everyone with access to the database can log in as the default user, which by default has administrator privileges.

Step 4/6. Set up mutual TLS

Export your Teleport cluster's db_client CA cert and concatenate it with your Redis Cluster's CA cert (in PEM format):

tctl auth export --type=db_client > db-client-ca.crt
cat /path/to/your/ca.crt db-client-ca.crt > pem-bundle.cas

Using your Redis Cluster's CA, issue server.crt for each of your Redis Cluster nodes and enable mutual TLS in your redis.conf configuration file, then restart each node:

tls-port 7001
port 0
cluster-enabled yes
tls-replication yes
tls-cluster yes
aclfile /path/to/users.acl
masterauth GENERATED_STRONG_PASSWORD
masteruser replica-user
tls-cert-file /usr/local/etc/redis/certs/server.crt
tls-key-file /usr/local/etc/redis/certs/server.key
tls-ca-cert-file /usr/local/etc/redis/certs/pem-bundle.cas
tls-protocols "TLSv1.2 TLSv1.3"

Once mutual TLS has been enabled, you will no longer be able to connect to the cluster without providing a valid client certificate. You can use the tls-auth-clients optional setting to allow connections from clients that do not present a certificate.

See TLS Support in the Redis documentation for more details.

Modify the Teleport Database Service to trust your Redis Cluster CA:

  databases:
  - name: "example-redis"
    protocol: "redis"
    uri: "rediss://redis.example.com:6379?mode=cluster"
    static_labels:
      "env": "example"
    tls:
      ca_cert_file: "/path/to/your/ca.crt"

Now the Teleport Database Service will trust certificates presented by your Redis Cluster.

Step 5/6. Create a cluster

To create the cluster after mutual TLS is enabled, you will need to use a certificate that the Redis nodes trust for client authentication. You can use a certificate that you already issued for one of the nodes, i.e. server.crt or you can issue a new client certificate using your Redis Cluster CA.

Use the following command to create the cluster. Please note redis-cli --cluster create accepts only IP addresses.

export REDISCLI_AUTH=STRONG_GENERATED_PASSWORD
export CERTS_DIR=/path/to/certs/
export IP1=10.0.0.1 # update with the real node 1 IP
export IP2=10.0.0.2 # update with the real node 2 IP
export IP3=10.0.0.3 # update with the real node 3 IP
export IP4=10.0.0.4 # update with the real node 4 IP
export IP5=10.0.0.5 # update with the real node 5 IP
export IP6=10.0.0.6 # update with the real node 6 IP
redis-cli --user alice --cluster-replicas 1 --tls --cluster-yes \ --cluster create ${IP1}:7001 ${IP2}:7002 ${IP3}:7003 ${IP4}:7004 ${IP5}:7005 ${IP6}:7006 \ --cacert ${CERTS_DIR}/ca.crt --key ${CERTS_DIR}/server.key --cert ${CERTS_DIR}/server.crt

Step 6/6. Connect

To enable Redis cluster mode in Teleport, add the mode=cluster parameter to the connection URI in your Teleport Database Service config file.

databases:
  - name: "redis-cluster"
    uri: "rediss://redis.example.com:6379?mode=cluster"

Log into your Teleport cluster and see available databases:

tsh login --proxy=teleport.example.com --user=alice
tsh db ls

Name Description Labels

------------- --------------- --------

example-redis Example Redis env=dev

tsh login --proxy=mytenant.teleport.sh --user=alice
tsh db ls

Name Description Labels

------------- --------------- --------

example-redis Example Redis env=dev

To connect to a particular database instance:

tsh db connect example-redis

You can optionally specify the database user to use by default when connecting to the database instance:

tsh db connect --db-user=alice example-redis

If flag --db-user is not provided, Teleport logs in as the default user.

Now you can log in as the previously created user using the below command:

AUTH alice STRONG_GENERATED_PASSWORD

To log out of the database and remove credentials:

Remove credentials for a particular database instance.

tsh db logout example-redis

Remove credentials for all database instances.

tsh db logout

Supported Redis Cluster commands

Redis in cluster mode does not support the following commands. If one of the listed commands above is called Teleport returns the ERR Teleport: command not supported error.

  • ACL
  • ASKING
  • CLIENT
  • CLUSTER
  • CONFIG
  • DEBUG
  • EXEC
  • HELLO
  • INFO
  • LATENCY
  • MEMORY
  • MIGRATE
  • MODULE
  • MONITOR
  • MULTI
  • PFDEBUG
  • PFSELFTEST
  • PSUBSCRIBE
  • PSYNC
  • PUNSUBSCRIBE
  • PUNSUBSCRIBE
  • READONLY
  • READWRITE
  • REPLCONF
  • REPLICAOF
  • ROLE
  • SCAN
  • SCRIPT DEBUG
  • SCRIPT KILL
  • SHUTDOWN
  • SLAVEOF
  • SLOWLOG
  • SSUBSCRIBE
  • SUNSUBSCRIBE
  • SYNC
  • TIME
  • WAIT
  • WATCH

Teleport conducts additional processing on the following commands before communicating with Redis Cluster:

CommandDescription
DBSIZESends the query to all nodes and returns the number of keys in the whole cluster.
KEYSSends the query to all nodes and returns a list of all keys in the whole cluster.
MGETTranslates the commands to multiple GETs and sends them to multiple nodes. Result is merged in Teleport and returned back to the client. If Teleport fails to fetch at least one key an error is returned.
FLUSHDBSends the query to all nodes.
FLUSHALLWorks the same as FLUSHDB.
SCRIPT EXISTSSends the query to all nodes. 1 is returned only if script exists on all nodes.
SCRIPT LOADSends the script to all nodes.
SCRIPT FLUSHSends the query to all nodes. ASYNC parameter is ignored.

Next steps

  • Take a look at the YAML configuration reference.