Skip to main content
Version: 19.x (unreleased)

GCP Database Discovery

Report an Issue

After you complete this guide, Teleport will automatically discover and register Cloud SQL for PostgreSQL and MySQL instances and proxy connections to them. Configure Teleport RBAC and database privileges separately.

For manual enrollment guides, including Cloud Spanner and AlloyDB, see Enroll Google Cloud Databases.

How it works​

Teleport database auto-discovery involves two components:

  1. The Teleport Discovery Service that watches for new databases or changes to previously discovered databases. It dynamically registers each discovered database as a db resource in your Teleport cluster. It does not need connectivity to the databases it discovers.
  2. The Teleport Database Service that monitors the dynamic db resources registered by the Discovery Service. It proxies communications between users and the database.

Prerequisites​

  • A running Teleport cluster accessible at a hostname with a valid TLS certificate. If you want to get started with Teleport, sign up for a free trial or set up a demo environment.

  • The tctl and tsh clients.

    Installing tctl and tsh clients

    1. Determine the version of your Teleport cluster. The tctl and tsh clients must be at most one major version behind your Teleport cluster version. Send a GET request to the Proxy Service at /v1/webapi/find and use a JSON query tool to obtain your cluster version. Replace teleport.example.com:443 with the web address of your Teleport Proxy Service:

      TELEPORT_DOMAIN=teleport.example.com:443
      TELEPORT_VERSION="$(curl -s https://$TELEPORT_DOMAIN/v1/webapi/find | jq -r '.server_version')"

    2. Follow the instructions for your platform to install tctl and tsh clients:

      Download the signed macOS .pkg installer for Teleport, which includes the tctl and tsh clients:

      curl -O https://cdn.teleport.dev/teleport-${TELEPORT_VERSION?}.pkg

      In Finder double-click the pkg file to begin installation.

      danger

      Using Homebrew to install Teleport is not supported. The Teleport package in Homebrew is not maintained by Teleport and we can't guarantee its reliability or security.

    Connecting with TLS routing disabled

    This guide's commands assume your Teleport cluster uses TLS routing (proxy_listener_mode: multiplex), where the tctl and tsh clients reach every Teleport service through the Proxy Service's web address on port 443. If you're not sure whether this applies to your cluster, check with whoever manages it.

    If your cluster uses separate listener ports instead, adjust ports as follows:

    • tsh commands (e.g., tsh login --proxy=...): continue using the Proxy Service web address on port 3080 (or 443 if behind a load balancer). Do not change these to port 3025.

    • Direct tctl or Auth Service API commands: use port 3025 for the Auth Service gRPC listener:

      tctl status --auth-server=teleport.example.com:3025
  • A Google Cloud project with one or more Cloud SQL instances.
  • Permission to create service accounts and grant IAM roles in that project.
  • A host to run the Teleport Discovery Service.
  • A host, such as a Compute Engine instance, to run the Teleport Database Service. The host must be able to reach your Cloud SQL instances over their public IP, private IP (PSA), or Private Service Connect endpoint; the Discovery Service does not need connectivity to your instances. For the requirements of each option, such as authorized networks for public IP, see Choose how to connect to Cloud SQL in the Google Cloud documentation.
  • Both hosts must be able to reach the Teleport Proxy and Google Cloud APIs over HTTPS.
  • A Google Cloud service account for the Discovery Service, granted the predefined Cloud SQL Viewer role (roles/cloudsql.viewer).
  • A Google Cloud service account for the Database Service, granted the predefined Cloud SQL Client role (roles/cloudsql.client).
  • Each host must be able to authenticate to Google Cloud as its service account using Application Default Credentials.
  • Check that you can connect to your Teleport cluster and verify that you can run tctl and tsh commands using your current credentials.

    1. Assign teleport.example.com to the domain name of the Teleport Proxy Service in your cluster and [email protected] to your Teleport username.

    2. Authenticate to your Teleport cluster. This depends on whether your shell is interactive or not.

      In an interactive shell: Run the following command. By default, this triggers a multi-factor authentication prompt:

      tsh login --proxy=teleport.example.com --user=[email protected]
      tctl status
      Cluster  teleport.example.com
      Version  19.0.0-dev
      CA pin   sha256:abdc1245efgh5678abdc1245efgh5678abdc1245efgh5678abdc1245efgh5678

      On non-interactive environments: If you are running tsh and tctl as an AI agent, in a CI/CD environment, or similar, make sure the TELEPORT_IDENTITY_FILE environment variable is assigned to a valid file path with credentials for your cluster. tsh and tctl read the file path from the environment variable and do not require a separate authentication step. If there is no identity file available, we recommend that you set up Machine ID to provision one automatically.

      When executing tctl commands with an identity file, you must pass the --auth-server flag to provide the Teleport Auth Service address, which is not included in the identity file. If you provide the Proxy Service address, tctl connects to the Proxy Service, which forwards traffic to and from the Teleport Auth Service. Update 443 to 3025 if you are contacting the Auth Service directly with tctl:

      tctl status --auth-server=teleport.example.com:443

      For tsh commands that read an identity file, you must pass the --proxy flag, which points tsh to the address of the Teleport Proxy Service:

      tsh status --proxy=teleport.example.com

      Ensure client commands can access your identity file. Replace path/to/identity/file with the path to your identity file:

      export TELEPORT_IDENTITY_FILE="${TELEPORT_IDENTITY_FILE:-path/to/identity/file}"

      Add the --auth-server or --proxy flags to all subsequent tctl and tsh commands.

    If you can connect to the cluster and run the tctl status command, you can use your current credentials to run subsequent tctl commands from your workstation. If you host your own Teleport cluster, you can also run tctl commands on the computer that hosts the Teleport Auth Service for full permissions.

Step 1/6. Install Teleport​

Install Teleport on the host or hosts that will run the Teleport Discovery Service and Teleport Database Service.

To install Teleport binaries on your Linux server, the recommended installation method is the cluster install script. This script is served by your Teleport cluster's Proxy Service and automatically selects the correct version, edition, and installation mode to match your cluster.

  1. Remove any existing Teleport binaries on your system:

    sudo rm -f /usr/local/bin/{tsh,teleport,tctl,tbot,fdpass-teleport,teleport-update}

  2. Assign teleport.example.com:443 to your Teleport cluster hostname and port, but not the scheme (https://).

  3. Run your cluster's install script:

    curl "https://teleport.example.com:443/scripts/install.sh" | sudo bash

Step 2/6. Deploy the Discovery Service​

Create a Teleport config file​

The following configuration uses a shared-secret join token. If you run the Discovery Service on a Google Compute Engine VM, you can instead use the GCP join method to join without a shared secret.

Create a teleport.yaml config file on the host that will run the Discovery Service:

version: v3
teleport:
  join_params:
    token_name: "/tmp/token"
    method: token
  proxy_server: "teleport.example.com:443"
auth_service:
  enabled: false
proxy_service:
  enabled: false
ssh_service:
  enabled: false
discovery_service:
  enabled: true
  discovery_group: "gcp-prod"

This enables the discovery_service and joins it to your Teleport cluster.

warning

Discovery Service exposes a configuration parameter - discovery_service.discovery_group - that allows you to group discovered resources into different sets. This parameter is used to prevent Discovery Agents watching different sets of cloud resources from colliding against each other and deleting resources created by another services.

When running multiple Discovery Services, you must ensure that each service is configured with the same discovery_group value if they are watching the same cloud resources or a different value if they are watching different cloud resources.

It is possible to run a mix of configurations in the same Teleport cluster meaning that some Discovery Services can be configured to watch the same cloud resources while others watch different resources. As an example, a 4-agent high availability configuration analyzing data from two different cloud accounts would run with the following configuration.

  • 2 Discovery Services configured with discovery_group: "prod" polling data from Production account.
  • 2 Discovery Services configured with discovery_group: "staging" polling data from Staging account.

Generate a join token​

The Discovery Service requires a valid join token to join your Teleport cluster. Run the following tctl command and save the token output in /tmp/token on the server that will run the Discovery Service:

tctl tokens add --type=discovery --format=text
abcd123-insecure-do-not-use-this

Start the Discovery Service​

Start the Discovery Service. The instructions depend on how you installed the Discovery Service and whether your system supports systemd:

Configure the Discovery Service to start automatically when the host boots up by creating a systemd service for it. On the host where you will run the Discovery Service, enable and start Teleport:

sudo systemctl enable teleport
sudo systemctl start teleport

You can check the status of the Discovery Service with systemctl status teleport and view its logs with journalctl -fu teleport.

Step 3/6. Configure Cloud SQL discovery​

Create a file named discovery-cloudsql.yaml with the same discovery_group you configured above:

kind: discovery_config
version: v1
metadata:
  name: discovery-cloudsql
spec:
  discovery_group: "gcp-prod"
  gcp:
    - types: ["cloudsql"]
      # The Google Cloud projects to search. Use ["*"] to search every project
      # the Discovery Service service account can access.
      project_ids: ["my-project"]
      # The Google Cloud regions to search. Use ["*"] to search all regions.
      # For Cloud SQL, locations must be regions (e.g. "us-central1"), not zones.
      locations: ["us-central1"]
      # Optional selector matched against imported user labels and Cloud SQL
      # metadata such as engine, region, and endpoint type. Defaults to a
      # wildcard that matches every instance.
      labels:
        "*": "*"

Create the discovery_config:

tctl create -f discovery-cloudsql.yaml

Discovery runs every five minutes by default, so matching instances can take up to five minutes to appear as Teleport db resources. The Discovery Service skips instances that are unsupported or unavailable or that expose no usable connection endpoint.

Step 4/6. List registered databases​

info

At this stage, tctl get db lists discovered database resources. A database appears in tsh db ls or tctl db ls only after a Database Service matches it.

Verify that the Discovery Service has created db resources for the instances you expect:

tctl get db

To check for a specific database, run the following command, assigning database-name to its Teleport resource name:

tctl get db/database-name

If you do not see the databases you expect, refer to Discovery Service troubleshooting.

Discovered database names

Discovered databases are named <instance>-cloudsql-<region>-<project-id>, for example my-postgres-cloudsql-us-central1-my-project. Teleport drops any suffix part the instance name already contains.

Both the full name and the instance name work with tsh and tctl commands. tsh db ls and tctl db ls show the instance name unless you pass --verbose; if an instance name matches more than one database, use the full name.

Applying the teleport-database-name user label to an instance overrides the name: Teleport uses the value verbatim, with no suffix, and the instance name no longer works as an alias for it. The value must be a valid, unique Teleport database name; Teleport skips instances with an invalid value.

For the labels applied to discovered databases, see the Google Cloud database labels reference.

Endpoint selection

When an instance has multiple endpoints, Teleport registers the first available in this order: public, private (PSA), then PSC. To override this selection, set the Cloud SQL instance's teleport-database-endpoint-type user label to public, private, or psc. Teleport does not try another endpoint type when an override is set.

The Database Service host must be able to reach whichever endpoint Teleport registers. If that is a public IP, the host's egress IP address must also be in the instance's authorized networks.

Step 5/6. Deploy the Database Service​

Create a Teleport config file​

The following configuration uses a shared-secret join token. If you run the Database Service on a Google Compute Engine VM, you can instead use the GCP join method to join without a shared secret.

Create a teleport.yaml config file on the host that will run the Database Service:

version: v3
teleport:
  join_params:
    token_name: "/tmp/token"
    method: token
  proxy_server: "teleport.example.com:443"
auth_service:
  enabled: false
proxy_service:
  enabled: false
ssh_service:
  enabled: false
db_service:
  enabled: true
  resources:
    - labels:
        "teleport.dev/origin": "cloud"
        "teleport.dev/cloud": "GCP"
        "teleport.dev/discovery-type": "cloudsql"
        "project-id": "my-project"
        "region": "us-central1"

The Discovery Service adds the label teleport.dev/origin: cloud to every database it registers. In this example, db_service.resources uses that label and the other selectors to match auto-discovered Cloud SQL databases in my-project and us-central1. For each matching db resource, the Database Service creates a short-lived db_server resource that announces to the Teleport cluster that it is ready to proxy connections for that db.

warning

Use more specific label selectors when needed so that a Database Service instance only matches databases it can actually reach. Broad selectors can cause unreachable databases to be registered and lead to connection failures.

Generate a join token​

The Database Service requires a valid join token to join your Teleport cluster. Run the following tctl command and save the token output in /tmp/token on the server that will run the Database Service:

tctl tokens add --type=db --format=text
abcd123-insecure-do-not-use-this

Start the Database Service​

Start the Database Service. The instructions depend on how you installed the Database Service and whether your system supports systemd:

Configure the Database Service to start automatically when the host boots up by creating a systemd service for it. On the host where you will run the Database Service, enable and start Teleport:

sudo systemctl enable teleport
sudo systemctl start teleport

You can check the status of the Database Service with systemctl status teleport and view its logs with journalctl -fu teleport.

Step 6/6. List database servers​

Confirm that the Database Service has registered servers for the discovered databases:

tctl db ls teleport.dev/origin=cloud,teleport.dev/cloud=GCP,teleport.dev/discovery-type=cloudsql,project-id=my-project,region=us-central1

If the expected databases are missing, refer to Database Service troubleshooting.

Troubleshooting​

Discovery Service troubleshooting​

First, check if any databases have been discovered. To do this, you can use the tctl get db command and check if the expected databases have already been registered with your Teleport cluster.

If some databases do not appear in the list, check if the Discovery Service selector matches the missing databases' provider labels or tags, or check the Discovery Service logs for permission errors.

Check that the Discovery Service is running with credentials for the intended GCP environment. To discover resources in another account or project, grant its principal access to the target account or project.

Check if there is more than one Discovery Service instance running:

tctl inventory status --connected

If you are running multiple Discovery Service instances, you must ensure that each service is configured with the same discovery_group value if they are watching the same cloud databases or a different value if they are watching different cloud databases. If this is not configured correctly, a typical symptom is db resources being intermittently deleted from your Teleport cluster's registry.

Database Service troubleshooting​

Databases do not appear in tctl db ls​

If the tctl get db command returns the discovered databases you expect, but the tctl db ls command does not include them, check that you have set the db_service.resources section correctly, for example:

db_service:
  enabled: true
  resources:
  - labels:
      "env": "prod"

If the section is correctly configured, but databases still do not appear, check that you have the correct permissions to list databases in Teleport. You should have a Teleport role that matches the database labels and allows the "read" and "list" verbs for db and db_server objects. Here's an example that grants those permissions for every database in your cluster:

kind: role
version: v6
metadata:
  name: view-all-databases
spec:
  allow:
    db_labels:
      '*': '*'
    rules:
      - resources: [db_server, db]
        verbs: [read, list]

Errors when connecting to a database​

note

This section assumes you have already provisioned a database user and configured Teleport RBAC for that database user by following the appropriate database enrollment guide.

If there are connection errors when you try to connect to a database, then first check if there are multiple db_server heartbeat resources for the target database: tctl get db_server/yourDatabaseName. If there are, it means that multiple Teleport Database Service instances are proxying the database - this is an HA setup that will complicate troubleshooting. Teleport will choose one of those Database Service instances at random to proxy the connection and if one of them can't reach the database endpoint or lacks permissions, then you will see random connection errors.

Even if connection errors are consistent, you should scale down or reconfigure your Teleport Database Service instances such that only one matches the target db while you are troubleshooting errors. Verify that there is only one db_server with tctl get db_server/yourDatabaseName and then try the connection again.

Check the Teleport Database Service logs with DEBUG level logging enabled and look for network or permissions errors.

Refer to the Database Service troubleshooting guide for more general troubleshooting steps.

The enrollment guide for your database may provide more specific troubleshooting information.

Next steps​