Skip to main content
Version: 18.x

Upgrading Compatibility Overview

Report an issue with this page

Since Teleport is a distributed system with a number of services that run on potentially many hosts, you should take care when upgrading the cluster to ensure that all components remain compatible.

This guide provides an overview of how to upgrade the components in your Teleport cluster while preserving compatibility.

Component compatibility​

Teleport uses Semantic Versioning. Version numbers include a major version, minor version, and patch version, separated by dots. When running multiple teleport binaries within a cluster, the following rules apply:

  • Servers support clients that are one major version behind, but do not support clients that are on a newer major version. For example, an 17.x.x Proxy Service instance is compatible with 16.x.x agents and 16.x.x tsh, but a 17.x.x agent will not work with an 16.x.x Proxy Service instance. This also means you must not attempt to upgrade from 16.x.x straight to 18.x.x. You must upgrade to 17.x.x first.
  • Proxy Service instances and agents do not support Auth Service instances that are on an older major version, and will fail to connect to older Auth Service instances by default. For example, an 18.x.x Proxy Service or agent is not compatible with an 17.x.x Auth Service.
  • Auth Service instances should always be the first component of the cluster that is upgraded, and you must upgrade all Auth Service instances to the target version before proceeding to upgrade Proxy Service instances, other agents, and client tools (tsh, tctl, tbot, Connect, etc).

You can think of Teleport cluster components in three tiers:

  1. The Auth Service is expected to be the newest component in the cluster. For this reason it should always be the first component that is upgraded.
  2. The Proxy Service is expected to run a version less than or equal to the Auth Service. The Proxy Service should never run a version newer than the Auth Service.
  3. Other Teleport agents (SSH Service, Database Service, etc.) should always run a version less than or equal to the Proxy Service.

In Teleport Enterprise Cloud, we manage the Auth Service and Proxy Service for you. You can determine the current version of these services by running the following command, where mytenant is the name of your Teleport Enterprise Cloud tenant:

curl -s https://mytenant.teleport.sh/webapi/find | jq '.server_version'

Upgrading a self-hosted Teleport cluster​

Because of the component compatibility guarantees we described in the previous section, you must adhere to the following sequence when upgrading a self-hosted Teleport cluster.

warning

If you are upgrading more then one major version, you must repeat the following steps for each major version until you reach your target version. For example, if your cluster is on v15 and you wish to upgrade to v18, you must first follow the sequence below for v16, then v17, before finally upgrading to v18. You must not upgrade directly from v15 to v18.

  1. Back up your Teleport cluster state as a precaution against any unforeseen incidents while upgrading the Auth Service, which may perform data migrations on its backend. Follow the instructions in Backup and Restore.
  2. Upgrade all Auth Service instances to the target version first. Auth Service instances may be upgraded in any sequence or at the same time. After the upgrade confirm that the cluster is in a healthy state before continuing.
  3. After all Auth Service instances are running the target version, upgrade Proxy Service instances to the same version as the Auth Service. Proxy Service instances are stateless and can be upgraded in any sequence or at the same time. After the upgrade confirm that the cluster is in a healthy state before continuing.
  4. After all Proxy Service instances are running the target version, upgrade your Teleport Agents to the same version as the Auth Service. You can upgrade agents in any sequence or at the same time. After the upgrade confirm that the cluster is in a healthy state before continuing.
  5. Upgrade your Teleport clients and plugins (tctl, tsh, tbot, terraform-provider, event-handler, etc.).

By following this process, your agents will never get ahead of your control plane components (even within the same major/patch version), ensuring that all expected APIs and migrations will be available by the time the agents upgrade.

Upgrading multiple Teleport clusters​

When upgrading multiple Teleport clusters with a trust relationship, you must upgrade in the following sequence to avoid compatibility issues.

You must upgrade all clusters one major version at a time. For example, if you would like to upgrade your clusters from v10 to v12, you must follow the sequence below to upgrade clusters from v10 to v11, then repeat the sequence to upgrade from v10 to v11.

  1. Upgrade the root cluster—that is, the cluster that other clusters trust—first.
  2. Verify the upgrade was successful.
  3. Upgrade the trusted leaf clusters.

Downgrading major versions​

In order to safely downgrade from one major version to another, i.e. from v17 to v16, you must restore the Teleport backend to the back up taken prior to upgrading. First, roll back all components to the exact version they were running prior to the upgrade in reverse order of the upgrade sequence above until only the Auth Service instances are running the new major version. Then stop the Auth Service and follow the Backup and Restore guidance to restore the backend to a point in time prior to the upgrade. Once the backup has been restored, downgrade the Auth Service instances to the exact version of Teleport they were running prior to the upgrade attempt, then start them again.

Next steps​

Return to the Upgrading Introduction for how to upgrade individual components within your Teleport cluster.