Teleport

Upgrade Teleport Cloud Agents on Linux

This guide explains how to upgrade Teleport Cloud agents running on Linux servers. On Teleport Cloud, Auth Service and Proxy Service upgrades are managed for you. To keep agents up to date, you can either enroll them in automatic upgrades or upgrade them manually using the method you used to install Teleport.

Prerequisites

Familiarity with the Upgrading Compatibility Overview guide, which describes the sequence in which to upgrade components in your cluster.
A Teleport Enterprise Cloud account. You can determine the current version of the Auth Service and Proxy Service by running the following command, where mytenant is the name of your Teleport Enterprise Cloud account. This requires the jq CLI tool:
```
curl -s https://mytenant.teleport.sh/webapi/ping | jq '.server_version'
```
The tctl and tsh client tools version >= 15.2.2.

You can download these from Teleport Cloud Downloads.
To check that you can connect to your Teleport cluster, sign in with tsh login, then verify that you can run tctl commands using your current credentials. tctl is supported on macOS and Linux machines.

For example:
```
tsh login --proxy=teleport.example.com --user=[email protected]
tctl status
Cluster  teleport.example.com
Version  15.2.4
CA pin   sha256:abdc1245efgh5678abdc1245efgh5678abdc1245efgh5678abdc1245efgh5678
```
If you can connect to the cluster and run the tctl status command, you can use your current credentials to run subsequent tctl commands from your workstation. If you host your own Teleport cluster, you can also run tctl commands on the computer that hosts the Teleport Auth Service for full permissions.

Teleport supports automatic agent updates for systemd-based Linux distributions using apt, yum, and zypper package managers. The Automatic Update Architecture guide describes how agent updating works. Automatic agent upgrades require:

A Teleport Cloud account that supports automatic updates. To determine if your account supports automatic updates, run the following command, replacing mytenant.teleport.sh with the address of your Teleport Cloud account:
```
curl -s https://mytenant.teleport.sh/webapi/ping | jq '.automatic_upgrades'
true
```
At least one Teleport Enterprise agent, started via systemd on a distribution using the apt or yum package managers.

So you can access each agent, we assume that each agent runs the Teleport SSH Service in addition to any other services. See the Deploy Teleport Agents with Terraform guide for an example.

Step 1/2. Find agents to upgrade

To list agents that are not enrolled in automatic updates, use the tctl inventory ls command with the --upgrader=none flag. This command also uses the --services flag to constrain the search to Teleport processes running agent services:

tctl inventory ls \  --services=node,kube,db,app,windows_desktop,discovery,okta \  --upgrader=none
Server ID                            Hostname      Services Version Upgrader------------------------------------ ------------- -------- ------- --------00000000-0000-0000-0000-000000000000 ip-10-1-6-130 Node     v13.4.5 none

If you have a lot of agents on different versions and want to prioritize enrolling your oldest agents, you can limit your search using the --older-than filter:

tctl inventory ls --upgrader=none --older-than=v1.2.3
Server ID                            Hostname        Services       Version Upgrader------------------------------------ --------------- -------------- ------- --------00000000-0000-0000-0000-000000000000 old.example.com Node           v1.1.1  none00000000-0000-0000-0000-000000000001 teleport-proxy  Proxy          v14.3.6 none00000000-0000-0000-0000-000000000002 teleport-auth   Auth,Discovery v14.3.6 none...

Note that the inventory ls command will also list teleport-auth and teleport-proxy services. You do not need to update these services or enroll them in automatic updates. Teleport Cloud manages updates for these services.

When you enroll each agent in automatic updates in the next section, you can run the following commands to fetch the hostname for each agent and access it via Teleport.

For each agent ID returned by the tctl inventory ls command, copy the ID and run the following tctl command to access the host via tsh:

HOST=00000000-0000-0000-0000-000000000000
USER=root
tsh ssh "${USER?}@${HOST?}"

You can then follow the steps in the next section on each agent to enroll it in automatic updates.

Step 2/2. Enroll agents in automatic updates

Complete the following instructions on each agent you would like to enroll into automatic updates:

Ensure the Teleport repository is added and Teleport Enterprise is installed.

To verify if the Teleport repository was added to the system, check if either of the follow files exists:
```
ls /etc/apt/sources.list.d/teleport.list
or
ls /etc/yum.repos.d/teleport.repo
```
The upgrader checks the repository for available releases, so make sure that it is up to date.
If the repository was added, make sure the Teleport binary installed on the agent can run the automatic updater:
```
which teleport-upgrade || echo "Install the upgrader"
Install the upgrader
```

If the Teleport repository is not found, or the Teleport binary you installed does not include the upgrader, add the appropriate repository and reinstall Teleport:

Add the Teleport repository to your repository list:

Download Teleport's PGP public key
sudo curl https://apt.releases.teleport.dev/gpg \-o /usr/share/keyrings/teleport-archive-keyring.asc
Source variables about OS version
source /etc/os-release
Add the Teleport APT repository for cloud.
echo "deb [signed-by=/usr/share/keyrings/teleport-archive-keyring.asc] \https://apt.releases.teleport.dev/${ID?} ${VERSION_CODENAME?} stable/cloud" \| sudo tee /etc/apt/sources.list.d/teleport.list > /dev/null

Provide your Teleport domain to query the latest compatible Teleport version
export TELEPORT_DOMAIN=example.teleport.com
export TELEPORT_VERSION="$(curl https://$TELEPORT_DOMAIN/v1/webapi/automaticupgrades/channel/default/version | sed 's/v//')"

Update the repo and install Teleport and the Teleport updater
sudo apt-get update
sudo apt-get install "teleport-ent=$TELEPORT_VERSION" teleport-ent-updater

Source variables about OS version
source /etc/os-release
Add the Teleport YUM repository for cloud.
First, get the OS major version from $VERSION_ID so this fetches the correct
package version.
VERSION_ID=$(echo $VERSION_ID | grep -Eo "^[0-9]+")
sudo yum install -y yum-utils
sudo yum-config-manager --add-repo "$(rpm --eval "https://yum.releases.teleport.dev/$ID/$VERSION_ID/Teleport/%{_arch}/stable/cloud/teleport-yum.repo")"

Provide your Teleport domain to query the latest compatible Teleport version
export TELEPORT_DOMAIN=example.teleport.com
export TELEPORT_VERSION="$(curl https://$TELEPORT_DOMAIN/v1/webapi/automaticupgrades/channel/default/version | sed 's/v//')"

Install Teleport and the Teleport updater
sudo yum install "teleport-ent-$TELEPORT_VERSION" teleport-ent-updater

Tip: Add /usr/local/bin to path used by sudo (so 'sudo tctl users add' will work as per the docs)
echo "Defaults    secure_path = /sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin" > /etc/sudoers.d/secure_path

Source variables about OS version
source /etc/os-release
Add the Teleport YUM repository for cloud.
First, get the OS major version from $VERSION_ID so this fetches the correct
package version.
VERSION_ID=$(echo $VERSION_ID | grep -Eo "^[0-9]+")
Use the dnf config manager plugin to add the teleport RPM repo
sudo dnf config-manager --add-repo "$(rpm --eval "https://yum.releases.teleport.dev/$ID/$VERSION_ID/Teleport/%{_arch}/stable/cloud/teleport-yum.repo")"

Provide your Teleport domain to query the latest compatible Teleport version
export TELEPORT_DOMAIN=example.teleport.com
export TELEPORT_VERSION="$(curl https://$TELEPORT_DOMAIN/v1/webapi/automaticupgrades/channel/default/version | sed 's/v//')"

Install Teleport and the Teleport updater
sudo dnf install "teleport-ent-$TELEPORT_VERSION" teleport-ent-updater

Tip: Add /usr/local/bin to path used by sudo (so 'sudo tctl users add' will work as per the docs)
echo "Defaults    secure_path = /sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin" > /etc/sudoers.d/secure_path

Source variables about OS version
source /etc/os-release
Add the Teleport Zypper repository for cloud.
First, get the OS major version from $VERSION_ID so this fetches the correct
package version.
VERSION_ID=$(echo $VERSION_ID | grep -Eo "^[0-9]+")
Use Zypper to add the teleport RPM repo
sudo zypper addrepo --refresh --repo $(rpm --eval "https://zypper.releases.teleport.dev/$ID/$VERSION_ID/Teleport/%{_arch}/stable/cloud/teleport-zypper.repo")

Provide your Teleport domain to query the latest compatible Teleport version
export TELEPORT_DOMAIN=example.teleport.com
export TELEPORT_VERSION="$(curl https://$TELEPORT_DOMAIN/v1/webapi/automaticupgrades/channel/default/version | sed 's/v//')"

Install Teleport and the Teleport updater
sudo zypper install "teleport-ent-$TELEPORT_VERSION" teleport-ent-updater

If you changed the agent user to run as non-root, create /etc/teleport-upgrade.d/schedule and grant ownership to your Teleport user. Otherwise, you can skip this step:
```
sudo mkdir -p /etc/teleport-upgrade.d/
sudo touch /etc/teleport-upgrade.d/schedule
sudo chown <your-teleport-user> /etc/teleport-upgrade.d/schedule
```
Verify that the upgrader can see your version endpoint by checking for updates:
```
sudo teleport-upgrade dry-run
```
You should see one of the following messages, depending on the target version you are currently serving:
```
no upgrades available (1.2.3 == 1.2.3)
an upgrade is available (1.2.3 -> 2.3.4)
```
The upgrader may display warnings about not having a valid upgrade schedule. This is expected immediately after install as the maintenance schedule might not be exported yet.
Ensure that the upgrader is running:
```
sudo systemctl status teleport-upgrade
```

Version locking

As of Teleport 15.1.10, a version locking mechanism is enabled by the updater. This mechanism locks the version of Teleport and prevents manual updates of the Teleport package. This prevents unintentional updates during routine system maintenance, or an accidental update by a user. The updater still has the capability to update the Teleport package, and all updates of Teleport are expected to be performed by the updater.

The version locking mechanism is implemented using the features of the package managers. In case a user would like to manually update the Teleport version, this can be done with the following commands.

With the APT package manager CLI, the --allow-change-held-packages flag must be provided to bypass the version lock.

apt-get install --allow-change-held-packages "teleport-ent=<target-version>"

With the YUM package manager CLI, the --disableexcludes="teleport" flag must be provided to bypass the version lock.

yum install --disablerepo="*" --enablerepo="teleport" --disableexcludes="teleport" "teleport-ent-<target-version>"

With the Zypper package manager CLI, the lock must be disabled and then re-enabled after the update.

zypper removelock "teleport-ent"
zypper install --repo="teleport" "teleport-ent-<target-version>"
zypper addlock "teleport-ent"

Updating the updater

The updater is designed to be minimal and relatively stable, but the updater will receive updates on occasion. Currently, the updater does not have the capability to update itself. Updates to the teleport-ent-updater package must be done manually.

The teleport-ent-updater will be backwards compatible with older versions of Teleport, so you can always update the teleport-ent-updater package to the latest available version.

Troubleshooting

The teleport-upgrade tool provides some basic commands to verify and perform an update of the Teleport agent.

teleport-upgrade help
USAGE: /usr/sbin/teleport-upgrade <command>
Tool for automatic upgrades of Teleport agents.
Commands:  run           check for and potentially apply a teleport upgrade.  dry-run       check for new teleport version but do not upgrade.  version       print the current version of /usr/sbin/teleport-upgrade.  help          show this help text.

The dry-run command can be used to check for an available update without performing an update.

Example output when teleport is already on the latest compatible version.
teleport-upgrade dry-run
[i] no upgrades available (14.3.14 == 14.3.14) [ 582 ]
Example output when an update is available.
teleport-upgrade dry-run
[i] an upgrade is available (13.4.14 -> 14.3.14) [ 585 ][i] within maintenance window, upgrade will be attempted. [ 596 ]

The run command performs an update if available.

Successful teleport update from 13.4.14 to 14.3.14.
teleport-upgrade run
[i] an upgrade is available (13.4.14 -> 14.3.14) [ 585 ][i] within maintenance window, upgrade will be attempted. [ 596 ][i] attempting apt install teleport-ent=14.3.14... [ 480 ][...][i] gracefully restarting Teleport (if already running) [ 449 ]
Teleport updates are not attempted when outside the maintenance window.
teleport-upgrade run
[i] an upgrade is available (13.4.14 -> 14.3.14) [ 585 ][i] upgrade is non-critical and we are outside of maintenance window, not attempting. [ 618 ]

Teleport agents are not updated immediately when a new version of Teleport is released, and agent updates can lag behind the cluster by a few days.

If the Teleport agent has not been automatically updating for several weeks, you can view the logs of the teleport-upgrade.service to help troubleshoot the problem.

journalctl -t teleport-upgrade
Apr 16 16:44:32 example teleport-upgrade[168983]: [i] upgrade is non-critical and we are outside of maintenance window, not attempting. [ 618 ]Apr 16 16:50:32 example teleport-upgrade[169084]: [i] an upgrade is available (13.4.14 -> 14.3.14) [ 585 ]Apr 16 16:50:33 example teleport-upgrade[169084]: [i] upgrade is non-critical and we are outside of maintenance window, not attempting. [ 618 ]Apr 16 16:56:32 example teleport-upgrade[169187]: [i] an upgrade is available (13.4.14 -> 14.3.14) [ 585 ]...

To suspend automatic updates, disable the teleport.upgrade.timer systemd timer:

sudo systemctl disable --now teleport-upgrade.timer

To enable and start the systemd timer after suspending:

sudo systemctl enable --now teleport-upgrade.timer

The Teleport Access Platform

Featured Resource

Integrations

Featured Resource

Use Cases

Featured Resource

Industries

Featured Resource

Compliance

Featured Webinar

Strategic Partners

Featured AWS Webinar

Read

Experience

Discover

Featured Blog Post

Company

Explore

Featured Webinar

Upgrade Teleport Cloud Agents on Linux

Prerequisites

Step 1/2. Find agents to upgrade

Step 2/2. Enroll agents in automatic updates

Version locking

Updating the updater

Troubleshooting