This guide will show you how to use the Teleport client tool, tsh.

You will learn how to:

• Log in to an interactive shell on remote cluster nodes.
• Copy files to and from cluster nodes.
• Connect to SSH clusters behind firewalls without any open ports using SSH reverse tunnels.
• Explore a cluster and execute commands on specific nodes in the cluster.
• Share interactive shell sessions with colleagues or join someone else's session.
• List and replay recorded interactive sessions.

In addition to this document, you can always simply type tsh into your terminal for the CLI reference.

Introduction

For the impatient, here's an example of how a user would typically use tsh:

Log into a Teleport cluster. This command retrieves the user's certificates
and saves them into ~/.tsh/teleport.example.com
tsh login --proxy=teleport.example.com
SSH into a Node as usual
tsh ssh [email protected]
`tsh ssh` takes the same arguments as the OpenSSH client:
tsh ssh -o ForwardAgent=yes [email protected]
tsh ssh -o AddKeysToAgent=yes [email protected]
You can even create a convenient symlink:
ln -s /path/to/tsh /path/to/ssh
... and now your 'ssh' command is calling Teleport's `tsh ssh`
ssh [email protected]
This command removes SSH certificates from a user's machine:
tsh logout

In other words, Teleport was designed to be fully compatible with existing SSH-based workflows and does not require users to learn anything new, other than to call tsh login in the beginning.

Installing tsh

Follow these install instructions to obtain the tsh binary. Ideally, install tsh of the same version as the version used in your Teleport cluster.

User identities

A user identity in Teleport exists in the scope of a cluster. The member nodes of a cluster may have multiple OS users on them. A Teleport administrator assigns allowed logins to every Teleport user account.

When logging into a remote node, you will have to specify both the Teleport login and the OS login. A Teleport identity will have to be passed via the --user flag while the OS login will be passed as [email protected] using syntax compatible with the traditional ssh command.

Authenticate against the "work" cluster as joe and then
log into the node as root:
tsh ssh --proxy=work.example.com --user=joe [email protected]

CLI Docs - tsh ssh

Logging in

To retrieve a user's certificate, execute:

Full form:
tsh login --proxy=proxy_host:<https_proxy_port>
Using default ports:
tsh login --proxy=work.example.com
Using custom HTTPS port:
tsh login --proxy=work.example.com:5000

CLI Docs - tsh login

The login command retrieves a user's certificate and stores it in ~/.tsh directory as well as in the ssh agent if there is one running.

This allows you to authenticate just once, maybe at the beginning of the day. Subsequent tsh ssh commands will run without asking for credentials until the temporary certificate expires. By default, Teleport issues user certificates with a time to live (TTL) of 12 hours.

A Teleport cluster can be configured for multiple user identity sources. For example, a cluster may have a local user called admin while regular users should authenticate via GitHub. In this case, you have to pass --auth flag to tsh login to specify which identity storage to use:

Log in using the local Teleport 'admin' user:
tsh --proxy=proxy.example.com --auth=local --user=admin login
Log in using GitHub as an SSO provider, assuming the GitHub connector is called "github"
tsh --proxy=proxy.example.com --auth=github login

When using an external identity provider to log in, tsh will need to open a web browser to complete the authentication flow. By default, tsh will use your system's default browser. If you wish to suppress this behavior, you can use the --browser=none flag:

Don't open the system default browser when logging in
tsh login --proxy=work.example.com --browser=none

In this situation, a link will be printed on the screen. You can copy and paste this link into a browser of your choice to continue the login flow.

CLI Docs - tsh login

Inspecting an SSH certificate

To inspect the SSH certificates in ~/.tsh, a user may execute the following command:

tsh status
> Profile URL:  https://proxy.example.com:3080
Logged in as: johndoe
Cluster:      proxy.example.com
Roles:        admin*
Logins:       root, admin, guest
Kubernetes:   disabled
Valid until:  2017-04-25 15:02:30 -0700 PDT [valid for 1h0m0s]
Extensions:   permit-agent-forwarding, permit-port-forwarding, permit-pty

CLI Docs - tsh status

SSH agent support

If there is an ssh agent running, tsh login will store the user certificate in the agent. This can be verified via:

ssh-add -L

The SSH agent can be used to feed the certificate to other SSH clients, for example to OpenSSH (ssh).

If you wish to disable SSH agent integration, pass --no-use-local-ssh-agent to tsh. You can also set the TELEPORT_USE_LOCAL_SSH_AGENT environment variable to false in your shell profile to make this permanent.

Identity files

tsh login can also save the user certificate into a file:

Authenticate the user against proxy.example.com and save the user
certificate to joe.pem
tsh login --proxy=proxy.example.com --out=joe
Use joe.pem to log in to the server 'db'
tsh ssh --proxy=proxy.example.com -i joe [email protected]

By default, the --out flag will create an identity file suitable for tsh -i. If compatibility with OpenSSH is needed, --format=openssh must be specified. In this case, the identity will be saved into two files, joe and joe-cert.pub:

tsh login --proxy=proxy.example.com --out=joe --format=openssh
ls -lh
total 8.0K
-rw------- 1 joe staff 1.7K Aug 10 16:16 joe
-rw------- 1 joe staff 1.5K Aug 10 16:16 joe-cert.pub

SSH certificates for automation

Regular users of Teleport must request an auto-expiring SSH certificate, usually every day. This doesn't work for non-interactive scripts, like cron jobs or a CI/CD pipeline.

For this kind of automation, it is recommended to create a separate Teleport user for bots and request a certificate for them with a long time to live (TTL).

In this example, we're creating a certificate with a TTL of one hour for the jenkins user and storing it in a jenkins.pem file, which can be later used with -i (identity) flag for tsh.

Log in to your cluster with tsh so you can use tctl from your local machine.
You can also run tctl on your Auth Service host without running "tsh login"
first.
tsh login --proxy=teleport.example.com --user=myuser
tctl auth sign --ttl=1h --user=jenkins --out=jenkins.pem

CLI Docs - tctl auth sign

Now jenkins.pem can be copied to the Jenkins server and passed to the -i (identity file) flag of tsh.

tctl auth sign is an admin's equivalent of tsh login --out and allows for unrestricted certificate TTL values.

For non-production usage, you can use Machine ID, currently in preview, to provide your bot user with automatically updated, short-lived credentials.

Exploring the cluster

In a Teleport cluster, all Nodes periodically ping the cluster's Auth Service and update their status. This allows Teleport users to see which Nodes are online with the tsh ls command:

This command lists all Nodes in the cluster you logged into via "tsh login":
tsh ls
Node Name     Address            Labels
---------     -------            ------
turing        10.1.0.5:3022      os:linux
turing        10.1.0.6:3022      os:linux
graviton      10.1.0.7:3022      os:osx

CLI Docs - tsh ls

tsh ls can apply a filter based on the node labels.

Only show Nodes with os label set to 'osx':
tsh ls os=osx
Nodename      UUID                   Address            Labels
---------     -------                -------            ------
graviton      33333333-aaaa-1284     10.1.0.7:3022      os:osx

CLI Docs -tsh ls

Interactive shell

To launch an interactive shell on a remote Node or to execute a command, use tsh ssh.

tsh tries to mimic the ssh experience as much as possible, so it supports the most popular ssh flags like -p, -l or -L. For example, if you have the following alias defined in your ~/.bashrc: alias ssh="tsh ssh" then you can continue using familiar SSH syntax:

Have this alias configured, perhaps via ~/.bashrc
alias ssh="/usr/local/bin/tsh ssh"
Login in to a cluster and retrieve your SSH certificate:
tsh --proxy=proxy.example.com login
These commands execute `tsh ssh` under the hood:
ssh [email protected]
ssh -p 6122 [email protected] ls
ssh -o ForwardAgent=yes [email protected]
ssh -o AddKeysToAgent=yes [email protected]

Proxy ports

By default, the Teleport Proxy Service listens on port 3080.

If a Teleport Proxy Service instance is configured to listen on non-default ports, they must be specified via --proxy flag as shown:

tsh --proxy=proxy.example.com:5000 <subcommand>

This tsh command will use port 5000 of the Proxy Service.

Port forwarding

tsh ssh supports the OpenSSH -L flag which forwards incoming connections from localhost to the specified remote host:port. The syntax of -L flag is as follows, where "bind_ip" defaults to 127.0.0.1:

-L [bind_ip]:listen_port:remote_host:remote_port

Example:

tsh ssh -L 5000:web.remote:80 node

This will connect to remote server node via the Proxy Service, then open a listening socket on localhost:5000. Finally, it will forward all incoming connections to web.remote:80 via this SSH tunnel.

It is often convenient to establish port forwarding, execute a local command which uses the connection, and then disconnect. You can do this with the --local flag.

Example:

tsh ssh -L 5000:google.com:80 --local node curl http://localhost:5000

This command:

1. Connects to node.
2. Binds the local port 5000 to port 80 on google.com.
3. Executes curl command locally, which results in curl hitting google.com:80 via node.

SSH jump host

While implementing ProxyJump for Teleport, we have extended the feature to tsh.

tsh ssh -J proxy.example.com telenode

Known limitations:

• Only one jump host is supported (-J supports chaining that Teleport does not utilize) and tsh will return with error in the case of two jump hosts, i.e. -J proxy-1.example.com,proxy-2.example.com will not work.
• When tsh ssh -J [email protected] is used, it overrides the SSH proxy defined in the tsh profile, and port forwarding is used instead of the existing Teleport proxy subsystem.

Resolving Node names

tsh supports multiple methods to resolve remote Node names.

1. Traditional: by IP address or via DNS.
2. Nodename setting: the teleport daemon supports the nodename flag, which allows Teleport administrators to assign alternative Node names.
3. Labels: you can address a Node by name=value pair.

If we have two Node, one with os:linux label and one Node with os:osx, we can log in to the OSX Node with:

tsh ssh os=osx

This only works if there is only one remote node with the os:osx label, but you can still execute commands via SSH on multiple Nodes using labels as a selector. This command will update all system packages on machines that run Linux:

tsh ssh os=ubuntu apt-get update -y

Short-lived sessions

The default TTL of a Teleport user certificate is 12 hours. This can be modified at login with the --ttl flag. This command logs you into the cluster with a very short-lived (1 minute) temporary certificate:

tsh --ttl=1 login

You will be logged out after one minute, but if you want to log out immediately, you can always run:

tsh logout

Copying files

To securely copy files to and from cluster Nodes, use the tsh scp command. It is designed to mimic OpenSSH's scp command as much as possible:

tsh scp example.txt [email protected]:/path/to/dest

Again, you may want to create a bash alias like alias scp="tsh --proxy=work scp" and use the familiar syntax:

scp -P 61122 -r files [email protected]:/path/to/dest

Starting from Teleport 9 the SCP and SFTP protocols are both supported by Server Access. OpenSSH scp or sftp commands can both be used in place of tsh scp if desired.

Sharing sessions

Suppose you are trying to troubleshoot a problem on a remote server. Sometimes it makes sense to ask another team member for help. Traditionally, this could be done by letting them know which host you're on, having them SSH in, start a terminal multiplexer like screen, and join a session there.

Teleport makes this more convenient. Let's log in to a server named luna and ask Teleport for our current session status:

tsh ssh luna
on host luna
teleport status
User ID    : joe, logged in as joe from 10.0.10.1 43026 3022
Session ID : 7645d523-60cb-436d-b732-99c5df14b7c4
Session URL: https://work:3080/web/sessions/7645d523-60cb-436d-b732-99c5df14b7c4

Now you can invite another user account to the work cluster. You can share the URL for access through a web browser, or you can share the session ID, and the other user can join you through their terminal by typing:

tsh join <session_ID>

Connecting to SSH clusters behind firewalls

Teleport supports creating clusters of servers located behind firewalls without any open listening TCP ports. This works by creating reverse SSH tunnels from behind-firewall environments into a Teleport Proxy Service you have access to.

These features are called Trusted Clusters. Refer to the Trusted Clusters guide to learn how a Trusted Cluster can be configured.

Assuming the Teleport Proxy Server called work is configured with a few Trusted Clusters, a user may use the tsh clusters command to see a list of all Trusted Clusters on the server:

tsh --proxy=work clusters
Cluster Name     Status
------------     ------
staging          online
production       offline

CLI Docs - tsh clusters

Now you can use the --cluster flag with any tsh command. For example, to list SSH nodes that are members of the production cluster, simply run:

tsh --proxy=work ls --cluster=production
Node Name     Node ID       Address            Labels
---------     -------       -------            ------
db-1          xxxxxxxxx     10.0.20.31:3022    kernel:4.4
db-2          xxxxxxxxx     10.0.20.41:3022    kernel:4.2

Similarly, if you want to SSH into db-1 inside the production cluster:

tsh --proxy=work ssh --cluster=production db-1

This is possible even if Nodes in the production cluster are located behind a firewall without open ports. This works because the production cluster establishes a reverse SSH tunnel back into the Proxy Service called work, and this tunnel is used to establish inbound SSH connections.

X11 forwarding

In order to run graphical programs within an SSH session, such as an IDE like Virtual Studio Code, you'll need to request X11 forwarding for the session with the -X flag.

tsh ssh -X node01

X11 forwarding provides the server with secure access to your local X Server so that it can communicate directly with your local display and I/O devices.

In order to use X11 forwarding, you'll need to enable it on the Teleport Node. You'll also need to ensure that your user has the permit_x11_forwarding role option:

tsh status
> Profile URL:        https://proxy.example.com:3080
  Logged in as:       dev
  ...
  Extensions:         permit-X11-forwarding

Custom aliases and defaults

You can configure tsh to define aliases, custom commands and command-specific flag defaults. Using aliases, you can run frequently used tsh commands more easily.

Aliases are defined in configuration files using the following syntax:

aliases:
    "<alias>": "<command>"

The <alias> can only be a top-level subcommand. In other words, you can define tsh mycommand alias but not tsh my command.

tsh loads two kinds of configuration files:

• global: by default /etc/tsh.yaml, unless overridden by the $TELEPORT_GLOBAL_TSH_CONFIG env var.
• user-specific: $TELEPORT_HOME/config/config.yaml, which by defaults resolves to ~/.tsh/config/config.yaml.

tsh merges the user-specific config with the global config. In case of conflicts (i.e. same alias defined in both files), the user-specific config has higher priority.

In either of those files you can add define an alias such as:

aliases:
    "l": "tsh login --auth=okta"

From now on, tsh l will resolve to tsh login --auth=okta.

You can also change the defaults for regular tsh commands:

aliases:
    "status": "tsh status --format=json"

Calling external programs other than tsh is also possible:

aliases:
    "connect": "bash -c 'tsh login $0 && tsh ssh $1'"

The example above demonstrates the usage of variables $0 and $1. They represent arguments provided to the alias. With the definition above, tsh connect foo bar resolves to bash -c 'tsh login foo && tsh ssh bar'.

The alias can use as many arguments as needed. If the alias is invoked with too few arguments, tsh will report an error. Conversely, providing additional arguments is not an error. tsh will append any additional arguments to the end of an alias definition.

Given the configuration:

aliases:
    "example": "bash -c 'echo first=$0 $0-$1 $3'"

tsh example 0 1 unused-2 3 unused-4 will expand to bash -c 'echo first=0 0-1 3 unused-2 unused-4'.

You can also add the $TSH variable to an alias definition. When invoking the alias, tsh will expand this to the absolute path to current tsh executable. This can be useful if there are multiple tsh versions installed, or the currently used version is not in PATH.

aliases:
    "status": "$TSH status --format=json"

The alias substitution happens before the command line flags are fully parsed. This means that it is not affected by the --debug flag. To troubleshoot your aliases, set the TELEPORT_DEBUG=1 environment variable instead. This will cause the tsh logs to be printed to the console:

$ TELEPORT_DEBUG=1 tsh status
DEBU [TSH]       Self re-exec command: tsh [status --format=json]. tsh/aliases.go:203
...

Interactive Recording

Within the Teleport cluster the Server and Kubernetes recorded sessions are available for listing and replay for authorized users via tsh. The same listings, including Desktop Access sessions, are available in the Web Console under Session Recordings in the Activity section.

List and Play Recordings

Get the list of recorded sessions. By default, recordings for the past 24 hours are shown. The time range can be customized with the --last, --from-utc, and --to-utc flags.

tsh recordings ls
ID                                   Type Participants Hostname Timestamp           
------------------------------------ ---- ------------ -------- ------------------- 
b0a04442-70dc-4be8-9308-7b7901d2d600 ssh  jeff         dev       Nov 26 16:36:16 UTC 
c0a02222-70dc-4be8-9308-7b7901d2d600 kube alice                  Nov 26 20:36:16 UTC 
d0a04442-70dc-4be8-9308-7b7901d2d600 ssh  navin        test      Nov 26 16:36:16 UTC 

To play sessions, use the tsh play <sessionid> command. The screen will reset playing the interactive session with the timestamp in the upper right.

tsh play c0a02222-70dc-4be8-9308-7b7901d2d600
[email protected]:/# df                                                                     2022-11-26T21:20:52.715Z
Filesystem     1K-blocks    Used Available Use% Mounted on
overlay         20959212 4387284  16571928  21% /
tmpfs              65536       0     65536   0% /dev
tmpfs            1982720       0   1982720   0% /sys/fs/cgroup
/dev/nvme0n1p1  20959212 4387284  16571928  21% /etc/hosts
shm                65536       0     65536   0% /dev/shm
tmpfs            3410436      12   3410424   1% /run/secrets/kubernetes.io/serviceaccount
tmpfs            1982720       0   1982720   0% /proc/acpi
tmpfs            1982720       0   1982720   0% /sys/firmware
[email protected]:/# exit
exit
end of session playback

Further reading

• CLI Reference.