Using the tsh Command Line Tool

This guide will show you how to use the Teleport client tool, tsh.

You will learn how to:

• Log in to an interactive shell on remote cluster nodes.
• Copy files to and from cluster nodes.
• Connect to SSH clusters behind firewalls without any open ports using SSH reverse tunnels.
• Explore a cluster and execute commands on specific nodes in the cluster.
• Share interactive shell sessions with colleagues or join someone else's session.
• List and replay recorded interactive sessions.

In addition to this document, you can always simply type tsh into your terminal for the CLI reference.

Introduction

For the impatient, here's an example of how a user would typically use tsh:

Log into a Teleport cluster. This command retrieves the user's certificates
and saves them into ~/.tsh/teleport.example.com
tsh login --proxy=teleport.example.com

SSH into a Node as usual
tsh ssh user@node

`tsh ssh` takes the same arguments as the OpenSSH client:
tsh ssh -o ForwardAgent=yes user@node
tsh ssh -o AddKeysToAgent=yes user@node

You can even create a convenient symlink:
ln -s /path/to/tsh /path/to/ssh

... and now your 'ssh' command is calling Teleport's `tsh ssh`
ssh user@host

This command removes SSH certificates from a user's machine:
tsh logout

In other words, Teleport was designed to be fully compatible with existing SSH-based workflows and does not require users to learn anything new, other than to call tsh login in the beginning.

Installing tsh

Follow the instructions below to install the tsh binary.

We recommend installing tsh of the same major version as the version used in your Teleport cluster.

To find the version number, either:

• In the Web UI, select your username in the upper right, then click Help & Support. You will see the version of your Teleport cluster under CLUSTER INFORMATION.

• Use curl and jq. Replace teleport.example.com with your Proxy Service address (e.g. mytenant.teleport.sh for Teleport Enterprise Cloud):

  curl https://teleport.example.com/webapi/find | jq '.server_version'
  "14.0.1"

curl -O https://cdn.teleport.dev/tsh-14.0.1.pkg

brew install teleport

curl.exe -O https://cdn.teleport.dev/teleport-v14.0.1-windows-amd64-bin.zip
Unzip the archive and move `tsh.exe` to your %PATH%

curl -O https://cdn.teleport.dev/teleport-v14.0.1-linux-amd64-bin.tar.gz
tar -xzf teleport-v14.0.1-linux-amd64-bin.tar.gz
cd teleport
sudo ./install
Teleport binaries have been copied to /usr/local/bin

User identities

A user identity in Teleport exists in the scope of a cluster. The member nodes of a cluster may have multiple OS users on them. A Teleport administrator assigns allowed logins to every Teleport user account.

When logging into a remote node, you will have to specify both the Teleport login and the OS login. A Teleport identity will have to be passed via the --user flag while the OS login will be passed as login@host using syntax compatible with the traditional ssh command.

Authenticate against the "work" cluster as joe and then
log into the node as root:
tsh ssh --proxy=work.example.com --user=joe root@node

CLI Docs - tsh ssh

Logging in

To retrieve a user's certificate, execute:

Full form:
tsh login --proxy=proxy_host:<https_proxy_port>

Using default ports:
tsh login --proxy=work.example.com

Using custom HTTPS port:
tsh login --proxy=work.example.com:5000

CLI Docs - tsh login

The login command retrieves a user's certificate and stores it in ~/.tsh directory as well as in the ssh agent if there is one running.

This allows you to authenticate just once, maybe at the beginning of the day. Subsequent tsh ssh commands will run without asking for credentials until the temporary certificate expires. By default, Teleport issues user certificates with a time to live (TTL) of 12 hours.

A Teleport cluster can be configured for multiple user identity sources. For example, a cluster may have a local user called admin while regular users should authenticate via GitHub. In this case, you have to pass --auth flag to tsh login to specify which identity storage to use:

Log in using the local Teleport 'admin' user:
tsh --proxy=proxy.example.com --auth=local --user=admin login

Log in using GitHub as an SSO provider, assuming the GitHub connector is called "github"
tsh --proxy=proxy.example.com --auth=github login

When using an external identity provider to log in, tsh will need to open a web browser to complete the authentication flow. By default, tsh will use your system's default browser. If you wish to suppress this behavior, you can use the --browser=none flag:

Don't open the system default browser when logging in
tsh login --proxy=work.example.com --browser=none

In this situation, a link will be printed on the screen. You can copy and paste this link into a browser of your choice to continue the login flow.

CLI Docs - tsh login

Inspecting an SSH certificate

To inspect the SSH certificates in ~/.tsh, a user may execute the following command:

tsh status

> Profile URL:  https://proxy.example.com:3080
Logged in as: johndoe
Cluster:      proxy.example.com
Roles:        admin*
Logins:       root, admin, guest
Kubernetes:   disabled
Valid until:  2017-04-25 15:02:30 -0700 PDT [valid for 1h0m0s]
Extensions:   permit-agent-forwarding, permit-port-forwarding, permit-pty

CLI Docs - tsh status

SSH agent support

If there is an ssh agent running, tsh login will store the user certificate in the agent. This can be verified via:

ssh-add -L

The SSH agent can be used to feed the certificate to other SSH clients, for example to OpenSSH (ssh).

If you wish to disable SSH agent integration, pass --no-use-local-ssh-agent to tsh. You can also set the TELEPORT_USE_LOCAL_SSH_AGENT environment variable to false in your shell profile to make this permanent.

Identity files

tsh login can also save the user certificate into a file:

Authenticate the user against proxy.example.com and save the user
certificate to joe.pem
tsh login --proxy=proxy.example.com --out=joe

Use joe.pem to log in to the server 'db'
tsh ssh --proxy=proxy.example.com -i joe joe@db

By default, the --out flag will create an identity file suitable for tsh -i. If compatibility with OpenSSH is needed, --format=openssh must be specified. In this case, the identity will be saved into two files, joe and joe-cert.pub:

tsh login --proxy=proxy.example.com --out=joe --format=openssh
ls -lh

total 8.0K
-rw------- 1 joe staff 1.7K Aug 10 16:16 joe
-rw------- 1 joe staff 1.5K Aug 10 16:16 joe-cert.pub

SSH certificates for automation

Regular users of Teleport must request an auto-expiring SSH certificate, usually every day. This doesn't work for non-interactive scripts, like cron jobs or a CI/CD pipeline.

The most secure way to generate certificates for automation purposes is to use Machine ID. This ensures that your automation is taking advantage of the security properties of short-lived credentials.

If Machine ID does not support your preferred CI/CD platform, you can create a local user for use in automation and request a long-lived certificate for that user.

In this example, we're creating a certificate with a TTL of one hour for the jenkins user and storing it in a jenkins.pem file, which can be later used with -i (identity) flag for tsh.

Log in to your cluster with tsh so you can use tctl from your local machine.
You can also run tctl on your Auth Service host without running "tsh login"
first.
tsh login --proxy=teleport.example.com --user=myuser
tctl auth sign --ttl=1h --user=jenkins --out=jenkins.pem

CLI Docs - tctl auth sign

Now jenkins.pem can be copied to the Jenkins server and passed to the -i (identity file) flag of tsh.

tctl auth sign is an admin's equivalent of tsh login --out and allows for unrestricted certificate TTL values.

Exploring the cluster

In a Teleport cluster, all Nodes periodically ping the cluster's Auth Service and update their status. This allows Teleport users to see which Nodes are online with the tsh ls command:

This command lists all Nodes in the cluster you logged into via "tsh login":
tsh ls

Node Name     Address            Labels
---------     -------            ------
turing        10.1.0.5:3022      os:linux
turing        10.1.0.6:3022      os:linux
graviton      10.1.0.7:3022      os:osx

CLI Docs - tsh ls

tsh ls can apply a filter based on the node labels.

Only show Nodes with os label set to 'osx':
tsh ls os=osx

Nodename      UUID                   Address            Labels
---------     -------                -------            ------
graviton      33333333-aaaa-1284     10.1.0.7:3022      os:osx

CLI Docs -tsh ls

Interactive shell

To launch an interactive shell on a remote Node or to execute a command, use tsh ssh.

tsh tries to mimic the ssh experience as much as possible, so it supports the most popular ssh flags like -p, -l or -L. For example, if you have the following alias defined in your ~/.bashrc: alias ssh="tsh ssh" then you can continue using familiar SSH syntax:

Have this alias configured, perhaps via ~/.bashrc
alias ssh="/usr/local/bin/tsh ssh"

Login in to a cluster and retrieve your SSH certificate:
tsh --proxy=proxy.example.com login

These commands execute `tsh ssh` under the hood:
ssh user@node
ssh -p 6122 user@node ls
ssh -o ForwardAgent=yes user@node
ssh -o AddKeysToAgent=yes user@node

tsh --proxy=proxy.example.com:5000 <subcommand>

Port forwarding

tsh ssh supports the OpenSSH -L flag which forwards incoming connections from localhost to the specified remote host:port. The syntax of -L flag is as follows, where "bind_ip" defaults to 127.0.0.1:

-L [bind_ip]:listen_port:remote_host:remote_port

Example:

tsh ssh -L 5000:web.remote:80 node

This will connect to remote server node via the Proxy Service, then open a listening socket on localhost:5000. Finally, it will forward all incoming connections to web.remote:80 via this SSH tunnel.

It is often convenient to establish port forwarding, execute a local command which uses the connection, and then disconnect. You can do this with the --local flag.

Example:

tsh ssh -L 5000:google.com:80 --local node curl http://localhost:5000

This command:

• Connects to node.
• Binds the local port 5000 to port 80 on google.com.
• Executes curl command locally, which results in curl hitting google.com:80 via node.

SSH jump host

While implementing ProxyJump for Teleport, we have extended the feature to tsh.

tsh ssh -J proxy.example.com telenode

Known limitations:

• Only one jump host is supported (-J supports chaining that Teleport does not utilize) and tsh will return with error in the case of two jump hosts, i.e. -J proxy-1.example.com,proxy-2.example.com will not work.
• When tsh ssh -J user@proxy is used, it overrides the SSH proxy defined in the tsh profile, and port forwarding is used instead of the existing Teleport proxy subsystem.

Resolving Node names

tsh supports multiple methods to resolve remote Node names.

• Traditional: by IP address or via DNS.
• Nodename setting: the teleport daemon supports the nodename flag, which allows Teleport administrators to assign alternative Node names.
• Labels: you can address a Node by name=value pair.

If we have two Node, one with os:linux label and one Node with os:osx, we can log in to the OSX Node with:

tsh ssh os=osx

This only works if there is only one remote node with the os:osx label, but you can still execute commands via SSH on multiple Nodes using labels as a selector. This command will update all system packages on machines that run Linux:

tsh ssh os=ubuntu apt-get update -y

Short-lived sessions

The default TTL of a Teleport user certificate is 12 hours. This can be modified at login with the --ttl flag. This command logs you into the cluster with a very short-lived (1 minute) temporary certificate:

tsh --ttl=1 login

You will be logged out after one minute, but if you want to log out immediately, you can always run:

tsh logout

Copying files

To securely copy files to and from cluster Nodes, use the tsh scp command. It is designed to mimic OpenSSH's scp command as much as possible:

tsh scp example.txt root@node:/path/to/dest

Again, you may want to create a bash alias like alias scp="tsh --proxy=work scp" and use the familiar syntax:

scp -P 61122 -r files root@node:/path/to/dest

Starting from Teleport 9 the SCP and SFTP protocols are both supported by Server Access. OpenSSH scp or sftp commands can both be used in place of tsh scp if desired.

Sharing sessions

Suppose you are trying to troubleshoot a problem on a remote server. Sometimes it makes sense to ask another team member for help. Traditionally, this could be done by letting them know which host you're on, having them SSH in, start a terminal multiplexer like screen, and join a session there.

Teleport makes this more convenient. Let's log in to a server named luna and ask Teleport for our current session status:

tsh ssh luna
on host luna
teleport status

User ID    : joe, logged in as joe from 10.0.10.1 43026 3022
Session ID : 7645d523-60cb-436d-b732-99c5df14b7c4
Session URL: https://work:3080/web/sessions/7645d523-60cb-436d-b732-99c5df14b7c4

Now you can invite another user account to the work cluster. You can share the URL for access through a web browser, or you can share the session ID, and the other user can join you through their terminal by typing:

tsh join <session_ID>

Connecting to SSH clusters behind firewalls

Teleport supports creating clusters of servers located behind firewalls without any open listening TCP ports. This works by creating reverse SSH tunnels from behind-firewall environments into a Teleport Proxy Service you have access to.

These features are called Trusted Clusters. Refer to the Trusted Clusters guide to learn how a Trusted Cluster can be configured.

tsh --proxy=work clusters

Cluster Name     Status
------------     ------
staging          online
production       offline

CLI Docs - tsh clusters

Now you can use the --cluster flag with any tsh command. For example, to list SSH nodes that are members of the production cluster, simply run:

tsh --proxy=work ls --cluster=production

Node Name     Node ID       Address            Labels
---------     -------       -------            ------
db-1          xxxxxxxxx     10.0.20.31:3022    kernel:4.4
db-2          xxxxxxxxx     10.0.20.41:3022    kernel:4.2

Similarly, if you want to SSH into db-1 inside the production cluster:

tsh --proxy=work ssh --cluster=production db-1

X11 forwarding

In order to run graphical programs within an SSH session, such as an IDE like Virtual Studio Code, you'll need to request X11 forwarding for the session with the -X flag.

tsh ssh -X node01

X11 forwarding provides the server with secure access to your local X Server so that it can communicate directly with your local display and I/O devices.

In order to use X11 forwarding, you'll need to enable it on the Teleport Node. You'll also need to ensure that your user has the permit_x11_forwarding role option:

tsh status
> Profile URL:        https://proxy.example.com:3080  Logged in as:       dev  ...  Extensions:         permit-X11-forwarding

Custom aliases and defaults

You can configure tsh to define aliases, custom commands and command-specific flag defaults. Using aliases, you can run frequently used tsh commands more easily.

Aliases are defined in configuration files using the following syntax:

aliases:
    "<alias>": "<command>"

The <alias> can only be a top-level subcommand. In other words, you can define tsh mycommand alias but not tsh my command.

tsh loads two kinds of configuration files:

• global: set via the $TELEPORT_GLOBAL_TSH_CONFIG env var if not provided it will default to /etc/tsh.yaml on non-Windows operating systems.
• user-specific: $TELEPORT_HOME/config/config.yaml, which by default resolves to ~/.tsh/config/config.yaml.

tsh merges the user-specific config with the global config. In case of conflicts (i.e. same alias defined in both files), the user-specific config has higher priority.

In either of those files you can add define an alias such as:

aliases:
    "l": "tsh login --auth=okta"

From now on, tsh l will resolve to tsh login --auth=okta.

You can also change the defaults for regular tsh commands:

aliases:
    "status": "tsh status --format=json"

Calling external programs other than tsh is also possible:

aliases:
    "connect": "bash -c 'tsh login $0 && tsh ssh $1'"

The example above demonstrates the usage of variables $0 and $1. They represent arguments provided to the alias. With the definition above, tsh connect foo bar resolves to bash -c 'tsh login foo && tsh ssh bar'.

The alias can use as many arguments as needed. If the alias is invoked with too few arguments, tsh will report an error. Conversely, providing additional arguments is not an error. tsh will append any additional arguments to the end of an alias definition.

Given the configuration:

aliases:
    "example": "bash -c 'echo first=$0 $0-$1 $3'"

tsh example 0 1 unused-2 3 unused-4 will expand to bash -c 'echo first=0 0-1 3 unused-2 unused-4'.

You can also add the $TSH variable to an alias definition. When invoking the alias, tsh will expand this to the absolute path to current tsh executable. This can be useful if there are multiple tsh versions installed, or the currently used version is not in PATH.

aliases:
    "status": "$TSH status --format=json"

The alias substitution happens before the command line flags are fully parsed. This means that it is not affected by the --debug flag. To troubleshoot your aliases, set the TELEPORT_DEBUG=1 environment variable instead. This will cause the tsh logs to be printed to the console:

$ TELEPORT_DEBUG=1 tsh status
DEBU [TSH]       Self re-exec command: tsh [status --format=json]. tsh/aliases.go:203
...

Interactive Recording

Within the Teleport cluster, server and Kubernetes recorded sessions are available for listing and replay for authorized users via tsh. The same listings, including Windows desktop sessions, are available in the Web Console under Session Recordings in the Activity section.

List and Play Recordings

Get the list of recorded sessions. By default, recordings for the past 24 hours are shown. The time range can be customized with the --last, --from-utc, and --to-utc flags.

tsh recordings ls
ID                                   Type Participants Hostname Timestamp------------------------------------ ---- ------------ -------- -------------------b0a04442-70dc-4be8-9308-7b7901d2d600 ssh  jeff         dev       Nov 26 16:36:16 UTCc0a02222-70dc-4be8-9308-7b7901d2d600 kube alice                  Nov 26 20:36:16 UTCd0a04442-70dc-4be8-9308-7b7901d2d600 ssh  navin        test      Nov 26 16:36:16 UTC

To play sessions, use the tsh play <sessionid> command. The screen will reset playing the interactive session with the timestamp in the upper right.

tsh play c0a02222-70dc-4be8-9308-7b7901d2d600
root@ubuntu:/# df                                                                     2022-11-26T21:20:52.715ZFilesystem     1K-blocks    Used Available Use% Mounted onoverlay         20959212 4387284  16571928  21% /tmpfs              65536       0     65536   0% /devtmpfs            1982720       0   1982720   0% /sys/fs/cgroup/dev/nvme0n1p1  20959212 4387284  16571928  21% /etc/hostsshm                65536       0     65536   0% /dev/shmtmpfs            3410436      12   3410424   1% /run/secrets/kubernetes.io/serviceaccounttmpfs            1982720       0   1982720   0% /proc/acpitmpfs            1982720       0   1982720   0% /sys/firmwareroot@ubuntu:/# exitexit
end of session playback

tsh configuration files

You can use a configuration file to control the behavior of tsh. The scope of the configuration file depends on its location:

• /etc/tsh.yaml is the default location for global, shared configuration settings. You can override the location with the TELEPORT_GLOBAL_TSH_CONFIG environment variable.
• $TELEPORT_HOME/config/config.yaml is the default location for user-specific configuration settings. The default location for TELEPORT_HOME is ~/.tsh.

tsh merges the settings from both configuration file locations, with the user configuration settings taking precedence.

Extra proxy headers

The tsh configuration file enables you to specify HTTP headers to be included in requests to Teleport Proxy Servers with addresses matching the proxy field.

add_headers:
  - proxy: "*.example.com" # matching proxies will have headers included
    headers: # headers are pairs to include in the http headers
      foo: bar # Key/Value to be included in the http request

For example, adding HTTP headers can be useful if an intermediate HTTP proxy is in place that requires setting an authentication token:

add_headers:
  - proxy: "*.infra.corp.xyz"
    headers:
      "Authorization": "Bearer tokentokentoken"

Aliases

Aliases allow you to define custom commands or change the default flag values for existing commands using the following syntax:

aliases:
    "<alias>": "<command>"

The <alias> can only be a top-level subcommand. In other words, you can define a tsh mycommand alias but not tsh my command.

New command tsh l:

aliases:
    "l": "tsh login --auth=okta"

Make tsh status use JSON as a default format:

aliases:
    "status": "tsh status --format=json"

The alias can use an arbitrary number of arguments. If an argument variable $N is referenced, tsh will check that at least N+1 arguments were given to the alias invocation. All arguments that were given but not referenced in the alias definition will be appended at the end.

Define a custom command using bash. The $0 and $1 variables will be substituted with command arguments.

aliases:
    "connect": "bash -c 'tsh login $0 && tsh ssh $1'"

Define a custom login command where first argument specifies --auth option.

aliases:
    "ap": "tsh login --auth=$0 --proxy=teleport.example.com"

Given the configuration:

aliases:
    "example": "bash -c 'echo first=$0 $0-$1 $3'"

tsh example 0 1 unused-2 3 unused-4 will expand to bash -c 'echo first=0 0-1 3 unused-2 unused-4'.

An alias definition can also reference the $TSH variable. If you use the $TSH variable in an alias, tsh expands the variable to the absolute path of the current tsh executable. This behavior can be useful if there are multiple tsh versions installed, or the version you're currently using is not in the PATH:

aliases:
    "status": "$TSH status --format=json"

To troubleshoot aliases, set the TELEPORT_DEBUG=1 environment variable. This will cause detailed logs to be printed to standard error:

$ TELEPORT_DEBUG=1 tsh status
DEBU [TSH]       Self re-exec command: tsh [status --format=json]. tsh/aliases.go:203
...

Proxy templates

With proxy templates, tsh dynamically determines the address of the Teleport Proxy Service to connect to based on the address of the destination host in your tsh ssh or tsh proxy ssh command:

proxy_templates:

# Regular expression that the host server address `%h:%p` is matched against.
# The "replace rules" below can reference capturing groups from this regular
# expression (`$1`, `$2`, etc.).
- template: '^(\w+)\.(\w+):([0-9]+)$' # <nodename>.<clustername>:<port>

  # Optional web proxy address to use for proxy jump (`--jumphost`, `-J`).
  #
  # Proxy Jump can be used to reduce latency in regionally distributed trusted
  # clusters by connecting to a leaf node through the leaf proxy instead of the
  # root proxy.
  proxy: "$2.eu.example.com:443"

  # Optional cluster name to connect to (`--cluster`).
  #
  # Cluster can be used to connect to leaf nodes from the root proxy without
  # first logging in to the leaf cluster. This may be useful in cases where
  # proxy jump is not applicable, such as when the leaf clusters do not have
  # their own public proxies.
  cluster: "$2"

  # Optional host server address to connect to (`%h:%p`).
  #
  # Port defaults to 3022 if not explicitly provided with `--port`.
  host: "$1:$3"

# Multiple templates can be provided. They are evaluated in order and the first
# match takes effect.
- template: ...

tsh -J {{proxy}} ssh and tsh -J {{proxy}} proxy ssh will attempt to match the host server address %h:%p with the configured templates. For each replace rule set, the corresponding cli value will be set.

If leaf certificates are required to connect to the node, tsh automatically retrieves leaf certificates from the root cluster:

$ tsh ssh -J {{proxy}} node1.leaf1
# becomes
$ tsh ssh -J leaf1.eu.example.com:443 --cluster leaf1 node1

If there is no template matched, an error is returned.

$ tsh ssh -J {{proxy}} node1.none.example.com
ERROR: proxy jump contains {{proxy}} variable but did not match any of the templates in tsh config

If you don't explicitly provide the proxy variable -J {{proxy}}, tsh still attempts to match a template, but won't fail if there isn't a match. Additionally, tsh won't replace the proxy value if it's explicitly set by the client:

$ tsh ssh -J leaf2.us.example.com:443 node1.leaf2
# becomes
$ tsh ssh -J leaf2.us.example.com:443 --cluster leaf2 node1

Proxy Templates can also be used with OpenSSH by setting the ProxyCommand in ~/.ssh/config to use tsh proxy ssh.

Host *.example.com
    Port 3022
    ProxyCommand tsh proxy ssh -J {{proxy}} %r@%h:%p

As a result, you can use tsh ssh and ssh interchangeably.

$ tsh ssh node1.leaf1
# is equivalent to
$ ssh node1.leaf1

Uninstalling tsh

To remove tsh and associated user data see Uninstalling Teleport.

Further reading

• CLI Reference.