Troubleshooting Server Access

This section describes common issues that you might encounter in managing access to servers with Teleport and how to work around or resolve them.

Starting SSH sessions fails

When you start a new SSH session, Teleport forks itself and the child process runs as the OS user who is attempting to connect. If the file system permissions on the teleport binary don't include execute permissions for this user, the session fails to start.

This issue could be caused by systems that prevent newly installed software from being executable by any user.

Symptom

This issue results in a disconnected session and an error message similar to the following:

Failed to launch: fork/exec /proc/self/exe: permission denied.
Process exited with status 255

Solution

You should check the permission settings for the teleport binary.

To check the file system permissions on the teleport binary:

1. Open a terminal shell on the computer where you have installed the teleport service.

2. Determine the location and file system permission of the Teleport binary by running the following command:

   ls -al $(which teleport)

   The command should return output similar to the following:

   -rwxr-xr-x  1 root  wheel  531849504 Aug 30 18:32 /usr/local/bin/teleport
   

   If you don't see the permission that allows other users to read and execute (-rwxr-xr-x), you should update the permissions. For example:

   sudo chmod go+rx $(which teleport)

3. Restart the teleport service.

Missing logins for single sign-on users

If you use an external identity provider to enable single sign-on for users, you should be sure to assign logins for those users.

Symptom

Users who have access to Teleport through an authentication connector for an external identity provider don't see any of the logins they need to access remote resources.

Solution

To fix this issue, you should check that the configuration of your auth connectors assigns logins to your single sign-on users or modify the traits in the Teleport roles assigned to users through their group membership in the external identity provider. For more information about using traits in roles, see Role Templates.

Offline servers are included in the server list

In some cases, running tsh ls or tctl nodes ls commands might include servers that are offline. For example, a server that has stopped sending a heartbeat to the Teleport Proxy Service might continue to be listed as available in the output of tsh ls or tctl nodes ls commands for 10 minutes or more.

Symptom

An unresponsive server is listed as available in the output of tsh ls or tctl nodes ls commands. For example, you might run either of these commands and see output similar to the following for servers that have previously sent a heartbeat signal to the Teleport Proxy Service even if one of these servers subsequently went offline:

Node Name      Address        Labels                  
-------------- -------------- ----------------------- 
ip-172-3-1-242 127.0.0.1:3022 hostname=ip-172-3-1-242 
ip-172-3-1-75  ⟵ Tunnel       hostname=ip-172-3-1-75  
ip-172-3-2-177 ⟵ Tunnel       hostname=ip-172-3-2-177

Solution

To investigate whether a server that previously sent a heartbeat has become unresponsive, you can run the tsh ls or tctl nodes ls command with the --format json command-line option to see additional information, including an expiration time. For example:

    "kind": "node",
    "version": "v2",
    "metadata": {
      "name": "c78612d9-dab4-497f-a4d8-59ddb7edc6e9",
      "labels": {
        "teleport.internal/resource-id": "3547a530-3b58-4f65-8335-c5cf99c7b374"
      },
      "expires": "2023-09-15T21:40:17.653190645Z",
      "id": 1694813417653574518
    },

If the server sends a regular heartbeat signal, the expires value should remain relatively consistent, for example, eight to ten minutes from the current time. If the time to expire is less than the typical expiration time—for example, within the next two or three minutes from the current time—it's likely that the server has stopped sending the heartbeat.

Unable to join a shared session

Teleport allows multiple users to observe or participate in active sessions. You can define rules and configure role-based policies to control which users can join other users' sessions from tsh and the Teleport Web UI. If you are unable to join a shared session, you should check your role assignments and ensure you have a role that include the join_session permission. For example:

kind: role
metadata:
  name: auditor
version: v6
spec:
  allow:
    join_sessions:
      - name: Join prod sessions
        roles : ['prod-access']
        kinds: ['k8s', 'ssh']
        modes: ['moderator', 'observer']

For more information about moderated sessions and session sharing, see Moderated Sessions.

Unable to connect to agentless OpenSSH server as root

You should check your sshd configuration in /etc/ssh/sshd_config for a setting like PermitRootLogin no or PermitRootLogin forced-commands-only - either of these settings will prevent login as root. If you wish to log in as root to an OpenSSH server via Teleport, we recommend changing this setting to PermitRootLogin prohibit-password.

You will need to restart sshd for the change to take effect:

sudo systemctl restart sshd