# Machine & Workload Identity with Ansible Ansible is a common tool for managing fleets of Linux hosts via SSH. In order to connect to the hosts, it requires a form of authentication. Machine & Workload Identity can be used to provide short-lived certificates to Ansible that allow it to connect to SSH nodes enrolled in Teleport in a secure and auditable manner. In this guide, you will configure the Machine & Workload Identity agent, `tbot`, to produce credentials and an OpenSSH configuration, and then configure Ansible to use these to connect to your SSH nodes through the Teleport Proxy Service. ## Prerequisites --- USING ANSIBLE AWX? If you're running your Ansible jobs under Ansible AWX or Ansible Automation Platform, consider reading our [dedicated guide](https://goteleport.com/docs/ver/19.x/machine-workload-identity/access-guides/ansible-awx.md). --- You will need the following tools to use Teleport with Ansible. - A running Teleport cluster. If you want to get started with Teleport, [sign up](https://goteleport.com/signup) for a free trial or [set up a demo environment](https://goteleport.com/docs/ver/19.x/get-started/deploy-community.md). - The `tctl` and `tsh` clients. Installing `tctl` and `tsh` clients 1. Determine the version of your Teleport cluster. The `tctl` and `tsh` clients must be at most one major version behind your Teleport cluster version. Send a GET request to the Proxy Service at `/v1/webapi/find` and use a JSON query tool to obtain your cluster version. Replace teleport.example.com:443 with the web address of your Teleport Proxy Service: ``` $ TELEPORT_DOMAIN=teleport.example.com:443 $ TELEPORT_VERSION="$(curl -s https://$TELEPORT_DOMAIN/v1/webapi/find | jq -r '.server_version')" ``` 2. Follow the instructions for your platform to install `tctl` and `tsh` clients: **Mac** Download the signed macOS .pkg installer for Teleport, which includes the `tctl` and `tsh` clients: ``` $ curl -O https://cdn.teleport.dev/teleport-${TELEPORT_VERSION?}.pkg ``` In Finder double-click the `pkg` file to begin installation. --- DANGER Using Homebrew to install Teleport is not supported. The Teleport package in Homebrew is not maintained by Teleport and we can't guarantee its reliability or security. --- **Windows - Powershell** ``` $ curl.exe -O https://cdn.teleport.dev/teleport-v${TELEPORT_VERSION?}-windows-amd64-bin.zip Unzip the archive and move the `tctl` and `tsh` clients to your %PATH% NOTE: Do not place the `tctl` and `tsh` clients in the System32 directory, as this can cause issues when using WinSCP. Use %SystemRoot% (C:\Windows) or %USERPROFILE% (C:\Users\) instead. ``` **Linux** All of the Teleport binaries in Linux installations include the `tctl` and `tsh` clients. For more options (including RPM/DEB packages and downloads for i386/ARM/ARM64) see our [installation page](https://goteleport.com/docs/ver/19.x/installation.md). ``` $ curl -O https://cdn.teleport.dev/teleport-v${TELEPORT_VERSION?}-linux-amd64-bin.tar.gz $ tar -xzf teleport-v${TELEPORT_VERSION?}-linux-amd64-bin.tar.gz $ cd teleport $ sudo ./install Teleport binaries have been copied to /usr/local/bin ``` * `ssh` OpenSSH tool * `ansible` >= 2.9.6 * Optional: [`jq`](https://stedolan.github.io/jq/) to process `JSON` output * `tbot` must already be installed and configured on the machine that will run Ansible. For more information, see the [deployment guides](https://goteleport.com/docs/ver/19.x/machine-workload-identity/deployment.md). * If you followed the above guide, note the `--destination-dir=/opt/machine-id` flag, which defines the directory where SSH certificates and OpenSSH configuration used by Ansible will be written. In particular, you will be using the `/opt/machine-id/ssh_config` file in your Ansible configuration to define how Ansible should connect to Teleport Nodes. * To check that you can connect to your Teleport cluster, sign in with `tsh login`, then verify that you can run `tctl` commands using your current credentials. For example, run the following command, assigning teleport.example.com to the domain name of the Teleport Proxy Service in your cluster and email\@example.com to your Teleport username: ``` $ tsh login --proxy=teleport.example.com --user=email@example.com $ tctl status Cluster teleport.example.com Version 19.0.0-dev CA pin sha256:abdc1245efgh5678abdc1245efgh5678abdc1245efgh5678abdc1245efgh5678 ``` If you can connect to the cluster and run the `tctl status` command, you can use your current credentials to run subsequent `tctl` commands from your workstation. If you host your own Teleport cluster, you can also run `tctl` commands on the computer that hosts the Teleport Auth Service for full permissions. ## Step 1/4. Configure RBAC As Ansible will use the credentials produced by `tbot` to connect to the SSH nodes, you first need to configure Teleport to grant the bot access. This is done by creating a role that grants the necessary permissions and then assigning this role to a Bot. In this example, access will be granted to all SSH nodes for the username root. Ensure that you set this to a username that is available across your SSH nodes and that will have the appropriate privileges to manage your nodes. Create a file called `role.yaml` with the following content: ``` kind: role version: v6 metadata: name: example-role spec: allow: # Allow login to the user 'root'. logins: ['root'] # Allow connection to any node. Adjust these labels to match only nodes # that Ansible needs to access. node_labels: '*': '*' ``` Replace `example-role` with a descriptive name related to your use case. For production use, you should use labels to restrict this access to only the hosts that Ansible will need to access. This is known as the principle of least privilege and reduces damage that exfiltrated credentials can do. Use `tctl create -f ./role.yaml` to create the role. --- TIP You can also create and edit roles using the Web UI. Go to **Access -> Roles** and click **Create New Role** or pick an existing role to edit. --- Now, use `tctl bots update` to add the role to the Bot. Replace `example` with the name of the Bot you created in the deployment guide and `example-role` with the name of the role you just created: ``` $ tctl bots update example --add-roles example-role ``` ## Step 2/4. Configure the `tbot` output service Now, `tbot` needs to be configured with an output service that will produce the credentials and SSH configuration that is needed by Ansible. For SSH, we use the `identity` service type. Output services must be configured with a destination. In this example, the `directory` destination will be used. This will write these credentials to a specified directory on disk. Ensure that this directory can be written to by the Linux user that `tbot` runs as, and that it can be read by the Linux user that Ansible will run as. Modify your `tbot` configuration to add an `identity` service: ``` services: - type: identity destination: type: directory # For this guide, /opt/machine-id is used as the destination directory. # You may wish to customize this. Multiple outputs cannot share the same # destination. path: /opt/machine-id ``` If operating `tbot` as a background service, restart it. If running `tbot` in one-shot mode, it must be executed before you attempt to execute the Ansible playbook. You should now see several files under `/opt/machine-id`: - `ssh_config`: this can be used with Ansible or OpenSSH to configure them to use the Teleport Proxy Service with the correct credentials when making connections. - `known_hosts`: this contains the Teleport SSH host CAs and allows the SSH client to validate a host's certificate. - `key-cert.pub`: this is an SSH certificate signed by the Teleport SSH user CA. - `key`: this is the private key needed to use the SSH certificate. Next, Ansible will be configured to use these files when making connections. ## Step 3/4. Configure Ansible Create a folder named `ansible` where all Ansible files will be collected. ``` $ mkdir -p ansible $ cd ansible ``` Create a file called `ansible.cfg`. We will configure Ansible to run the OpenSSH client with the configuration file generated by `tbot`, `/opt/machine-id/ssh_config`. Note, `example.com` here is the name of your Teleport cluster. ``` [defaults] host_key_checking = True inventory=./hosts remote_tmp=/tmp [ssh_connection] scp_if_ssh = True ssh_args = -F /opt/machine-id/ssh_config ``` You can then create an inventory file called `hosts`. This should refer to the hosts using their hostname as registered in Teleport and the name of your Teleport cluster should be appended to this. For example, if your cluster is called `teleport.example.com` and your host is called `node1`, the entry in `hosts` would be `node1.teleport.example.com`. You can generate an inventory file for all your nodes that meets this requirement with a script like the following: ``` Source tsh env to get the name of the current Teleport cluster. $ eval "$( tsh env )" You can modify the `tsh ls` command to filter nodes based on the label. $ tsh ls --format=json | jq --arg cluster $TELEPORT_CLUSTER -r '.[].spec.hostname + "." + $cluster' > hosts ``` Not seeing Nodes? When the Teleport Auth Service receives a request to list Teleport-connected resources (e.g., to display resources in the Web UI or via `tsh ls`), it only returns the resources that the current user is authorized to view. For each resource in the user's Teleport cluster, the Auth Service applies the following checks in order and, if one check fails, hides the resource from the user: - None of the user's roles contain a `deny` rule that matches the resource's labels. - At least one of the user's roles contains an `allow` rule that matches the resource's labels. If you are not seeing resources when expected, make sure that your user's roles include the appropriate `allow` and `deny` rules as documented in the [Access Controls Reference](https://goteleport.com/docs/ver/19.x/reference/access-controls/roles.md). ## Step 4/4. Run a playbook Finally, let's create a simple Ansible playbook, `playbook.yaml`. The example playbook below runs `hostname` on all hosts. ``` - hosts: all remote_user: root tasks: - name: "hostname" command: "hostname" ``` From the folder `ansible`, run the Ansible playbook: ``` $ ansible-playbook playbook.yaml PLAY [all] ***************************************************************************************************************************************** TASK [Gathering Facts] ***************************************************************************************************************************** ok: [terminal] TASK [hostname] ************************************************************************************************************************************ changed: [terminal] PLAY RECAP ***************************************************************************************************************************************** terminal : ok=2 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0 ``` You are all set. You have provided your machine with short-lived certificates tied to a machine identity that can be rotated, audited, and controlled with all the familiar access controls. ## Troubleshooting In case if Ansible cannot connect, you may see error like this one: ``` example.host | UNREACHABLE! => { "changed": false, "msg": "Failed to connect to the host via ssh: ssh: Could not resolve hostname node-name: Name or service not known", "unreachable": true } ``` You can examine and tweak patterns matching the inventory hosts in `ssh_config`. Try the SSH connection using `ssh_config` with verbose mode to inspect the error: ``` $ ssh -vvv -F /opt/machine-id/ssh_config root@node-name.example.com ``` If `ssh` works, try running the playbook with verbose mode on: ``` $ ansible-playbook -vvv playbook.yaml ``` If your hostnames contain uppercase characters (like `MYHOSTNAME`), please note that Teleport's internal hostname matching is case sensitive by default, which can also lead to seeing this error. If this is the case, you can work around this by enabling case-insensitive routing at the cluster level. **Self-hosted Teleport** Edit your `/etc/teleport.yaml` config file on all servers running the Teleport `auth_service`, then restart Teleport on each. ``` auth_service: case_insensitive_routing: true ``` **Managed Teleport Enterprise/Cloud** Run `tctl edit cluster_networking_config` to add the following specification, then save and exit. ``` spec: case_insensitive_routing: true ``` ## Next steps - Read the [configuration reference](https://goteleport.com/docs/ver/19.x/reference/machine-workload-identity/configuration.md) to explore all the available configuration options. - If you use Ansible AWX or Ansible Automation Platform, read our [dedicated Ansible AWX Guide](https://goteleport.com/docs/ver/19.x/machine-workload-identity/access-guides/ansible-awx.md).