Using Teleport Machine ID with Jenkins

Jenkins is an open source automation server that is frequently used to build Continuous Integration and Continuous Delivery (CI/CD) pipelines.

In this guide, we will demonstrate how to migrate existing Jenkins pipelines to utilize Machine ID with minimal changes.

Prerequisites

You will need the following tools to use Teleport with Jenkins.

• The Teleport Auth Service and Proxy Service version >= 9.0.0, deployed on your own infrastructure or managed via Teleport Cloud.
• ssh OpenSSH tool
• Jenkins
• The tctl admin tool version >= 11.3.2

version: v1
proxy_service:
  enabled: "yes"
  listen_addr: "0.0.0.0:3023"
  ...

tsh login --proxy=teleport.example.com [email protected]
tctl status
Cluster  teleport.example.com
Version  11.3.2
CA pin   sha256:abdc1245efgh5678abdc1245efgh5678abdc1245efgh5678abdc1245efgh5678

tsh login --proxy=myinstance.teleport.sh [email protected]
tctl status
Cluster  myinstance.teleport.sh
Version  11.2.1
CA pin   sha256:sha-hash-here

Architecture

Before we begin, it should be noted that Jenkins is a tool that is notoriously difficult to secure. Machine ID is one part of securing your infrastructure, but it alone is not sufficient. Below we will provide some basic guidance which can help improve the security posture of your Jenkins installation.

Single-host deployments

The simplest Jenkins deployments have the controller (process that stores configuration, plugins, UI) and agents (process that executes tasks) run on the same host. This deployment model is simple to get started with, however any compromise of the jenkins user within a single pipeline can lead to the compromise of your entire CI/CD infrastructure.

Multihost deployments

A slightly more complex, but more secure deployment is running your Jenkins controllers and agents on different hosts and pinning workloads to specific agents. This is an improvement over the simple deployment because you can limit the blast radius of the compromise of a single pipeline to a subset of your CI/CD infrastructure instead of all of your infrastructure.

Best practices

We strongly encourage the use of the second deployment model whenever possible, with ephemeral hosts and IAM joining when possible. When using Machine ID with this model, create and run Machine ID bots per-host and pin particular pipelines to a worker. This will allow you to give each pipeline the minimal scope for server access, reduce the blast radius if one pipeline is compromised, and allow you to remotely audit and lock pipelines if you detect malicious behavior.

Step 1/2 Configure and start Machine ID

First, determine if you would like to create a new role for Machine ID or use an existing role. You can run tctl get roles to examine your existing roles.

In the example below, create a file called api-workers.yaml with the content below to create a new role called api-workers that will allow you to log in to Nodes with the label group: api and Linux user jenkins.

kind: "role"
version: "v3"
metadata:
  name: "api-workers"
spec:
  allow:
    logins: ["jenkins"]
    node_labels:
      "group": "api"

Connect to the Teleport Auth Server and use tctl to examine what roles exist on your system.

tctl create -f api-workers.yaml
tctl bots add jenkins --roles=api-workers

Machine ID allows you to use Linux Access Control Lists (ACLs) to control access to certificates on disk. You will use this to limit the access Jenkins has to the short-lived certificates Machine ID issues.

In the example that follows, you will create a Linux user called teleport to run Machine ID but short-lived certificates will be written to disk as the Linux user jenkins.

sudo adduser \    --disabled-password \    --no-create-home \    --shell=/bin/false \    --gecos "" \    teleport

Create and initialize the directories you will need using the tbot init command.

sudo tbot init \    --destination-dir=/opt/machine-id \    --bot-user=teleport \    --owner=teleport:teleport \    --reader-user=jenkins

Next, you need to start Machine ID in the background of each Jenkins worker.

First create a configuration file for Machine ID at /etc/tbot.yaml.

auth_server: "auth.example.com:3025"
onboarding:
  join_method: "token"
  token: "00000000000000000000000000000000"
  ca_pins:
  - "sha256:1111111111111111111111111111111111111111111111111111111111111111"
storage:
  directory: /var/lib/teleport/bot
destinations:
  - directory: /opt/machine-id

Next, create a systemd.unit file at /etc/systemd/system/machine-id.service.

[Unit]
Description=Teleport Machine ID Service
After=network.target

[Service]
Type=simple
User=teleport
Group=teleport
Restart=on-failure
ExecStart=/usr/local/bin/tbot start -c /etc/tbot.yaml
ExecReload=/bin/kill -HUP $MAINPID
PIDFile=/run/machine-id.pid
LimitNOFILE=524288

[Install]
WantedBy=multi-user.target

Finally, run the following commands to start Machine ID.

sudo systemctl daemon-reload
sudo systemctl start machine-id
sudo systemctl status machine-id

Step 2/2. Update and run Jenkins pipelines

Using Machine ID within a Jenkins pipeline is now a one-line change. For example, if you want to run the hostname command on a remote host, add the following to your Jenkins pipeline.

steps {
  sh "ssh -F /opt/machine-id/ssh_config [email protected] hostname"
}

You are all set. You have provided Jenkins with short-lived certificates tied to a machine identity that can be rotated, audited, and controlled with all the familiar Teleport access controls.