Skip to main content
Version: 17.x (unreleased)

HSM Support

This guide will show you how to set up the Teleport Auth Service to use a hardware security module (HSM) to store and handle private keys.

Prerequisites​

  • Teleport v17.0.0-dev Enterprise (self-hosted).
  • To check that you can connect to your Teleport cluster, sign in with tsh login, then verify that you can run tctl commands using your current credentials. For example: 
    $ tsh login --proxy=teleport.example.com [email protected]
    $ tctl status
    # Cluster  teleport.example.com
    # Version  17.0.0-dev
    # CA pin   sha256:abdc1245efgh5678abdc1245efgh5678abdc1245efgh5678abdc1245efgh5678
    If you can connect to the cluster and run the tctl status command, you can use your current credentials to run subsequent tctl commands from your workstation. If you host your own Teleport cluster, you can also run tctl commands on the computer that hosts the Teleport Auth Service for full permissions.
  • An HSM reachable from your Teleport auth server.
  • The PKCS#11 module for your HSM.
Compatibility Warning

Teleport Cloud and Teleport Open Source do not currently support HSMs or Key Management Services.

While most PKCS#11 HSMs should be supported, the Teleport team tests with AWS CloudHSM, YubiHSM2, and SoftHSM2.

Step 1/5. Set up your HSM​

You will need to set up your HSM and make sure that it is accessible from your Teleport Auth Server. You should create a unique HSM user or token for Teleport to use.

  1. Create a CloudHSM cluster in the VPC where you will run your Teleport Auth Server. https://docs.aws.amazon.com/cloudhsm/latest/userguide/create-cluster.html

  2. Wait for the newly created cluster to enter the "Uninitialized" state.

  3. Add an HSM to the new cluster, using the AWS Console or the AWS CLI:

    $ aws --region region cloudhsmv2 create-hsm --cluster-id cluster ID --availability-zone availability zone
    {
        "Hsm": {
            "AvailabilityZone": "ca-central-1a",
            "ClusterId": "cluster-6uysmebmutd",
            "SubnetId": "subnet-0c535b67a117f7186",
            "HsmId": "hsm-ppzzfxbleki",
            "State": "CREATE_IN_PROGRESS"
        }
    }

  4. Optionally verify the identity and authenticity of your new HSM https://docs.aws.amazon.com/cloudhsm/latest/userguide/verify-hsm-identity.html

  5. Wait for the state of the HSM to become ACTIVE:

    $ aws --region region cloudhsmv2 describe-clusters --filters clusterIds=cluster ID \
          --query 'Clusters[].Hsms[].State'

  6. To initialize the cluster, you must download and sign a certificate signing request (CSR) that is generated by the cluster's first HSM. Download the CSR from the AWS Console or via the AWS CLI:

    $ aws --region region cloudhsmv2 describe-clusters --filters clusterIds=cluster ID \
          --query 'Clusters[].Certificates.ClusterCsr' --output text \
          > ClusterCsr.csr

  7. Choose an appropriate RSA 2048 or RSA 4096 key and self-signed certificate to sign the HSM CSR. For a production cluster, AWS recommends a secured offsite and offline HSM. For a demonstration or test cluster, you can create a key and self-signed certificate with the following openssl commands:

    $ openssl genrsa -aes256 -out customerCA.key 2048
    $ openssl req -new -x509 -days 3652 -key customerCA.key -out customerCA.crt

  8. Sign the cluster CSR using the key and self-signed cert from the previous step. A demonstration openssl command to sign the CSR is:

    $ openssl x509 -req -days 3652 -in ClusterCsr.csr \
          -CA customerCA.crt \
          -CAkey customerCA.key \
          -CAcreateserial \
          -out CustomerHsmCertificate.crt

  9. Initialize the CloudHSM cluster by uploading the signed cert to the AWS Console or with the following AWS CLI command:

    $ aws --region region cloudhsmv2 initialize-cluster --cluster-id cluster ID \
          --signed-cert file://CustomerHsmCertificate.crt \
          --trust-anchor file://customerCA.crt
    {
        "State": "INITIALIZE_IN_PROGRESS",
        "StateMessage": "Cluster is initializing. State will change to INITIALIZED upon completion."
    }

  10. A security group with the same name as your CloudHSM cluster will have been automatically created when you created the cluster. Attach this security group to the EC2 instance where you will run your Teleport Auth Server to allow traffic between the Auth Server and your HSM.

  11. On the Auth Server EC2 instance, install the CloudHSM CLI for the CloudHSM Client SDK 5. https://docs.aws.amazon.com/cloudhsm/latest/userguide/gs_cloudhsm_cli-install.html

    Bootstrap the CLI by configuring the IP address of the new HSM:

    $ sudo /opt/cloudhsm/bin/configure-cli -a HSM IP address

  12. Copy the self-signed certificate from step 6 (customerCA.crt) to the EC2 instance, save it at /opt/cloudhsm/etc/customerCA.crt.

  13. Activate the CloudHSM cluster with the CloudHSM CLI by creating an admin user with a new password:

    $ sudo /opt/cloudhsm/bin/cloudhsm-cli interactive
    aws-cloudhsm > cluster activate
    Enter password:
    Confirm password:
    {
      "error_code": 0,
      "data": "Cluster activation successful"
    }

  14. Login to the CloudHSM CLI as the new admin user and create a Crypto User to be used by Teleport. Remember this new password because Teleport will use it later to authenticate to the PKCS#11 library.

    aws-cloudhsm > login --username admin --role admin
    Enter password:
    {
      "error_code": 0,
      "data": {
        "username": "admin",
        "role": "admin"
      }
    }
    aws-cloudhsm > user create --username teleport --role crypto-user
    Enter password:
    Confirm password:
    {
      "error_code": 0,
      "data": {
        "username": "teleport",
        "role": "crypto-user"
      }
    }
    aws-cloudhsm > quit

  15. Install the PKCS#11 library for the Client SDK 5 on the same Auth Server EC2 instance https://docs.aws.amazon.com/cloudhsm/latest/userguide/pkcs11-library-install.html

    Bootstrap the PKCS#11 library by configuring the HSM IP address. If you only have one HSM in the cluster, you must include the --disable-key-availability-check flag.

    $ sudo /opt/cloudhsm/bin/configure-pkcs11 --disable-key-availability-check -a HSM IP address

Step 2/5. Configure Teleport​

To configure Teleport to use an HSM for all CA private key generation, storage, and signing, include the ca_key_params section in /etc/teleport.yaml on the auth server.

# /etc/teleport.yaml
teleport:
  ...

auth_service:
  enabled: true
  ...

  ca_key_params:
    pkcs11:
      # this is the default install location of the PKCS#11 module for the
      # CloudHSM Client SDK 5
      module_path: /opt/cloudhsm/lib/libcloudhsm_pkcs11.so

      # token_label should always be "hsm1" for the CloudHSM SDK 5
      # if you prefer to use SDK version 3, replace this with "cavium"
      token_label: "hsm1"

      # pin should be set to the username and password of the Crypto User
      # created earlier in the guide
      pin: "<CU_username>:<CU_password>"
      # pin_path can optionally be used to read the pin from a file
      # pin_path: /path/to/pin_file

Step 3/5. (Re)start Teleport Auth​

If this is a new Teleport Auth Service which has not been started yet, starting a brand new cluster with an empty backend, HSM keys will be automatically generated at startup and no further action is required, skip to step 5. Otherwise, continue reading.

If you are connecting an HSM to an existing Teleport cluster, restart the auth server for the configuration changes to take effect. New CA keys will automatically be generated in the HSM during the next CA rotation. Until a CA rotation is completed, the Auth Service will continue signing new certificates with the existing software keys.

Step 4/5. Certificate Rotation with HSM​

When adding a new HSM to an existing Teleport cluster, or adding a new HSM-connected Auth Service to an HA Teleport cluster, you will need to rotate all Certificate Authorities in order for new certificates to be issued and trusted.

teleport start will print a warning during startup if any CA needs to be rotated. Any users allowed the update verb for cert_authority resources in any of their Teleport roles will also see a cluster alert reminding them to rotate the CAs.

CA rotation can be performed manually or semi-automatically, see our admin guide on Certificate rotation. All CAs listed in the output of tctl status must be rotated.

Step 5/5. Confirm that Teleport is using your HSM​

You are all set! You can confirm that HSM keys are being used a few different ways:

  1. tctl status will show PKCS#11 HSM as the storage method for all Certificate Authority keys.
  2. Teleport Auth service logs will contain Creating new HSM key pair.
  3. Your HSM's admin tool should show the newly created keys.