Secure Bots and Service Account Access With Ephemeral Certificates

"Create cryptographically secure access keys" and "Rotate keys frequently" — These are probably the best security recommendations one can get for bot authentication security. In reality, while these are good practices, they simply aren't enough. First, cryptographically secure keys only prevent simple guessable or brute-force attacks, but the impact is catastrophic if the keys themselves are stolen. Second, the meaning of "frequently" itself is abused and ignored by practitioners because there's a lot of operational complexity involved in the key rotation. Key rotation is easier on paper than in practice!

This blog post will explore why static keys and credentials are inadequate for bot authentication and how scoped, ephemeral, and auto-renewing certificates are the best form of authentication for bot access.

Bot? You mean like a chatbot?? Bot is a broad topic. But in this post, we refer to a bot as a computer program that can automate and scale manual tasks. These bots usually perform machine-to-machine access. Examples include service accounts used in CI/CD pipelines, CLI tools, server configuration tools, etc.

What’s wrong with popular authentication methods for bot access?

Let's look at this recent blog post published by NCC on 10 real-world stories of how they compromised CI/CD pipelines. According to the post, 3 out of 10 attacks involved direct compromise of secrets and credentials used in the CI/CD account:

• Attack 1: "It Always Starts with an S3 Bucket…"
• Attack 2: Privilege Escalation in Hardened Environment
• Attack 7: Runners Exposed Secrets to Untrusted CI/CD Jobs

We can also take a reference from the recent Heroku and Travis-CI credential compromise incident. The first step in that attack began with stolen OAuth user tokens issued to Heroku and Travis-CI. These are just a few recent examples of the impact a stolen credential can have on overall security posture.

Below are the three major security risks to static credentials for bot authentication:

Infinite lifetime by default

Keys, tokens, passwords, credentials; all forms of static credentials have an infinite validity period by default, which also gives attackers an infinite period of time to misuse and exploit these creds.

Insecure key storage

In most cases, keys are stored as an environment variable or in a text file. Environment variables and files are readable by any process running under the same UID or root processes. These risks can be partially contained by using secret managers like AWS secret manager, HashiCorp Vault, etc. The problem with these services, however, is that each of them still requires a "master key" managed by the bots that need to access the creds behind the vault. This means that if the "master key" is compromised, every credential in the vault is compromised as well. Although this is not specific to bots and applies to secret management in general, the point to note here is that the method involved in key storage is usually insecure and prone to compromise.

Only single-factor authentication

In 2019, Microsoft reported that MFA could block over 99.9 percent of account compromise attacks. Two-factor authentication (2FA) and multi-factor authentication (MFA) definitely save us from the security risks associated with password-based authentication. However, 2FA and MFA require us to perform specific actions such as entering TOTP codes, approving a push notification in mobile phones, tapping a YubiKey, etc., or providing biometrics such as FaceID and TouchID. Unfortunately, a bot cannot perform these out-of-band actions during the authentication process. Hence, dual-factor authentication is not applicable to bot authentication.

To summarize, static credentials like keys and tokens are bad for bot authentication, so what does the perfect form of authentication look like? Where we humans struggle with password rotation that forces us to use a new password every time we log in, this process is a cakewalk for the bots!

Teleport's answer to enhance bot authentication security

Using the combination of ephemeral certificates with automated renewals, we can elevate the security of bot authentication while also addressing the operational complexity that comes with a manual key rotation. With ephemeral certificates and an automated renewal process, admins need to configure the certificate issuance process only once, and the authentication method's security is taken care of by the process itself!

Let's see the properties of an ephemeral certificate and automated renewal process below:

Ephemeral certificates

Ephemeral certificates have this unique property that they automatically lose validity after a configurable short TTL according to access requirements. The primary security advantage of this method is it creates a short window where it is possible for credentials to be compromised and misused. Remember, the goal is not to make the credential 100% theft-proof but rather to limit the chances of theft and limit the risks of misuse if they are stolen.

Automated renewals

Administrators tend to favor long-standing credentials like passwords and API tokens because it relieves the manual operational hassle of credential rotation and renewals. Teleport's automated certificate renewal feature silently renews certificates in the background with a secure certificate renewal process, which allows the use of ephemeral certificates without the risk of losing access due to expired certificates.

How does it work?

The ephemeral certificate issuance and automated renewal process are implemented as a feature set named Teleport Machine ID. Machine ID takes the certificate-based authentication for bots and machines even further by implementing an on-air certificate locking feature that can immediately block the certificate and bot access if they are compromised.

The core functionality of Teleport Machine ID is a certificate renewal bot that communicates with the Teleport cluster and securely renews access certificates in an automated fashion. The clients (service accounts, configuration management tools, SSH client, database client, Kubernetes client, API client, etc.) use these access certificates to access servers (API servers, SSH servers, Kubernetes clusters, databases, etc.).

By assigning a short-lived certificate and supporting an automated certificate renewal process, Machine ID helps to enforce just-in-time access and just-enough-privilege principles to machine-to-machine access. For example, suppose a service account needs to access the CI/CD server between 9 AM and 6 PM. In that case, administrators can create a short-lived certificate that renews every 6 hours and then automates role assignments during working hours and revokes them during off-hours.

Furthermore, this feature is integrated with Teleport's existing modern privileged access management features and regular RBAC, SSO and audits.

Conclusion

Pre-shared static credentials (API key, OAuth tokens, etc.) are the primary way we authenticate bots in machine-to-machine access. Although credentials for bot authentication usually have a higher entropy due to larger length and randomness, the security risks are still comparable with passwords. The fact that bot access is not monitored for privileged access and does not go through second-factor authentication means that the risks associated with the stolen access keys are much more significant.

At Teleport, our solution to this problem is to issue an ephemeral certificate for authentication, which can be automatically renewed by a certificate renewal bot that ensures security without risking the genuine bot access getting locked out.

Try Machine ID today!

Teleport Machine ID is an open-source implementation. Find more details on getting started with Machine ID guide. If you are new to Teleport, learn how Teleport works.