Hardened Teleport Access

Hardened Teleport Access - Overview

This webinar is a deep dive into how companies can harden access to Teleport with two new features in Teleport 6.1. These include creating dual & multi authorization rules, requiring multiple team members to approve role escalation with Pull Requests for Infrastructure. This is an important FedRAMP control (AC-3) and increases the visibility and audibility for access. We’ll show how to enforce per-session MFA with the aid of hardware tokens; reducing the scope and risk related to certificate exfiltration.

Key Topics on Hardened Access - Dual Authorization for Roles & Per-Session MFA with Yubikeys

• Two new features were released in Teleport 6.1: creating dual & multi Authorization rules and enforcing per-session MFA with the aid of hardware tokens.
• Teleport is an access platform which allows engineers and security professionals to unify access across all environments behind the net.
• Teleport started off with server access, then launched Kubernetes access, application access, and database access.
• Teleport is already very secure by default, but there are extra steps you can take to get extra security and strength.
• Access requests let team members request access to new and elevated roles. This feature is available with Teleport Pro (Cloud) and Enterprise editions.
• Teleport supports U2F authentication on every SSH and Kubernetes "connection".
• It's recommended to give employees two hardware tokens: one they can keep on their keychain, and one they can keep in a secure location.

Expanding Your Knowledge on Hardened Access - Dual Authorization for Roles & Per-Session MFA with Yubikeys

• Teleport Application Access
• Get started in 10 minutes
• Documentation
• Teleport Quick Start
• Teleport Unified Access Platform
• Teleport Kubernetes Access

Learn More About Hardened Access - Dual Authorization for Roles & Per-Session MFA with Yubikeys

• Read the Teleport Admin Guide
• Download Teleport

Introduction - Hardened Access - Dual Authorization for Roles & Per-Session MFA with Yubikeys

(The transcript of the session)

Ben: Thanks for joining today. Today, we'll be covering how to harden Teleport, and we'll go through some older features that are available in Teleport around sort of networking, but we'll also look at a few different things that have been newly released in more recent releases in 6 and some upcoming releases, which can really help you strengthen your position at Teleport. It's going to be primarily high-level tips. All of these are sort of a deep rabbit hole. If you have specific questions about getting either FedRAMP or PCI, we're happy to bring in some of our solution engineers who've worked with many sort of large banks or people selling SaaS off into the federal government who can recommend sort of better architectures. And I have the Q&A box open here, so for free, I'll try and answer them as I go through this. So I have it open for free; just ping me. But if you have any questions afterwards or if you're watching this on the YouTube recording, just email me, ben@goteleport, and I'm also on Twitter, @benarent and same with @goteleport, where we share a bunch of other tips.

A Helpful Security Metaphor v

Ben: So before we dive too deep into hardening Teleport, let's start off with a small story about a miscellaneous field with a very large manhole cover. And under this manhole cover is one of the many nuclear missile sites in America. And one of the interesting things about nuclear missile silos is it has things like this, which is actually restricted based upon two locks, and these two locks require that two people in charge will be able to go in and get keys and then start the nuclear launch process. And the reason for this is — per this Air Force instruction that only two people — is to prevent accidental or malicious launch of a nuclear weapon by a single individual. When I was researching the “Two-Person Rule”, there's another rule called the “No Lone Zone”, which is similar to the two-person rule, but in this area, you need multiple people there. You can't be there on your own. So why did I start off — in the world of Teleport, we're about securing infrastructure, and so we have questions like: who logged into the server's EC2 user? Why did Bob export the MySQL database? Why did Alice have access to ‘system:masters’ to access Kubernetes? Or when did Jim access the admin panel? And this could both be potentially malicious or accidental access, and we were trying to address how we can sort of solve some of these questions to stop either accidental or malicious sort of launch of servers, Kubernetes clusters, or databases.

Teleport in Brief

Ben: For people who are new to Teleport, Teleport is an access platform which allows engineers and security professionals to unify access across all environments behind the net, and it's been around for six years. We started off with Server Access as sort of an upgraded version of OpenSSH, migrating customers from public-private key cryptography to short-lived certificates, and we have extended this concept of short-lived certificates to other areas. So for Kubernetes, instead of giving your team members a long-lived kubeconfig, you can give them a kubeconfig for the eight hours in which they work. And I'll dive in a bit later about how you sort of link this to your identity provider, so as you have team members come and leave, you can give them access or remove access without having to log in to host and have to do a whole bunch of cleanup. And last year, we had an Application Access primarily designed for internal applications and dashboards, and Database Access is another new addition. So like I said, this webinar's going to focus a lot on hardening Teleport. Teleport's already very secure by default, but these are extra steps that you can go to get extra security and strength.

The OSI Model

Ben: So to kick it off, we'll talk about the OSI model. The OSI model can be critiqued to saying it's possibly out-of-date with the movement to the cloud. Many of these things in the lower levels, such as the physical data network layer are managed by hosting providers, but we have customers who are still running on-premises and there's still some sort of concerns in which they need to get physical access to hosts and keep it secure. Cloud-native people will be more familiar with what has become sort of the industry standard around sort of Shared Responsibility. This is AWS's Shared Responsibility slide, and this means when you sign up to a cloud provider, AWS in this case is responsible for the security of hardware and global infrastructure. So whoever is the security guards at Virginia who's protecting the data center, who's getting access to the racks, people can't access your storage. This is all of Amazon's responsibility, but you can see the customer's responsibility is equally large. So it's still your responsibility to ensure that you encrypt all of your data, you have server-side encryption. You might want to use AWS's KMS or all these other tools that they have available to make sure that everything is secure. And even simple things like setting up the correct [inaudible] PCs and firewall configurations is still the responsibility of you as the customer. And one of the benefits of using Teleport is that out of the box, we can help you strengthen this position sort of as the customer and have to worry about one less thing.

Teleport Attack Tree

Ben: One thing I'm going to be touching on a lot in this webinar is the Teleport Attack Tree. I know this slide is very hard to see, but I have another version here. And the Attack Tree is something that we use when we're talking to prospects around what's all the possible ways in which someone could potentially get access and have a malicious session. So if I start with something sort of easy, so if someone hacks into a user's computer, they could steal the session ID, exploit the proxy, and this could start with a malicious session. And what we tell our customers is that all these different factors, you want to kind of reduce the scope of this potentially happening, so you could consider adding like a device management to make sure that antivirus is secure on a user's computer. We have things such as stealing or bruteforce a node join token, and you can improve this by using dynamic tokens instead of static tokens. I won't be covering tokens as much, but I'll mainly be focusing on some of the more user aspects of our attack tree.

Ben: So let's get back to the OSI model. So at the physical layer, starting at the bottom, Teleport doesn't require public internet access, and we do have some people who have deployed Teleport in the field without any internet connection, but they just connect via ethernet cable. This is very common for certain field IoT devices. And the benefit of deploying Teleport in this mode is that these field devices still have a full audit log of who accessed which system and what happened. And you can take this another step in a different direction and consider air-gap systems. It can all work within a completely confined environment, which is completely air-gapped off the public internet.

AWS Shared Responsibility

Ben: Next up, we have network isolation. Network isolation is important as we went back to the AWS sort of Shared Responsibility. Teleport's sort of core brain is consisted of a proxy server and an auth server. You can run this as one service. If you go through our Getting Started guide, we run both the proxy, auth, and node in one service, but as you expand, we recommend splitting up the proxy and the auth server. And I have a better diagram here of how you can also split up Teleport to be both highly available. In this case, these two orange boxes represent two availability zones, and there's two different proxies which have two different autoscaling groups. And you can see that we have elastic IPs on the Teleport proxy, but the auth server is in a private subnet which has more limited access only from the sort of public endpoint. And so the idea is — you're pretty free to put your public endpoint on the internet, and even if it gets compromised, it's much more difficult to sort of get access to the private auth subnet and its secrets and other things sort of live, in this case, in DynamoDB and the recorded sessions. We have customers also doing things such as sending all of the recorded data into a dedicated AWS account which the main account can access. Yeah, feel free to ask any questions. I don't see any coming in yet, but if I'm going too fast or too slow, please let me know. Happy to adjust my pace.

Trusted Clusters

Ben: Another feature you can leverage is trusted clusters. Trusted clusters is this pretty cool feature that lets you connect multiple LEAF clusters into one proxy. You can think of LEAFs in many different ways, but I like to use one of our customers who's deployed Teleport in a hospital. Each hospital has its own individual Teleport cluster and all routes back to a main route in their AWS account. And what this lets this customer do is they can monitor and update all of their medical devices in all of these different hospital networks, but they don't have to go through all of the network restrictions and work [inaudible] to get access to that hardware. And we have other customers using trusted clusters for both different cloud providers, make it very easy to switch and reverse those different networks and easily get access to your sort of inventory of hosts. Sort of next up, this is sort of much higher level in our OSI model, which is around identity. And this is sort of a cool way in which Teleport works. For open source edition, we support GitHub, but for Enterprise, per users, you can use SAML, OpenID, or Active Directory workplaces. And Teleport works by connecting — your users in your organization will sign up to Google Workspaces. So as I join sort of Acme Inc., I will then be added to Google Groups, and these groups will then link to Teleport roles, and based upon these Teleport roles, these will provide access. And up until this latest release, these are pretty much static. So I would be added to the SRE team and then I would have access to all of the SRE resources. Then if I was to leave Acme Inc., I would be removed. In the case of Teleport, I would be issuing the eight-hour certificate for that data. Once that day ended, I would no longer have access to any of the infrastructure.

Multi-Factor Authentication

Ben: But there's a few things in this sort of Attack Tree which we're going to talk about. One is an exploit in the identity provider and possibly hacking into a user's computer. When we work with many organizations, they could have 2,000, 3,000 employees, and working with IT and the identity provider, it can be difficult to really enforce best second-factor practices at the identity provider. And in the same degree, control over the user's computer can differ based upon that organization. And for anyone who is touching production systems, you want to strengthen that position as much as possible. So one thing that we have added in Teleport 6 is multifactor authentication. We've recently started publishing our RFDs and sort of background about how we design and work upon Teleport, so if you're interested, you can check out our RFD on our GitHub. But I'm going to just show a quick video that'll outline this feature.

Ben: Introducing Teleport's additional second-factor authentication. Teleport uses identity providers such as Google Workspaces, Active Directory, or your own SAML or OIDC provider, combined with Teleport roles to provide fine-grained access. As new team members join, they can be added to external groups. And these can be easily mapped to Teleport roles, creating rules such as engineers can only obtain 4-hour certificate for access [inaudible] 48 hours for the SRE team. Many IDPs let teams set up best practices for securing accounts such as requiring a second factor, but sometimes, that's not enough. Teleport 6.1 lets you add another layer of protection by requiring users to present a hardware token when starting a new SSA Kubernetes session. By using a hardware token, theft of credentials becomes much harder. These policies can be enforced by the ops team without having to get buy-In from IT. As I start a new session, it will require me to tap my YubiKey. Teams can set up this feature to use both hardware and software tokens. Teleport supports all major hardware tokens such as YubiKeys, SoloKeys, or Titan tokens. You can even limit to specific provider requiring FIP-certified keys and even keys only issued by one company. Teleport supports all software token apps and can be used with Google Authenticator, Authy, or many of the OTP applications. Thanks for watching. Per-session MFA is available in Introducing Teleport —

Ben: So I'm going to do a quick demo here. I have the same host. And this has been set up with the second factor. So I'm going to log in. I don't need my 1password. And for people who are unfamiliar with Teleport, what I'm doing here is I'm using tsh, which is our client tool for logging in to Teleport. So sort of the start of my day, sort of I come in and I log in. And then here, I'm going to tap my security key. So you can see I have one host here. Let's make this a bit bigger for you watching. So if I go try to access root on this host, you can see it's going to have me to “Tap any security key”. And I have my small token which [inaudible] actually plugged into my machine. I've tapped it, and I'm now on this Linux box. So if I exit, if I come back into Teleport itself, you can see that this last session I started has been recorded, and it says what happened during that session. Actually, we didn't log if it has been a session event, but in my case, it set up cluster-wide. So even if I log in on the UI here, it's going to ask me to tap it in the browser as well. And this is very helpful because we have some customers who have deployed Chromebooks, and so they can't install a command line client unless you take this second layer of protection also to the browser itself. So this is an example. You can go to this part of our Docs per-session MFA to learn more and you can require session MFA. Let's have a question from Alexei. Hackers can easily circumvent MFA by man in the middle. Do you alert that, and what defense mechanism do you use to counter it?" I think we do have failed order events which you can send in. Let me double check. I can also check with our developer as well. The hardware token is harder to circumvent than our ODP clients. Let me just try and use an unregistered one. Okay. So you can see this is one that's not allowed. Let me see if it captured the audit log. It doesn't look like we omit it, but I think there's an open ticket for admitting failed hardware devices to alert if that issue has happened.

Hardware Tokens

Ben: And then just a bit of background on hardware tokens. I would assume most people are sort of familiar, but as we’ve sort of gone through this process, we've learned a few things around the pros and cons. I've arranged them so I actually have a Titan token here, have a bunch of YubiKeys, and the Solo tokens. This feature supports all common hardware tokens. Price range can differ quite widely between the Solo and the YubiKey. The thing I like about the YubiKeys is you can get a FIP-certified key and you can also get Yubico to add your company CA, so not only can you specify, "I want a YubiKey — I want a YubiKey to be my second factor, but a YubiKey issued by sort of Acme Inc." But I have heard of possible supply chain issues for larger rollouts, so if you want to roll it to an organization with more than 500 people, definitely make this part of your planning. The Titan token's the same. They have also quite good industrial design. It's engineered by Google, so have some great engineers. We also use their BoringCrypto library for Teleport. If you are using the Bluetooth token, the app support in Bluetooth is kind of lacking. And I think last up is SoloKey, which is open-source and open-audited token. Version 1 lacks a lot in its industrial design, but Version 2 is coming and there's lots of great improvement, so we really like this SoloKey project. Another note: if you're considering [inaudible] out, I'd recommend giving employees two hardware tokens: one they can keep on their keychain, and one they can keep in a secure location. Depending upon what your sort of OPSEC is in your organization, it definitely is easier to have sort of a hidden hardware token as a backup for such environments than having an admin go in and sort of restore and change access.

Ben: So there's some limitations. It's currently only supported on Windows. tsh ssh, as I showed, only works with tsh ssh. It doesn't work with OpenSSH supports. You can configure hardware tokens with OpenSSH natively, but the way in which we've built it, you need to migrate to tsh ssh. And then kubectl is only per-session; it's not per-action. And last up, database in applications doesn't currently support it. Will likely look into this for 7.0. If you have a very specific comment, you can ping us and we can look into it.

Access Requests

Ben: Okay. So going back to attack tree. This next feature, I guess, covers the hack into a user's computer, and this is around access requests. There's a few RFDs that cover access requests. There's dual authorizations, script approvals, and workflow UI. You can check out the RFD to sort of learn more about the problem and how we try to solve it, but I'm going to give you a quick demo video to explain this feature.

Ben: Introducing access requests: a new way to request access to infrastructure. Access requests let team members request access to new and elevated roles. This feature lets your team quickly and easily request access from the terminal or browser. Administrators can provide users with a list of predefined roles with wildcard requests. When a team member wants to get access, they have the option to add a reason to the request. Here, Alice is trying to request the dba role to run a debug tool. Next up, Ben needs access to Kubernetes. He can request the super-maintenance role that'll let him assume the role as system:masters. Because the super maintenance roles allows unfettered access to Kubernetes, it requires at least two teammates to approve the request. The rest of the SRE team are alerted in Slack and the access is quickly reviewed by the reviewers. After it's approved, Ben logs in again, pulling down the new kubeconfig that'll provide him six hours of access to get the job done. All of the access requests and access are recorded by Teleport, providing a complete audit log of what happened during the session. This video scratched the surface of what's possible with access requests. Access requests is available with Teleport Pro (Cloud) and Enterprise editions. Thanks for watching. Give it a try today.

Ben: Okay. So let me dive into a little demo. Actually, before going to a demo, let me explain sort of how it works a bit more. So in this demo here — a lot happened in that video, but let's say I join the DevOps team. And as part of DevOps role, I have access to be able to request two roles, so I can access the dbadmin and the super-maintenance role. And as part of this request roles, there's also a threshold of the amount of people who you need to approve that role. And so in my case, I have Samir and Gibbons, who are my reviewers and they're in the Ops team, and they can also review these requests and approve them. So if I just go ahead and come in here. There's two ways in which you can request it. You can request that using the UI. So I'm going to request the dba, run migration [command]. You can also add reviewers. And I send this request; it's pending. And we know that you don't do all your work in Teleport all day, but we have multiple integrations. And here we have my run migration request, and this links back to Teleport. I won't go through this full flow, though, because I'm currently logged in as my other user doesn't work. But this is a way in which the team can be alerted. You can come in. Once approved, I'll be able to then assume the role and go about my day as the dbadmin. And what's a very cool feature about this is you can do things like — you can change the labels you can get access to. You change the principles and do things like you could set the super-maintenance role to have a very short ttl. So if someone does want to get access to the super maintenance role, like system officers, you may want to say that they can use this word for one hour. If they need more time, they have to go through the request again.

Adding More Approvers

Ben: Another feature in here which is very powerful is the ability to add more approvers. So in this case, the previous iteration, only one person could approve it, but you could have three or four different people, which goes back to our two-man rule, and then limit the amount of people who can deny access to this role. So it's very flexible and powerful. There's a lot more in this feature set based upon sort of some of our early customer feedback, so I highly recommend checking the RFD and our Docs to learn more. And next, dual auth [inaudible], and you can learn more about it.

HSM Support

Ben: So next up — let me come back to the Attack Tree — is giving direct access to unencrypted backends or the backend encrypted keys. This has been a possible attack effect if someone was to get access. Once you get access to the encryption keys, you can get access to private key material and start a malicious session. To solve this, we are going to be adding HSM support. We're adding support for both physical hardware HSMs such as the YubiHSM, which is a relatively affordable HSM, but also CloudHSMs. Think we're starting with AWS CloudHSMs since it's a pretty well-supported one, but we should support most HSMs as long as they have PKCS #11 support, which is the majority of them. SoftHSM is another one. And yes, so if anyone was to get access to — makes getting access to that sort of backend private store CA much more difficult.

Ben: And then last up, another sort of honorable mention as far as hardening Teleport. For a long time, we've had a FIPS binary which has helped customers get FedRAMP support. This is a great place to start if you want to sort of start with a very secure, locked-down version. It helps enforce many best practices such as making sure that encryption arrest for DynamoDB is always turned on. It won't start. Same with changing TLS version, and then it removes non-compliant algorithms and gets everything sort of up to the NIST recommendations. So compiled with BoringCrypto, which is one of the NIST requirements you have to go through an authorized crypto library, and we also enforce 248 RSA private keys. So that brings me to the end of today's webinar. I have lots of different links in there. Sometimes I have one call to action, but I recommend really checking out the RFDs. And like I said, if you’re going through the setup and you're sort of going through your architecture, feel free to reach out to us. We have a Slack room which we're happy to chat to, or you can have a more private conversation with our sales engineers and myself, and we'll hopefully find the right person in our organization to sort of answer your questions and get you set up. I think, on another note, majority of these features, the hardware token support's all in our community edition, so you can get started off the bat without having to talk to anyone. You can try it out. And all these features are available today, apart from HSM support. So that brings me to any questions.

Anadelia: Excellent. Thank you, Ben. One of the questions we had — I think you may have already answered — was around which of these features are available in the community edition.

Ben: Yes. So I believe HSM, the YubiKey sort of hardware token support, but the access workflow's only in our Pro Edition.

Anadelia: Great. Thank you. Next question we have here is: "How do I obtain FIPS?"

Ben: Sorry, what was the question?

Anadelia: Yes. Next question is: "How do I obtain FIPS?"

Ben: Oh. The FIPS Library is available through our sort of control panel. You'll need to reach out to our sales team and they can help set it up for you. But our Docs — I think our Docs here. This also will be very helpful if you're thinking about setting up Teleport. And then also looking at these NIST controls, this links to the specific control and what are the limits that — these may be things that you've not necessarily considered, such as how do you define concurrent session control, and then you can go to the NIST certified guideline around what are they recommending. Some of these, it's kind of a bit open to interpretation, but we try and follow them as much as possible, and we work with our customers who've gone through FedRAMP certification. And yeah, if there's something that you go through an auditor and you have any kind of questions, we're happy to help you out.

Anadelia: Excellent. Thank you. And that is it for the questions today. Thank you for an excellent presentation, and thank you, everyone, for joining us today.

Ben: Oh, thanks, everyone.