Why The Four Eyes principle is critical for access

Introduction

The four-eyes principle means an activity must be approved by two people, or from Argus Panoptes if the ancient Greeks needed access controls. This principle is commonly used in both routine and non-routine scenarios.

On the routine side are “Business Execution” processes. Here the Four Eyes principle is used to stop negative outcomes as the result of poor execution of a regular business task. For example, if accounting wants to wire $50k to a new account, three people should review the transaction (the originator plus two approvers). This helps prevent internal fraud and mistakes due to something like a business email compromise attack. One individual can’t easily execute a fraudulent transaction without approval.

On the non-routine side, there are procedures for remote nuclear missile launch sites. If a launch is ordered, a lengthy verification process starts, culminating with the act of turning the two launch keys positioned so far away from each other that it requires two people to start the launch. But that’s not all. Another crew in another launch control center must do the same thing, requiring four keys, and four people to initiate the launch. This prevents a rogue individual, or rogue silo from launching a missile.

The missile silo is an extreme example, so let’s focus on a relevant example for the DevOps crowd. Let’s say you hire a contractor to develop a microservice. During the engagement, they needed access to a production database, but the Ops team forgot to revoke their access after the project was complete. A few months later, the company finds that the customer database has been put up for sale on the dark web. Did the contractor go rogue? No, the contractor had their laptop stolen and their credentials were used to dump the db. How could the Four Eyes principle prevent this?

I can already hear the sound of a thousand cybersecurity vendors pitching solutions to fix this problem: endpoint protection, zero trust, data exfil prevention, etc…and as you're reading this — written by a vendor of an access solution — we’ve got our take too. Let’s take a moment to break down the problem and address the problem with snake oil pitches.

Let’s Pitch Teleport to solve the contractor problem: Teleport is the easiest, most secure way to access infrastructure. It provides a centralized way to both access and audit access. Teleport even has a full recording for each individual session.

You can think of this as a ‘DVR’, or Twitch recording for all activities. But it has a few disadvantages, since there are a variety of ways a user can bypass the Teleport session recording using obfuscation, disabling terminal commands or simply uploading a shell script to do the dirty work for them.

Teleport 0 - Snake Oil 1

But wait… We solved this problem by launching Enhanced Session Recording.

This feature leverages the power of BPF to capture low level network, disk and execs so obfuscation of the command line itself doesn’t hide anything. By transferring user input into structured events, it makes it easier to set up real-time machine processing to monitor and alert on.

Teleport 1 - Snake Oil 1

*side note: A common request is to have the ability to turn off SCP to prevent data exfiltration. For example, say you want a list of all users on a remote host but the SCP binary has been removed from disk. You can simply run the following command to download a copy.

$ ssh example.com cat /etc/passwd > passwd.copy

This is one of the many issues with SSH Restricted Shells.

We can now assume that Teleport is capable of capturing all the audit information needed to capture the malicious activity outlined in our scenario. But there are more inherent problems with using technology to solve a human problem. Unless you roll out restrictive AppArmour policies, you might be out of luck to let engineers do their job while providing the oversight and restrictive control you need.

eBPF transferred our PTY recordings into something that’s easily consumed by machines, piped on-mass to SIEM solutions and easily alerted on. Fine-tuning your SIEM can be tough, and ultimately, humans will need to tweak alerts and alarms based on activity. This is why humans are best at detecting suspicious behavior and it's good to have another pairs of eyes on the sessions running on critical systems. By having one or multiple people observe an action. Square created a sudo_pair to help solve this problem to ...

sudo is used by engineers daily to run commands as privileged users. But on some sensitive systems, you really want to ensure that no individual can act entirely autonomously. At Square, this includes applications that manage our internal access-control systems, store accounting ledgers, or even move around real money. This plugin allows us to ensure that no user can act entirely on their own authority within these systems.

sudo_pair is a good example of a tool that allows multiple users to moderate and observe actions during a SSH session. sudo_pair has a lot of options but by its nature, only limited to ssh sessions. Plus, in 2022, isn’t it an anti-pattern to SSH into boxes? There are many exceptions to the rule that operators shouldn’t access machines. I worked in an organization that would mark a machine for termination within an hour of an SSH session, viewing that node as tainted by the human touch. This rule is unlikely to work for everyone, as even with good DevOps practices, there are break-glass scenarios where human access is required. Plus, these break-glass scenarios can get more complicated. For example, debugging a Kubernetes system can be much quicker using kubectl exec, than requiring a million +1 logging and monitoring systems.

Ultimately, you’ll need to give people access to a system, and who accesses it is important. When someone is touching a sensitive system, you’ll want someone to watch, and unless you’re working in a physical NOC or SSH Room with a drop cam, it’s unlikely that the person will be beside you.

To solve this problem we developed multi-party sessions for Kubernetes and SSH sessions. This feature lets teams require one or multiple people to join a session to initiate it. Let’s take a look at the following scenario.

Session creation can happen with the existing flow using kubectl exec, but the wrapper command can provide extra context when accessing a system.

tsh kube exec [email protected],[email protected] --reason="Need to fix this pod" -- database_pod -- /bin/bash 

This provides greater context to people who have requested access and can alert a team using Teleport plugins. Teleport administrators can specify session rules within the Teleport role template. Below we’ve created a role that’ll require two senior developers and one FedRAMP auditor to let the developer start the session.

kind: role
metadata:
  name: prod-access
spec:
  allow:
  require_session_join:
    - count: 2
      filter: contains(user.roles, "secops") || contains(user.roles, "audit")

This example shows how moderated sessions can provide four-eyes (and more) moderation for both Kubernetes and SSH sessions.

Are humans the cure for cybersecurity snake oil?

If you’ve made it this far, you might be wondering how this applies to cybersecurity snake oil pitches. The problem with cybersecurity tooling is that they often market and sell technology to problems that can never be fully solved by technology alone. A sophisticated internal user, or group of teenagers, will always find a way around snake oil. This is why easy-to-use tooling to provide secure access is the only best practice that works. This might start with an Access Request and for sensitive systems require moderated sessions.

Teleport 2 - Snake Oil 1

Thanks for reading my slighting tongue in cheek review of Teleport Access Controls. Teleport Moderated Sessions is a new enterprise feature available in Teleport 9. If you would like a demo, please get in touch with us.