Skip to main content
Version: 18.x

Access Monitoring

Report an issue with this page

Access Monitoring allows users to understand and analyze the access patterns in a Teleport cluster.

When Access Monitoring is enabled in a cluster, Teleport runs a background service that collects data from different input sources, builds graphical reports highlighting usage patterns within the cluster, and allows users to create automated response actions.

The primary data source for Access Monitoring is the Teleport audit log. Out of the box, Teleport includes a report that searches the audit log for events indicating risky usage patterns (such as SSH or database sessions without MFA) and provides users with recommendations on suggested actions.

Users are able to write their own custom access monitoring queries by querying the audit log.

note

Access Monitoring is not currently supported with External Audit Storage in Teleport Enterprise (Cloud). This functionality will be enabled in a future Teleport release.

Prerequisites​

  • A running Teleport cluster. If you do not have one, read Getting Started.

  • The tctl and tsh clients.

    Installing tctl and tsh clients

    1. Determine the version of your Teleport cluster. The tctl and tsh clients must be at most one major version behind your Teleport cluster version. Send a GET request to the Proxy Service at /v1/webapi/find and use a JSON query tool to obtain your cluster version. Replace teleport.example.com:443 with the web address of your Teleport Proxy Service:

      TELEPORT_DOMAIN=teleport.example.com:443
      TELEPORT_VERSION="$(curl -s https://$TELEPORT_DOMAIN/v1/webapi/find | jq -r '.server_version')"

    2. Follow the instructions for your platform to install tctl and tsh clients:

      Download the signed macOS .pkg installer for Teleport, which includes the tctl and tsh clients:

      curl -O https://cdn.teleport.dev/teleport-${TELEPORT_VERSION?}.pkg

      In Finder double-click the pkg file to begin installation.

      danger

      Using Homebrew to install Teleport is not supported. The Teleport package in Homebrew is not maintained by Teleport and we can't guarantee its reliability or security.

Configuration​

Access Monitoring is enabled by default for all Teleport Enterprise (cloud-hosted) accounts.

Access Monitoring RBAC Permissions​

To access the Access Monitoring interface, a user must have a role that allows list, read and use verbs on the security_report and audit_query resources. The preset auditor role has these permissions by default. Alternatively, you can create a custom role with these permissions:

kind: role
metadata:
  name: my-role
spec:
  allow:
    rules:
    - resources:
      - security_report
      - audit_query
      verbs:
      - list
      - read
      - use
tip

You can also create and edit roles using the Web UI. Go to Access -> Roles and click Create New Role or pick an existing role to edit.

Query Editor​

The Query Editor in Access Monitoring provides users with an interface to interactively query audit logs and generate reports. Users can write custom SQL queries for these views to build custom reports akin to querying a relational database.

Within the Query Editor, users have access to a number of SQL views representing audit events captured by Teleport. Below is a list of the available SQL views:

Available SQL views
access_list_create
access_list_delete
access_list_member_create
access_list_member_delete
access_list_member_update
access_list_review
access_list_update
access_request_create
access_request_review
auth
bot_join
cert_create
db_session_query
db_session_query_failed
db_session_start
device_authenticate
device_enroll
exec
instance_join
join_token_create
kube_request
lock_created
lock_deleted
recovery_code_used
reset_password_token_create
saml_idp_auth
session_command
session_join
session_rejected
session_start
user_create
user_login
user_password_change
windows_desktop_session_end
windows_desktop_session_start

To access the Query Editor, navigate to the Access Monitoring section in the Teleport UI and click on the Query Editor tab.

SQL query examples​

  • Query for unique users who executed a ssh command involving /etc/passwd file in some way:
SELECT
  DISTINCT user
FROM
  exec
WHERE
  command LIKE '%/etc/passwd%';

  • Select the count of unique IP addresses associated with each user cert over different event dates:
SELECT
  event_date,
  identity_user AS user,
  COUNT(DISTINCT cert_create.identity_client_ip) AS unique_ip_count
FROM
  cert_create
GROUP BY
  event_date,
  identity_user
ORDER BY
  event_date;
  • Selects users who interacted with the Teleport cluster from more than one different IP address:
SELECT * FROM (SELECT
  event_date,
  identity_user AS user,
  COUNT(DISTINCT cert_create.identity_client_ip) AS unique_ip_count
FROM
  cert_create
GROUP BY
  event_date,
  identity_user
ORDER BY
  event_date) AS data WHERE unique_ip_count > 1;
  • Show all the unique IP addresses that were used by the user 'admin-annie':
SELECT
  DISTINCT identity_client_ip
FROM
  cert_create
WHERE identity_user = 'admin-annie'
  • Show Access Requests and their reviews:
SELECT
  *
FROM
  access_request_create, access_request_review
WHERE
  access_request_create.id = access_request_review.id
  • Show details about an Access Request and its reviews:
SELECT
  request.user, request.reason, request.roles, request.resource_ids, review.reviewer, review.state
FROM
  access_request_create as request, access_request_review as review
WHERE
  request.id = review.id

Access Monitoring Reports

Access Monitoring provides a number of pre-built reports.

Privileged Access Report​

The Privileged Access Report reports offers valuable insight into identifying weak security events across infrastructure. The report allows to identify the following weak security events:

Database sessions with weak security​

The following query identifies database sessions with weak security such as as sessions with missing Access Requests, MFA, impersonation, and trusted device identification.

SELECT
    event_date,
    count(*) as count,
    user
FROM
    db_session_start
WHERE
    CARDINALITY(access_requests) IS NULL
AND
    with_mfa IS NULL
AND
    impersonator IS NULL
AND
    trusted_device_device_id IS NULL
GROUP BY
    event_date,
    user
ORDER BY
    event_date

Suggestion: Set up Access Requests, Device Trust and per-session MFA.

SSH sessions with weak security​

The following query identifies SSH sessions with weak security, such as as sessions with missing Access Requests, MFA, impersonation, and trusted device identification.

SELECT
    event_date,
    count(*) as count,
    user
FROM
    session_start
WHERE
    CARDINALITY(access_requests) IS NULL
AND
    proto='ssh'
AND
    with_mfa IS NULL
AND
    impersonator IS NULL
AND
    trusted_device_device_id IS NULL
GROUP BY
    event_date,
    proto,
    user
ORDER BY
    event_date

Suggestion: Set up Access Requests, Device Trust and per-session MFA.

Kubernetes API calls with weak security​

The following query identifies Kubernetes sessions with weak security, such as sessions with missing Access Requests, MFA, impersonation, and trusted device identification.

SELECT
    event_date,
    count(*) as count,
    user
FROM
    kube_request
WHERE
    CARDINALITY(access_requests) IS NULL
AND
    with_mfa IS NULL
AND
    impersonator IS NULL
AND
    trusted_device_device_id IS NULL
GROUP BY
    event_date,
    user
ORDER BY
    event_date

Suggestion: Set up Access Requests, Device Trust and per-session MFA.

Privileged Postgres sessions​

The following query identifies privileged PostgreSQL sessions initiated by the 'postgres' user.

SELECT
	event_date,
	COUNT(*) AS count,
	user
FROM
    db_session_start
WHERE
    db_protocol='postgres' and db_user='postgres'
GROUP BY
    event_date,
    user
ORDER BY
    event_date

Suggestion: Downgrade database connections to less privileged database user.

Kube Execs​

The following query identifies Kubernetes exec usage.

SELECT
    event_date,
    count(*) as count,
    user
FROM
    exec
WHERE
    proto='kube'
GROUP BY
    event_date,
    proto,
    user
ORDER BY
    event_date

Suggestion: Eliminate usage of kube exec.

Long lived certificates​

The following query identifies long-lived certificates, where the certificate expiration time is greater than 1 day.

SELECT DISTINCT
    event_date,
    COUNT(*) AS count,
    identity_user AS user
FROM
    cert_create
WHERE
    from_iso8601_timestamp(identity_expires) - from_iso8601_timestamp(time) > INTERVAL '1' DAY
GROUP BY
    event_date,
    identity_user
ORDER BY
    event_date

Suggestion: Use short lived certificates less than a working day. Check out Machine ID to get short lived certificates for your automation.

Long-lived join tokens​

The following query identifies long-lived join tokens where the token expiration time is greater than 1 day.

SELECT
    event_date,
    COUNT(*) as count,
    node_name,
    host_id
FROM
    instance_join
WHERE
    FROM_ISO8601_TIMESTAMP(token_expires) - FROM_ISO8601_TIMESTAMP(time) > INTERVAL '1' DAY
GROUP BY
    event_date,
    node_name,
    host_id
ORDER BY
    event_date

Suggestion: Use short-lived tokens to reduce risk of compromise or delegated joining when possible.

Root SSH sessions​

The following query identifies SSH sessions initiated by the 'root' user.

SELECT
    event_date,
    COUNT(*) as count,
    user
FROM
    session_start
WHERE
    login='root'
GROUP BY
    event_date,
    user
ORDER BY
    event_date

Suggestion: Don’t use root for SSH sessions, downgrade access to users with sudo privileges instead.

System Kubernetes API calls​

The following query identifies Kubernetes API calls initiated by users in the 'system:master' group.

SELECT
	event_date,
	COUNT(*) as count,
	user
FROM
    kube_request
WHERE
    CONTAINS(kubernetes_groups, 'system:masters')
GROUP BY
    event_date,
    user
ORDER BY
    event_date

Suggestion: Don't use system:masters group for Kubernetes API calls, downgrade to view or edit instead.

Executing Access Monitoring Audit Queries from the CLI​

Run custom audit query using tctl audit query exec command:

tctl audit query exec 'select event_time,identity_user from cert_create order by event_date limit 1;'
[    {        "data": [            "event_time",            "identity_user"        ]    },    {        "data": [            "2024-03-20 12:12:03.374",            "alice"        ]    }]

The query result is returned as a JSON array of arrays. The first array contains the column names, and the subsequent arrays contain the query results.

By default, tctl audit query exec will search over the past 7 days. You can specify an alternate time frame with the --days=N flag.