Fork me on GitHub
Teleport

Backup and Restore

Improve

When planning a backup of Teleport, it's important to know what is where and the importance of each component. Teleport's Proxies and Nodes are stateless, and thus only teleport.yaml should be backed up.

The Auth server is Teleport's brains, and depending on the backend should be backed up regularly.

For example, a customer running Teleport on AWS with DynamoDB have these key items of data:

WhatWhere ( Example AWS Customer )
Local Users ( not SSO )DynamoDB
Certificate AuthoritiesDynamoDB
Trusted ClustersDynamoDB
Connectors: SSODynamoDB / File System
RBACDynamoDB / File System
teleport.yamlFile System
teleport.serviceFile System
license.pemFile System
TLS key/certificate( File System / Outside Scope )
Audit logDynamoDB
Session recordingsS3

For this customer, we would recommend using AWS best practices for backing up DynamoDB. If DynamoDB is used for the audit log, logged events have a TTL of 1 year.

BackendRecommended backup strategy
dir ( local filesystem )Backup /var/lib/teleport/storage directory and the output of tctl get all --with-secrets.
DynamoDBFollow AWS Guidelines for Backup & Restore
etcdFollow etcD Guidleines for Disaster Recovery
FirestoreFollow GCP Guidlines for Automated Backups

Teleport resources

Teleport uses YAML resources for roles, trusted clusters, local users, and auth connectors. These could be created via tctl or the UI.

GitOps

If you're running Teleport at scale, your teams need to have an automated way to restore Teleport. At a high level, this is our recommended approach:

  • Persist and backup your backend
  • Share that backend among auth servers
  • Store your configs as discrete files in VCS
  • Have your CI run tctl create -f *.yaml from that git directory

Migrating backends

As of version v4.1, you can now quickly export a collection of resources from Teleport. This feature was designed to help customers migrate from local storage to etcd.

Using tctl get all --with-secrets will retrieve the below items:

  • Users
  • Certificate Authorities
  • Trusted Clusters
  • Connectors:
    • Github
    • SAML [Teleport Enterprise]
    • OIDC [Teleport Enterprise]
    • Roles [Teleport Enterprise]

When migrating backends, you should back up your auth server's data_dir/storage directly.

Example of backing up and restoring a cluster.

Export dynamic configuration state from old cluster

tctl get all --with-secrets > state.yaml

Prepare a new uninitialized backend (make sure to port

any non-default config values from the old config file)

mkdir fresh && cat > fresh.yaml << EOFteleport: data_dir: freshEOF

bootstrap fresh server (kill the old one first!)

sudo teleport start --config fresh.yaml --bootstrap state.yaml

from another terminal, verify state transferred correctly

tctl --config fresh.yaml get all

<your state here>

The --bootstrap flag has no effect, except during backend initialization (performed by auth server on first start), so it is safe for use in supervised/High Availability contexts.

Limitations

  • The --bootstrap flag doesn't re-trigger trusted cluster handshakes, so trusted cluster resources need to be recreated manually.
  • All the same limitations around modifying the config file of an existing cluster also apply to a new cluster being bootstrapped from the state of an old cluster. Of particular note:
    • Changing cluster name will break your CAs (this will be caught and teleport will refuse to start).
    • Some user authentication mechanisms (e.g. u2f) require that the public endpoint of the web ui remains the same (this can't be caught by teleport, be careful!).
  • Any node whose invite token is defined statically (in the config file of the auth server) will be able to join automatically, but nodes that were added dynamically will need to be re-invited
Have a suggestion or can’t find something?
IMPROVE THE DOCS