Bring your own Inference to Teleport Beams

Report an Issue

Is this page helpful?

How can we improve this page?

View as Markdown

This guide explains the steps to provision an Anthropic model through Amazon Bedrock to be used by Claude Code running within a Beam. This guide covers setting up Beams as an OIDC provider in AWS, granting the required permissions and creating an LLM proxy. The steps are the same for any compatible model.

To get you started, Beams comes with Claude Code and Codex out of the box (subject to rate limits and fair usage). This is useful for evaluation and experimentation, but may not provide the scalability, privacy controls, or flexibility required for enterprise workloads. Once past the evaluation period, customers should bring their own inference provider to continue seeing value from beams for AI workloads.

Beams offers a convenient connection to Amazon Bedrock via an AWS OIDC integration. To view the latest supported models, visit AWS: Models at a glance.

How it works

Beam instances access inference models securely through an LLM proxy hosted by your Teleport cluster. The proxy authorizes and forwards requests to the inference provider (Amazon Bedrock) and passes responses back to the beam. Tools and SDKs running in a beam that support the Anthropic Messages API or OpenAI Responses API reach these endpoints transparently.

Switching inference providers is simply a process of creating a new LLM proxy for the provider and configuring beams to use it. Beams have two provider slots, one supporting providers using the Anthropic Messages API and the other supporting providers using the OpenAI Responses API.

Existing beams will continue to use the providers configured when they were created, while new beams will use the newly set LLM proxy.

Prerequisites

• A Teleport Beams cluster. Start your free trial.
• The tctl and tsh clients
• An active Teleport session (tsh login)
• aws CLI installed
• An active aws session
• AWS permissions to create an OIDC provider and role, edit its trust policy, and attach an inline policy
• A chosen model, ensuring the following:
  • The model is available in Amazon Bedrock via the bedrock-mantle endpoint
  • For Anthropic-compatible models, the Messages API is supported
  • For OpenAI-compatible models, the Responses API is supported
  • The model is available in a region that suits your use case (default is us-east-1)

CLI Tools

Beams requires tsh version 18.11.0 or newer. To install tsh, visit Download Client Tools, select your platform, and choose "CLI Client Tools" in the Download Type menu.

Step 1/4. Configure an AWS OIDC integration

This guide assumes you will not be using an existing AWS OIDC integration. To list existing integrations use tctl get integration and look out for aws-oidc as the integration type. Skip this step if you wish to use an existing integration.

To configure a new integration, start by creating an identity provider in AWS. Use the aws iam create-open-id-connect-provider CLI command, or create the identity provider through the AWS Console. Use the following configuration:

• Provider type: OpenID Connect
• Provider URL: https://teleport.example.com
• Audience: discover.teleport
• Tags:
  • teleport.dev/cluster: teleport.example.com
  • teleport.dev/origin: integration_awsoidc
  • teleport.dev/integration: my-integration

For example:

aws iam create-open-id-connect-provider \  --url https://teleport.example.com \  --client-id-list discover.teleport \  --tags \    Key=teleport.dev/cluster,Value=teleport.example.com \    Key=teleport.dev/origin,Value=integration_awsoidc \    Key=teleport.dev/integration,Value=my-integration

Next, create an IAM role using aws iam create-role or the AWS Console to establish a trust relationship between AWS and the Beams cluster.

Note the role's ARN as this will be used later.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::aws-account-id:oidc-provider/teleport.example.com"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "teleport.example.com:aud": "discover.teleport"
        }
      }
    }
  ]
}

Complete the integration by creating an integration resource. Create a file called awsoidc-integration.yaml with the following content:

kind: integration
sub_kind: aws-oidc
version: v1
metadata:
  name: my-integration
spec:
  aws_oidc:
    role_arn: "arn:aws:iam::aws-account-id:role/beams-oidc"

Create the resource:

tctl create awsoidc-integration.yaml

Validate the new integration is operational using the tctl integration test utility:

tctl integrations test my-integration

See the dedicated AWS OIDC guide for more details.

Step 2/4. Grant Amazon Bedrock permissions

The integration needs permissions to invoke Bedrock models. These are granted by attaching a policy to the identity provider role created earlier. The policy can be appended using the aws iam put-role-policy command or using the AWS Console.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["bedrock-mantle:CreateInference"],
      "Resource": "arn:aws:bedrock-mantle:*:*:project/*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "aws-marketplace:Subscribe",
        "aws-marketplace:ViewSubscriptions"
      ],
      "Resource": "*",
      "Condition": {
        "StringEquals": {
          "aws:CalledViaLast": "bedrock-mantle.amazonaws.com"
        }
      }
    }
  ]
}

The policy above gives access to all Bedrock Mantle projects. Projects provide a useful way to track token usage and costs. Currently, the Amazon Bedrock integration uses the default project only.

The aws-marketplace:Subscribe and aws-marketplace:ViewSubscriptions actions are required for Mantle to automatically create subscriptions when models are first used.

For example:

aws iam put-role-policy \  --role-name beams-oidc \  --policy-name beams-bedrock \  --policy-document '{ policy json here }'

Step 3/4. Configure an LLM proxy

An LLM proxy exposes Bedrock models to Beam instances. It can also be used by authenticated cluster users via a local proxy.

Create a file called llm-proxy.yaml with the following content:

kind: app_server
version: v3
metadata:
  name: beams-anthropic
spec:
  host_id: <generate-uuid-here>
  app:
    kind: app
    sub_kind: llm
    version: v3
    metadata:
      name: beams-anthropic
      labels:
        teleport.internal/beams/app-type: llm
    spec:
      public_addr: beams-anthropic.teleport.example.com
      integration: my-integration
      aws:
        region: us-east-1
      inference:
        format: anthropic
        provider: bedrock
        models:
          - name: claude-opus-4-8
            provider_name: anthropic.claude-opus-4-8
        quota: # Optional
          intervals:
            - interval: lifetime
              input_tokens: 5_000_000
              output_tokens: 15_000_000

Make sure to generate a fresh UUID for spec.host_id.

The teleport.internal/beams/app-type: llm label allows users with the built-in beams-user role to access the LLM proxy without changes to their roles.

This config whitelists the claude-opus-4-8 model. The LLM proxy will allow only requests for this model. Remove the spec.inference.models list entirely to expose all available models. The model name is the name used in an inference request, while the model provider_name is the Amazon Bedrock ID for the model. Discover model IDs in the Amazon Bedrock model cards.

The config includes an optional lifetime usage quota. Remove this section to allow unrestricted token usage. Note that without quotas set, usage is not tracked and cannot be reported. Read the resource reference for all available options.

Token usage per period can be a challenge to forecast. One approach is to grant generous quotas initially and refine the limits based on actual usage. You can use tctl llm usage to view consumption at any time. Note that reports are not realtime and can take some time to propagate.

Create the resource:

tctl create llm-proxy.yaml

Validate the new LLM proxy is operational by sending a test request via a local app proxy:

Check for an app named "beams-anthropic"
tsh apps ls

Start a local proxy and leave it running
tsh proxy app --port 56789 beams-anthropic

In a second terminal, set the API URL to the local proxy address (http)
export ANTHROPIC_API_URL=http://localhost:56789

Send a test request
curl --fail-with-body --silent --show-error \  -X POST "$ANTHROPIC_API_URL/v1/messages" \  -H "Content-Type: application/json" -H "anthropic-version: 2023-06-01" \  -d '{"model":"claude-opus-4-8","max_tokens":256,"messages":[{"role":"user","content":"Hello!"}]}' \  | jq -r '.content[0].text'

A successful response from the model means the LLM proxy is operational and your user has permission to access it.

To expose models supporting the OpenAI Responses API, repeat this step to create another LLM proxy, changing the inference.format to openai.

Step 4/4. Update the global beams config

Update the beams global config to route Anthropic traffic through the new LLM proxy for all future beams.

Fetch the existing beams config resource to a local file. If the resource does not yet exist, it'll be created in the coming steps.

tctl get beams_config > beams-config.yaml

Update (or create) beams-config.yaml with the following contents:

kind: beams_config
version: v1
metadata:
    name: beams-config
spec:
    llm:
        anthropic:
-            app_name: anthropic
+            app_name: beams-anthropic
        openai:
            app_name: openai

Update (or create) the beams config resource:

tctl create -f beams-config.yaml

Alternatively, for an interactive editing experience:

tctl edit beams_config

Validate the beams config changes have had an effect by inspecting the config available to a new beam and sending a test request:

Create a new beam
tsh beams add --no-console

The ANTHROPIC_BASE_URL environment variable should contain the new LLM proxy URL
tsh beams exec beam-id 'echo $ANTHROPIC_BASE_URL'

Test Claude Code
tsh beams exec beam-id 'echo "tell me a joke about an AI" | claude -p --model claude-opus-4-8'

View quota usage (if configured)
tctl llm usage

You've provisioned an Anthropic model for use within a Beam and configured new beams to use the new provider. You've upgraded the beams experience by giving you control over model selection, region routing, and usage.