Computer use tool

Claude can interact with computer environments through the computer use tool, which provides screenshot capabilities and mouse/keyboard control for autonomous desktop interaction.

The computer use tool is an Anthropic-defined client toolset: one {"type": "computer_toolset_20260801"} entry in tools gives Claude 17 member tools such as screenshot, left_click, type, and zoom, and your application runs every call in an environment you control. It isn't currently available in Claude Managed Agents. Claude's calls are tool_use blocks whose name is the member and which carry "toolset_name": "computer", often several per turn (a batch action).

For tasks that stay inside webpages, the browser use tool is the closer fit: its member tools read and act on the page itself, and it doesn't need a full desktop environment.



Security considerations

Computer use has unique risks distinct from standard API features. These risks are heightened when interacting with the internet.

In some circumstances, Claude will follow commands found in content even when they conflict with your instructions. For example, instructions on webpages or contained in images might override your instructions or cause Claude to make mistakes. Take precautions to isolate Claude from sensitive data and actions to avoid risks related to prompt injection.

Anthropic has trained the model to resist these prompt injections and has added an extra layer of defense. If you use the computer use tools, classifiers will automatically run on your prompts to flag potential instances of prompt injections. When these classifiers identify potential prompt injections in screenshots, they will automatically steer the model to ask for user confirmation before proceeding with the next action. This extra protection won't be ideal for every use case (for example, use cases without a human in the loop), so if you'd like to opt out and turn it off, contact support.

These precautions remain important even with the classifier defense layer in place.

Inform end users of relevant risks and obtain their consent prior to enabling computer use in your own products.



Quick start

Add the computer use toolset to the tools array of a Messages API request as {"type": "computer_toolset_20260801"}. The request needs no beta header. This example also declares the text editor tool and bash tool, which Claude typically uses alongside computer use:

client = anthropic.Anthropic()

response = client.messages.create(
    model="claude-opus-5",
    max_tokens=1024,
    tools=[
        {"type": "computer_toolset_20260801"},
        {"type": "text_editor_20250728", "name": "str_replace_based_edit_tool"},
        {"type": "bash_20250124", "name": "bash"},
    ],
    messages=[{"role": "user", "content": "Save a picture of a cat to my desktop."}],
)
print(response)

When Claude acts on the desktop, the response has a stop_reason of tool_use and contains one or more member tool_use blocks, each naming a member tool and carrying "toolset_name": "computer". Partway through this task, after Claude has seen a screenshot of the desktop, a response might look like this:

{
  "id": "msg_01UZ3bXcQH8mTqNhVfL9eK2p",
  "type": "message",
  "role": "assistant",
  "model": "claude-opus-5",
  "content": [
    {
      "type": "text",
      "text": "I'll open the web browser to find a picture of a cat."
    },
    {
      "type": "tool_use",
      "id": "toolu_01WkoTUvSHDzTBu2xnGk8Ep8",
      "name": "left_click",
      "toolset_name": "computer",
      "input": { "coordinate": [512, 742] }
    },
    {
      "type": "tool_use",
      "id": "toolu_017nJn3RgSCkTMwuZDb4uUov",
      "name": "screenshot",
      "toolset_name": "computer",
      "input": {}
    }
  ],
  "stop_reason": "tool_use",
  "stop_sequence": null
}

Your application runs each call in order in your own environment, returns one tool_result block per tool_use block, and calls the API again; How computer use works describes that loop, and the rest of this page shows how to implement it.



How computer use works

The repetition of steps 3 and 4 without user input is referred to as the "agent loop" (that is, Claude responding with a tool use request and your application responding to Claude with the results of evaluating that request).



Batch actions

Claude can plan a short sequence of actions, such as click, type, and then take a screenshot, and return them together in one response. This is called a batch action; it uses the same response shape as parallel tool use with one difference: you run the blocks in order rather than concurrently.

A response with a three-action batch looks like this:

{
  "role": "assistant",
  "content": [
    {
      "type": "tool_use",
      "id": "toolu_01HqCF3nJ4Vzr8sTkPZ2wxYA",
      "name": "left_click",
      "toolset_name": "computer",
      "input": { "coordinate": [640, 60] }
    },
    {
      "type": "tool_use",
      "id": "toolu_01Ppr3sZ3TnE9m6VUu4RyH2K",
      "name": "type",
      "toolset_name": "computer",
      "input": { "text": "pictures of cats" }
    },
    {
      "type": "tool_use",
      "id": "toolu_01Xf5W1sD8Q9aBcJ7kLmN2pQ",
      "name": "screenshot",
      "toolset_name": "computer",
      "input": {}
    }
  ]
}

Return one tool_result block for each tool_use block, matched by tool_use_id, all in the next user message. Every result for a member tool must carry "toolset_name": "computer"; a result that omits it, or that names a different toolset than its tool_use block, is rejected. Only screenshot and zoom results need an image; for the other members, a short text acknowledgment such as OK is enough (cursor_position returns the coordinates as text):

{
  "role": "user",
  "content": [
    {
      "type": "tool_result",
      "tool_use_id": "toolu_01HqCF3nJ4Vzr8sTkPZ2wxYA",
      "toolset_name": "computer",
      "content": [{ "type": "text", "text": "OK" }]
    },
    {
      "type": "tool_result",
      "tool_use_id": "toolu_01Ppr3sZ3TnE9m6VUu4RyH2K",
      "toolset_name": "computer",
      "content": [{ "type": "text", "text": "OK" }]
    },
    {
      "type": "tool_result",
      "tool_use_id": "toolu_01Xf5W1sD8Q9aBcJ7kLmN2pQ",
      "toolset_name": "computer",
      "content": [
        {
          "type": "image",
          "source": {
            "type": "base64",
            "media_type": "image/png",
            "data": "iVBORw0KGgo..."
          }
        }
      ]
    }
  ]
}

Run blocks in order and stop at the first failure. Later actions in a batch usually depend on earlier ones: the type in this example enters text into whatever the preceding click focused. Run the blocks sequentially in the order they appear in content, and if one fails, don't run the rest. Every tool_use block still needs a tool_result, so answer the batch as follows:

• For each action that succeeded, return its normal result.
• For the action that failed, return is_error: true with a text description of what went wrong.
• For every later action in the batch, return is_error: true with exactly this text (the browser use tool uses its own halt text):

{
  "type": "tool_result",
  "tool_use_id": "toolu_01Xf5W1sD8Q9aBcJ7kLmN2pQ",
  "toolset_name": "computer",
  "is_error": true,
  "content": "Not executed: an earlier computer action in this turn failed."
}

Claude then sees which actions succeeded, which one failed, and which were skipped, and replans on its next turn. A request that leaves any tool_use block in the batch unanswered is rejected with an invalid_request_error, so an agent loop that reads only the first block fails on its next call. If your application asks a human to confirm consequential actions, make that check before each block runs, because a batch can complete a multistep action within one turn.

Claude typically finishes a batch with screenshot so it can observe the outcome before deciding what to do next. When a batch doesn't end with one, your application can attach a screenshot as an extra image block on the last result in the batch so that Claude always sees the current state of the screen, which saves a round trip compared with waiting for Claude to ask. You can also prompt Claude to end every batch with a screenshot (see Optimize model performance with prompting).



The computing environment

Computer use requires a sandboxed computing environment where Claude can safely interact with applications and the web. This environment includes:

1. Virtual display: A virtual X11 display server (using Xvfb) that renders the desktop interface Claude will see through screenshots and control with mouse/keyboard actions.

2. Desktop environment: A lightweight UI with window manager (Mutter) and panel (Tint2) running on Linux, which provides a consistent graphical interface for Claude to interact with.

3. Applications: Pre-installed Linux applications such as Firefox, LibreOffice, text editors, and file managers that Claude can use to complete tasks.

4. Tool implementations: Integration code that translates Claude's abstract tool requests (such as "move mouse" or "take screenshot") into actual operations in the virtual environment.

5. Agent loop: A program that handles communication between Claude and the environment, sending Claude's actions to the environment and returning the results (screenshots, command outputs) back to Claude.

When you use computer use, Claude doesn't directly connect to this environment. Instead, your application:

1. Receives Claude's tool use requests
2. Translates them into actions in your computing environment
3. Captures the results (such as screenshots and command outputs)
4. Returns these results to Claude

For security and isolation, the reference implementation runs all of this inside a Docker container with appropriate port mappings for viewing and interacting with the environment.



How to implement computer use

Upgrading an existing computer_20251124 integration? Start with Migrate from computer_20251124; the rest of this section applies to both new and migrated integrations.



Understand the agent loop

The core of computer use is the "agent loop": a cycle where Claude requests tool actions, your application runs them, and returns results to Claude. The loop uses the client you created in the Quick start, a tools array that declares only the computer use toolset, and the tool-call processing helper under Implement the computer use tool. If you also declare other tools, such as the Quick start's bash and text editor tools, dispatch their tool_use blocks in the same pass; the helper answers only computer use member calls, and the loop treats a turn with no answered calls as finished. Here's a simplified example:

def sampling_loop(model: str, messages: list[MessageParam], max_iterations: int = 10):
    """
    Run the computer-use agent loop until Claude stops requesting tools
    or the iteration limit is reached.
    """
    for _ in range(max_iterations):
        response = client.messages.create(
            model=model,
            max_tokens=4096,
            messages=messages,
            tools=TOOLS,
        )

        # Add Claude's response to the conversation history
        messages.append({"role": "assistant", "content": response.content})

        # Run the actions Claude requested, in order, and collect the results
        tool_results = process_tool_calls(response)
        if not tool_results:
            return messages  # No more tool use; task complete

        # Send every result back to Claude in a single user message
        messages.append({"role": "user", "content": tool_results})

    return messages

The loop continues until either Claude responds without requesting any tools (task completion) or the maximum iteration limit is reached. This safeguard prevents potential infinite loops that could result in unexpected API costs.



Optimize model performance with prompting

1. Specify simple, well-defined tasks and provide explicit instructions for each step.
2. Claude sometimes assumes outcomes of its actions without explicitly checking their results. To prevent this you can prompt Claude with After each step, take a screenshot and carefully evaluate if you have achieved the right outcome. Explicitly show your thinking: "I have evaluated step X..." If not correct, try again. Only when you confirm a step was executed correctly should you move on to the next one.
3. Some UI elements (such as dropdowns and scrollbars) might be tricky for Claude to manipulate using mouse movements. If you experience this, try prompting the model to use keyboard shortcuts.
4. For repeatable tasks or UI interactions, include example screenshots and tool calls of successful outcomes in your prompt.
5. If you need the model to log in, provide it with the username and password in your prompt inside XML tags such as <robot_credentials>. Using computer use within applications that require login increases the risk of bad outcomes as a result of prompt injection. Review Mitigate jailbreaks and prompt injections before providing the model with login credentials.
6. When constructing a user turn's content array, place the instruction text before the screenshot image. Providing the target description before the image is processed improves click accuracy.
7. Claude uses the zoom action to inspect a region at full resolution when asked about small text or specific UI elements that aren't legible at the screenshot's default resolution, such as file names in a sidebar, tab titles, status-bar text, line numbers, or button labels. If Claude isn't zooming when you expect, ask about a specific region or element rather than the screen as a whole.
8. If you want every batch action to end with a screenshot, say so in the system prompt, for example, End each group of actions with a screenshot so you can verify the result before continuing.



System prompts

When you include the computer use tool in a request, the API generates a computer use-specific system prompt. It's similar to the tool use system prompt but starts with:

You have access to a set of functions you can use to answer the user's question. This includes access to a sandboxed computing environment. You do NOT currently have the ability to inspect files or interact with external resources, except by invoking the below functions.

As with regular tool use, the user-provided system parameter is still respected and used in the construction of the combined system prompt.



Available actions

Each action is a member tool of the computer use toolset: Claude names the member in a tool_use block that carries "toolset_name": "computer", and the block's input holds only that member's parameters, with no action field. The toolset has 17 member tools:

Keep the following in mind when implementing the members:

• Coordinates are in screenshot pixels. Every coordinate, start_coordinate, and region value, and the position that cursor_position reports, is in the pixel space of the full-display screenshots you return, with the origin at the top left. Zoom images don't change this: after a zoom, Claude still expresses coordinates in the full screenshot's space, never relative to the zoomed image. If you scale screenshots down before returning them, scale Claude's coordinates back up before applying them to the real display (see Size screenshots to fit image limits).
• All members are enabled by default, including zoom. If your environment can't produce zoom images, withhold the member with configs (see Tool parameters) rather than leaving it enabled and returning errors. If Claude calls a member that you have withheld or don't implement, return a tool_result with is_error: true for that block.
• Dispatch on the pair (toolset_name, name). toolset_name is what marks a block as a computer action: a custom tool in the same request can share a member's name, and a later toolset version can add members (see Client toolsets).



Tool parameters

The toolset entry in the tools array accepts four parameters; the rules they share with the browser use toolset are listed under Client toolsets.

For example, this entry withholds zoom for an environment that doesn't implement it and sets a cache breakpoint at the toolset definition:

{
  "type": "computer_toolset_20260801",
  "configs": {
    "zoom": { "enabled": false }
  },
  "cache_control": { "type": "ephemeral" }
}

If your agent loop can run only one action per round trip, set disable_parallel_tool_use to true in tool_choice; Claude then returns at most one member tool_use block per turn (see Disable parallel tool use).

The entry rejects these parameters from earlier tool versions, and a request that includes any of them returns an invalid_request_error:

• name: member names are fixed by the toolset version.
• display_width_px, display_height_px, and display_number: coordinates are always in the pixel space of the screenshots you return.
• enable_zoom: zoom is a member tool that you control through configs.

The entry also can't be declared in the same request as a computer_20251124 entry or another tool named computer. For strict, input_examples, defer_loading placement, tool_choice, streaming, and caller restrictions, see Client toolsets.



Combining with thinking

For combining computer use with thinking, see Thinking.



Augmenting computer use with other tools

To add other tools alongside computer use, include them in the same tools array. The Quick start section shows this pattern with the bash tool and text editor tool. You can add your own custom tool definitions the same way.

For tasks that stay inside webpages, you can also declare the browser use tool in the same request: the two toolsets work independently, each in its own coordinate frame, and calls to members that share a name, such as screenshot or key, are told apart by toolset_name.



Build a custom computer use environment

The reference implementation is meant to help you get started with computer use. It includes all of the components needed to have Claude use a computer. However, you can build your own environment for computer use to suit your needs. You'll need:

• A virtualized or containerized environment suitable for computer use with Claude
• An implementation of the computer use tool's actions
• An agent loop that interacts with the Claude API and runs the tool_use results using your tool implementations
• An API or UI that allows user input to start the agent loop



Implement the computer use tool

The computer use tool is implemented as a schema-less tool. When using this tool, you don't need to provide an input schema as with other tools; the schema is built into Claude's model and can't be modified.



Handle errors

Report a failed action to Claude as a tool_result with is_error: true and a short description, and include "toolset_name": "computer" as on any other member result. If the failed action was part of a batch action, answer the remaining blocks in the batch with the halt text shown there instead of running them.

For example, when screenshot capture fails:

{
  "role": "user",
  "content": [
    {
      "type": "tool_result",
      "tool_use_id": "toolu_01A09q90qw90lq917835lq9",
      "toolset_name": "computer",
      "content": "Error: Failed to capture screenshot. Display may be locked or unavailable.",
      "is_error": true
    }
  ]
}

Use the same shape for coordinates outside the display bounds and for actions that fail to run, with a message that says what went wrong.



Size screenshots to fit image limits

Screenshots and zoom images that you return to the computer use toolset must already fit within your model's image size limits: the toolset takes no display dimensions and the API doesn't downscale for you, so an oversized tool_result image is rejected with a validation error. Because Claude returns coordinates in the pixel space of the image it sees, keep the scale factor you used so you can map those coordinates back to your screen.

If your screen is larger than the limit, resize each screenshot before returning it and scale Claude's returned coordinates back to the original screen space. Because the toolset takes no display dimensions, the resize and the coordinate scaling in your application code are all you need:

import math

screen_width, screen_height = 1512, 982


def get_scale_factor(width, height):
    """Calculate scale factor to meet API constraints."""
    long_edge = max(width, height)
    total_pixels = width * height

    long_edge_scale = 1568 / long_edge
    total_pixels_scale = math.sqrt(1_150_000 / total_pixels)

    return min(1.0, long_edge_scale, total_pixels_scale)


# When capturing screenshot
scale = get_scale_factor(screen_width, screen_height)
scaled_width = int(screen_width * scale)
scaled_height = int(screen_height * scale)

# Resize image to scaled dimensions before sending to Claude
screenshot = capture_and_resize(scaled_width, scaled_height)


# When handling Claude's coordinates, scale them back up
def execute_click(x, y):
    screen_x = x / scale
    screen_y = y / scale
    perform_click(screen_x, screen_y)

When you choose a display resolution and return screenshots:

• For general desktop tasks, use 1024x768 or 1280x720; for web applications, use 1280x800 or 1366x768.
• Avoid resolutions above 1920x1080 to prevent performance issues.
• Encode screenshots as base64 PNG or JPEG, and consider compressing large screenshots to improve performance.
• Include relevant metadata such as timestamp or display state.
• If you use higher resolutions, ensure coordinates are accurately scaled.



Manage screenshot history

Long agent loops accumulate screenshots quickly (roughly 1,000–1,800 input tokens each). The API's request limits also apply. Once a single request carries more than 20 images, every image in it is held to a stricter per-side limit. A loop that keeps its screenshot history reaches that count within a few dozen turns, so either resize each screenshot so that neither side exceeds 2000 px or prune older screenshots to keep 20 or fewer in the request.

To keep Prompt caching effective while bounding context:

• Place one cache_control breakpoint after the system prompt and tool definitions, and up to three more on the last tool_result block of each of the most recent turns, advancing them each turn. Within a batch action, markers on several blocks act as a single breakpoint but each still counts toward the limit of four, so use one per turn.
• Prune old screenshots in batches, not one each turn. Dropping a screenshot every turn changes the prefix every turn and invalidates the cache. A reasonable default is to keep the last three screenshots and prune every 25 turns, so the prefix stays byte-identical between prune events; if your screenshots exceed 2000 px on either side, choose an interval that keeps each request at 20 or fewer images.



Diagnose click issues

If clicks miss their targets, the cause is usually one of the following:



Follow implementation best practices



Migrate from computer_20251124

Upgrading from computer_20251124 to the toolset is optional: the models listed for computer_20251124 under Earlier tool versions keep accepting it with its beta header, so an existing integration keeps working until you change it. To upgrade, make the following changes together:

1. Remove the beta header. Drop anthropic-beta: computer-use-2025-11-24 from your requests. In the SDKs, remove the betas parameter and call the Messages API through the standard client rather than the beta namespace.
2. Change the tools entry. Set type to computer_toolset_20260801 and delete name, display_width_px, display_height_px, display_number, and enable_zoom. The toolset rejects each of these fields.
3. Choose whether to keep zoom enabled. Zoom is enabled by default on the toolset, whereas enable_zoom defaults to false. If your environment doesn't implement zoom, add "configs": {"zoom": {"enabled": false}} to keep the previous behavior; otherwise implement it (see Available actions).
4. Handle every block in a turn. Update your agent loop to iterate over every tool_use block in a response rather than reading only the first, and to dispatch on the block's name together with toolset_name instead of on input.action. Member inputs no longer contain an action field; the remaining fields are unchanged.
5. Run blocks in order and use the halt text. Run the blocks sequentially, stop at the first failure, and answer the remaining blocks with Not executed: an earlier computer action in this turn failed. as described in Batch actions. If your loop can't run batches yet, Tool parameters explains how to limit Claude to one action per turn.
6. Echo toolset_name on results. Add "toolset_name": "computer" to every tool_result that answers a member call. Results may contain only text and image content.
7. Support repeat on key. The key member accepts an optional repeat count from 1 to 100. A handler that ignores unrecognized fields would press the key once, so make your key handler honor repeat.
8. Resize screenshots yourself. The toolset rejects a screenshot or zoom image that exceeds the model's image limits instead of downscaling it. Resize before returning the image and keep scaling coordinates as described in Size screenshots to fit image limits.
9. Remove unsupported options. Move any defer_loading from the entry into configs, with the same value on every enabled member. The other options not supported on toolset entries are listed under Client toolsets.

This is the tools entry before the change, sent with the anthropic-beta: computer-use-2025-11-24 header:

{
  "type": "computer_20251124",
  "name": "computer",
  "display_width_px": 1024,
  "display_height_px": 768,
  "display_number": 1
}

This is the tools entry after the change, sent with no beta header. The configs object keeps zoom off to match the earlier entry, which doesn't set enable_zoom; omit configs entirely to accept the default and let Claude zoom:

{
  "type": "computer_toolset_20260801",
  "configs": {
    "zoom": { "enabled": false }
  }
}

The following pair shows a tool_use block before and after the change. The action name moves from input.action to name, and the block gains toolset_name:

{
  "type": "tool_use",
  "id": "toolu_01A9r5kQm2LxWc7vT3nZ4bJs",
  "name": "computer",
  "input": { "action": "left_click", "coordinate": [500, 300] }
}

{
  "type": "tool_use",
  "id": "toolu_01A9r5kQm2LxWc7vT3nZ4bJs",
  "name": "left_click",
  "toolset_name": "computer",
  "input": { "coordinate": [500, 300] }
}



Earlier tool versions

Two earlier versions of the computer use tool remain available in beta for existing integrations, for models that don't support the toolset, and on platforms where the toolset isn't currently available. Each requires its beta header on every request, and their parameters are documented in the beta Messages API reference. In the SDKs, pass the header through the betas parameter and use the beta namespace; only the computer use tool needs the header, not the bash or text editor tools in the same request.



Limitations

1. Latency: The current computer use latency for human-AI interactions might be too slow compared to regular human-directed computer actions. Focus on use cases where speed isn't critical (for example, background information gathering, automated software testing) in trusted environments.
2. Computer vision accuracy and reliability: Claude might make mistakes or hallucinate when outputting specific coordinates while generating actions. Claude's summarized thinking output can help you understand the model's reasoning and identify potential issues; set display: "summarized" on the thinking configuration, because the models that support the toolset omit thinking text by default.
3. Tool selection accuracy and reliability: Claude might make mistakes or hallucinate when selecting tools while generating actions or take unexpected actions to solve problems. Additionally, reliability might be lower when interacting with niche applications or multiple applications at once. Prompt the model carefully when requesting complex tasks.
4. Scrolling reliability: The scroll action supports direction control (up, down, left, right) and a specified amount. In applications where scrolling doesn't take effect, keyboard alternatives such as Page Down can help.
5. Spreadsheet interaction: Use the fine-grained mouse control actions (left_mouse_down, left_mouse_up) and modifier-key combinations to select individual cells. Complex spreadsheet operations might still require multiple attempts.
6. Account creation and content generation on social and communications platforms: Although Claude visits websites, its ability to create accounts, generate and share content, or otherwise engage in human impersonation across social media websites and platforms is limited.
7. Vulnerabilities: Jailbreaks and prompt injection can affect computer use as they can any frontier AI system, including through instructions embedded in webpages or images; apply the precautions in Security considerations.
8. Inappropriate or illegal actions: Under Anthropic's Terms of Service, you must not employ computer use to violate any laws or the Acceptable Use Policy.

Always carefully review and verify Claude's computer use actions and logs. Do not use Claude for tasks requiring perfect precision or sensitive user information without human oversight.



Data retention

Computer use is a client-side tool. All screenshots, mouse actions, keyboard inputs, and any files involved in a session are captured and stored in your environment, not by Anthropic. Anthropic processes the screenshot images and action requests in real time as part of the API call. Retention for those API requests is governed by API and data retention.

Because your application controls where and how computer use data is stored, computer use is ZDR eligible. For ZDR eligibility across all features, see API and data retention.



Pricing

Computer use follows the standard tool use pricing. When using the computer use tool:

Toolset definition overhead: Declaring computer_toolset_20260801 with its default members adds about 4,500 input tokens to a request (about 4,520 on Claude Fable 5, Claude Mythos 5, Claude Opus 5, and Claude Opus 4.8, and about 4,590 on Claude Sonnet 5), which covers the member tool definitions and the tool use system prompt. Disabling zoom with configs removes about 410 of those tokens. The exact count for a request is reported in the response usage, and you can estimate it in advance with the token counting endpoint.

Earlier tool versions: The following figures apply to the computer_20251124 and computer_20250124 tool versions, not to computer_toolset_20260801:

• System prompt overhead: 466–499 tokens added to the system prompt
• Tool definition: about 735 input tokens per tool definition (measured with computer_20250124)

Additional token consumption:

• Screenshot and zoom images returned in tool results, billed as image input (see Vision pricing)
• Tool execution results returned to Claude



Next steps