Loading
Loading
Loading
Loading
Loading
Loading
Loading
Loading
Loading
Loading
Loading
Loading
Loading
Loading
Loading
Loading
Claude can interact with computer environments through the computer use tool, which provides screenshot capabilities and mouse/keyboard control for autonomous desktop interaction.
On Claude Sonnet 4.5, Claude Haiku 4.5, Claude Opus 4.1 (retired, except on Bedrock and Google Cloud), Claude Sonnet 4 (retired, except on Bedrock and Google Cloud), and Claude Opus 4 (retired, except on Google Cloud), use the earlier computer-use-2025-01-24 beta header instead of computer-use-2025-11-24.
Reach out through the feedback form to share your feedback on this feature.
Computer use is a beta feature that enables Claude to interact with desktop environments. This tool provides:
While computer use can be augmented with other tools such as bash and text editor for more comprehensive automation workflows, computer use specifically refers to the computer use tool's capability to see and control desktop environments.
For model support, see the Tool reference.
Computer use is a beta feature with unique risks distinct from standard API features. These risks are heightened when interacting with the internet.
To minimize risks, consider taking precautions such as:
In some circumstances, Claude will follow commands found in content even when they conflict with your instructions. For example, instructions on webpages or contained in images might override your instructions or cause Claude to make mistakes. Take precautions to isolate Claude from sensitive data and actions to avoid risks related to prompt injection.
Anthropic has trained the model to resist these prompt injections and has added an extra layer of defense. If you use the computer use tools, classifiers will automatically run on your prompts to flag potential instances of prompt injections. When these classifiers identify potential prompt injections in screenshots, they will automatically steer the model to ask for user confirmation before proceeding with the next action. This extra protection won't be ideal for every use case (for example, use cases without a human in the loop), so if you'd like to opt out and turn it off, contact support.
These precautions remain important even with the classifier defense layer in place.
Inform end users of relevant risks and obtain their consent prior to enabling computer use in your own products.
Get started with the computer use reference implementation that includes a web interface, Docker container, example tool implementations, and an agent loop.
Here's how to get started with computer use:
client = anthropic.Anthropic()
response = client.beta.messages.create(
model="claude-opus-5", # or another compatible model
max_tokens=1024,
tools=[
{
"type": "computer_20251124",
"name": "computer",
"display_width_px": 1024,
"display_height_px": 768,
"display_number": 1,
},
{"type": "text_editor_20250728", "name": "str_replace_based_edit_tool"},
{"type": "bash_20250124", "name": "bash"},
],
messages=[{"role": "user", "content": "Save a picture of a cat to my desktop."}],
betas=["computer-use-2025-11-24"],
)
print(response)A beta header is only required for the computer use tool.
The preceding example shows all three tools being used together, which requires the beta header because it includes the computer use tool.
Provide Claude with the computer use tool and a user prompt
Claude selects the computer use tool
stop_reason of tool_use, signaling a tool use request.Extract tool input, evaluate the tool on a computer, and return results
user message containing a tool_result content block.Claude continues calling computer use tools until it's completed the task
tool_use stop_reason and you should return to step 3.The repetition of steps 3 and 4 without user input is referred to as the "agent loop" (that is, Claude responding with a tool use request and your application responding to Claude with the results of evaluating that request).
Computer use requires a sandboxed computing environment where Claude can safely interact with applications and the web. This environment includes:
Virtual display: A virtual X11 display server (using Xvfb) that renders the desktop interface Claude will see through screenshots and control with mouse/keyboard actions.
Desktop environment: A lightweight UI with window manager (Mutter) and panel (Tint2) running on Linux, which provides a consistent graphical interface for Claude to interact with.
Applications: Pre-installed Linux applications such as Firefox, LibreOffice, text editors, and file managers that Claude can use to complete tasks.
Tool implementations: Integration code that translates Claude's abstract tool requests (such as "move mouse" or "take screenshot") into actual operations in the virtual environment.
Agent loop: A program that handles communication between Claude and the environment, sending Claude's actions to the environment and returning the results (screenshots, command outputs) back to Claude.
When you use computer use, Claude doesn't directly connect to this environment. Instead, your application:
For security and isolation, the reference implementation runs all of this inside a Docker container with appropriate port mappings for viewing and interacting with the environment.
A reference implementation is available that includes everything you need to get started with computer use:
The core of computer use is the "agent loop": a cycle where Claude requests tool actions, your application runs them, and returns results to Claude. The loop uses the client you created in the Quick start, a tool list shaped like the Quick start's tools array, and the tool-call processing helper defined in Process Claude's tool calls. Here's a simplified example:
def sampling_loop(model, messages, max_iterations=10):
"""
Run the computer-use agent loop until Claude stops requesting tools
or the iteration limit is reached.
"""
for _ in range(max_iterations):
response = client.beta.messages.create(
model=model,
max_tokens=4096,
messages=messages,
tools=TOOLS,
betas=["computer-use-2025-11-24"],
)
# Add Claude's response to the conversation history
messages.append({"role": "assistant", "content": response.content})
# Run any tools Claude requested and collect results
tool_results = process_tool_calls(response)
if not tool_results:
return messages # No more tool use; task complete
# Send tool results back to Claude for the next iteration
messages.append({"role": "user", "content": tool_results})
return messagesThe loop continues until either Claude responds without requesting any tools (task completion) or the maximum iteration limit is reached. This safeguard prevents potential infinite loops that could result in unexpected API costs.
Try the reference implementation out before reading the rest of this documentation.
Here are some tips on how to get the best quality outputs:
After each step, take a screenshot and carefully evaluate if you have achieved the right outcome. Explicitly show your thinking: "I have evaluated step X..." If not correct, try again. Only when you confirm a step was executed correctly should you move on to the next one.<robot_credentials>. Using computer use within applications that require login increases the risk of bad outcomes as a result of prompt injection. Review Mitigate jailbreaks and prompt injections before providing the model with login credentials.content array, place the instruction text before the screenshot image. Providing the target description before the image is processed improves click accuracy.computer_20251124 with enable_zoom: true set, Claude zooms in on a region when asked about small text or specific UI elements that aren't legible at the screenshot's default resolution, such as file names in a sidebar, tab titles, status-bar text, line numbers, or button labels. If Claude isn't zooming when you expect, ask about a specific region or element rather than the screen as a whole.If you repeatedly encounter a clear set of issues or know in advance the tasks Claude will need to complete, use the system prompt to provide Claude with explicit tips or instructions on how to do the tasks successfully.
For agents that span multiple sessions, run end-to-end verification at the start of each session, not only after implementation. Browser-based checks catch regressions from prior sessions that code-level review alone misses. See Effective harnesses for long-running agents for details.
When one of the Anthropic-schema tools is requested through the Claude API, a computer use-specific system prompt is generated. It's similar to the tool use system prompt but starts with:
You have access to a set of functions you can use to answer the user's question. This includes access to a sandboxed computing environment. You do NOT currently have the ability to inspect files or interact with external resources, except by invoking the below functions.
As with regular tool use, the user-provided system parameter is still respected and used in the construction of the combined system prompt.
The computer use tool supports these actions:
Basic actions (all versions)
[x, y]Enhanced actions (computer_20250124 and later)
Available in computer_20250124 and computer_20251124:
Enhanced actions (computer_20251124)
Available in Claude Opus 5, Claude Sonnet 5, Claude Opus 4.8, Claude Opus 4.7, Claude Opus 4.6, Claude Sonnet 4.6, and Claude Opus 4.5:
computer_20250124enable_zoom: true in tool definition. Takes a region parameter with coordinates [x1, y1, x2, y2] defining top-left and bottom-right corners of the area to inspect.| Parameter | Required | Description |
|---|---|---|
type | Yes | Tool version (computer_20251124 or computer_20250124) |
name | Yes | Must be "computer" |
display_width_px | Yes | Display width in pixels |
display_height_px | Yes | Display height in pixels |
display_number | No | Display number for X11 environments |
enable_zoom | No | Enable zoom action (computer_20251124 only). Set to true to allow Claude to zoom into specific screen regions. Default: false |
Important: Your application must explicitly run the computer use tool; Claude cannot run it directly. You are responsible for implementing the screenshot capture, mouse movements, keyboard inputs, and other actions based on Claude's requests.
For combining computer use with thinking, see Thinking.
For computer use specifically, internal benchmarking suggests these effort settings:
high as the default; use low for high-throughput or cost-sensitive workloads.medium as the default (best accuracy-to-cost ratio). Avoid max, which adds token cost without improving accuracy on UI tasks. On these models, low uses fewer output tokens than disabling thinking entirely (fewer mistakes mean fewer retries), making it a strong option for cost-sensitive loops.To add other tools alongside computer use, include them in the same tools array. The Quick start section shows this pattern with the bash tool and text editor tool. You can add your own custom tool definitions the same way.
The reference implementation is meant to help you get started with computer use. It includes all of the components needed to have Claude use a computer. However, you can build your own environment for computer use to suit your needs. You'll need:
tool_use results using your tool implementationsThe computer use tool is implemented as a schema-less tool. When using this tool, you don't need to provide an input schema as with other tools; the schema is built into Claude's model and can't be modified.
Set up your computing environment
Create a virtual display or connect to an existing display that Claude will interact with. This typically involves setting up Xvfb (X Virtual Framebuffer) or similar technology.
Implement action handlers
Create functions to handle each action type that Claude might request:
def capture_screenshot():
return "<screenshot data>"
def click_at(x, y):
return f"clicked at ({x}, {y})"
def type_text(text):
return f"typed: {text}"
def handle_computer_action(action_type, params):
if action_type == "screenshot":
return capture_screenshot()
elif action_type == "left_click":
x, y = params["coordinate"]
return click_at(x, y)
elif action_type == "type":
return type_text(params["text"])
# Handle other actions as needed
return f"unhandled action: {action_type}"Process Claude's tool calls
Extract and run tool calls from Claude's responses:
def process_tool_calls(response):
tool_results = []
for block in response.content:
if block.type == "tool_use":
action = block.input["action"]
result = handle_computer_action(action, block.input)
tool_results.append(
{
"type": "tool_result",
"tool_use_id": block.id,
"content": result,
}
)
return tool_resultsImplement the agent loop
Create a loop that continues until Claude completes the task:
def sampling_loop(model, messages, max_iterations=10):
"""
Run the computer-use agent loop until Claude stops requesting tools
or the iteration limit is reached.
"""
for _ in range(max_iterations):
response = client.beta.messages.create(
model=model,
max_tokens=4096,
messages=messages,
tools=TOOLS,
betas=["computer-use-2025-11-24"],
)
# Add Claude's response to the conversation history
messages.append({"role": "assistant", "content": response.content})
# Run any tools Claude requested and collect results
tool_results = process_tool_calls(response)
if not tool_results:
return messages # No more tool use; task complete
# Send tool results back to Claude for the next iteration
messages.append({"role": "user", "content": tool_results})
return messagesWhen implementing the computer use tool, various errors might occur. Here's how to handle them:
Screenshots sent to the computer tool should fit within Claude's image size limits (see image size limits). The API downscales oversized images before Claude sees them, and Claude returns coordinates for the image it sees, so relying on the server-side downscale leaves you without the scale factor you need to map those coordinates back to your screen. Only images over the API's separate request limits (for example, more than 8,000 px on a side) are rejected with a validation error rather than downscaled.
Limits vary by model. Claude Opus 5, Claude Sonnet 5, Claude Opus 4.8, and Claude Opus 4.7 accept up to 2576 pixels on the long edge; earlier models accept up to 1568 pixels on the long edge and approximately 1.15 megapixels total. The following example uses the earlier-model 1568 px / 1.15 MP limits; substitute your model's limit.
If your screen is larger than the limit, resize the screenshot before sending it, set display_width_px/display_height_px to the resized dimensions, and scale Claude's returned coordinates back to the original screen space:
import math
def get_scale_factor(width, height):
"""Calculate scale factor to meet API constraints."""
long_edge = max(width, height)
total_pixels = width * height
long_edge_scale = 1568 / long_edge
total_pixels_scale = math.sqrt(1_150_000 / total_pixels)
return min(1.0, long_edge_scale, total_pixels_scale)
# When capturing screenshot
scale = get_scale_factor(screen_width, screen_height)
scaled_width = int(screen_width * scale)
scaled_height = int(screen_height * scale)
# Resize image to scaled dimensions before sending to Claude
screenshot = capture_and_resize(scaled_width, scaled_height)
# When handling Claude's coordinates, scale them back up
def execute_click(x, y):
screen_x = x / scale
screen_y = y / scale
perform_click(screen_x, screen_y)macOS Retina displays capture screenshots at a device pixel ratio of 2, so the image is twice the resolution of the logical screen coordinates. Either downscale the screenshot by 2x before sending, or halve the coordinates Claude returns before issuing the click.
If clicks miss their targets, the cause is usually one of the following:
| Symptom | Likely cause | Try |
|---|---|---|
| Clicks consistently offset in one direction | display_width_px/display_height_px don't match the image dimensions actually sent | Ensure display dimensions exactly match the screenshot you send |
| Clicks land in the right area but miss the target | Target is very small, detail was lost downscaling a 4K+ source, or aspect ratio was distorted | Set enable_zoom: true; capture at lower DPI or crop to the relevant region; preserve aspect ratio when resizing |
| Claude clicks the wrong element entirely | Ambiguous instruction, or visually similar elements nearby | Use positional prompts ("the blue Submit button in the bottom-right"); break the interaction into smaller steps |
| Accuracy is consistently poor | Resolution too low | Try 1280x720 as a baseline |
Model choice affects click precision. Claude Sonnet 4.6 is more mechanically precise at clicking than Claude Opus 4.6 and is more robust when screenshots require heavy downscaling. Claude Opus 4.7 narrows that gap: its click precision is roughly comparable to Sonnet 4.6, and its higher resolution limit means less downscaling is needed.
Computer use is in beta. Keep the following limitations in mind:
Latency: The current computer use latency for human-AI interactions might be too slow compared to regular human-directed computer actions. Focus on use cases where speed isn't critical (for example, background information gathering, automated software testing) in trusted environments.
Computer vision accuracy and reliability: Claude might make mistakes or hallucinate when outputting specific coordinates while generating actions. Extended thinking can help you understand the model's reasoning and identify potential issues.
Tool selection accuracy and reliability: Claude might make mistakes or hallucinate when selecting tools while generating actions or take unexpected actions to solve problems. Additionally, reliability might be lower when interacting with niche applications or multiple applications at once. Prompt the model carefully when requesting complex tasks.
Scrolling reliability: The scroll action supports direction control (up, down, left, right) and a specified amount. In applications where scrolling doesn't take effect, keyboard alternatives such as Page Down can help.
Spreadsheet interaction: Use the fine-grained mouse control actions (left_mouse_down, left_mouse_up) and modifier-key combinations to select individual cells. Complex spreadsheet operations might still require multiple attempts.
Account creation and content generation on social and communications platforms: While Claude will visit websites, Claude's ability to create accounts or generate and share content or otherwise engage in human impersonation across social media websites and platforms is limited. This capability might be updated in the future.
Vulnerabilities: Vulnerabilities such as jailbreaking or prompt injection might persist across frontier AI systems, including the beta computer use API. In some circumstances, Claude will follow commands found in content, sometimes even when they conflict with your instructions. For example, instructions on webpages or contained in images might override your instructions or cause Claude to make mistakes. Consider the following:
Inappropriate or illegal actions: Under Anthropic's Terms of Service, you must not employ computer use to violate any laws or the Acceptable Use Policy.
Always carefully review and verify Claude's computer use actions and logs. Do not use Claude for tasks requiring perfect precision or sensitive user information without human oversight.
Computer use is a client-side tool. All screenshots, mouse actions, keyboard inputs, and any files involved in a session are captured and stored in your environment, not by Anthropic. Anthropic processes the screenshot images and action requests in real time as part of the API call. Retention for those API requests is governed by API and data retention.
Because your application controls where and how computer use data is stored, computer use is ZDR eligible. For ZDR eligibility across all features, see API and data retention.
Computer use follows the standard tool use pricing. When using the computer use tool:
System prompt overhead: The computer use beta adds 466–499 tokens to the system prompt
Computer use tool token usage:
| Model | Input tokens per tool definition |
|---|---|
| Claude 4.x models | 735 tokens |
Additional token consumption:
If you're also using bash or text editor tools alongside computer use, those tools have their own token costs as documented in their respective pages.
Fix the most common tool-use errors with symptom-to-fix diagnostic tables.
Get started with the complete Docker-based implementation
Connect Claude to external tools and APIs. See where tools execute, when Claude calls them, and which tool fits your task.
Benchmarked recommendations for resolution, thinking effort, and context management
| Supported models |
|
|---|---|
| Supported platforms |
|
Was this page helpful?