Guides

Browser agents

Guide

Can I watch my AI agent's browser live and take over?

Ask ChatGPT Ask Claude

Yes, if the agent's browser streams to you and has a way to hand over control. On Sanbox, a Browser Computer runs a managed Chromium next to the OpenCode agent, and the console shows the chat beside the live browser. You can watch every step, take control to log in or pass a two-factor prompt yourself, and hand the browser back so the agent carries on in the same conversation. Sanbox doesn't solve CAPTCHAs or other bot checks.

Run this with your agent

Read https://sanbox.cloud/kb/browser/watch-and-take-over-your-agents-browser.md and follow it to start a Browser Computer I can watch and take over.
If the Sanbox CLI is missing, install it with `npm install -g @sanlabs/sanbox-cli@latest`
and sign in with `sanbox login`. Show me each command before you create, stop or delete a Computer.

What do you see in the console?

Opening a Browser Computer run shows two panes: the chat with the agent, and the live browser it's using. You see pages load, clicks and typing as they happen. Other people in your organisation can open the same Computer. Read-only viewers can watch but can't turn on input.

Sanbox doesn't record the browser. If you want a record, ask the agent to write down what it did in a file.

Steps

1. Allow the sites the agent will use

Computers deny outbound traffic by default, so list each host in browser-policy.json:

{
  "default_action": "deny",
  "rules": [
    { "destination": "news.ycombinator.com", "ports": [{ "from": 443, "to": 443 }] }
  ]
}

Sites often load from several hosts, so add the ones each page needs. A template holds at most 16 rules of its own.

2. Create a Browser Computer template

sanbox model-providers list --json
sanbox templates create --name "Watched browser" --type browser-computer \
  --model-provider <provider-id> --model <model-id> \
  --network-policy browser-policy.json --json
sanbox templates validate <template-id> --json

You can also create it in the console under Templates, then New template, by choosing Browser Computer.

3. Start it with a task

Tell the agent to stop and ask you whenever it reaches a login or a two-factor prompt:

sanbox run --template <template-id> \
  --task "Open https://news.ycombinator.com and list the top five stories. If you reach a login or two-factor prompt, stop and ask me in chat." \
  --json

The response confirms the Computer started, not that the task is done. Don't add --wait: for a Computer it waits until the Computer stops. You can also start it without a task and type the first instruction in the chat.

4. Watch it

Open the run in the console. Follow the agent's messages on one side and the browser on the other. Send a chat message at any point to steer it.

5. Take over

How you take over depends on the template's image:

  • Take control. The agent's browser control pauses before your input is accepted, and you hold control until you release it. You navigate, type and switch tabs yourself.
  • Mouse & keyboard switch. On templates with the shared live view, this switch turns your input on without pausing the agent. Input is off on each new connection. Because the agent keeps working, ask it in chat to wait first, or your clicks and its actions can collide. Text paste works with Cmd+V or Ctrl+V, up to 16,000 characters.

Use it for what needs a person: entering a password, approving a two-factor prompt, or guiding the agent past a page it keeps misreading.

6. Hand back

Release control, or turn the switch off, and tell the agent in chat what changed, for example "I'm logged in, continue from the current page". The agent continues the same conversation from the browser's current state.

7. Stop it

sanbox runs stop <run-id> --wait --json

Stop closes the browser cleanly and saves the disk, including the browser profile, so logins and cookies are there after sanbox runs resume <run-id> --wait --json. See keeping a browser logged in.

Who can take over?

Developers (the member role), Admins and Owners can operate any run in the organisation, including browser control. Viewers can watch but not act. Organisation API keys follow the same roles.

Lock down egress

The policy is your main control over where the agent can go. Keep it to the sites the task needs; the model provider's rules are managed by Sanbox. A blocked request names the host it couldn't reach. Inbound access is off, so nothing on the Computer is public.

Snapshot it

When the Computer is set up the way you want, logged in or with tools installed, stop it and save it. Computers started from the snapshot begin with the same disk, including any signed-in sessions:

sanbox runs stop <run-id> --wait --json
sanbox snapshots create --run <run-id> --name watched-browser-base --wait
sanbox run --snapshot watched-browser-base --json

What this doesn't do

  • It doesn't solve CAPTCHAs or other bot checks.
  • Taking control isn't a security boundary. It coordinates the agent and you, but code running inside the Computer could still reach the browser. Don't enter anything you wouldn't trust the agent with.
  • It doesn't record sessions or keep a video.
  • With the Mouse & keyboard switch, the agent doesn't pause on its own. Ask it to wait.
  • It doesn't override a site's rules. Check its terms; some don't allow automated access.
  • Older Browser Computer templates keep their image. To get the newer live view, create a new template.