Sanbox By Sanlabs

Guides

Building agent products

Guide

How do I use my own model endpoint (Azure OpenAI, vLLM, LiteLLM) with an agent computer?

By the Sanbox team updated

Ask ChatGPT Ask Claude

Register the endpoint once as a provider and point your agent templates at it. In Sanbox, sanbox model-providers create takes any OpenAI-compatible base URL, such as an Azure OpenAI deployment, a LiteLLM gateway or a vLLM server, with its key on stdin; the provider ID becomes custom:<name>, and Computers built from templates on that provider use your endpoint for their model calls. That's how you keep model calls in a region you choose, for example on an Azure OpenAI deployment in the EU Data Zone.

Run this with your agent

Read https://sanbox.cloud/kb/build/bring-your-own-model-endpoint.md and follow it to connect my model endpoint to Sanbox.
If the Sanbox CLI is missing, install it with `npm install -g @sanlabs/sanbox-cli@latest`
and sign in with `sanbox login`. Show me each command before you create, stop or delete a Computer.

What do I need?

  • An OpenAI-compatible endpoint over HTTPS. Sanbox validates the key and discovers models by calling GET <base-url>/models, so the endpoint has to be reachable from Sanbox.
  • The endpoint's API key, in an environment variable or a secret manager. Never put it in a command argument, print it or write it to a file.
  • CLI 0.0.22 or later and the owner or admin role, from an organisation API key or a sanbox login session. A member key gets 403.
  • A free slot. An organisation can connect up to 20 of these proxies.

What base URL and model ID do I use?

EndpointBase URLModel ID
Azure OpenAI (v1 API)https://<resource>.openai.azure.com/openai/v1Your deployment name
vLLMhttps://<your-host>/v1The model the server lists at /v1/models
LiteLLM gatewayhttps://<your-gateway>/v1A model name your gateway lists at /v1/models
  • Azure OpenAI: Microsoft's v1 API takes the resource endpoint with /openai/v1 appended, and the model you send is your deployment name.
  • vLLM: start the server with an API key, for example vllm serve <model> --api-key <key>. Agents call tools, and vLLM's docs require --enable-auto-tool-choice and a matching --tool-call-parser for automatic tool calling.
  • LiteLLM: the gateway gives one OpenAI-compatible endpoint in front of many providers. A virtual key limits which models and budgets the key can use.

Steps

1. Add the endpoint

printf '%s' "$PROXY_API_KEY" | sanbox model-providers create --name azure-eu \
  --base-url https://<resource>.openai.azure.com/openai/v1 --api-key-stdin --json

The name is 1 to 48 lowercase letters, digits and single hyphens, starting with a letter, and can't be changed later. If /models doesn't list a model you need, such as an Azure deployment name, add --model-id <id> (repeatable) to the same command. If that route refuses the request without rejecting the key, Sanbox checks the first --model-id with one chat request instead.

2. Check the models

sanbox model-providers models custom:azure-eu --json

3. Create a template on it

sanbox templates create --name "EU review" --type opencode-computer \
  --model-provider custom:azure-eu --model <model-id> \
  --network-policy policy.json --json
sanbox templates validate <template-id> --json

For an OpenCode Computer, repeat --allowed-model <model-id> to offer more models from the same endpoint. If you drive the Computer through the OpenCode SDK and pick a model per prompt, use providerID: "custom" for every Custom OpenAI Proxy, whatever its name.

4. Start Computers as usual

sanbox run --template <template-id> --json

Like other model provider keys, the endpoint's key stays in Sanbox's control plane, not in your template or task.

How do I keep model calls in the EU?

Sanbox Computers, disks, snapshots, files, run records, accounts, logs and monitoring are stored and processed in EU data centres. What leaves the EU is only what agents send: calls to the model provider you configure, and to the destinations you allow. So the model endpoint decides where prompts go.

  • Azure OpenAI: Microsoft says Global deployment types may process prompts in any Azure region, Data Zone types only within the chosen data zone, and Standard types within the Azure geography you pick. The EU data zone follows the EU Data Boundary, which can also include EFTA countries such as Norway and Switzerland. For EU-only processing, deploy as Standard in an EU-country geography. Data Zone (EU) can also include EFTA countries.
  • vLLM: prompts go wherever you run the server.
  • LiteLLM: the gateway forwards to the providers in its configuration, so check where each of those processes data.

What do the errors mean?

ErrorWhat it means
invalid_resource_nameThe name breaks the naming rule
custom_provider_existsThat name is taken; choose another
custom_provider_limitThe organisation already has 20
provider_validation_failed (400)The endpoint rejected the key or the request
provider_validation_failed (502)Sanbox couldn't reach the endpoint
403The key or session has the member role, not owner or admin

To replace a proxy's endpoint or key, use the console.

What this doesn't do

  • Only OpenAI-compatible endpoints. Other provider credentials are connected in the console.
  • Not for the OpenCode 2 Computer beta, which supports only the anthropic, openai and google providers.
  • No model hosting. Sanbox doesn't sell or host models, and doesn't train on your data. Model quality, tool calling and rate limits come from your endpoint.
  • No renaming. The name is part of the provider ID your templates use.

Sources