Reference
Model providers
Every model an agent can use: Anthropic, OpenAI and any OpenAI-compatible server, Gemini, Mistral, local Ollama and llama.cpp, and more, plus fallback and routing.
Naming a model
An agent's model is "provider:model". The provider is the text before the first colon when it
is one of the names below; the rest is that provider's own model id, colons included
("ollama:llama3.2:3b"). Bare Claude names ("claude-opus-5-5") and "mock" need no prefix.
Keys, base URLs and headers come from the environment, never from the source, so the same program runs against
another server by changing the environment only.
Every provider
| Provider | Example | Speaks | Environment | Browser |
|---|---|---|---|---|
| offline stand-in | "mock" | nothing | nothing | yes |
| Anthropic | "claude-opus-5-5" | Messages (SDK in the CLI, fetch in the browser) | ANTHROPIC_API_KEY, ANTHROPIC_BASE_URL | yes (sends the direct-browser-access header) |
| OpenAI, or any OpenAI-compatible server | "openai:gpt-5.1" | Chat Completions | OPENAI_API_KEY, OPENAI_BASE_URL, OPENAI_HEADERS | depends on the endpoint's CORS policy |
| OpenAI Responses API | "openai-responses:gpt-5.1" | Responses (stateless, store: false) | OPENAI_API_KEY, OPENAI_BASE_URL, OPENAI_HEADERS | depends on the endpoint's CORS policy |
| Google Gemini | "gemini:gemini-3-pro" | generateContent (streamed) | GEMINI_API_KEY, GOOGLE_API_KEY, GEMINI_BASE_URL | depends on the API's CORS policy |
| Mistral | "mistral:mistral-large-latest" | Chat Completions (Mistral dialect) | MISTRAL_API_KEY, MISTRAL_BASE_URL | depends on the API's CORS policy |
| Ollama | "ollama:llama3.2" | native /api/chat | runs locally; OLLAMA_HOST, OLLAMA_API_KEY | if OLLAMA_ORIGINS allows the page |
| llama.cpp server | "llamacpp:default" | llama-server /v1/chat/completions | runs locally; LLAMACPP_BASE_URL, LLAMACPP_API_KEY | if the server allows the page |
| Llama API | "llamaapi:Llama-4-Maverick-17B-128E-Instruct-FP8" | native event stream | LLAMA_API_KEY, LLAMA_API_BASE_URL | depends on the API's CORS policy |
| LiteLLM proxy | "litellm:anthropic/claude-opus-5-5" | Chat Completions, routed by the proxy | runs locally; LITELLM_BASE_URL, LITELLM_API_KEY | if the proxy's CORS settings allow the page |
| Writer | "writer:palmyra-x5" | Chat Completions | WRITER_API_KEY, WRITER_BASE_URL | depends on the API's CORS policy |
| Vercel AI Gateway | "vercel:anthropic/claude-opus-5-5" | Chat Completions via the gateway | AI_GATEWAY_API_KEY, VERCEL_OIDC_TOKEN, AI_GATEWAY_BASE_URL | depends on the gateway's CORS policy |
| Amazon Bedrock | "bedrock:us.anthropic.claude-haiku-4-5-20251001-v1:0" | ConverseStream (binary event stream), Bearer API key or SigV4 | AWS_BEARER_TOKEN_BEDROCK, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN, AWS_REGION, AWS_DEFAULT_REGION, BEDROCK_BASE_URL | yes with a Bedrock API key (the runtime endpoint allows any origin, checked 2026-11-16); access keys work too, but keep long-lived keys out of web pages |
Every provider streams reply text to on text, passes tool calls and results, and reports token usage and
the stop reason (finished, tool call, cut off, refused) the same way, so maxTokens, typed replies and
A.try behave alike everywhere. Every one is plain fetch; only Claude in the CLI uses an SDK
(skip it with TIN_PROVIDER=fetch). Each is tested against recorded or scripted HTTP, and the local ones
against a small mock server; none of the tests calls a live model.
Claude gets two extras, in the CLI and the browser alike. Prompt caching is on: the system prompt,
the tools and each turn's conversation are marked for the cache, so a long agent run pays full price only for what is
new (TIN_PROMPT_CACHE=off turns it off). Extended thinking is off until
TIN_THINKING=adaptive or a budget such as TIN_THINKING=8000; thinking is sent back unchanged on
the next turn and never reaches on text. Gemini, the Responses API and Mistral's reasoning models also keep
their thinking out of the reply and pass it back as the provider requires.
OpenAI-compatible servers
openai: reaches anything that speaks Chat Completions. Point OPENAI_BASE_URL at it, up to and
including the version segment:
OPENAI_BASE_URL=http://localhost:8000/v1 tin run main.tin # vLLM, with model: "openai:Qwen/Qwen3-8B"
OPENAI_BASE_URL=https://api.groq.com/openai/v1 OPENAI_API_KEY=… tin run main.tin
OPENAI_BASE_URL=https://openrouter.ai/api/v1 OPENAI_API_KEY=… tin run main.tin
OPENAI_HEADERS='{"api-key":"…"}' OPENAI_BASE_URL=https://my-gateway/v1 tin run main.tin # a key in another header Away from OpenAI's own API the key is optional and the output limit is sent as max_tokens.
Local models
- Ollama:
ollama pull llama3.2, thenmodel: "ollama:llama3.2".OLLAMA_HOSTtakes Ollama's own forms (gpu-box,0.0.0.0:11434). - llama.cpp:
llama-server -m model.gguf --jinja, thenmodel: "llamacpp:default". Tool calls need--jinja; the error says so if it is missing. - LiteLLM:
litellm --config config.yaml, thenmodel: "litellm:<a model in its config>".
Amazon Bedrock
bedrock: speaks the ConverseStream API over fetch, with no SDK. Its binary event stream is decoded
in src/event-stream.ts, and both of each frame's checksums are checked, so a cut stream fails instead of losing
text. Name a model id or an inference profile id; most current models need the profile (us.,
eu., global. …):
AWS_BEARER_TOKEN_BEDROCK=… AWS_REGION=us-west-2 tin run main.tin # model: "bedrock:us.anthropic.claude-haiku-4-5-20251001-v1:0"
AWS_ACCESS_KEY_ID=… AWS_SECRET_ACCESS_KEY=… AWS_SESSION_TOKEN=… tin run main.tin
eval "$(aws configure export-credentials --format env)" && tin run main.tin # an SSO or profile login - Keys: a Bedrock API key (
AWS_BEARER_TOKEN_BEDROCK) is used first. Otherwise access keys are signed with SigV4 (src/aws-sign.ts, WebCrypto). Tin reads only these variables, not~/.awsprofiles; export a profile's keys as above. - Region:
AWS_REGION, thenAWS_DEFAULT_REGION, then the region in an ARN model id, thenus-west-2.BEDROCK_BASE_URLreplaces the endpoint (a VPC endpoint or a proxy). - Prompt caching: Claude models get cache points on the system prompt, the tools and the latest
user turn. Nova Pro, Lite, Micro and 2 Lite get them only after text, which is the only place Nova accepts them.
Other models get none.
TIN_PROMPT_CACHE=offturns it off.TIN_THINKINGworks for Claude models. - Output limit: if a model says the requested
maxTokensis over its limit (Nova Lite allows 10000), the call is sent once more at that limit, and later calls use it straight away. - Errors say what to do: an unknown model id, a model that needs an inference profile, a model your account has not enabled, a refused key, and throttling (retried three times first).
Checked live on 2026-11-16 with a Bedrock API key in us-west-2: the tool-using agent program on Claude Haiku 4.5 and Nova Lite, a cache write then a cache read on both, and the error text for an unknown id and for a model called without its profile.
Fallback and routing
agent Helper { model: "openai:gpt-5.1 | anthropic:claude-opus-5-5 | ollama:llama3.2" }
agent Triage { model: "route(by: openai:gpt-5.1-mini; quick: ollama:llama3.2; deep: claude-opus-5-5)" } a | b moves to the next model when one fails (no key, rejected key, missing model, rate limited after
retries, unreachable), starting each call with the model that has failed least recently. Refusals and cut-off
replies are answers, not failures. route(…) asks the by model, once per question, which labelled
candidate should answer; if it fails or is unclear, the first candidate answers. "openai:gpt-5.1 | mock"
runs offline when there is no key.
When a call fails
A failed call is a panic that names the agent, or Err(AgentError) from A.try. The message names the
provider and the case, and says what to do:
Calc: no model key: set GEMINI_API_KEY (or GOOGLE_API_KEY), or run offline with `model: "mock"`
Calc: model call failed: openai did not accept the key for gpt-5.1 (401): Incorrect API key provided — check your API key
Calc: model call failed: ollama has no model 'llama3.2' — pull it with `ollama pull llama3.2`
Calc: model call failed: could not reach llama.cpp at http://localhost:8080/v1/chat/completions (fetch failed) — is llama-server running? … 429 and 5xx answers and network errors are retried twice, with a growing delay, before the call fails.
In the browser
The playground lets you pick any provider, paste a key and set a base URL. They stay in the
tab (in localStorage only if you tick "remember") and go only to that provider. A provider that does not
allow calls from web pages fails with "could not reach"; the CLI has no such limit. Bedrock's runtime endpoint allows
any origin, so a Bedrock API key works from the playground; pick another region with its endpoint as the base URL.
From JavaScript
envModels(env) from src/env-models.ts gives a host every provider, configured from the variable names
above, with routing; languageModels({ name: model }) from src/vercel-provider.ts takes any AI SDK model. Each
provider's options are in docs/HOST-API.md.
Trying them live
npm run smoke:bedrock runs the Bedrock checks above with your key (at most 20 calls, on Haiku and Nova
Lite). npm run smoke:providers runs one small agent program against every provider whose key (or local server) is set
in your environment, and skips the rest. It is not part of npm test, and it costs a few tokens per provider.