For the complete documentation index, see llms.txt. Markdown versions of all docs pages are available by appending .md to any docs URL.
Providers
Configure agentgateway for first-class provider shortcuts and OpenAI-compatible fallbacks.
Learn how to configure agentgateway for a particular LLM.
First-class providers
Prefer the dedicated provider pages when the AgentgatewayBackend API has a first-class provider type for the upstream, with built-in host, path, and request-format defaults.
OpenAI-compatible providers
Many providers, such as Cohere, DeepSeek, Groq, Mistral, Together AI, and xAI, expose the OpenAI Chat Completions API but do not have a first-class type in the AgentgatewayBackend API. Configure these with the ai.provider.openai shape. For the list of supported hosts and paths, see OpenAI-compatible providers.
Custom providers
Use custom providers for unsupported, non-standard, or self-hosted targets when you need to declare formats, upstream paths, or backend targets explicitly.
Amazon Bedrock
Configure Amazon Bedrock as an LLM provider for agentgateway.
Anthropic
Configure Anthropic Claude as an LLM provider for agentgateway.
Azure
Configuration and setup for Azure AI services provider
Gemini
Configure Google Gemini as an LLM provider for agentgateway.
OpenAI
Configure OpenAI as an LLM provider for agentgateway.
OpenAI-compatible providers
Configure providers without built-in support that expose the OpenAI API format, such as Perplexity.
Vertex AI
Configure Google Vertex AI as an LLM provider for agentgateway.
Custom providers
Configure unsupported, self-hosted, and non-standard LLM providers with explicit API formats, paths, …
Ollama
Configure agentgateway to route LLM traffic to Ollama for local model inference
vLLM
Configure agentgateway to route traffic to vLLM for self-hosted model inference
Multiple endpoints
Configure access to multiple OpenAI API endpoints like chat completions, embeddings, and models …
Mock LLM with httpbun
Set up httpbun as a mock OpenAI-compatible LLM backend for testing without API keys.
