TrueFoundry (US) is an AI gateway that sits between applications and model providers. The company describes the product as the proxy layer that also brokers MCP servers, putting LLMs, tools and agents under the same policies.
What it does
- Exposes an OpenAI-compatible endpoint for different providers
- Creates virtual models that group versions and providers of the same model
- Routes by weight, latency, priority or adaptively by cost and availability
- Fails over and retries automatically when a provider breaks
- Sets budgets, spend limits and cost attribution per team or project
- Applies built-in, external or bring-your-own guardrails
- Manages virtual keys, role-based access and rate limits
- Offers MCP and agent gateways to expose tools with policy
How it works
- Applications point at the gateway endpoint, which resolves the provider
- The gateway runs in the vendor cloud or inside the customer infrastructure
- Metrics and request logs are exported to observability tooling
- Semantic caching and batch processing cut repeated calls
- Access and spend policies apply before the call leaves
Models and providers
- A declared catalog of more than a thousand LLMs
- Providers such as OpenAI, Anthropic, Gemini, Vertex AI, Bedrock, Azure OpenAI, Mistral, Cohere, Groq, Together, DeepInfra, SambaNova and Cerebras
- Self-hosted models join as their own provider
Availability and license
- Proprietary product, with a limited free plan and per-user paid plans
- Managed deployment or inside the customer infrastructure
- The gateway is not open source; adjacent company projects are
Points to note
- Not open source, which limits auditability and unconstrained self-hosting
- The free plan caps users, model accounts and MCP servers
- Overage billing means watching consumption