Mantis | Mantis Skip to content SELF_HOSTED: TRUECLOUD_PROVIDER: AWSAPI_STYLE: OPENAI<br>Mantis

A self-hosted LLM gateway for routing, caching, guardrails, and observability across model providers.<br>Case Study &rarr; Get Started

01 // UNIFIED_API<br>One Interface<br>A stable chat completions endpoint that sits in front of multiple model targets. Switch providers without changing a single line of client code.<br>OPENAICLAUDE+MORE

02 // INFRASTRUCTURE<br>AWS-Native<br>Built for the AWS ecosystem. Deploys via Terraform to ECS, using ElastiCache for fast caching and Bedrock for foundational models.<br>ECSCLOUDWATCHBEDROCK

03 // LOGIC_ENGINE<br>Policy Driven<br>Routing, retry, and fallback logic. Define timeouts, cooldown periods, and cache behavior directly in your config.<br>ROUTINGRETRYCACHING

// Capabilities<br>01<br>Configurable Routing<br>Route requests by metadata, model aliases, weighted targets, and fallback chains.<br>multi-providerweighted routingfallback chains

Configurable routing: stepped medallion with a pulsing core node

02<br>Gateway Orchestration<br>Coordinate validation, cache checks, cooldowns, provider calls, retries, and terminal responses.<br>lifecycle controlshort circuitretry logic

Gateway orchestration: square wave field in a diamond aperture with one packet on the centre line

03<br>OpenAI-style API Surface<br>Send chat completion requests through a single gateway endpoint with optional routing metadata.<br>streaming supportunified schemasingle endpoint

OpenAI style API surface: split circle with paired registers and a packet on the axis

04<br>Python SDK<br>Call Mantis from application code without manually constructing each HTTP request.<br>pip installableasync clientrouting metadata

Python SDK: call grid with one active cell, routed through by a packet

// Team

Hubert de Parseval Europe

Rey van den Berg Dublin, Ireland

Riz Syed London, UK

Samuel Meka Taipei, Taiwan

// Next Steps<br>Route. Cache. Guard. Observe.<br>Read Case Study→View on GitHub