Manifest

Manifest functions as an open-source LLM Gateway, addressing the common issue of unreliable LLM API calls in production environments. It aims to prevent the 5% of LLM API calls that typically fail, which are often avoidable errors. The platform provides a robust infrastructure for managing and optimizing interactions with various LLM providers, ensuring that AI agents and applications maintain continuous operation.

How Manifest Works

The core functionality of Manifest includes an Autofix mechanism that repairs broken requests on the fly before they reach the agent, ensuring zero downtime. It also offers Model Fallbacks, allowing the system to automatically retry requests on alternative providers if one fails. For comprehensive oversight, Manifest provides Full Body Logs to inspect complete request and response bodies, alongside Cost Visualization to track spending by agent, key, and provider.

Key Use Cases

• AI Builders: Connect agents and harnesses to any LLM provider, customize routing, and maximize subscription plans.
• Teams & Startups: Manage inference consumption, track every request and token through a unified interface, and control AI application settings.
• Platforms & Providers: Facilitate smooth connections to models for users, preventing client errors that could impact user experience and revenue.

Manifest supports self-hosting, allowing users to run the LLM router on their own infrastructure or machines. It also integrates with local models like Ollama, LM Studio, and llama.cpp, enabling users to decide which data stays local and which goes to the cloud. Users can bring their own API keys for various providers, paying usage directly.