Models, Skills, Agents and DLP.
One governed graph.
Orch Engine runs open-source LLMs, MCP tool servers and the agents you write, on your own Kubernetes or a single laptop. It wires them together, waits until every dependency is ready, and puts a DLP proxy in front of every model call.
The real dashboard, playing a recorded session. Nothing runs behind this page. Use ◀ ▶ or click any node.
Four building blocks, all plain Kubernetes resources
Model
One command pulls an open model from Hugging Face and serves it with an OpenAI-compatible API.
Skill
Any MCP server you package as a container becomes a discoverable, health-checked tool.
Agent
Your code, any framework. Endpoints are injected for you, and it never starts half-wired.
DLP
A reusable policy, enforced by a sidecar. It scans prompts and answers, streams included. Redact, block or audit.
Owner-approved sharing
An agent can use another team's model only after the owner runs bairav grant. Until then it waits, and tells you the exact command to ask for.
See everything
bairav dashboard is a local, read-only console: live graph, waiting reasons, grants and every DLP decision. No server, no login: it uses your kubeconfig.
Others own one layer. Orch Engine governs the graph.
Model servers make tokens fast, gateways route calls, frameworks help you write agents. Orch Engine is the control plane that runs all of it on your infrastructure, and puts governance on the connections between them.
| Model serversvLLM, KServe, Ollama | LLM gatewaysLiteLLM, Portkey | Agent frameworksLangGraph, CrewAI | Orch EngineBAIRAVS | |
|---|---|---|---|---|
| Serve open models on your own hardware | ✓Their core job | —Routes to servers | — | ✓llama.cpp today |
| Route to OpenAI / Anthropic | — | ✓Their core job | ✓Via SDKs | ✓Behind the DLP gateway |
| Run MCP tools as managed services | — | — | ◐In-process | ✓Skill CRD |
| Deploy agents with dependencies wired | — | — | ◐You deploy them | ✓Waits until ready |
| DLP on prompts and answers, incl. streams | — | ◐Plugins / guardrails | — | ✓Built in |
| Owner-approved cross-team sharing | — | ◐Keys and budgets | — | ✓AccessGrant |
| Kubernetes-native resources | ◐KServe: yes | ◐Runs as a proxy | — | ✓Five CRDs |
| Maximum GPU throughput | ✓vLLM excels | — | — | —Not our focus |
✓ built for it · ◐ partly or with add-ons · — not its job. By category: individual products vary. Orch Engine runs alongside them, and uses llama.cpp (and later vLLM) underneath.
A governed agent in five commands
Each step is a normal Kubernetes resource you can see with kubectl and keep in Git. No cluster yet? Drop --target k8s and the same commands run on Docker.
- Agents wait for their dependencies instead of crashing
- Cross-team access is explicit, and so is revoking it
- DLP is configured once, as a policy, not in agent code
# a shared model for the company $ bairav model deploy Qwen/Qwen2.5-0.5B-Instruct-GGUF \ --target k8s --name qwen -n shared-models # a team's MCP tool $ bairav skill deploy bairav/mcp-echo:v1 --target k8s -n team-a # their agent: it waits for access $ bairav agent deploy ghcr.io/team-a/bot:v1 --target k8s -n team-a \ --model shared-models/qwen --dlp-policy prod-guardrails # the owner approves; the agent binds in seconds $ bairav grant team-a --model qwen -n shared-models ✓ team-a may now use Model/qwen $ bairav dashboard
From source, in a few minutes
You need Go 1.22+ and Docker; add k3d for a local Kubernetes. Prebuilt binaries and brew install bairav are on the way.
$ git clone https://github.com/bairav-foundation/orch-engine $ cd orch-engine $ make rebuild-all $ make dev $ bin/bairav dashboard --demo
Alpha, and tested end-to-end
Works today
- Models, Skills and Agents with dependency waiting
- DLPPolicy with request, response and streaming redaction
- AccessGrants for cross-team sharing
- Docker and Kubernetes targets, same CLI
- Local dashboard with a demo mode
Every feature passes a smoke test against a real cluster before release.
Not yet
- Prebuilt binaries and Homebrew
- Per-team rate limits on shared models
- DLP on Skill traffic
- A shared, in-cluster dashboard
Follow the releases on GitHub.