Manifold: Chaining Seven Proxy Layers Into a Single Transparent LLM Pipeline

Every LLM proxy tool you install — a redactor, a router, a rate limiter, a cache — spins its own HTTP server. Each one speaks the OpenAI or Anthropic API. Each one expects to be the last stop before the cloud. Wire three of them together and you’re hand-editing YAML files, praying the port numbers line up, and discovering at 2 AM that your rate limiter was sending traffic directly to api.anthropic.com because someone forgot to update the upstream.

We built Manifold to make this problem disappear. It’s a control plane and thin entry proxy that chains multiple LLM proxy services into a single transparent pipeline. Point your agent at http://127.0.0.1:9000 and every request flows through seven independent services before it ever touches a cloud API. No MCP tool registration. No manual config wiring. One port, one pipeline.

The pipeline, layer by layer#

Here’s what a request sees on its way through our production manifold config:

Agent → Veritas → LLM-Redactor → Lattice → Local-Splitter → Entropy Gate → Palisade → Hivemind → Cloud API

Each layer is an independent service with its own port, its own config, its own health check. Manifold starts them all, patches each one’s upstream to point to the next in the chain, monitors health, and rewires around failures. Here’s what each layer does:

1. Veritas — Implementation Fidelity#

LLMs cut corners. They skip steps, elide details, write placeholder comments instead of real code. Veritas sits at the front of the pipeline and inspects every response for corner-cutting patterns. When it catches one, it sends the request back with an enforcement directive — “no, really, implement this properly” — and the model gets a second chance. One retry, then pass through. It’s the quality gate that refuses to accept // TODO: implement as a deliverable.

2. LLM-Redactor — Privacy#

Every prompt you send to a cloud LLM is plaintext on someone else’s server. LLM-Redactor runs NER + regex detection locally, replaces PII and secrets with typed placeholders ([PERSON_7], [AWS_KEY_3]), forwards the sanitized prompt, and restores the real values in the response. The cloud model never sees your customer names, your API keys, or your internal codenames. We wrote about this in detail — the combined leak rate on PII drops to 0.6% with the full stack.

3. Lattice — Knowledge Graph Memory#

Lattice is a privacy-preserving knowledge graph that builds a model of you over time — your projects, preferences, rejected approaches, security findings — without storing PII. In the pipeline, it enriches prompts with relevant context from your graph and records patterns from the conversation. Every 60 minutes it runs an auto-reflection pass, consolidating new observations into the graph. The model gets smarter about you without you repeating yourself.

4. Local-Splitter — Cost Optimization#

Not every request needs a frontier model. “What does this function return?” doesn’t need Claude Opus. Local-Splitter uses a 3B parameter model running locally on Ollama to triage every request: trivial ones get answered locally (zero cloud tokens), complex ones get their prompts compressed before forwarding. We measured this too — routing plus compression saves 56% of cloud tokens on average, with the local model handling the easy stuff and the cloud handling what matters.

5. Entropy Gate — Information Density Filtering#

Some prompts are noise. Entropy Gate measures the information density of each request and response, building a memory of what “normal” looks like for your workload. Low-signal requests get flagged, compressed, or routed differently. It’s the layer that asks “is this request actually worth the tokens it’ll cost?”

6. Palisade — Query Reformulation#

Cloud models have safety filters. Sometimes those filters block legitimate security research. Palisade reformulates and encodes queries — binary, base64, morse, compression — so the cloud model processes the content without the surface-level filter triggering. The response decodes back transparently. It’s a research tool, not an evasion tool: the model still sees the content, just in a form its safety classifier doesn’t pattern-match.

7. Hivemind — Admission Control#

The last stop before the cloud. Hivemind manages concurrency with AIMD (additive increase, multiplicative decrease), enforces rate limits, applies backpressure when the cloud is slow, and retries with exponential backoff. Our production config runs 120–240 concurrent connections with a 12-second latency target, 12 max retries, and a 60-second retry window. It’s the traffic controller that keeps you from getting rate-limited or billed into oblivion.

One pipeline, four cloud models#

Here’s where it gets interesting. We run the same seven-layer pipeline against four different cloud backends, each with its own manifold config:

The only difference between these configs is the fallback_upstream URL. Every service, every port, every layer stays identical. Switching models is a config file change:

# Claude today
manifold up -c manifold-anthropic-new.yaml

# DeepSeek tomorrow
manifold up -c manifold.yaml

# Z.AI for experimental workloads
manifold up -c manifold-zai.yaml

The agent doesn’t know or care which cloud model is on the other end. It talks to http://127.0.0.1:9000 and the pipeline handles the rest.

The Buzz setup: Red Team, Blue Team, Purple Team#

We took this a step further and configured a set of agents on Buzz — each one routed through the Manifold pipeline to a different model backend. The agents are organized into three teams that communicate with each other:

🔴 Red Team — “The Breach Boys” 🚪💥

• Havoc — “I don’t knock. I test the locks.”
• Breach — “First one through the door.”
• Ghostwire — “In and out. Never a trace.”
• Blackout — “You can’t defend what you can’t see coming.”
• Cipher — “Every secret has a shelf life.”

🔵 Blue Team — “The Deadbolt Mob” 🔒🛡️

• Sentinel — “Eyes on every wire, all night.”
• Warden — “Nobody gets a key they didn’t earn.”
• Kevlar — “Nothing gets through what isn’t there.”
• Ironclad — “When it breaks, I’m already moving.”
• Vault — “Locked down. Sealed tight. Sleep easy.”

🟣 Purple Team — “The Handshake” 🤝♻️

• Arbiter — “Two crews, one scoreboard.”

Each agent is backed by a different model (Claude, DeepSeek, Kimi, Zai) through the same Manifold pipeline. The Red Team probes and attacks, the Blue Team defends and hardens, and the Purple Team adjudicates. They talk to each other on Buzz in real time — Red finds a gap, Blue patches it, Arbiter scores the exchange.

The screen below shows them in action — a live conversation between all three teams, with each agent’s model backend transparently routed through the seven-layer Manifold pipeline:

Why this matters#

The LLM infrastructure space is fragmenting fast. Every week there’s a new proxy tool that does one thing well — redact, route, cache, rate-limit, validate. Each one wants to be the middleware. Each one adds another port, another config file, another thing to remember to start.

Manifold says: let them all be middleware. Chain them. Stack them. Switch the cloud model without changing anything else. The pipeline is the product.

And when you put agents on top of that pipeline — agents that talk to each other, argue, collaborate, compete — you get something that’s more than the sum of its layers. You get a Red Team that actually finds things, a Blue Team that actually fixes them, and a Purple Team that keeps score. All through one port.

Manifold is open source. uv sync && manifold up and you’re running.