Misceo

English · 简体中文

[!IMPORTANT] This is the official public documentation, issue tracker, and release home for Misceo. The product is distributed as prebuilt binaries through npm. Source code is not included in this repository.

Cut AI-agent costs without breaking conversations or tool loops.

Misceo is a local Anthropic-compatible gateway. It routes eligible requests through a lower-cost model, checks the completed candidate, and escalates to a stronger model when policy requires it.

Unlike a request-only router, Misceo also protects agent state: session continuity, cross-model transcript compatibility, and ownership of active tool loops.

Why Misceo

Strong-only routing is simple but pays strong-model cost for every request. Cheap-only routing lowers spend but cannot recover when the answer is empty, malformed, or inadequate.

A static router decides before generation. Misceo can also inspect the first completed candidate before choosing which answer the client receives.

Strategy	Decision point	Trade-off
Strong-only	Before generation	Consistent quality posture; strong-model cost on every request.
Cheap-only	Before generation	Lowest routing overhead; no automatic recovery.
Static router	Before generation	Uses request metadata but cannot inspect the generated answer.
Misceo	Before and after the first attempt	Accepts a checked candidate or regenerates with a stronger model.

Misceo does not make a cheaper model equivalent to a stronger model. It makes the cost-versus-quality policy explicit, observable, and configurable.

How a request flows

flowchart LR
    A["Anthropic-compatible client"] --> B["Misceo local gateway"]
    B --> C{"Protected route?"}
    C -->|"Yes"| S["Strong backend"]
    C -->|"No"| L["Lower-cost attempt"]
    L --> G{"Structural gate"}
    G -->|"Fail"| S
    G -->|"Pass"| J{"Judge policy"}
    J -->|"Accept or skip"| R["Return candidate"]
    J -->|"Reject or fail closed"| S
    S --> R2["Return regenerated answer"]
    B --> O["Local history and dashboard"]

1. Protected routes, open tool loops, active pins, and explicit rules run before the cascade.
2. Eligible traffic first reaches the configured lower-cost backend.
3. A structural gate rejects upstream failures, empty output, and malformed tool calls.
4. Depending on the selected mode, an optional judge scores the visible candidate against the latest user turn.
5. A passing candidate is returned. A rejected candidate is discarded and a stronger backend generates the served answer.
6. Escalated conversations remain on the stronger backend for a configurable number of fresh turns.

See Routing and handoffs for decision precedence, session identity, tool-loop protection, and mode-specific failure behavior.

Agent-safe routing

Conversation continuity

Misceo groups requests by structured session identity when the client provides it. Bootstrap, title-generation, and visible requests from one launch remain part of the same conversation.

Cross-model handoff

Provider-specific thinking signatures, cache markers, model names, and message invariants are handled at model-family boundaries. The original local audit record remains available for inspection.

Tool-loop ownership

When a model opens a tool loop, that backend keeps ownership until the loop is complete. Misceo does not move an active tool_use / tool_result exchange to another model family halfway through.

First-visible-reply protection

For eligible structured sessions, known launch bootstrap traffic is separated from the user's first visible request. The first visible reply uses the strong backend before later requests enter normal routing.

Where data goes

The proxy, embedded dashboard, configuration, and traffic logs run locally. Local does not mean offline: inference requests go to the providers selected by your routing policy.

For an eligible cascade, the first provider receives the request. If the candidate is rejected, the escalation provider also receives the request. A configured judge receives the capped latest user turn, visible candidate, and tool names.

Read Privacy and data before using Misceo with sensitive prompts or exposing either listener beyond loopback.

Highlights

• One Anthropic-compatible endpoint for Claude, GLM, and custom backends.
• Three explicit postures: quality-first, balanced, and savings-first.
• Structural validation plus an optional LLM judge.
• Session-aware model-family handoffs and tool-loop protection.
• Embedded dashboard with live traffic, history, cost, and deletion controls.
• Observable serving backend, judge result, latency, usage, and route reason.
• Manual backend handoff from the CLI or dashboard.

Requirements

• Node.js 18 or newer and npm for installation and the npm launcher.
• Provider credentials for the backends you enable.
• No separate Bun, Docker, or UI installation.

Published platform packages currently cover:

Operating system	Architectures
macOS	arm64, x64
Linux	glibc arm64, glibc x64
Windows	x64

See Compatibility for client and platform boundaries.

Start in five minutes

Install the CLI:

npm install --global @misceo/cli

Do not install with --omit=optional; the platform executable is delivered as an optional package.

Create a private configuration:

mkdir my-misceo
cd my-misceo
misceo init --mode balanced

Add the provider credentials you use to .env:

ANTHROPIC_API_KEY=your-anthropic-key
ZAI_API_KEY=your-zai-key

Validate and start:

misceo doctor
misceo start

doctor performs static checks. It does not contact providers, validate live credentials, or prove model availability.

Connect Claude Code on macOS or Linux:

ANTHROPIC_BASE_URL=http://127.0.0.1:4141 \
ANTHROPIC_AUTH_TOKEN=misceo \
claude

For Windows PowerShell and a complete verification procedure, read Getting started.

Default local addresses:

• Proxy: http://127.0.0.1:4141
• Dashboard: http://127.0.0.1:5141

Verify routing in 60 seconds

Send a normal prompt in Claude Code, followed by a second small request. In an eligible structured session, the first visible response is protected and a later request can exercise the cascade.

Open the dashboard and inspect Live traffic. Confirm the serving backend, route or cascade note, status, latency, and judge score when a judge ran.

In another terminal:

curl http://127.0.0.1:4141/healthz
misceo status

A healthy response, a traffic row, and a visible session confirm that the client is using Misceo instead of contacting a provider directly.

Routing modes

Mode	Judge behavior	If the judge is unavailable
quality-first	Required	Escalate to the strong backend.
balanced	Used when configured	Serve a structurally valid lower-cost candidate. This is the default.
savings-first	Skipped	Continue from the structural gate without a judge call.

Select a mode during initialization:

misceo init --mode quality-first
misceo init --mode balanced
misceo init --mode savings-first

Dashboard

The management UI is embedded in the installed executable. It includes:

• live routing state, sessions, handoffs, and pins;
• traffic detail with request, response, judge, status, and latency;
• token and cost fields with contextual explanations;
• searchable, paginated conversation history;
• deletion for one conversation or all recorded history.

The dashboard has no user login. Keep it on loopback unless trusted authentication and network controls are placed in front of it.

Read Dashboard for listener options and data controls.

Evidence and limitations

Misceo does not publish a universal savings or quality percentage. Results vary with tasks, models, providers, prices, region, and sampling behavior.

The public benchmark contract compares four policies on the same versioned task set:

1. strong-only
2. cheap-only
3. cheap-first + structural gate
4. cheap-first + judge

Reports must include task success, quality loss, cost, latency, escalation rate, judge error rates, and handoff retention. No public result is included until the corresponding task set and raw report can be published together.

See Benchmark methodology.

CLI quick reference

Command	Purpose
misceo init	Create a private starter .env.
misceo doctor	Perform static configuration checks.
misceo start	Start the proxy and embedded dashboard.
misceo status	Show current routing and session state.
misceo handoff BACKEND	Switch the active backend.
misceo cost	Report token cost and configured savings.
misceo audit	Run explicit paid compatibility probes.
misceo logs	Prune, reindex, or forget local history.

Detailed flags and configuration examples are in Getting started and Configuration.

Documentation

Guide	Purpose
Getting started	Install, configure, connect, verify, update, and uninstall.
Routing and handoffs	Cascade decisions, sessions, judges, and tool loops.
Configuration	Environment variables, endpoints, pricing, and listeners.
Compatibility	Supported platforms, clients, and protocol boundaries.
Dashboard	UI behavior, history, cost fields, and exposure safety.
Privacy and data	Provider flow, local logs, retention, and deletion.
Troubleshooting	Common installation, endpoint, port, and history problems.
Benchmark methodology	Reproducibility rules, metrics, and claim boundaries.

Source availability, license, and support

This repository does not contain Misceo source code. Published binaries are licensed under the Functional Source License 1.1 with the Apache 2.0 Future License. Review the license before redistribution or competing use.

Use GitHub Issues for reproducible bugs and documentation problems. Read SUPPORT.md before filing.

Report security vulnerabilities privately as described in SECURITY.md.