Skip to content

Send Half of Your Claude Code Traffic to a Local Model

Hybrid Claude Code: Route the Cheap Half Locally

How to split Claude Code between the hosted API and a local Docker model, with the version traps that every older guide still gets wrong.

· · 9 min read

Quick Take

Going fully local loses on speed, quality and even cost. Splitting the work does not. Claude Code stamps a model tier on every request it sends, a router can rewrite that stamp, and the boring tier can land on the model already sitting on your disk. Here is the wiring, installed and screenshotted rather than copied from a README.

TL;DR: Install claude-code-router on Node 22, add your local endpoint as a custom provider, and write one Global Routing rule: if request.body.model contains haiku, rewrite it to ai/gpt-oss. That is the entire split. Two things every older guide still gets wrong: version 3 has no config.json and no ccr code command, and the Claude Code login it offers to import cannot authenticate, so the hosted half needs a real API key.

The previous article ended with a working local setup and a recommendation not to use it as your only model. Full local loses on wall-clock, on multi-file reasoning, and once you buy hardware that holds a decent model, on money too.

None of which means the local model should sit idle. Claude Code does not send one kind of request. It sends the hard ones you asked for, and a steady trickle of housekeeping around them, and only the first kind justifies a frontier model.

I installed the router to write this. Half of what I had planned to write turned out to be wrong, which is the useful part.

What the router actually decides

The thing to understand before touching anything: you do not classify tasks. Claude Code already stamps each request with a model tier, and the router matches on that stamp. Cheap housekeeping goes out marked for the small model; your actual work goes out marked for the big one.

A rule is a condition plus a rewrite. Match on something in the request, change something in the request, forward it on.

Claude Code Router rule editor showing a condition on request.body model containing haiku and a rewrite setting request.body.model to ai/gpt-oss
The whole hybrid setup is this one rule.

Notice what this is not. Nothing inspects your prompt and decides it looks easy. A short question about a subtle race condition carries the normal tier stamp like everything else, which is correct, and a long mechanical summary carries the cheap one, which is also correct, but neither decision involved judgement about the work.

Step 0: the version traps

Deal with these first, because both produce errors that point somewhere else.

Node 20 is not enough. The router needs 22 or newer. My system Node is 20.20.0 and ccr was unusable until I switched:

nvm use 22
npm install -g @musistudio/claude-code-router

Version 3 threw away the config file. Every guide written before it, including the ones near the top of the search results, tells you to edit ~/.claude-code-router/config.json with a Providers array and a Router block holding default, background, think and longContext. That file does not exist any more. Version 3.0.22 keeps its configuration in config.sqlite and expects you to use a web UI. The README is explicit that you should not edit the live SQLite yourself.

The CLI changed with it. There is no ccr code:

Usage:
  ccr start [--host <host>] [--port <port>] [--open|--no-open] [--gateway|--no-gateway]
  ccr ui [--host <host>] [--port <port>]
  ccr stop
  ccr <profile-name-or-id> [cli|app] [-- <agent args>]

You launch a named profile, not a fixed subcommand.

Step 1: check the local side before adding a hop

A router in front of a broken endpoint produces errors that blame the router.

docker model ps
curl -s http://localhost:12435/anthropic/v1/messages \
  -H "Content-Type: application/json" \
  -d '{"model":"ai/gpt-oss","max_tokens":16,"messages":[{"role":"user","content":"Reply with ready."}]}'

Port 12435 is the schema-stripping proxy from the previous article, not Model Runner's own 12434. Point the router at 12434 and the local branch will 400 on the first tool-bearing request with failed to parse grammar, for exactly the reasons bisected there.

If the model is not resident, load it with the window it needs and a generous keep-alive, because routed requests arrive sporadically and a model that unloads between them turns every one into a cold start:

docker model configure --context-size 65536 --keep-alive 30m ai/gpt-oss

Step 2: start the service

ccr start --no-open
CCR service started at http://127.0.0.1:3458/?ccr_web_token=<token> (pid 49437).

The gateway that Claude Code will talk to runs on 127.0.0.1:3456; the management UI is the 3458 URL above. That token is a credential. Do not paste the URL into a ticket, and keep both listeners on localhost.

Step 3: the import that looks like it solves authentication

The first-run wizard scans your machine and offers this:

Claude Code login detected. Click Import to add it as a gateway provider.

It is the most inviting button on the screen and it does not work. The import succeeds, the provider appears, and then the account check fails:

Claude Code Router provider list showing the imported Claude Code API provider failing with HTTP 401 and a working Local DMR provider on localhost port 12435
The imported subscription login returns 401. The local provider is fine.
Account endpoint returned HTTP 401: OAuth access token has expired.
Re-authenticate to continue.

Claude Code authenticates a subscription with a short-lived OAuth token that it refreshes itself. The router copies the token and has no way to refresh it. So the hosted half of a hybrid setup needs a real Anthropic API key, billed separately from a Max subscription. That is a genuine cost the routing guides do not mention: you are not splitting a subscription, you are adding metered API spend and hoping the local branch offsets it.

Step 4: add the local provider, the right way

In Providers, click Add. The preset list offers Anthropic, and picking it feels right because the local proxy speaks the Anthropic messages protocol. It is a trap: a preset hard-codes its vendor URL and the form shows no endpoint field at all. I saved a provider called Local DMR that was quietly pointing at api.anthropic.com.

Choose Other / custom API endpoint instead. The endpoint field appears:

Provider edit dialog with the custom endpoint preset selected and the API endpoint field set to http://localhost:12435
Only the custom preset exposes an endpoint field.

Set the endpoint to http://localhost:12435, give it any placeholder API key (nothing checks it locally, but an empty value fails validation), and add ai/gpt-oss with Custom model, because the runner returns no model list to enumerate.

Protocol detection then runs against the proxy and reports Anthropic Messages available alongside both OpenAI dialects. No transformer needed, which is the payoff for Model Runner carrying three dialects on one port.

Step 5: write the rule

Open Global Routing and add a rule. Mine:

FieldValue
Condition sourcerequest.body
Pathmodel
Operatorcontains
Valuehaiku
RewriteSet request.body.model = ai/gpt-oss
Global Routing list showing one enabled rule named haiku to local with its condition and rewrite summarised
Rules are ordered and individually toggleable, so you can disable the split without unpicking anything.

The condition can also read request.header or request.auth, and the rewrite can delete keys or push into arrays, so the same engine covers considerably more than a model swap. For hybrid routing you need exactly one rule.

Leave On failure off while you are testing. Retry and fallback chains hide a broken local branch behind a silent redirect to the hosted one, and then you cannot tell whether the split is working.

Step 6: a profile that does not hijack everything

In Agent Profiles, create a Claude Code profile. The field that matters is effect scope: set it to Only opened from CCR, not System default. System default rewires every claude invocation on the machine, which is exactly the mistake the previous article warned about with a stray ANTHROPIC_BASE_URL.

Agent profile card named hybrid-local tagged Claude Code, Only opened from CCR, CLI and APP
Scope it to CCR so a plain `claude` still behaves normally.

Then launch by profile name:

ccr hybrid-local

Also unset the variable from the previous article, or it wins and everything goes local:

unset ANTHROPIC_BASE_URL

Step 7: prove the split is real

This is the step people skip, and then they run for a month convinced they are saving money.

The router's Overview page counts requests, tokens and cost per provider, which is the easiest instrument you will get:

Claude Code Router overview dashboard with per-provider request, token and cost widgets
Provider Analysis is where the split shows up, or does not.

Cross-check it from the other end, so you are not trusting one counter:

docker model requests --follow --model ai/gpt-oss | jq -c '{in: .usage.input_tokens}'

No lines at all means the split is not happening, and there are only three plausible reasons: ANTHROPIC_BASE_URL is still set, the rule's rewrite value does not exactly match a model you declared on the local provider, or the session produced no cheap-tier traffic, which happens in short sessions.

What I cannot give you is my own ratio. The hosted branch never authenticated, for the reason in step 3, so I have a verified local branch and a verified rule but no live session split. Anyone quoting you a percentage without saying how they counted is quoting a README. Count your own.

The parts that do not improve

Routing changes where requests go, not what happens when they arrive, so every measurement from the previous article still holds on the local branch.

The 35,482-token handshake still ships. A routed request carries the same system prompt and the same tool schemas as any other, so a 64k window still leaves about 30k of usable room, and /clear still does not give the handshake back.

The speed is unchanged. My laptop answered a one-file question in 4 minutes 45 seconds, and a routed request on the same hardware takes the same kind of time. In a hybrid setup that appears as a stall in the middle of a session that was otherwise fast, which is arguably worse than a slow session you had braced for. It is the argument for a real GPU rather than a laptop.

Should you bother

Honestly, it depends on two numbers you do not have yet: your own cheap-tier share, and what the hosted branch costs you at API rates rather than subscription rates. The second one is the awkward part. If you are on a Max plan today, hybrid routing does not reduce that bill, it opens a second one.

Measure before committing. Step 7 costs half an hour and answers it properly, which beats reasoning from someone else's 21x claim.

For choosing which local model to sit behind the rule, the comparison of local coding models applies, though the criteria shift: summarisation quality matters more than agentic editing, so the smaller and faster options look better here than they do as a primary model.

Rolling it back

ccr stop
npm uninstall -g @musistudio/claude-code-router
rm -rf ~/.claude-code-router

Then run claude directly again. Nothing on the Docker side needs undoing unless you want the disk space back, and the local model stays useful for the offline case whether or not you keep the router.

Frequently Asked Questions

Which requests can be routed to the local model?
The ones Claude Code stamps as its cheapest tier, which it uses for housekeeping rather than for the work you asked for: summarising a tool result, condensing conversation history, and similar filler between steps. You do not classify anything by hand. A routing rule matches on the model name in the request body and rewrites it, so the split is a string match on a tier, not a judgement about difficulty.
Can the router reuse my Claude Code subscription instead of an API key?
No, and it looks like it can, which is the trap. The setup wizard scans your machine, finds the Claude Code login and offers to import it as a provider. Importing works, and then every request against it fails with HTTP 401 and OAuth access token has expired. That credential is a short-lived OAuth token the router cannot refresh, so the hosted half of a hybrid setup needs a real Anthropic API key with its own billing.
Do I still need the schema-stripping proxy?
Yes, for the local branch. The pattern and maxLength keywords break llama.cpp's grammar compilation regardless of what sits in front of it, so the provider has to point at the proxy on port 12435 rather than at Model Runner's own 12434. Chain them: Claude Code to the router on 3456, router to the proxy on 12435, proxy to Model Runner on 12434.
Why does the provider form not let me change the endpoint?
Because a preset provider hard-codes its vendor URL and hides the field entirely. Picking Anthropic to get the right protocol looks correct and leaves you pointing at api.anthropic.com with no way to change it. Choose Other slash custom API endpoint instead, then type the local URL; protocol detection runs against whatever you enter, and it found Anthropic Messages on the local proxy without any transformer configured.
Is the router a security risk on a shared machine?
It stores upstream provider credentials in its own local data directory and its management URL carries an auth token in the query string, so treat both like passwords and keep the listener bound to 127.0.0.1. If several people need routing, that is the case for LiteLLM instead, which was built for team use and has authentication and audit logging.
Does routing fix the slow local model problem?
It contains it rather than fixing it. A routed request still runs at local speed, so if your machine answers in four minutes, four minutes appear in the middle of an otherwise fast session. Hybrid routing only makes sense on hardware where the local branch answers in seconds, which in practice means a real GPU rather than a laptop.