Open-Weight Models That Can Actually Drive a Harness

Not another leaderboard

Open-weight models get re-ranked every week — on chat arenas, on code-completion suites, on whatever index is trending. Almost nobody ranks them on the question that decides whether one belongs in your stack: can it hold the engine slot of an agent harness? Meaning: will it still be calling tools correctly on step 37, ninety minutes into a refactor, with 200K tokens of accumulated context leaning on its instruction-following. Chat quality and agent quality are different properties, and the gap between them is where most “I tried a local model and it was useless” stories are born.

You have seen the failure mode. A model writes lovely prose about your bug, then emits tool-call JSON with a trailing comma, patches the wrong file, forgets the system-prompt rule it obeyed twenty steps earlier, and — when the shell hands back an error — apologizes warmly and does the same thing again. That model may sit above half the frontier on a chat leaderboard. It cannot drive a harness.

So this is the agent-first tour of the open-weight field as of August 2026: agent-fitness defined and made measurable, profiles of the families that actually have it — DeepSeek V4, Qwen3-Coder-Next, Kimi K3, GLM-5.2, gpt-oss, and the Western bench — a method for reading agentic benchmarks without being fooled, and then the concrete part: serving, hardware tiers, and wiring these engines into the harness you already run.

Agent-fitness: what open-weight models must prove

Agent-fitness is a model’s capacity to run reliably inside an agent loop: emitting schema-valid tool calls with sane arguments thousands of times without drift, obeying system instructions as context accumulates, producing diffs that apply cleanly to the right files, and recovering from its own errors by reading them and changing course. It is not chat quality, and chat scores do not predict it.

Four properties, and each one breaks engines on its own:

1. Structured tool calling. Valid schema, the right tool, plausible arguments — at call 1,400, not just call 4. Open-source LLM tool calling is measurable, and it varies across families far more than chat scores do .
2. Instruction adherence over long context. The system prompt’s “never push to main” has to bind at 200K tokens exactly as it did at 2K.
3. Diff and patch discipline. Search-and-replace blocks that match, unified diffs that apply, no phantom files.
4. Recovery. Reading a traceback or a failed patch apply, forming a different plan, refusing to loop.

Chat quality predicts none of this because helpfulness tuning rewards none of it. Agentic post-training — reinforcement learning over long tool-use trajectories, error-recovery episodes, real repository tasks — is a separate and expensive investment, and not every lab made it. The families profiled below are the ones that visibly did.

The useful news: you can measure agent-fitness in an afternoon, and your own probe beats any leaderboard screenshot. The procedure, expanded in our guide to evals for AI agents:

1. Script 20 harness tasks you actually do — small fixes, a test-first feature, a three-file refactor, at least one task that requires reading a failing CI log.
2. Run them with the candidate engine in your daily harness, default settings, one attempt each.
3. Count three things only: malformed or mis-aimed tool calls, patch applies that fail, and abandoned loops (the model gives up, or repeats itself until the harness does).
4. Set the bar before you run — roughly one malformed call per 50 is already disqualifying for unattended work — and re-run the probe after any change of model, quant, or serving stack.

Agent-fitness on one card: the four properties chat benchmarks never measure.

DeepSeek V4: the price-performance anchor

Every open-model conversation starts with price, and price starts with DeepSeek. The V3/R1 generation reset the industry’s expectations; the V4 generation (first released April 24, 2026, GA July 19) is now the entire story, because DeepSeek deprecated V3 and R1 on July 24, 2026. Anything in your stack still pinned to them is running on borrowed time.

• Sizes: DeepSeek V4 Pro, 1.6T total / 49B active parameters (MoE); V4 Flash, 284B total / 13B active. 1M context on both.
• License: MIT, no scale carve-outs.
• Agent strengths: V4 Pro posts ~80.6% on SWE-bench Verified (vendor-run) per the mid-2026 open-model roundup; diff discipline and code quality strong for the price; reasoning folded into the main line rather than split into a separate R-series, which keeps mid-loop latency sane .
• Limits: even Flash’s 284B total parameters makes self-hosting a cluster problem, not a workstation one. Most teams consume V4 hosted, which changes the privacy calculus.
• Where to run: DeepSeek’s first-party API — V4 Flash at $0.14 per million input tokens and $0.28 per million output, the credible price floor for agentic coding — or Western re-hosts when data must stay out of first-party hands.

Flash is the interesting one for agent work. Thirteen billion active parameters serve fast and cheap, and for the bulk mechanical layer of agent workloads — test runs, lint sweeps, mass renames — it delivers most of Pro’s usefulness at a price that rewrites the economics of running agents outright.

Qwen3-Coder-Next: the one-box frontier

If DeepSeek anchors the price floor, Alibaba’s Qwen3-Coder-Next anchors the local story. It is the model behind most of the credible “my coding agent runs on my own machine” claims of 2026.

• Sizes: 80B total / ~3B active (MoE); the full build fits in roughly 46GB of unified memory, and the 30B Flash variant runs in ~18GB.
• License: Apache-2.0, like the whole Qwen3 line.
• Agent strengths: ~70.6% SWE-bench Verified (vendor-run); trained against real tool loops — Alibaba ships its own harness, Qwen Code, with tuned tool-call parsers, and the QwenLM repositories are where serving fixes land first. The most-forked, most-quantized open base means the best GGUF coverage and the fastest community patches when a chat template breaks.
• Limits: ~3B active parameters shows on long-horizon planning. It is a superb executor and a mediocre architect; pair it with a stronger planner for multi-hour work.
• Where to run: locally on a 64GB Mac, a Ryzen AI Max box, or a DGX Spark; hosted via Alibaba Cloud at $0.11 in / $0.80 out per million tokens.

“Frontier-adjacent on one box” is a factual claim here, not marketing. A model at ~70% Verified that fits beside its KV cache in 64GB of unified memory did not exist a year ago. For most readers this is the default local engine, and the Flash variant is the default answer on a 24GB GPU.

Kimi K3 and GLM-5.2: the agentic MoE bets

The two most aggressive agent-first bets in the open field are Chinese MoE flagships, and they bracket the category from opposite ends — one chasing the absolute frontier, the other chasing your Claude subscription.

Kimi K3 (Moonshot AI, July 16, 2026) — the frontier chase:

• Sizes: 2.8T-parameter MoE; 1M context.
• License: modified MIT expected, with weights staged to follow the API launch — as of late August they are promised, not posted .
• Agent strengths: ~93.4% SWE-bench Verified in Vals AI’s independent run; the first open model to lead a frontend-code arena ahead of Claude Fable 5; post-trained hard on agentic trajectories.
• Limits: hosted-or-nothing — nothing you own runs 2.8T parameters. Until the weights land, “open” describes a trajectory, not a download.
• Where to run: Moonshot’s API today; expect third-party re-hosting the week the weights actually post.

GLM-5.2 (Zhipu’s Z.ai, June 16, 2026) — the commercial squeeze:

• Sizes: 744B total / 40B active; 1M context.
• License: MIT.
• Agent strengths: #1 open model on Artificial Analysis’s index at launch; ~62.1% on SWE-bench Pro (vendor-run) and ~78.7% Verified in Epoch AI’s independent reproduction. It is the engine behind the Z.ai GLM Coding Plan, which means Zhipu’s subscription revenue depends directly on tool calls working inside real harnesses — and that commercial pressure shows in its function calling and patch discipline.
• Limits: 744B still means multi-GPU serving; local GLM-5.2 is for labs, not laptops.
• Where to run: Z.ai’s API at roughly a quarter of frontier output prices, the flat-rate GLM Coding Plan, or Western re-hosts.

Both are excellent engines, and both sit under a jurisdiction question — training provenance, data-handling terms, procurement rules — that deserves its own analysis; we cover it in our review of Chinese frontier models for agent work. The short version: MIT weights re-hosted on infrastructure you choose answer most of it. First-party APIs answer less.

The Western bench: gpt-oss, Llama 4.x, and Mistral’s Vibe line

The Western open-weight bench is thinner at the top, but one entry punches far above its size class.

gpt-oss (OpenAI):

• Sizes: gpt-oss-120b and gpt-oss-20b (MoE), shipped natively in MXFP4 low precision — the 120b fits a single 80GB GPU or a big-memory Mac, the 20b runs in ~16GB, per OpenAI’s release materials.
• License: Apache-2.0.
• Agent strengths: tool use trained as a first-class objective; in harness work it outperforms its active-parameter count, and it remains the strongest fully-Western local engine .
• Limits: the bespoke Harmony chat format. If your serving stack does not render it exactly, tool calls degrade into decorative text — use a runtime with native Harmony support and never hand-roll the template.
• Where to run: local from 16GB up on the major runtimes; hosted on the usual performance providers.

Llama 4.x (Meta):

• Sizes and license: the MoE generation under Meta’s community license — scale thresholds and branding terms, not OSI-open .
• Agent strengths: ecosystem gravity. Every runtime, every fine-tuning shop, every enterprise legal review already knows it.
• Limits: teams pick it for familiarity more than agentic edge; its coding and tool-use reception has been mixed since launch, and it rarely wins an agent-fitness probe against the families above.
• Where to run: everywhere — which is honestly its argument.

Mistral — the Vibe line:

• Sizes and license: small Apache-2.0 variants for single-GPU local lanes, larger API-tier models hosted; the line now sits under Mistral’s “Vibe” brand per mid-2026 reporting .
• Agent strengths: the Devstral lineage — models built with an agent-harness team and optimized for real SWE-agent workflows, with standout patch discipline for the size class.
• Limits: top-end capability trails the Chinese MoEs; the small variants are executors, not planners.
• Where to run: local on a 24GB card for the small variants; Mistral’s EU-hosted API where a European compliance posture is precisely the point.

Reading agentic benchmarks without being fooled

Every “best open-source coding model” claim rests on one of four kinds of numbers, and they do not measure the same thing:

• SWE-bench Verified — real GitHub issues resolved end to end. The closest thing to an agent-fitness headline number, and the most gamed.
• Terminal-bench-class suites — multi-step shell work; probes loop behavior directly.
• Tau-class tool-use evals — policy-bound tool orchestration; probes instruction adherence under rules.
• BFCL-class function-calling suites — schema validity and argument accuracy in isolation; necessary, never sufficient.

The standard distortions, all alive in 2026:

1. Harness sensitivity. The same model swings by double digits across scaffolds. A vendor’s number was produced on the vendor’s scaffold, not on yours.
2. Undisclosed retry budgets. pass@k quietly reported where you assumed pass@1.
3. Contamination. Public test sets leak into training corpora, and saturated suites stop discriminating at the top.
4. Vendor-run versus independent. Notice how often the profiles above flag “vendor-run.” That flag is doing real work.

The reading protocol: prefer third-party reproductions (the Epoch AI and Vals AI class); demand the scaffold be named; treat single-digit gaps as noise; and date every number, because this field re-sorts quarterly — our open-weight scorecard for mid-2026 is the dated snapshot of exactly these standings. Then break the remaining ties with your own 20-task probe, which runs on the only scaffold that matters: yours.

The serving decision tree for open-weight models

Open weights do not mean local. Most open-model tokens in 2026 are bought from hosts, and that is often correct. The fork has three prongs, and usually one binding constraint picks for you.

Hosted, on someone else’s silicon. Performance hosts (the Together and Fireworks class) serve the big MoEs at full precision with tool-call parsing already handled. Fast-silicon hosts (the Groq and Cerebras class) trade model selection for loop speed you can feel. First-party APIs — DeepSeek, Z.ai, Alibaba Cloud, Moonshot — are usually the price floor. OpenRouter-style aggregators put one key, unified billing, and automatic fallbacks in front of all of them .

Local, daily-driver grade. Ollama and LM Studio run quantized weights from Hugging Face behind an OpenAI-compatible endpoint on one machine. Setup is minutes; throughput is one-user grade.

Local, fleet grade. vLLM or SGLang on owned or rented GPUs: continuous batching, tensor parallelism, tool-call parser plugins. This is serving infrastructure, not an app — the runtime layer gets full treatment in our open-source AI agent stack guide.

The tree itself:

1. Must the work stay on your hardware — legally, contractually, or by policy? Yes → local. One seat → Ollama or LM Studio. A team or a CI fleet → vLLM or SGLang on your own GPUs.
2. No, but is token volume large and steady? Sustained load → vLLM-class serving on rented GPUs beats per-token prices.
3. Neither binds? Hosted: an aggregator for resilience, first-party APIs for the lowest floor.

Product note: The local-first kinship runs through this whole lane: if your models run locally for privacy, your session data should live locally too. Automater Lite archives every agent session and meters tokens per provider on your own disk — local by design. Free on automater.ai.

Hardware realism: 24GB, 48GB, 96GB, and beyond

Under every “run a coding agent locally” search sits a memory question. Here is the honest tier map for a local LLM coding agent, as of August 2026 :

Tier	Typical hardware	What honestly runs	Agent-grade pick
24GB	RTX 4090/5090; 32GB Macs	30B-class MoE coders at Q4–Q5; gpt-oss-20b	Qwen3-Coder-Next Flash (~18GB)
48GB	2×24GB GPUs; 64GB unified memory	70B dense at Q4; the 80B Qwen3-Coder-Next (~46GB) with a tight KV budget	Qwen3-Coder-Next, modest context
96GB+	DGX Spark; 128GB Macs and Ryzen AI Max boxes	gpt-oss-120b comfortably; 100B-class MoE at Q5–Q6 with six-figure context	gpt-oss-120b, or the 80B with room to breathe
Cluster / hosted	Multi-node datacenter GPUs	GLM-5.2 (744B), DeepSeek V4 (1.6T), Kimi K3 (2.8T)	Buy these as tokens

Choosing and pricing the box itself — a DGX Spark versus a Ryzen AI Max machine versus a Mac — is a separate decision with real trade-offs; our local AI workstation guide walks the 2026 field.

Now the finding generic roundups keep missing, and this article’s most practical claim: aggressive quantization degrades tool calling and structured output before prose quality visibly suffers. A checkpoint can still chat fluently after its function calling has gone unreliable — JSON drifts, argument fields swap, the schema validator starts bouncing calls — because the fine-grained token distinctions quantization sands away are exactly the ones structured output leans on. Community evals and harness bug trackers reproduce the pattern regularly .

The working rule: treat Q4 as the floor for an agent engine, prefer Q5–Q6 whenever memory allows, and re-run your 20-task probe after any quant change. Chat-testing a new quant tells you nothing you need to know.

One more line the demos omit: the KV cache eats memory beside the weights, so a setup that fits at 8K context can OOM forty steps into a real session. Budget for the context you will actually accumulate, not the context the demo used.

The 2026 tier map: what each memory class honestly runs — and where quantization starts costing you tool calls.

Wiring open weights into harnesses

The happy path is genuinely happy now. Every serious runtime and host exposes an OpenAI-compatible endpoint, and per the mid-2026 harness survey, every serious CLI harness accepts OpenAI-compatible or Anthropic-Messages APIs. Base URL, key, model name, done. Per-harness detail lives in our review of open-source harnesses; the load-bearing recipes :

Aider — point it at any OpenAI-compatible server, and declare context limits so its repo-map budget is right:

aider --model openai/qwen3-coder-next \
  --openai-api-base http://localhost:8000/v1 \
  --openai-api-key local

// .aider.model.metadata.json
{ "openai/qwen3-coder-next": { "max_input_tokens": 262144, "max_output_tokens": 32768 } }

OpenCode — a custom provider entry in opencode.json:

{
  "provider": {
    "local": {
      "npm": "@ai-sdk/openai-compatible",
      "options": { "baseURL": "http://localhost:8000/v1" },
      "models": { "qwen3-coder-next": {} }
    }
  }
}

Anthropic-format harnesses — a LiteLLM or claude-code-router-class proxy translates the Messages API, so open weights can drive those too:

litellm --model openai/glm-5.2 --port 4000
export ANTHROPIC_BASE_URL=http://localhost:4000

Where wiring actually breaks is tool-call formats, and the breakage is family-shaped:

Family or artifact	Quirk	Symptom	Fix
gpt-oss	Harmony format must render exactly	Tool calls arrive as prose	Runtime with native Harmony support; never hand-roll the template
Qwen3-Coder	XML-style tool tags	Empty or unparsed calls behind generic parsers	Serve via vLLM with the qwen3-coder parser and auto tool choice
Community GGUFs	Broken chat template embedded in the file	Model narrates tool calls instead of making them	Re-pull official quants; pin a known-good template
DeepSeek V4	Strict tool-name matching	“Unknown tool” errors on aliased names	Mirror the harness’s tool names exactly in your schemas
Mixed families	Parallel tool calls unevenly supported	Second call silently dropped	Disable parallel calls harness-side for that engine

The one test to run after any wiring change: ask for a two-tool sequence — read a file, then edit it. If the second call arrives malformed, the fault is almost always the template or the parser, not the model.

The honest gap, the routing answer, and the cost math

The even-handed sentence first: as of August 2026, closed frontier models — Claude Fable 5, GPT-5.6, Gemini 3.1 — still lead on long-horizon agent work, and the gap concentrates exactly there . Multi-hour autonomy, recovery from compounding errors, step-40 reliability: the open field’s best (Kimi K3 aside, and K3 you rent anyway) trails where loops get long. Single-step quality is close to a solved comparison. Endurance is not.

But “trails at the frontier” is not “unfit.” Open weights are already the right engine for bulk mechanical steps, for privacy-bound work that cannot leave the building, and for cost-floor workloads where frontier list prices make the unit economics absurd.

So the working answer is routing, not allegiance: an open engine for the grind lane, a frontier model for the judgment steps, and evals gating which tasks graduate between lanes. That is a portfolio, and running a portfolio of engines across several harnesses is exactly the operating problem a multi-agent command center exists to solve. There is also a third lane most teams forget: when no shelf weight fits a narrow niche, training your own mini-model by fine-tuning or distillation is now a weekend project, not a research program.

The month-of-heavy-use math, three ways, for a workload around 500M input and 100M output tokens :

Lane	Up-front	What the month costs	Ceiling
Self-hosted (Qwen3-Coder-Next on a ~$4,000, 128GB box)	~$4K hardware	~$180 amortized over two years, power included; marginal cost near zero	One box’s throughput — and you are the SRE
Hosted open weights (DeepSeek V4 Flash floor)	none	~$98 ($70 in + $28 out); Pro-class engines run 3–8× that	Rate limits; provider trust
Flat plans (GLM Coding Plan class)	none	Tens of dollars, quota-bound	Quotas, throttles, plan-change risk

The same month at frontier list prices lands in four figures, which is the whole reason this article exists. Open-model API pricing fell roughly 80% year over year per mid-2026 reporting, and the floor is still dropping.

The portfolio sentence, then: choose engines on fitness first, jurisdiction second, price third — because a cheap engine that fumbles tool calls is the most expensive thing you can run.

Model facts, prices, and standings verified as of August 27, 2026.

FAQ: open-weight models for agents

What is the best open-source model for coding agents?

There is no single crown — fit decides. As of August 2026: Qwen3-Coder-Next for local work (the Flash variant on 24GB cards), DeepSeek V4 Flash as the hosted budget engine, Kimi K3 or GLM-5.2 where you want maximum open-model capability and are buying tokens anyway. Probe before committing.

What is the difference between open-weight and open-source models?

Open-weight means the trained weights are downloadable under some license; open-source, strictly used, means OSI-style openness including training code and data information. Most 2026 “open” models — even MIT-licensed ones — are open-weight only. The distinction matters for audits, reproduction, and fine-tuning rights.

Can I run a coding agent locally?

Yes, within hardware tiers. 24GB of VRAM runs 30B-class MoE coders like Qwen3-Coder-Next Flash; 48–64GB runs the full 80B build (~46GB); 96GB and up handles gpt-oss-120b with long context. The flagship MoEs — GLM-5.2, DeepSeek V4, Kimi K3 — stay hosted regardless of your hardware.

Do quantized models work for AI agents?

Yes, with care. Quantization degrades tool calling and structured output before prose visibly suffers, so a quant that chats well can still fail as an engine. Treat Q4 as the floor, prefer Q5–Q6 when memory allows, and re-run a scripted 20-task probe after every quant change.

Sources

• Best Open-Source Coding Models 2026 — Morph
• Epoch AI — independent benchmark reproductions
• QwenLM on GitHub — Qwen3-Coder repositories
• OpenAI — gpt-oss release materials
• Hugging Face — the open-weight model hub
• Ollama — local model runtime
• vLLM documentation
• Coding CLIs in Mid-2026: The Engineer’s Map — dev.to