Grok 4.5 Ranks Second on APEX-SWE, First on a Rival Benchmark

Grok 4.5 finished second on Mercor’s APEX-SWE leaderboard this week, posting a 51.2% Pass@1 score against Claude Fable 5’s 65.5%. The 14-point gap looks decisive enough to end most model rivalries on the spot. A day earlier, on a different benchmark, xAI’s model finished first.

APEX-SWE, built by hiring platform Mercor with coding startup Cognition, grades the unglamorous work real engineers do: wiring cloud services together and debugging production failures from logs and dashboards. Grok 4.5’s split result, runner-up on one scoreboard and winner on another within 48 hours, is turning into the defining shape of the 2026 AI coding race.

What Does APEX-SWE Actually Measure?

APEX-SWE is a 200-task benchmark that Mercor built with Cognition and launched in March 2026 to grade models on integration work and production debugging instead of isolated coding puzzles. Each task is scored Pass@1, meaning a model gets credit only if it solves the problem correctly on the first attempt, with no retries.

Traditional coding tests had a saturation problem. GPT-4 climbed from 67% to 90% on HumanEval in just two years, and OpenAI has since flagged some SWE benchmarks as contaminated after finding models that could reproduce original patches verbatim from task IDs. APEX-SWE was built to route around that.

Half the test is Integration work: stitching databases, ticketing systems and cloud emulators into something that actually runs. The other half is Observability work, debugging with production-style telemetry such as logs and dashboards rather than a tidy bug report. A human-written rubric grades functional correctness, robustness and code style on top of automated unit tests.

The 14-Point Gap

When APEX-SWE went public in March, GPT-5.3 Codex held the top spot at 41.5% Pass@1, with plenty of room left to climb. Four months later, Claude Fable 5, Anthropic’s flagship released in early June 2026 for high-capability reasoning and agentic work, sits at 65.5%, plus or minus 6.2 percentage points. Grok 4.5 trails at 51.2%, plus or minus 6.0.

Factor in those error bars and the real gap could run anywhere from about 2 points to roughly 26, a reminder that a single leaderboard number carries more uncertainty than its ranking implies.

The previous Grok generation did far worse on this exact test. In Mercor’s original technical paper, Grok 4 scored 5.7% Pass@1 on the Observability tasks, the weakest showing among eleven frontier models graded. That is a steep climb for its successor to make in one release.

Grok 4.5 Wins the Very Next Benchmark

Score Grok 4.5 on a different test and the placement flips. On AutomationBench-AA, an Artificial Analysis evaluation covering 657 tasks across 40 simulated business apps including Gmail, Slack and Salesforce, Grok 4.5 scored 51.4%, ahead of Fable 5 at 48.6% and Claude Opus 4.8 at 48.5%, while costing $0.34 per task against $1.35 and $1.46 for its rivals.

On xAI’s own SWE Marathon test, which measures long-horizon issue resolution, Grok 4.5 posted a 29.0% resolution rate, ahead of Opus 4.8 at 26.0% and Fable 5 at 24.0%, though that score comes from xAI’s own testing and has not been independently verified.

Four benchmarks, four different places on the podium. That spread is the actual story behind this week’s headlines, more than any single row of the table.

Price Cuts Through the Rankings

None of that fragmentation matters much if the price gap stays this wide. xAI prices Grok 4.5 at $2 per million input tokens and $6 per million output tokens. Anthropic charges $10 and $50 for Fable 5, five times more on input and more than eight times more on output.

• $2.49 average cost per completed coding task inside Grok Build, against $11.80 for Fable 5 running inside Claude Code, per Artificial Analysis.
• 4 to 17 times lower cost per task than rival models depending on workload, according to xAI’s own comparisons.
• 91.3 output tokens per second measured independently, versus the 80 tokens per second xAI advertises.
• 500,000 tokens of context for Grok 4.5, smaller than Fable 5’s window beyond 1 million.

Those numbers hold under 200,000 tokens of context. Push past that threshold and Grok 4.5’s rate doubles to $4 and $12, still well below Fable 5’s flat pricing regardless of length.

What the Discount Doesn’t Cover

The savings track a real tradeoff. On Artificial Analysis’s AA-Omniscience Index, Grok 4.5’s accuracy rose from 35% to 52% over its predecessor. Its hallucination rate on the same index jumped from 25% to 54%. The newer model answers more questions correctly, and states its wrong answers with more conviction than before.

The same tension showed up on AutomationBench-AA. Grok 4.5 logged 0.63 guardrail violations per task, above Opus 4.8’s 0.55 and Google’s Gemini 3.5 Flash at 0.46, a gap that matters for any team running agents near live financial systems.

The best PS2 game ever, but the PS3 has been out for two months.

That’s developer Theo’s verdict after testing Grok 4.5 through early Cursor access on the t3.gg podcast. His complaint wasn’t raw coding ability. It was orchestration: Grok 4.5, he found, cannot break a complex job into sub-agents and hand off pieces the way Fable 5 and GPT-5.6 can.

Cursor’s fingerprints are all over Grok 4.5 for a reason. SpaceX now owns xAI outright after going public on the Nasdaq this year at a $1.77 trillion valuation, and it paid $60 billion in June for Anysphere, Cursor’s parent company, folding trillions of tokens of Cursor’s developer-interaction data into the training run. An earlier snapshot of Cursor’s own codebase accidentally entered that training data too, inflating results on an internal Cursor benchmark xAI otherwise would have topped. xAI disclosed the contamination publicly and removed it from future training runs, a move developers largely praised even as they discounted that particular score.

Not everyone is worried about the tradeoffs. Cursor chief executive Michael Truell, whose company jointly trained the model, said Grok 4.5 has already “become the daily driver for many on our team.”

The pattern lines up with a broader industry finding. A 2026 benchmark report tracking more than 250,000 developers found AI coding tools cut time to a pull request by up to 58%, but the same pull requests introduce 15 to 18% more vulnerabilities and wait far longer in review. Cheaper, faster code is not automatically safer code.

The Office Push Meets a Trust Problem

xAI is not stopping at coding. Grok 4.5 is now the default engine inside Grok Build, and it ships this week in three new places.

• Word, Excel and PowerPoint – the model builds multi-sheet formulas, native diagrams and slide content directly inside Microsoft’s office apps.
• Notion – a new integration lets Grok 4.5 manage meetings, documents and company knowledge.
• Convex – developers can use the model to build full-stack applications on the backend platform.

That expansion arrives while trust questions about xAI keep multiplying elsewhere. A Canadian body penalized a lawyer for using Grok in a filing this year, an early preview of the liability that follows any model into professional work. In the UK, lawmaker Jess Asato has sued xAI over Grok’s deepfake design choices, and in Ireland, lawmaker Niamh Smyth is at the center of a fight reshaping the country’s pending AI bill.

The Coding Work No Benchmark Tests

One gap sits inside APEX-SWE, and inside nearly every mainstream coding benchmark: blockchain-specific work. No major evaluation suite yet treats smart-contract programming as its own category, even as AI coding agents get deployed on Solidity audits and protocol upgrades.

• Solidity development – smart-contract code carries vulnerability classes that general-purpose engineering tasks do not reproduce.
• Cross-chain integration testing – moving assets or data between separate blockchains has no equivalent among APEX-SWE’s cloud-service integration tasks.
• On-chain state management – checking that a code change holds up against a live, consensus-driven ledger is not part of any current observability test.

Mercor’s own open dataset shows the benchmark is already brushing against that world without formally testing it. Two of its published observability tasks debug real blockchain client code, including Polygon’s Bor client and ChainSafe’s Gossamer client, yet Solidity work, cross-chain testing and on-chain state checks still are not scored as their own disciplines.

How Buyers Are Actually Deciding

OpenAI did not wait long to answer. Its GPT-5.6 family, in Sol, Terra and Luna tiers priced from $1 to $30 per million tokens, reached general availability the same week, after a delayed rollout tied to national security review. OpenAI reported Sol hitting 91.9% on Terminal-Bench 2.1 through a new mode that splits work across parallel subagents.

Elon Musk’s own framing of Grok 4.5 has stayed modest by his standards: “roughly comparable to Opus 4.7, but much faster.”

Forrester’s Biswajeet Mahapatra, a principal analyst at the firm, put the caution more bluntly. Cheaper tokens change the economics of agentic coding, he said, but “enterprise buyers should focus on cost per successful outcome rather than cost per token.” A model that needs three attempts to ship working code is not actually the cheap option.

Analysts increasingly describe the market that way: not one winner, but a routing problem, with inexpensive tiers like Grok 4.5 absorbing bulk work while pricier models handle the jobs nothing else can solve. Mercor updates the APEX-SWE leaderboard on a rolling basis, and whichever lab ships the next frontier model will get graded against the same 200 tasks.

Frequently Asked Questions

What Is APEX-SWE?

APEX-SWE is a 200-task benchmark that hiring platform Mercor built with coding startup Cognition and launched in March 2026 to grade AI models on integration and production-debugging work instead of isolated coding puzzles. Each task carries a human-written rubric plus automated unit tests, and Mercor has published its task set and evaluation harness on Hugging Face and GitHub so outside researchers can check the scores themselves.

Why Did Grok 4.5 Rank Second Instead Of First?

Grok 4.5’s overall score reflects a split performance across APEX-SWE’s two task types. Models across the industry consistently score higher on integration work than on observability debugging, and Grok’s predecessor was especially weak on the observability half, scoring just 5.7% in Mercor’s original technical paper. xAI has not published a task-by-task breakdown for Grok 4.5, so which half is still dragging down its average is not public yet.

Is Grok 4.5 Cheaper Than Claude Fable 5?

Yes, by a wide margin. Grok 4.5 costs $2 per million input tokens and $6 per million output tokens, against $10 and $50 for Fable 5. The tradeoff is context: Grok 4.5 holds 500,000 tokens in its window, while Fable 5 stretches past 1 million, useful for holding an entire large codebase in view at once.

Does Grok 4.5 Handle Blockchain Or Smart Contract Coding?

Not as a tested category. No mainstream software engineering benchmark, APEX-SWE included, scores Solidity development, cross-chain integration testing or on-chain state management as its own discipline yet, even though APEX-SWE’s task set already borrows debugging problems from blockchain client software. Crypto-native teams currently lean on general coding scores as a rough stand-in.

Which Model Currently Leads The APEX-SWE Leaderboard?

Claude Fable 5 leads at 65.5% Pass@1, ahead of Claude Opus 4.8 and GPT-5.3 Codex, with Grok 4.5 now in second. The top score has moved fast: it stood at 41.5% when the benchmark launched in March 2026, so a 24-point climb in four months suggests whoever ships next will not hold the top spot for long.