Grok 4.5 vs GLM 5.2 on real-world agentic tasks

Grok 4.5 is xAI’s new frontier model for coding, STEM, knowledge work, and agentic tasks. It is available through xAI and gateways like OpenRouter. Pricing is simple: $2/M input, $6/M output, with a 500K context window on OpenRouter. xAI’s docs also list function calling, web search, X search, and code execution in the stack. (openrouter.ai)

GLM 5.2 takes another path. Z.ai pitches it for long-horizon work. The weights are on Hugging Face under the MIT license, and the model card lists 753B parameters with a 1M-token context target. Hosted pricing is lower in the lanes I would actually use: Z.ai lists $1.40/M input and $4.40/M output, while OpenRouter currently shows $0.84/M input and $2.64/M output for z-ai/glm-5.2. (huggingface.co)

The comparison gets interesting once agent usage enters the picture. Grok has the frontier badge and first-party agent tooling. GLM’s appeal is practical: open weights, a longer context window, and a cheaper meter for token-heavy loops that burn tokens on retries, tool schemas, intermediate reasoning, and dumb little SaaS confirmations.

So I ran both behind the same agent on the same real tool-use benchmark. The SaaS accounts, task set, tool surface, and evaluator stayed fixed. I only swapped the model.

I wanted the answer I need for production SaaS workflows: which model actually gets the account into the right final state, and whether the higher token price buys fewer headaches.

The two models

Grok 4.5

Grok 4.5 is the xAI frontier pick here. xAI positions it as the flagship for code, agentic tool calling, knowledge work, and STEM, with configurable reasoning plus built-in function calling, web search, X search, and code execution. The model card lists a 500K context window, $2.00 per million input tokens, and $6.00 per million output tokens. (docs.x.ai)

Going in, it looked like the safer, more expensive bet. The pitch is direct: give it engineering work and tools, then let it grind. The detail that made me slow down was caching. xAI explicitly recommends setting prompt_cache_key or x-grok-conv-id, because otherwise a long agent loop can land on a cache-cold server and pay full input price. That is exactly the kind of small integration footgun that can quietly turn a good model into a dumb bill. (docs.x.ai)

GLM 5.2

GLM 5.2 is Z.ai’s open-weight heavyweight. OpenRouter describes it as a large-scale reasoning model for long-horizon agent workflows and project-level software engineering, with high and xhigh reasoning modes for complex multi-step automation. It has a 1M-token context window, and the OpenRouter card for z-ai/glm-5.2 is currently marked 40% off at $0.84 per million input tokens and $2.64 per million output tokens. Z.ai’s own pricing page lists $1.40 input and $4.40 output per million tokens. (openrouter.ai)

GLM 5.2 is what made the comparison interesting. The weights are on Hugging Face under an MIT license, with local serving paths shown for vLLM and SGLang, so the escape hatch is real if you want to experiment outside a hosted API. The 1M context window also changes how I would run it before any test starts. You can dump a whole messy workspace into it, which is useful when the workspace matters and expensive when the agent starts looking productive instead of making progress. (huggingface.co)

Using OpenRouter’s current cards, Grok 4.5 is roughly 2.4x GLM 5.2’s input price and 2.3x its output price: $2.00 vs $0.84 in, $6.00 vs $2.64 out. If you price GLM 5.2 at Z.ai’s direct list rate instead of the temporary OpenRouter discount, the gap shrinks to about 1.4x on input and 1.36x on output. (openrouter.ai)

Composio GoldenEval: Real-world tool-use benchmark

Composio Golden Eval is a real-account tool-use benchmark. Claude Code runs as the agent, communicates with tools via the hosted Composio MCP router, and must complete complex, multi-step SaaS tasks on live accounts.

The run spans 38 scenarios and 76 trials across normal work apps, including Salesforce, HubSpot, Jira, Linear, GitHub, Zendesk, Slack, Gmail, Google Sheets, Notion, and Airtable, as well as ops and product tools like PagerDuty, PostHog, LaunchDarkly, and Datadog via Composio MCP.

Pass/fail comes from discovery plus API readback against the real account state. The evaluator performs additional API reads to identify the run’s tagged writes, checks the expected state, and fails if anything is extra or missing. No LLM transcript judge gets to skim the conversation and decide whether the intent looked good enough. Each write is uniquely tagged and cleaned up after the run. That is how you test noisy live accounts without trashing them.

The tasks hit production failure modes. Exact-set reconciliation can fail due to an extra Gmail message with a label. Dedup requires identity normalization before duplicate detection. Scoped mutation prevents the agent from updating rows that were supposed to remain unchanged. Audit work forces full collection, including paginated results. The verifier treats almost-right as a fail.

What the tasks actually look like

In Salesforce, one task asks the agent to find Account records whose website domain normalises to a target and whose name carries the run tag. Then it has to treat the normalised domain as the identity, report duplicate groups, and pick the most recently created record as the canonical one. That leaves a lot of room for dumb mistakes: comparing raw websites, missing the tag scope, or choosing the wrong canonical record.

The Gmail task is deceptively small. Find every message whose subject contains a specific token, create the label if it does not exist, and apply it to exactly those messages and no others. If the agent grabs the first matching page, or labels a nearby message because the subject looks similar, the account readback catches it.

Linear tests the same thing from the mutation side. In a team backlog, the agent must set a 1-point estimate for every tagged issue that lacks one, while leaving already-estimated issues untouched. Bulk edits are where agents love to overreach, so the scoped part matters.

GitHub makes the agent audit repos for hardcoded credentials by searching Python files for a token = ... pattern. The catch is coverage: it has to walk the full paginated result set instead of stopping at whatever comes back first.

With Google Sheets, the precision is tiny. Using data filters and grid ranges, the agent must change the value of a single target row while leaving the rest unchanged. An incorrect row or a broad-range update gives it away in the final state of the sheet, and a loose filter does too.

How I ran it

I ran both models as the agent behind the same Claude Code + Composio MCP setup, one attempt per scenario, across the same 38 scenarios. I pointed Claude Code at OpenRouter for Grok 4.5 and used Z.ai‘s Claude Code runtime for GLM 5.2. I kept the same tool surface and scenario set, with identical grading. Only the model changed.

Grading was pass/fail per trial using real-account API readback verifiers, with tag-scoped cleanup.

GLM 5.2 came out ahead, but barely on pass rate: 37/38 versus 36/38, or 97% versus 95%. The bigger gap was the amount of work it took to get there. GLM 5.2 averaged 101.4 seconds per scenario, 7.1 turns, 6.1 tool calls, and 320,480 tokens. Grok 4.5 averaged 164.2 seconds, 13.7 turns, 12.7 tool calls, and 508,421 tokens for the same 38-scenario run.

Findings

Pass rate was close enough to make the ranking fragile. GLM 5.2 finished 37/38 (97%), while Grok 4.5 finished 36/38 (95%). Both cleared almost the whole suite, and the misses were exactly the kind of real-account failures this benchmark was meant to catch: exact cell targeting, exact-set completion, and paginated recall.

Grok 4.5 missed two scoped mutation tasks. In Google Sheets, it changed the wrong cells in a data filter grid range scenario, where only one target row per tab was supposed to change. In Linear, it failed exact-set completion for backlog estimates: not every required tagged backlog issue received an estimate of 1, and already-estimated issues needed to remain untouched. GLM 5.2 passed both.

GLM 5.2’s miss was different: paginated GitHub recall. In the gh code search secrets scenario, the search returned only 18 files, while the task required at least 38 and the evaluator’s baseline was 48, stopping after the first page of results. Grok 4.5 passed that one, but it paid for its extra recall with much heavier runs: about 19.3M tokens, 104 minutes, and 483 tool calls across the suite versus GLM’s 12.2M tokens, 64 minutes, and 232 tool calls.

What it costs

For cost, I used runtime token counts and set a ceiling of each model’s listed input rate for every runtime token. Prompt caching can reduce repeated input charges, so I treat the dollar figures as upper-bound estimates. The ratio is the part I’d trust more.

Grok 4.5 comes out about 3.8x more expensive for the 38-scenario suite. That gap has two parts: Grok 4.5 costs about 2.4x as much per token, and it processes about 60% more tokens.

The extra spend did not buy the better result here. Grok 4.5 was the more expensive, heavier-running model, and it still finished one scenario behind GLM 5.2.

At volume, I’d care a lot more about the ratio than the exact dollars. A few thousand runs turns $1.02 vs $0.27 per scenario into a budget line pretty quickly.

Verdict

GLM 5.2 won the run-on pass rate (37/38 vs 36/38) and was much leaner while doing so: 101.4 seconds vs 164.2 seconds on average, 320,480 vs 508,421 runtime tokens, and 6.1 vs 12.7 tool calls. If your cost tracks token burn, the price pressure points the same way as the speed result.

Use Grok 4.5 when recall is the scary part, especially if the workflow looks like paginated search. The one GLM miss was gh code search secrets, where it returned only 18 files when the task needed at least 38 and the evaluator’s baseline was 48. It stopped at the first page. Grok passed that. If I’m sweeping repos for hardcoded token = ... patterns and missing page two is the failure mode that can bite me later, Grok’s slower, heavier run is easier to justify.

For most normal agent workflows in this run, I’d use GLM 5.2. It passed more scenarios, used about 12.2M total tokens instead of Grok’s 19.3M, finished in 64 minutes instead of 104, and made 232 tool calls instead of 483. The gap also showed up in boring, real work, such as the Gmail label batch task. Both passed there, but GLM finished in 81 seconds with 6 tool calls and 306k tokens. Grok took 379 seconds, 25 tool calls, and about 1.18M tokens for the same passing result.

For my own workflow, I’d start with GLM 5.2 as the default because it costs less for this usage and still yields one more pass with shorter runs. I’d keep Grok 4.5 in the rotation for jobs where incomplete search results are worse than extra tokens and time.