What can Fable 5 do that GLM-5.2 can't, when you hand them real agentic work?
To answer that question, we connected Fable 5 and GLM-5.2 to 17 SaaS tools and gave them 47 tasks.
TLDR: As expected, Fable 5 solved all 47 tasks. GLM-5.2 solved 45, but the two misses tell an important story. They showed us exactly how open-weight models still fall short of matching SOTA performance. Let’s dig in.
Background
Each model ran as an agent connected to 17 live SaaS accounts: Airtable, Datadog, GitHub, Gmail, Google Calendar, Google Drive, Google Sheets, HubSpot, Jira, LaunchDarkly, Linear, Notion, PagerDuty, PostHog, Salesforce, Slack, and Zendesk.
The tasks are the kind of work you'd actually delegate to an agent:
Find every file in this repository that leaks a credential
Deduplicate these CRM records
Repair this broken recurring calendar event.
Every task had a known correct answer baked in ahead of time. In this post, we looked at the traces to analyse how exactly GLM-5.2 “failed” compared to Fable 5. GLM-5.2 solved 45/47 tasks and Fable 5 had a perfect 100% score. In addition:
Fable averaged 84 seconds per task; GLM averaged 148. Across the full suite, Fable finished in nearly half the total time (66 minutes vs 116).
Fable was the faster model in 43 of the 47 scenarios.
Fable used about 20% fewer tokens overall
Fable needed fewer tool calls (239 vs 294) and fewer conversation turns (6.1 vs 7.3 on average) to get to an answer
The most interesting part comes from digging deeper into the stack traces. That revealed some interesting gaps:
Gap #1: Knowing when the job isn't finished
One of the tasks GLM-5.2 failed was a GitHub security audit. The instruction was to find every Python file in a repository that contains a hardcoded `secret_key`. The repository had been seeded with exactly 130 such files, so the correct answer was known in advance.
Fable 5 found all 130 of them. This took 3 tool calls and 68 seconds: Fable constructed an effective search query on its first attempt, pulled every page of results, deduplicated the paths, and answered the question.
GLM-5.2 found 120 files and reported those 120 as the complete answer, without ever questioning whether it might have missed something.
Both models had access to identical tools. GLM used a slightly different search query that returned fewer results, and it simply trusted what came back. Along the way, it also lost track of a results file it had saved earlier and spent turns searching the filesystem to find it again, and it also hit two failed tool calls while trying to fetch file contents. In essence, it ended up spending 262 seconds and three and a half times the tokens to deliver 92% of the answer.
Ninety-two per cent sounds close, but in a real security audit, that gap is 10 leaked credentials making it into production.
Gap #2: Judgment when the criteria are fuzzy
The second failed task is more unsettling because GLM did almost everything right yet still failed to reach a complete answer.
The task was a Zendesk SLA audit: find open billing tickets where no support agent had posted a public reply within 24 hours of the ticket's creation. This requires reading each ticket's actual conversation history and making a judgment call about whether a genuine agent reply happened.
GLM-5.2 inspected every candidate ticket, exactly as instructed. It also computed breach timestamps correctly. It also produced perfectly structured output in exactly the requested format. But then it classified the wrong tickets as breached. GLM spent 927,000 tokens and six and a half minutes producing a wrong answer that looked correct on the surface.
Fable 5 identified the exact set of breached tickets in 131 seconds.
What makes this failure mode dangerous is precisely how presentable the wrong answer was. The formatting was right, the timestamps were right, the structure was also right; a human skimming the output would almost certainly have approved it. A human would identify the error after carefully analysing the stack traces.
Gap #3: Efficiency, compounded
Even on the 45 tasks both models passed, the traces often looked very different, and one task made the difference quite visible.
The task was a LaunchDarkly configuration change applied via JSON Patch, a format that demands strict precision. Fable 5 completed it in 45 seconds, using 3 tool calls and 181,000 tokens. GLM-5.2 got the same correct result after 8.8 minutes, 17 tool calls, and 982,000 tokens. That's 11.7 times longer and more than five times the tokens for an identical outcome.
Looking at the largest speed gaps across the whole run: the LaunchDarkly change at 11.7x, the GitHub secrets audit at 3.9x, a Google Calendar recurring-event repair at 3.6x, a free/busy scheduling task at 3.4x, an Airtable batch-isolation task at 3.4x, the Zendesk SLA audit at 3.0x.
The pattern underneath all of these is that Fable tends to reach the right tool with the right parameters on the first attempt, while GLM takes a more exploratory path, doing extra searches, extra retries, occasional detours to recover from its own missteps.
This difference barely matters in a single chat exchange, but in an agent workflow, where every step feeds into the next, the time compounds across the entire task. That's how you end up finishing the same suite of work in half the time and at 80% of the token cost.
What all this actually tells us
The interesting conclusion here isn't "the closed model beat the open one.", but *where* it beat it.
Both models can definitely use tools, navigate real APIs, handle authentication, parse messy responses, and chain steps together. The real gaps were things like:
Knowing when a job isn't actually finished yet.
Verifying its own work before committing to an answer,
Treating "the output looks plausible" and "the work is complete" as different things
Getting judgment calls right when the criteria are fuzzy
In other words, Fable 5 scored higher in the places where small mistakes are hardest to spot and most costly to miss.