This is the English reading version of the Dutch news item Drie CAD-benchmarks: wat ze meten, en wat niet.
There are now three benchmarks that measure whether an AI model really does CAD work: Parametric CAD Bench, CADGenBench and AutoCAD Bench. Each measures something different, and that is why you cannot line their figures up side by side.
How that is measured differs per benchmark.
| Benchmark | What it measures |
|---|---|
| Parametric CAD Bench (gNucleus) | 100 FreeCAD tasks: 30 creating from text, 30 creating and then editing, 40 creating from an engineering drawing. The code really runs and the geometry is compared with the task and with an example model. |
| CADGenBench (Hugging Face) | Whether a model turns a design task into a valid and geometrically correct 3D part. Invalid output gets no score. There are two kinds of task: generating and editing. |
| AutoCAD Bench (Markov Studios) | 50 drawing tasks in AutoCAD 2019, 21 in 2D and 29 in 3D. The agent sees only a screen and may click, type and drag: no automation, no MCP. |
Parametric CAD Bench: 61% with an agent around it
At the top of the V3 leaderboard is Claude Opus 5.5 at maximum effort with Claude Code as its tool, at 61.03%. The same Opus reaches 57.96% with mini-swe-agent, and GPT-6 Astra comes in with Codex at 56.87%. What stands out inside those figures: going from an engineering drawing to CAD works best by far at 84.66%, and changing an existing model stalls at 39.69%. The whole run of 100 tasks cost 573.94 dollars in tokens at the model provider, about 5.74 dollars per task, and a failed or unscored attempt counts as zero. Those costs are reported for the full cohort and depend on the model and agent configuration: in this list they run from 196.74 to 1,056.04 dollars.
CADGenBench: that 86.9% is a partial figure
In reports about CADGenBench, 86.9% comes up for build123d-mcp with Claude Opus 5.5. In the public list of submissions that is the figure on the generation part, not the total: across the whole benchmark that submission comes to 82.98%, with 77.0% on editing. At the top of the total score stands a submission named deeprd-v2 at 84.82%, marked as not checked; the build123d-mcp submission is checked.
The benchmark runs each submission itself, but the participant supplies their own combination of model and tools. So the list is a signal that a combination works, not a ranking of models.
AutoCAD Bench: 46%, and that is the highest
This one is strictest in what it allows: fifty tasks in AutoCAD 2019 in which the agent has only a mouse and a keyboard, no MCP and no automation. At the top stands Sol at 46%, so 23 of the 50 tasks, and the split shows where it goes wrong: 14 of 21 in 2D against 9 of 29 in 3D. Terra reaches 14%, Fable 5 10%, Luna 8%, and Opus 4.8, Kimi K2.5 and Qwen3.7 do not come above the threshold of 75 points in any task.
The makers say themselves what their benchmark does not see. The evaluator looks only at the saved drawing file, not at how it was made. In their own example Sol keeps the outline and every dimension but replaces two radius-driven arcs with a single spline; that still got through with 86.3 points. What also stands out is that 88.8% of all actions are typed commands, and that the models often send several actions from one screenshot.
Why you cannot compare the figures
In one benchmark an agent writes FreeCAD code, in another it types and clicks in AutoCAD itself, and in the third only the geometry that comes out counts. A figure therefore says as much about the task and the setup as about the model, and the tool around it weighs in.
My verdict: the three lists together show something no single figure says. Models are strong where they may write code and weak where they have to work in the interface themselves: FreeCAD code goes reasonably, driving AutoCAD by hand stalls at 46%, and that is still with an evaluator that looks only at the final file. For anyone who wants to vibecad, that means the connection matters more than the figure: a program an agent can drive from outside gives it a route that works.
The leaderboards are at cadbench.ai, on the CADGenBench leaderboard of Hugging Face, and on the page of AutoCAD Bench by Markov Studios. What CADGenBench measures exactly is in What is CADGenBench?