# What is CADGenBench?

- Link: https://vibecad.nl/en/what-is-cadgenbench/
- Published: 2026-10-22T21:44:02+02:00
- Author: Marijn van den Heuvel \+ AI

_This is the English reading version of the Dutch explainer [Wat is CADGenBench?](https://vibecad.nl/2026/10/21/wat-is-cadgenbench/)._

In reports about AI and CAD, benchmark figures come up, and CADGenBench is one of
them. It measures whether a model turns a design task into a real CAD model. Below
is what is actually measured, and why you should take such a figure with a pinch
of salt.

## What it measures

CADGenBench asks a model to turn a task into a CAD model. Not a picture, and not
a mesh: real geometry, a solid you can open in a CAD program and carry on editing.
After that it checks whether that worked.

That makes it a different question from most AI tests. A model that can give a convincing
description of a part does not get far here; it has to deliver something that works
as a model.

## How it judges

The judging is not the opinion of a chatbot but a test. First the model has to be
a valid solid: closed, without loose faces or holes in the wall. Then the shape 
has to be right compared with what was asked for. And the build-up has to be right:
is a hole in the right place, is the wall thickness right, do the faces meet.

That last part is exactly where models stumble. A model can look good from the outside
and still fail, because the build-up underneath is wrong. For a CAD user that is
recognisable: a part you cannot edit is worth little in practice.

## Where it comes from

CADGenBench comes from Mecado and is on Hugging Face. It is a public benchmark with
a leaderboard where results are submitted. There is also a variant that measures
not only making a model but also editing an existing STEP file.

## Why one figure says little

A benchmark can only be compared fairly if everyone measures under the same conditions,
and that is not always the case here. xAI measured its Grok 4.7 with a tool of its
own, while the models it compared itself with used theirs. Then the ranking says
something, but not everything.

On top of that, the leaderboard is still thin. There are few models on it at the
moment, and that makes a high position less meaningful than it looks. A score shows
that a model can do something, not that it is the best.

## Not the same as BenchCAD

BenchCAD is a different benchmark with a different setup. The figures of the two
do not say the same thing, even though you often see them in the same list. So do
not compare numbers from different benchmarks with each other.

The benchmark is at [CADGenBench on Hugging Face](https://github.com/huggingface/cadgenbench),
with the leaderboard at [Mecado](https://www.mecado.com/). The figures I mention
here come from the providers’ own model cards.