Don't Mesh with Me: Generating Constructive Solid Geometry Instead of Meshes by Fine-Tuning a Code-Generation LLM

Maximilian Mews, Ansar Aynetdinov, Vivian Schiller, Peter Eisert, Alan Akbik
Humboldt-Universität zu Berlin
mewsmaxi@hu-berlin.de
AI4CC @ CVPR 2025
arXiv Poster GitHub

Geometry Input

Text Input

Ground Truth

Result

L-shaped bracket input

"Thick, solid ring."

L-shaped bracket ground truth
L-shaped bracket result
Ring input

"L-shaped corner bracket."

Ring ground truth
Ring result
U-shaped bracket input

"U-shaped bracket with a hole and notches."

U-shaped bracket ground truth
U-shaped bracket result

Motivation

"Software developers use AI tools like ChatGPT to write code.
Mechanical engineers don't have similar AI tools to design parts."
Existing approaches generate Neural Pointclouds, Voxel Grids, or Radiance Fields or Meshes.
Point Cloud
✗
Point Cloud
Voxel
✗
Voxel
Mesh
✗
Mesh
Engineers need true parametric curves and surfaces.
Brep
✓
Brep
Constructive Solid Geometry (CSG) combines simple shapes.
CSG
✓
CSG
Our Approach generates precise 3D geometries from natural language.

Contributions

CSG Pipeline

Developed a pipeline to convert BREP geometries from CAD software into surface-based CSG Python scripts.

Annotation Generation

Generated natural language descriptions of 3D geometries using GPT-4o.

LLM Fine-tuning

Fine-tuned a 1.3B code generation LLM on 37,000 geometry-description pairs to enable plausible 3D geometry completion from text.

Pipeline

Pipeline Diagram

Data Sample

Input Geometry


x_plane_1 = XPlane(x0=40.0)
y_plane_1 = YPlane(y0=-70.0)
x_cylinder_1 = XCylinder(y0=-35.0, z0=0.0, r=20.0)
z_plane_1 = ZPlane(z0=30.0)
z_plane_2 = ZPlane(z0=0.0)
y_plane_2 = YPlane(y0=-10.0)
x_plane_2 = XPlane(x0=70.0)
cell_1 = Cell(region=(+x_plane_1 & +y_plane_1 & +x_cylinder_1 
                      & -z_plane_1 & +z_plane_2 & +y_plane_2 & -x_plane_2))
x_plane_3 = XPlane(x0=-40.0)
y_cylinder_1 = YCylinder(x0=0.0, z0=60.0, r=20.0)
z_plane_3 = ZPlane(z0=60.0)
cell_2 = Cell(region=(+x_plane_3 & +y_plane_1 & +x_cylinder_1 
                      & +y_cylinder_1 & +z_plane_2 & -z_plane_3 & +x_plane_1))
x_plane_4 = XPlane(x0=-70.0)
cell_3 = Cell(region=(+y_plane_1 & +x_cylinder_1 & -z_plane_1 
                      & +z_plane_2 & -y_plane_2 & -x_plane_3 & +x_plane_4))
y_cylinder_2 = YCylinder(x0=55.0, z0=45.0, r=5.0)
y_cylinder_3 = YCylinder(x0=-55.0, z0=45.0, r=5.0)
y_plane_3 = YPlane(y0=0.0)
cell_4 = Cell(region=(+y_plane_2 & +y_cylinder_2 & +y_cylinder_3 
                      & +x_plane_4 & +z_plane_3 & +z_plane_2 & -y_plane_3 & -x_plane_2))
Input Geometry

Output Geometry


y_plane_4 = YPlane(y0=-80.0)
y_cylinder_4 = YCylinder(x0=0.0, z0=60.0, r=30.0)
cell_5 = Cell(region=(+y_plane_4 & +y_cylinder_1
& +y_cylinder_4 & -z_plane_3 & -y_plane_1))
Output Geometry

Demo Video

Experimental Results

  1. Geometric Coherence: 98% of generated models are geometrically valid (finite, connected cells with no overlaps).
  2. Surface Adherence: 83% of models conformed to the input surface geometry.
  3. Text Guidance: Textual input improves ground truth alignment, but only in low-complexity (low cell count) cases.
Share of Geometries Matrix

Share of Geometries, where Output = Ground Truth.
Text Annotations increase the likelihood of hitting the ground truth for low cell counts.

Distribution of Cell Counts

Distribution of the number of cells generated by ground truth cell count.
Our model generates cell counts closely matching the ground truth.

Enhanced Eye Catcher Grid

Qualitative Results

Geometry Input

Text Input

Ground Truth

Result

L-shaped bracket input

"Thick, solid ring"

L-shaped bracket ground truth
L-shaped bracket result
Ring input

"L-shaped corner bracket."

Ring ground truth
Ring result
Rectangular prism input

"A cylinder held by an axle."

Rectangular prism ground truth
Rectangular prism result
U-shaped bracket input

"U-shaped bracket with a hole and notches"

U-shaped bracket ground truth
U-shaped bracket result
Cylinder held by axle input

"Six cylindrical discs arranged in a 2x3 grid."

Cylinder held by axle ground truth
Cylinder held by axle result
Six cylindrical discs input

"Two disks."

Six cylindrical discs ground truth
Six cylindrical discs result
Two disks input

"Rectangular prism."

Two disks ground truth
Two disks result

Citation

BibTeX

@misc{mews2025dontmeshmegenerating,
  title={Don't Mesh with Me: Generating Constructive Solid Geometry Instead of Meshes by Fine-Tuning a Code-Generation LLM}, 
  author={Maximilian Mews and Ansar Aynetdinov and Vivian Schiller and Peter Eisert and Alan Akbik},
  year={2025},
  eprint={2411.15279},
  archivePrefix={arXiv},
  primaryClass={cs.LG},
  url={https://arxiv.org/abs/2411.15279}, 
}