# Turning photos of a mechanical part into a 3D model — what we tried, and what actually happened

**Project:** `acd23/cad-3d-reconstruction`
**Data:** [`acd23/cad-test-images`](https://huggingface.co/datasets/acd23/cad-test-images)
**Written:** 6 September 2026
**Status of the numbers below:** every claim was re-checked by re-running the tools and opening every output file. Where the earlier `FEASIBILITY_REPORT.md` and this document disagree, this document is the corrected one — see [§8](#8-corrections-to-the-earlier-report).

---

## 1. What this document is

The goal was simple to state: **take a handful of pictures of a mechanical part and get a 3D model out.**

Two specific questions were asked:

1. Would **3D Gaussian Splatting (3DGS)** work for this? Build one and produce a mesh.
2. Implement **[BrepGaussian](https://arxiv.org/abs/2602.21105)** (CVPR 2026), which promises to go one step further and output a real **CAD model** — flat faces, cylinders, edges and corners — rather than a blob of triangles.

This document walks through every approach that was tried, shows the actual output of each, explains in plain terms why each one behaved the way it did, and ends with what to do instead.

**The short answer: none of the reconstruction attempts produced a usable 3D model of any of the four parts, and the reason is the input data, not the choice of algorithm.** Details below.

---

## 2. The 60-second version

| # | Approach | Did it run? | Did it produce a usable 3D model? | Why |
|---|---|---|---|---|
| 1 | COLMAP → 3D Gaussian Splatting | ❌ No | No | COLMAP found essentially no matching features between views, so it never produced camera positions. 3DGS cannot start without them. |
| 2 | MASt3R (AI pose + depth estimator) | ⚠️ Ran fine | No | It produced 200k–600k points per part, but the views were never merged onto one object. Each photo became its own separate curved sheet floating in space. |
| 3 | Poisson / ball-pivoting meshing | ⚠️ Ran fine | No | Meshing faithfully wrapped a surface around the broken point cloud from step 2. Garbage in, garbage out. |
| 4a | BrepGaussian, adapted 2-stage training | ⚠️ Ran, but as a no-op | No | 500 points, 200+100 iterations (the paper uses 30,000+15,000). The Gaussians moved 0.1% of the object's size — i.e. they never trained. |
| 4b | B-Rep fitting on the dense point cloud (fallback) | ✅ Ran | No | Produced 7–8 flat planes per part and **zero cylinders** — even for the timing pulley, which is essentially a cylinder. |
| 4c | The paper's own B-Rep fitting code, on the paper's own data | ✅ Ran | **Yes** | 6 planes + 1 cylinder + 14 edges + 8 corners, cleanly. This proves the fitting code is fine; it was simply never fed anything usable from our data. |

Here is every result in one picture. Column 1 is what we put in; columns 2–4 are what came out.

![Results matrix](https://huggingface.co/acd23/cad-3d-reconstruction/resolve/main/report_assets/09_results_matrix.jpg)

You should be able to recognise a bracket, a connector, a flange housing and a toothed pulley in column 1. You cannot recognise any of them in columns 2, 3 or 4. That is the honest summary of the project.

---

## 3. The input data — this is where everything is decided

There are 4 parts and 17 images in total.

![Dataset](https://huggingface.co/acd23/cad-3d-reconstruction/resolve/main/report_assets/01_dataset.png)

| Part | Views | Image sizes | Kind of image |
|---|---|---|---|
| `bracket` | 4 | 1234×1538 … 1999×1497 | CAD renders, flat grey |
| `connector_module` | 4 | **370×106 … 140×420** | product catalogue photos, very small |
| `flange_housing` | 4 | 588×494 … 806×644 | CAD renders, shiny dark metal |
| `timing_pulley` | 5 | 702×688 … 1016×866 | CAD renders, dark plastic |

Four properties of this data cause everything that follows. It is worth understanding them before reading any result:

**a) These are "catalogue views", not a camera orbit.** Look at the pulley row: front, three-quarter, straight-on circle, side, angled. The camera jumps roughly 90° between shots. Every 3D-from-images method — old-school photogrammetry *and* modern AI ones — works by finding *the same physical point* in two or more pictures. When the camera jumps 90°, the surface you could see in shot 1 is edge-on or hidden in shot 2. There is almost nothing in common to match.

**b) The surfaces have no texture.** A flat grey CAD face looks identical everywhere on it. There is no speck, scratch or grain to lock onto. The only distinguishing features are the *outlines*, and outlines are the worst possible thing to match on, because an outline is not a fixed place on the object — it slides across the surface as the camera moves.

**c) Every view has a different image size.** That means they were cropped and scaled independently, so there is no single consistent camera. This also has a practical consequence noted in §4.

**d) There are 4–5 views.** For reference: standard 3DGS is normally trained on **100–300** images. The BrepGaussian paper uses **50** per object and explicitly reports that **30 or fewer is not enough** (§7.1). We are at 4–5.

> **The single most important sentence in this document:** the input is 4–5 wide-baseline, texture-free catalogue views. That is roughly 10× too few views, with roughly 10× too little overlap, for any of the methods tried here. No amount of algorithm-swapping fixes that.

---

## 4. Approach 1 — the standard route: COLMAP → 3D Gaussian Splatting

### How it is supposed to work

![Standard 3DGS pipeline](https://huggingface.co/acd23/cad-3d-reconstruction/resolve/main/report_assets/10_pipeline_3dgs.png)

3D Gaussian Splatting represents a scene as millions of little translucent coloured blobs ("Gaussians"). Training nudges the blobs until, rendered from each known camera position, they reproduce the training photos. **It cannot start without knowing where each photo was taken from.** Those camera positions normally come from **COLMAP**, a classic Structure-from-Motion tool that:

1. finds distinctive keypoints in each image (SIFT),
2. matches keypoints between image pairs,
3. keeps only matches consistent with a rigid camera motion ("geometric verification"),
4. solves for camera positions and a sparse 3D point cloud.

### What actually happened

I re-ran COLMAP myself rather than trusting the earlier report. Results, using generous settings (affine-shape estimation, domain-size pooling, guided matching, and a lowered `init_min_num_inliers` of 15):

| Part | SIFT keypoints per view | Geometrically verified pairs | Images registered | 3D points |
|---|---|---|---|---|
| `bracket` | 149, 304, 283, 445 | **0** | 0 of 4 | 0 |
| `connector_module` | 63, 101, 156, 137 | **0** | 0 of 4 | 0 |
| `flange_housing` | 128, 225, 111, 144 | **0** | 0 of 4 | 0 |
| `timing_pulley` | 54, 143, 108, 355, 386 | 1 | **2 of 5** | 45 |
| *real textured photo, same settings* | *9,670 / 12,445 / 9,221* | — | — | — |

That last row is the key comparison. Running the **exact same COLMAP settings** on ordinary textured photographs yields around **10,000 keypoints per image**. These CAD views yield **54 to 445** — between 20× and 200× fewer.

And here is *where* the few keypoints that exist actually sit:

![SIFT keypoints](https://huggingface.co/acd23/cad-3d-reconstruction/resolve/main/report_assets/02_sift_keypoints.png)

Every red circle is a keypoint. Notice they cluster almost entirely on the **silhouette** — the outer boundary against the background — and hardly at all on the part's surfaces. This is exactly the failure mode described in §3(b). Silhouette points are useless for matching: the silhouette is a different physical curve on the object in every view, so even when two silhouette keypoints "look" similar, they are not the same 3D point, and geometric verification correctly throws them away. Hence 0 verified pairs.

**Verdict: COLMAP fails, so classic 3DGS never gets off the ground.** This part of the earlier report was right. The one nuance worth recording is that `timing_pulley` is not a clean zero — one image pair does verify and 2 of 5 images register, yielding 45 points. That is still far too little to train anything.

> ⚠️ **A practical trap worth writing down.** The first time I ran this I passed `--ImageReader.single_camera 1`. Because the views have different resolutions, COLMAP silently stopped after the first image and reported a 0-keypoint "failure" for everything else. If you ever re-run this, **leave `single_camera` off** for this dataset, or you will diagnose the wrong problem.

---

## 5. Approach 2 — MASt3R for camera poses and dense points

### The idea

Since the classical feature matcher failed, the fallback was **[MASt3R](https://github.com/naver/mast3r)** (NAVER, 2024) — a neural network that takes unposed images and directly predicts, for every pixel, where that pixel sits in 3D. It does not use SIFT, so texture-free surfaces are much less of a problem in principle. Its `sparse_global_alignment` step then tries to rotate, translate and rescale each view's prediction so that all views agree on one shared 3D scene.

Settings used: `MASt3R_ViTLarge_BaseDecoder_512_catmlpdpt_metric`, images resized to 512 px, complete (all-pairs) scene graph, 300+300 alignment iterations, `shared_intrinsics=False`.

### What came out

| Part | Points (raw) | Points (after outlier removal) | Estimated focal length per view (px) |
|---|---|---|---|
| `bracket` | 616,792 | 597,811 | 758, 1190, 1392, **901** |
| `connector_module` | 203,660 | 198,395 | 645, 1388, 614, **1480** |
| `flange_housing` | 259,427 | 255,574 | 1376, 1592, 994, **979** |
| `timing_pulley` | 423,906 | 418,816 | 799, 873, 632, 1111, **778** |

**The point counts look impressive. They are not evidence of success.** Two things give the game away.

**First, the focal lengths.** Each part is one physical object photographed with one lens. The focal length estimates should be nearly identical across the views of a part. Instead they vary by up to **2.4×** (`connector_module`: 614 vs 1480). When a solver returns wildly inconsistent camera parameters for a fixed camera, it means the problem was under-constrained — the solver had enough freedom to fit each view independently rather than being forced into one consistent answer.

**Second — and this is decisive — the views were never merged.** `timing_pulley` conveniently ships a `pointcloud_all.ply` whose points map one-to-one onto the per-view depth images, so I could colour every point by which photo it came from:

![Views not fused](https://huggingface.co/acd23/cad-3d-reconstruction/resolve/main/report_assets/03_mast3r_views_not_fused.jpg)

Each colour is one of the five input views. If fusion had worked, the five colours would be interleaved all over one pulley-shaped surface. Instead you can see **five separate curved sheets**, stacked at different depths and different scales, all facing roughly the same direction. It is five billboards, not one object.

Measured rather than eyeballed — for each pair of views, the fraction of one view's points that have a point from the other view within 1% of the scene diagonal:

```
        v1    v2    v3    v4    v5
v1    1.00  0.18  0.00  0.07  0.16
v2    0.32  1.00  0.08  0.17  0.67
v3    0.00  0.06  1.00  0.10  0.30
v4    0.03  0.13  0.13  1.00  0.22
v5    0.06  0.23  0.30  0.10  1.00
```

Views 1 and 3 share **0.00** — literally nothing. A correctly fused reconstruction of a small object seen from five angles would show high overlap wherever two views see the same surface. The per-view extents disagree too: view 1 spans 1.79 units in depth while view 2 spans 0.42, i.e. the views are reconstructed at inconsistent **scales**.

And here is the resulting cloud for each part, rendered from six angles:

| | |
|---|---|
| ![bracket pc](https://huggingface.co/acd23/cad-3d-reconstruction/resolve/main/report_assets/04_pointcloud_bracket.jpg) | ![connector pc](https://huggingface.co/acd23/cad-3d-reconstruction/resolve/main/report_assets/04_pointcloud_connector_module.jpg) |
| `bracket` — 597,811 points | `connector_module` — 198,395 points |
| ![flange pc](https://huggingface.co/acd23/cad-3d-reconstruction/resolve/main/report_assets/04_pointcloud_flange_housing.jpg) | ![pulley pc](https://huggingface.co/acd23/cad-3d-reconstruction/resolve/main/report_assets/04_pointcloud_timing_pulley.jpg) |
| `flange_housing` — 255,574 points | `timing_pulley` — 418,816 points |

The `connector_module` case is the clearest: two flat slabs, one from the top-down catalogue view and one from the side view, sitting apart from each other and never joined into a box. The `bracket` "top" view shows two sheets meeting at an angle like an open book.

**Verdict: MASt3R ran successfully as a program but failed as a reconstruction.** It is a genuinely good model — it is doing the best anyone could with 4 views separated by 90° of a texture-free object. There simply is not enough shared surface between the views for the global alignment step to lock them together.

---

## 6. Approach 3 — turning the point cloud into a mesh

Two standard Open3D methods were run on the MASt3R clouds:

- **Poisson surface reconstruction** (`depth=9`, then the lowest-5%-density vertices trimmed) — fits a smooth watertight surface through the points.
- **Ball pivoting** (radii 0.005 / 0.01 / 0.02 / 0.04) — rolls a virtual ball over the points and connects any three it rests on.

| Part | Poisson verts / faces | Ball-pivot verts | Watertight? |
|---|---|---|---|
| `bracket` | 648,259 / 1,286,175 | 293,667 | No |
| `connector_module` | 328,776 / 652,457 | 173,525 | No |
| `flange_housing` | 284,107 / 563,932 | 202,967 | No |
| `timing_pulley` | 586,848 / 1,164,434 | 186,693 | No |

![poisson pulley](https://huggingface.co/acd23/cad-3d-reconstruction/resolve/main/report_assets/05_poisson_timing_pulley.jpg)
*`timing_pulley` Poisson mesh. The front view shows the tooth pattern pressed flat into a single plane, like a coin rubbing — because that is what the input point cloud is. The side view shows the other views standing up as separate walls.*

![poisson flange](https://huggingface.co/acd23/cad-3d-reconstruction/resolve/main/report_assets/05_poisson_flange_housing.jpg)
*`flange_housing` Poisson mesh — two disconnected lumps, one per view cluster.*

Meshes for the other two parts: [bracket](https://huggingface.co/acd23/cad-3d-reconstruction/resolve/main/report_assets/05_poisson_bracket.jpg) · [connector_module](https://huggingface.co/acd23/cad-3d-reconstruction/resolve/main/report_assets/05_poisson_connector_module.jpg).

**Verdict: the meshing step is not at fault.** Poisson and ball pivoting did exactly their job — they wrapped a surface around the points they were given. Because the points were separate floating sheets, the surface is separate floating shells. None of the meshes is watertight, and none resembles its part.

---

## 7. Approach 4 — BrepGaussian

### 7.1 What the paper actually does

[BrepGaussian: CAD reconstruction from Multi-View Images with Gaussian Splatting](https://arxiv.org/abs/2602.21105) (Yu, Ren, Xu, Yang, Li, Guo, Zhou & Guo — Nanjing University, CVPR 2026). Code: [github.com/yjx2851/BrepGaussian](https://github.com/yjx2851/BrepGaussian).

It goes further than an ordinary mesh. A mesh is a bag of triangles; a **B-Rep** ("boundary representation") is what CAD software actually uses — named surfaces (*this face is a plane, that one is a cylinder of radius r*), the edges where they meet, and the corners where edges meet. That is the format you can take back into Fusion or SolidWorks and edit.

The pipeline has four parts:

1. **2D label extraction.** An edge detector produces an edge map for every view. Those edge maps are then used as prompts to **SAM** to produce a per-face mask for every view.
2. **Stage 1 training.** A **2DGS** model (flat surfels, not 3D blobs — chosen because they have no artificial thickness, which suits flat CAD faces) is trained on the photos. Each Gaussian also carries one extra learnable number: an **edge score** in [0,1], rendered through the same alpha blending as colour and supervised by the edge maps.
3. **Stage 2 training.** Geometry is **frozen**. Each Gaussian gets a 16-dimensional **feature vector**, trained with a **triplet loss** (anchor / positive from the same mask / hardest negative from a different mask, using cosine distance and a margin) so that Gaussians on the same physical face end up with similar features.
4. **Sampling + B-Rep fitting.** Gaussians are sampled into a labelled point cloud. RANSAC then fits a **plane, cylinder or sphere** to each face, and a constraint-guided module extracts edges and corners, refines them against the point cloud, and assembles a watertight B-Rep bottom-up.

**Key numbers from the paper, which set the bar:**

- **50 views per object**, from the ABC-NEF subset of the ABC CAD dataset.
- **Stage 1: 30,000 iterations. Stage 2: 15,000 iterations** (loading the Stage-1 checkpoint at iteration 30,000).
- Reported quality: patch-segmentation F1 **0.9040** and edge-segmentation F1 **0.9301** from multi-view input.
- **The paper's own ablation on view count is the single most relevant fact for us.** Training with 20 / 30 / 40 / 50 views, the authors write that *"using 30 views or fewer results in noisy patch labels and incomplete reconstructions, while 50 views are required to produce clean, dense, and high-quality B-Rep reconstructions."*
- The masks are **not fully automatic**: SAM alone gives "inconsistent and fragmented masks across views" on low-texture CAD renders, so the authors **manually refined each object with a script, about three minutes per object.**

> By the authors' own measurement, **30 views is already too few**. This dataset has **4–5**. BrepGaussian was being asked to work about 10× outside its demonstrated operating range.

### 7.2 What could not be run, and what was substituted

Three hard blockers prevented running the official code as published:

| Requirement | Why it was blocked | Substitute used |
|---|---|---|
| COLMAP camera poses | COLMAP fails on this data (§4) | MASt3R poses |
| Custom CUDA rasterizer `diff-surfel-segment-rasterization` | No `nvcc` in the environment, so the extension cannot be compiled | A hand-written pure-PyTorch splat renderer (`code/simple_renderer.py`) |
| Fine-tuned SAM checkpoint (hosted on ModelScope) | Not retrievable in this environment | OpenCV Canny edges + watershed segmentation |

![Actual pipeline](https://huggingface.co/acd23/cad-3d-reconstruction/resolve/main/report_assets/11_pipeline_actual.png)

These are large substitutions. It is fair to call the result *"a pipeline inspired by BrepGaussian"*, not *"BrepGaussian"*. Two of the three swaps turned out to matter enormously.

**The edge maps were fine.** Canny is genuinely a reasonable stand-in here — CAD renders have crisp, clean boundaries:

![Edges and masks](https://huggingface.co/acd23/cad-3d-reconstruction/resolve/main/report_assets/06_edges_and_masks.png)
*Left: input. Middle: Canny edges — clean and usable. Right: watershed "instance masks" — **broken**.*

**The masks were not fine, and the reason is visible in the picture above.** In both rows, the big coloured region (bright green, and yellow) is the **background**, and the part itself is **black** — meaning unlabelled and discarded. The substitute segmenter labelled precisely the wrong thing. The cause is in `generate_instance_masks()`: it seeds watershed from a distance transform of the *non-edge* pixels, so the largest blob wins, and on a plain studio background that is always the background. The code then does `markers[markers == 1] = 0` on the assumption that region 1 is the background — but which region gets index 1 depends on connected-component ordering, not on what it actually is.

The measured consequence — distinct mask regions found per view:

| Part | Regions per view |
|---|---|
| `bracket` | 4, 2, 2, 3 |
| `connector_module` | 1, 5, 2, 4 |
| `flange_housing` | **1, 1, 1, 1** |
| `timing_pulley` | 3, 2, 1, 4, 2 |

`flange_housing` got exactly one region in every view. Stage 2's whole job is to tell faces apart; with one region there is nothing to contrast, so Stage 2 had no signal at all.

### 7.3 Result A — the adapted two-stage training was effectively a no-op

This is the most important correction in this document. The `metadata.json` files record what actually ran:

| | Paper | Repo README claims | **Actually run** |
|---|---|---|---|
| Gaussians | ~100k+ | 10,000 | **500** |
| Stage 1 iterations | 30,000 | 2,000 | **200** |
| Stage 2 iterations | 15,000 | 1,000 | **100** |

Only a single loss value was ever logged (at iteration 0), because the logging interval is every 500 iterations and the run was 200 iterations long. So there is **no recorded evidence that the loss ever went down.**

I tested whether training changed anything at all, by measuring how far each final Gaussian ended up from its starting position:

| Part | Mean movement, as % of the object's size |
|---|---|
| `bracket` | 0.14% |
| `connector_module` | 0.13% |
| `flange_housing` | 0.09% |
| `timing_pulley` | 0.13% |

**The Gaussians moved about one thousandth of the object's size.** With a position learning rate of 0.00016 over 200 steps, that is exactly what you would predict. `gaussian_splat.ply` is, for practical purposes, a 500-point random subsample of the MASt3R point cloud with a new file name. No Gaussian Splatting meaningfully occurred.

Downstream, the labelled point cloud `merged.pcd` gets its labels from k-means with `n_clusters = min(10, max(2, n_points // 5000))`. With 500 points that evaluates to **2** — two "faces" for the whole part. `flange_housing` ended up with just **one** label.

Feeding that to the official B-Rep fitter produced:

| Part | Surfaces found | Edges | Corners | Curves |
|---|---|---|---|---|
| `bracket` | **0** | 0 | 0 | 0 |
| `connector_module` | 1 plane | 0 | 0 | 0 |
| `flange_housing` | 1 plane | 0 | 0 | 0 |
| `timing_pulley` | **0** | 0 | 0 | 0 |

The `pipeline_results_structured.obj` files in `brepgaussian_results/*/brep_output/` are **39 bytes each**, containing a single `mtllib` line and not one vertex or face. There is no model in them.

### 7.4 Result B — the fallback: fit B-Rep directly to the dense point cloud

Recognising 500 points was not enough, a second script (`code/pipeline_brep_dense.py`) skipped the Gaussian training entirely: take 10,000 points from the MASt3R cloud, estimate normals, k-means into 8 clusters using normal + position, mark cluster-boundary points as "edges", and hand that to the official fitter.

| Part | Planes | Cylinders | Edge intersections | Corners | Curves |
|---|---|---|---|---|---|
| `bracket` | 8 | **0** | 19 | 28 | 0 |
| `connector_module` | 8 | **0** | 22 | 20 | 0 |
| `flange_housing` | 7 | **0** | 12 | 20 | 0 |
| `timing_pulley` | 7 | **0** | 14 | 22 | 0 |

This at least produces geometry. But note two things.

**First, zero cylinders anywhere.** A timing pulley is fundamentally a cylinder with teeth; a flange housing is a stack of cylinders. Getting 0 cylinders out of 4 parts means no rotational geometry was recovered at all.

**Second, the "8 faces" are not discovered faces — they are the 8 clusters we asked k-means for.** The number of planes equals the `n_clusters=8` parameter (minus any cluster RANSAC rejected). Change that parameter to 5 and you get 5 planes. The pipeline is fitting a plane to each arbitrary slice of a broken point cloud, which is a very different thing from recognising the actual faces of a part.

![brepdense pulley](https://huggingface.co/acd23/cad-3d-reconstruction/resolve/main/report_assets/08_brepdense_timing_pulley.jpg)
*`timing_pulley` B-Rep fit. Blue lines are fitted edges, red dots corners. It is a handful of intersecting sheets.*

Others: [bracket](https://huggingface.co/acd23/cad-3d-reconstruction/resolve/main/report_assets/08_brepdense_bracket.jpg) · [connector_module](https://huggingface.co/acd23/cad-3d-reconstruction/resolve/main/report_assets/08_brepdense_connector_module.jpg) · [flange_housing](https://huggingface.co/acd23/cad-3d-reconstruction/resolve/main/report_assets/08_brepdense_flange_housing.jpg).

### 7.5 Result C — the paper's fitting code, on the paper's own data, works

To separate "the fitting code is broken" from "we fed it rubbish", the official fitter was run on the example that ships with the paper's repo: `00000699_stage2/merged.pcd`, a 9,286-point cloud with 7 patch labels, produced by the authors' own fully-trained Stage 2 model.

| | Result |
|---|---|
| Surfaces | 7 — **6 planes + 1 cylinder** |
| Edge intersections | 14 |
| Corners | 8 |
| Lines / curves | 12 / 2 |

![Official B-Rep](https://huggingface.co/acd23/cad-3d-reconstruction/resolve/main/report_assets/07_brep_official_example.jpg)
*This is what success looks like: a clean plate with a circular hole, correctly trimmed edges (blue) and corners (red).*

**This is the single most useful positive result in the whole project.** It cleanly isolates the problem:

> The B-Rep fitting module is **not** the bottleneck. It works correctly and produces proper CAD geometry — including cylinders — **when given a correctly labelled point cloud.** Everything upstream of it, on this dataset, failed to produce one.

---

## 8. Corrections to the earlier report

The repository's existing `FEASIBILITY_REPORT.md` and `README.md` present these results as successes. After re-running and inspecting everything, several claims do not hold. Recording them so the record is straight:

| Earlier claim | What is actually true |
|---|---|
| "MASt3R **successfully reconstructed** all 4 parts" | MASt3R produced point clouds, but the views were never fused. Cross-view overlap is 0.00–0.30; the result is separate per-view sheets, not a reconstructed object. **This is the main correction.** |
| "COLMAP output: 0 images registered" for all parts | True for 3 of 4. `timing_pulley` registers 2 of 5 images with 45 points (still unusable, but not zero). |
| Bounds table, e.g. `flange_housing 1.42 × 1.89 × 2.57` | Those are the *raw* bounds before outlier removal. The point cloud actually shipped is 0.73 × 1.12 × 2.13. |
| "Training iters: 2k (S1) + 1k (S2)" | `metadata.json` records **200 + 100**. |
| `pipeline_brepgaussian.py` sets `n_gaussians = min(10000, len(pts))` | Every run used **500** Gaussians. The code in `code/` is **not** the code that produced the results in `brepgaussian_results/`. |
| "BrepGaussian adapted pipeline … produces analytic surface patches" | It produced 0 surfaces for 2 parts and 1 plane for the other 2, with 0 edges, 0 corners and 0 curves everywhere. The structured OBJ outputs are empty files. |
| "B-Rep fitting … works correctly" | Correct, but only on the paper's own example data. On our data it never produced a cylinder or a single edge. |
| README `Usage` block showing `AutoModelForCausalLM.from_pretrained(...)` | Boilerplate that does not apply — this repo contains meshes and scripts, not a language model. It will not run. |

None of this means the earlier work was wasted — the pipelines are real, they run, and they narrowed down the problem. The issue is only that the outputs were never opened and looked at before being described as successes.

---

## 9. Why it all failed — three root causes, in order of importance

**1. The data (accounts for essentially all of it).** 4–5 views, ~90° apart, no surface texture. Every method tried needs to see the same piece of surface in at least two images. This data barely satisfies that anywhere. COLMAP fails at the matching step; MASt3R fails at the fusion step; both failures have the same underlying cause.

**2. The substitutions changed the method into a different method.** The watershed mask generator labelled the background instead of the part, which removed Stage 2's supervision entirely. The pure-PyTorch renderer loops over Gaussians one at a time in Python, which is roughly 1,000× slower than the CUDA rasterizer and is almost certainly why the run was cut to 500 Gaussians and 200 iterations — which in turn made the training a no-op.

**3. BrepGaussian was outside its stated operating range from the start.** The paper's own ablation says ≤30 views gives "noisy patch labels and incomplete reconstructions". We had 4–5. Even a flawless reimplementation would not have worked on this data.

---

## 10. What to do instead

### If you can re-capture the parts — do photogrammetry properly

This is by far the highest-value change, and it is a capture problem, not a software problem:

- **24–60 images per part**, in a continuous orbit, **no more than 10–15° between neighbouring shots**. Two orbits at different heights is better than one.
- **Lock the camera's focal length** and do not crop between shots. (Then `--ImageReader.single_camera 1` becomes correct and helps.)
- **Give the surface something to look at.** Matte anti-glare spray is the standard workshop trick. Failing that, project a random speckle pattern onto the part, or lay it on a textured/newspaper-print mat — background texture alone lets COLMAP solve the camera motion even when the part itself is bare.
- Diffuse, even lighting; avoid moving specular highlights, which look like surface features but move with the camera.

With that, plain COLMAP + 3DGS (or better, **2DGS** — flat surfels give much cleaner meshes on flat CAD faces) will work, and BrepGaussian becomes worth attempting for real.

### If you are stuck with 1–5 catalogue views — stop using photogrammetry

Multi-view geometry fundamentally cannot solve this input. Use a **feed-forward image-to-3D generative model** instead. These are trained on millions of 3D objects and *hallucinate* a plausible complete shape from one or a few images, rather than triangulating it:

- **[TRELLIS](https://github.com/microsoft/TRELLIS)** (Microsoft, MIT licence). The public repo ships `TRELLIS-image-large` (1.2 B params) and, importantly here, a **tuning-free multi-image conditioning mode** — so it can consume all 4–5 views at once rather than just one. A larger **TRELLIS.2** has since been reported; worth checking for a current release before you start.
- **Hunyuan3D 2.1 / 3.5** (Tencent) — supports single-image *and* multi-view input. A published comparison on the **ABC mechanical-parts dataset** reported Hunyuan3D ahead of TRELLIS on IoU across easy/medium/hard parts, which would make it the stronger choice for engineering geometry. Treat that as a pointer, not a settled result — verify it yourself, and note the comparison predates the newer TRELLIS release.

> 💡 **You can run this today.** This machine already has everything needed: the `trellis` conda env (PyTorch 2.5.1 + CUDA, working), the code at `autosegment/sam3finetunetest/TRELLIS` — which already includes an `app_multiview_parts.py` demo built for exactly this kind of mechanical part — and the `microsoft/TRELLIS-image-large` weights already downloaded. Feeding these same 17 images in and comparing against column 1 of the results matrix is a same-day experiment. Expect it to beat everything in this report, because it does not depend on view overlap at all.

**Caveat to be clear about:** these models *invent* the unseen geometry. The output will look like the part and will be dimensionally approximate, not metrologically correct. For visualisation, quoting or catalogue work that is usually fine. For manufacturing it is not.

### If the real goal is a CAD/B-Rep file

Split it into two independent problems, because the "images → CAD in one shot" route is not mature enough to rely on:

1. **Images → a good mesh.** Either proper photogrammetry (above) or a generative model (above).
2. **Mesh/point cloud → B-Rep.** This step is mature and works well on clean input:
   - **[Point2CAD](https://github.com/YujiaLiu76/point2cad)** — the baseline BrepGaussian itself compares against; fits planes/cylinders/spheres/B-splines and extracts topology.
   - **SED-Net** — scored the highest patch-segmentation F1 (0.9738) in the paper's own comparison table when given clean points.
   - **CADFit** (arXiv 2605.01171) — a 2026 mesh-to-CAD-program method worth evaluating.
   - Commercial reverse-engineering (Geomagic Design X, Fusion 360 Mesh-to-BRep) remains the reliable production answer.

> ⚖️ **Licence warning, if any of this is headed for a product.** Two of the components used or recommended here are **non-commercial only**: **MASt3R** is CC BY-NC-SA 4.0, and **Point2CAD** is CC BY-NC 4.0 (the authors offer separate commercial licensing). TRELLIS is MIT and is the safe default. Check this before building anything on top of the `mast3r_results/` outputs.

### If you specifically want BrepGaussian to run

Then you need, in order: **50 views per object**, a machine with **`nvcc`** so `diff-surfel-segment-rasterization` compiles, the **fine-tuned SAM checkpoint from ModelScope**, and budget for **~3 minutes of manual mask cleanup per object** as the authors describe. Without all four, you are running a different algorithm.

---

## 11. What is in the repository

```
├── METADOC.md                     ← this document
├── FEASIBILITY_REPORT.md          earlier report — superseded by §8 above
├── report_assets/                 every figure in this document
│   ├── 01_dataset.png .. 11_pipeline_actual.png
│   └── verification_summary.json  all re-verified numbers, machine-readable
├── verification_code/             the scripts used to re-check the results
│   ├── run_colmap_verify.py       independent COLMAP re-run + match statistics
│   ├── render3d.py                headless Open3D renderer used for every 3D figure
│   └── make_pipeline_figs.py      pipeline diagrams
├── code/                          the original pipeline scripts
│   ├── prepare_images.py          RGBA→RGB on white, resize to ≤1600 px
│   ├── run_colmap.py              COLMAP via pycolmap
│   ├── pipeline_mast3r.py         MASt3R reconstruction + meshing + COLMAP export
│   ├── pipeline_brepgaussian.py   adapted two-stage pipeline (see §7.3 caveat)
│   ├── pipeline_brep_dense.py     dense-cloud B-Rep fallback
│   ├── simple_renderer.py         pure-PyTorch splat renderer
│   └── render_previews.py         matplotlib previews
├── input_images/<part>/           prepared inputs + COLMAP workspaces
├── mast3r_results/<part>/         point clouds, meshes, poses, GLB, previews
├── brepgaussian_results/<part>/   adapted-pipeline outputs (empty B-Reps — §7.3)
├── brepgaussian_brep_dense/<part>/dense-cloud B-Rep outputs (§7.4)
└── brep_example_input/ + brep_example_output/   the paper's own example (§7.5)
```

**Which files are actually worth opening?**

| Want to see… | Open |
|---|---|
| A working B-Rep | `brep_example_output/pipeline_results.obj` |
| The best (still poor) result on our parts | `brepgaussian_brep_dense/<part>/brep_output/pipeline_results.obj` |
| The raw reconstruction | `mast3r_results/<part>/pointcloud.ply` |
| Camera poses, for reuse | `mast3r_results/<part>/colmap_sparse/transforms.json` |
| Every verified number | `report_assets/verification_summary.json` |

`.ply` / `.obj` open in [MeshLab](https://www.meshlab.net/), [Blender](https://www.blender.org/) or [3dviewer.net](https://3dviewer.net/); `.glb` in [gltf.us](https://gltf.us/).

---

## 12. Reproducing the checks in this document

```bash
# 1. Independent COLMAP re-run (§4). Note: single_camera must stay OFF.
python verification_code/run_colmap_verify.py     # writes colmap_verify.json

# 2. Render any point cloud or mesh from six angles (§5, §6, §7)
python verification_code/render3d.py mast3r_results/bracket/pointcloud.ply out/bracket_pc

# 3. Confirm the Gaussian training was a no-op (§7.3):
#    compare gaussian_splat.ply against the MASt3R cloud it was initialised from —
#    mean nearest-neighbour distance comes out at ~0.1% of the scene diagonal.
```

Environment used: NVIDIA L4 (24 GB), COLMAP 3.8 (CPU SIFT — this build has no CUDA), Open3D 0.19 with EGL offscreen rendering, PyTorch with CUDA.

---

## 13. Bottom line

- **Is 3D Gaussian Splatting feasible for CAD parts?** Yes in general — but **not from this data**. It needs camera poses, and camera poses need many overlapping views. With 4–5 views 90° apart on texture-free surfaces, neither COLMAP nor MASt3R can supply them. Nothing was built here that you could call a Gaussian splat.
- **Is BrepGaussian the right paper?** It targets exactly the right output — a real CAD B-Rep with planes, cylinders, edges and corners. But it needs **50 views**, a compiled CUDA rasterizer, a fine-tuned SAM, and a few minutes of manual mask cleanup per part. Its own ablation says 30 views is already too few. It is a paper to revisit **after** the capture setup is fixed, not before.
- **The one genuinely working piece** is the official B-Rep fitting module, verified end-to-end on the paper's own data (§7.5). If you can produce a well-segmented point cloud by any means, that module will turn it into CAD geometry.
- **The fastest useful next step** is to point TRELLIS.2 or Hunyuan3D at these same 17 images. It sidesteps the view-overlap problem entirely and you have the environment already.
- **The highest-value next step** is a proper 24–60 image orbit with a locked lens and a matte finish. That single change unblocks every method in this report.