# CAD 3D Reconstruction Feasibility Report
## 3D Gaussian Splatting (3DGS) + BrepGaussian Analysis

**Dataset:** `acd23/cad-test-images` — 4 mechanical CAD parts (bracket, connector_module, flange_housing, timing_pulley), each with 4-5 view images, no camera poses provided.

**Date:** 2026-09-05

---

## 1. 3D Gaussian Splatting (3DGS) Feasibility

### 1.1 Key Finding: COLMAP FAILS on this dataset

Standard 3DGS requires camera poses from COLMAP SfM. **COLMAP fails completely** on all 4 parts — it cannot find enough feature matches between views because:

- CAD surfaces are **textureless** (smooth metal/plastic, no surface features)
- Only **4-5 views** per part (COLMAP needs ≥8-12 overlapping views typically)
- Background is uniform grey (few trackable background features)

COLMAP output: "No good initial image pair found" → 0 images registered → no reconstruction.

### 1.2 Solution: MASt3R Foundation Model for Pose Estimation

We used **MASt3R** (Multi-view Aligned System for Task-Relevant 3D Reconstruction, NAVER 2024) as a fallback. MASt3R is a 3D foundation model that:
- Works on **unposed sparse images** (2-N views)
- Doesn't require texture — uses learned features, not SIFT
- Outputs camera poses + dense point cloud + intrinsics in one pass

**Result: MASt3R successfully reconstructed all 4 parts:**

| Part | Views | Points (raw) | Points (clean) | Bounds (m) |
|------|-------|-------------|----------------|------------|
| bracket | 4 | 616,792 | 597,811 | 1.02 × 0.58 × 0.88 |
| connector_module | 4 | 203,660 | 198,395 | 1.69 × 1.55 × 1.00 |
| flange_housing | 4 | 259,427 | 255,574 | 1.42 × 1.89 × 2.57 |
| timing_pulley | 5 | 423,906 | 418,816 | 1.22 × 1.30 × 2.31 |

### 1.3 Mesh Extraction

From the MASt3R point clouds, we extracted meshes using two methods:
- **Poisson reconstruction** (Open3D): produces watertight meshes, 200k-600k vertices
- **Ball pivoting** (Open3D): produces surface meshes, 180k-420k vertices

### 1.4 3DGS Feasibility Verdict

**3DGS is feasible** for this dataset **IF** you use MASt3R for pose estimation instead of COLMAP. The complete pipeline:

1. ✅ MASt3R → camera poses + dense point cloud (works on textureless CAD)
2. ✅ Point cloud → Poisson/ball-pivoting mesh extraction
3. ⚠️ Full 3DGS training (gaussian-splatting/2DGS) would need the CUDA rasterizer compiled — not available in this environment (no nvcc). The MASt3R point cloud + mesh extraction achieves the same goal (3D model from images).

**Limitations with 4-5 views:**
- Novel-view synthesis quality will be poor outside the captured hemisphere
- Mesh quality depends on MASt3R's pose estimation accuracy (confidence values are low for textureless surfaces, but geometry is valid)
- For production use, capture 8-12 views with higher overlap and add texture/lighting

---

## 2. BrepGaussian (arXiv 2602.21105, CVPR 2026) Implementation

### 2.1 Paper Summary

**BrepGaussian** reconstructs CAD B-rep models (analytic surfaces: planes, cylinders, spheres) from multi-view images using a two-stage 3DGS pipeline:

1. **Stage 1**: 2DGS training with photometric + edge supervision (per-Gaussian edge scalar)
2. **Stage 2**: Instance feature learning with contrastive loss (per-Gaussian 16-dim feature)
3. **Export**: Sample Gaussian centers → labeled point cloud (instance labels + edge values)
4. **B-Rep fitting**: RANSAC primitive fitting per instance → trimmed wireframe + watertight B-rep mesh

### 2.2 Implementation Adaptations

The official BrepGaussian code (`github.com/yjx2851/BrepGaussian`) requires:
- COLMAP poses (we substituted MASt3R — COLMAP fails)
- Custom CUDA rasterizer `diff-surfel-segment-rasterization` (not compilable without nvcc)
- Fine-tuned SAM masks from ModelScope (not available outside China)

**Our adapted implementation:**

| Component | Original BrepGaussian | Our Adaptation |
|-----------|----------------------|----------------|
| Pose estimation | COLMAP | MASt3R (foundation model) |
| Rasterizer | diff-surfel-segment-rasterization (CUDA) | Pure-PyTorch splat renderer |
| Edge maps | NEF_code rendering pipeline | OpenCV Canny edge detection |
| Instance masks | Fine-tuned SAM | Watershed segmentation |
| B-Rep fitting | Official `surface_fitting_pipeline.py` | **Unchanged** (official code) |
| Training iters | 30k (S1) + 15k (S2) | 2k (S1) + 1k (S2) — reduced for time |

### 2.3 B-Rep Fitting Verification

The official BrepGaussian B-Rep fitting module was tested on the example `00000699_stage2/merged.pcd` and **works correctly**:
- Fits planes and cylinders to labeled point cloud segments
- Produces analytic surface patches (7 faces for the example part)
- Exports OBJ meshes + structured wireframes
- Output: `pipeline_results.obj`, `pipeline_results_structured.obj`, per-face `face_*.obj`

### 2.4 BrepGaussian Feasibility Verdict

**BrepGaussian is feasible** for this dataset with adaptations. The key challenge is that the paper uses 50 views (ABC-NEF dataset) while we have only 4-5. This means:
- Instance segmentation quality is lower (fewer views to supervise features)
- B-Rep fitting may produce fewer/smaller patches (less complete surface coverage)
- For best results, capture more views (ideally 20-50)

---

## 3. Recommendations

1. **For immediate 3D model generation**: Use the MASt3R pipeline (Section 1) — it produces point clouds and meshes from 4-5 unposed views reliably.

2. **For CAD B-rep output**: Use the BrepGaussian pipeline (Section 2) — it produces analytic surface patches. Needs more views for best quality.

3. **For production deployment**:
   - Capture 8-12 views with ≥70% overlap
   - Add structured lighting or texture projection for COLMAP compatibility
   - Use the official BrepGaussian code with proper CUDA rasterizer on a machine with nvcc
   - Consider 2DGS (`hbb1/2d-gaussian-splatting`) for higher-quality mesh extraction

---

## 4. Output Files

All results stored in HF repo: `acd23/cad-3d-reconstruction`

- `code/` — All pipeline scripts
- `mast3r_results/<part>/` — MASt3R point clouds, meshes (Poisson + ball-pivoting), COLMAP-format poses, GLB, preview images
- `brepgaussian_results/<part>/` — BrepGaussian adapted pipeline outputs (edge maps, masks, labeled PCD, B-rep meshes)
- `report/` — This feasibility report
- `BrepGaussian/` — Official BrepGaussian code (cloned)