sfm-core
Incremental structure-from-motion written from first principles: every estimator hand-built in NumPy, unit-tested against synthetic ground truth.
- 1.6 px
- mean reprojection error
- ~1,000
- 3D points from 5 images
- < 4 s
- full pipeline on a laptop
The rule: no black boxes
This is incremental structure-from-motion with a constraint attached: no
cv2.findFundamentalMat, no cv2.solvePnP, no reconstruction libraries.
Every geometric estimator is implemented by hand and unit-tested against a
synthetic rig with known ground truth: the eight-point algorithm,
essential-matrix decomposition, triangulation, PnP, and sparse bundle
adjustment with analytic Jacobians.
Five images of a building become ~1,000 3D points and 5 camera poses with a mean reprojection error of 1.6 px, in under 4 seconds on a laptop.
The details that actually moved the numbers
Textbook SfM is three formulas. Working SfM is the paragraph under each formula that nobody writes down. Each of these measurably changed the result:
- Track merging. Pairwise match files duplicate physical points; naive concatenation starves PnP of long tracks. A union-find pass keyed on shared keypoints merges them, rejecting any merge that would give a track two positions in one image.
- Iterated PnP registration. A single RANSAC + refine pass left the last camera with 5 of 291 inliers. Alternating LM refinement with inlier re-selection recovers 211 of 285.
- Gauge fixing. Bundle adjustment pins camera 0, but global scale is still free: left alone, the optimizer shrank the scene ~100× between rounds. Scale is re-pinned to the seed baseline after every BA pass.
- Outlier pruning. After each BA round, tracks whose worst observation still exceeds 8 px are dropped rather than left to bias the next round.
Where it came from
The starting point was coursework from RBE/CS549 (Computer Vision) at WPI, then rebuilt from scratch, properly this time: clean geometry modules, a real test suite, and honest documentation of what it took to make each stage converge.