Machine Learning & Matching: High Volume Data Acquisition and Reconstruction Pipelines

This project develops efficient 3D acquisition systems that use machine learning to reconstruct artifacts. A notable project developed a system used by archeologist at the Akrotiri excavation site in Greece to reconstruct the Thera Frescoes. There were several challenges faced working with the frescos. The range of fragment sizes, effects of erosion, and large regions of pure white posed challenges.

In my contribution to this project, I developed an algorithm that computes 3D surface normals from 2D scans with sub-millimeter accuracy, capturing geometric discontinuities, textures and brushstroke orientations that prove more effective than traditional color and 3D shape for matching. I also developed a multifeature matching pipeline that uses logistic regression to find matches. The system out-performed state-of-the-art 3D matching algorithms, solving problems with scalability, efficiency, data validation and usability.  New matches were found in seconds (manual methods take weeks to years). 

Learning How to Match Fresco Fragments

Learning How to Match Fresco Fragments

Thomas Funkhouser, Hijung Shin, Corey Toler-Franklin, Antonio Garcia Castaneda, Benedict Brown, David Dobkin, Szymon Rusinkiewicz, Tim Weyrich.
Eurographics 2011 Special Area Track on Cultural Heritage, Llandudno, UK, April 2011

Journal Papers: Learning How to Match Fresco Fragments

Journal Papers
Learning How to Match Fresco Fragments

Thomas Funkhouser, Hijung Shin, Corey Toler-Franklin, Antonio Garcia Castaneda, Benedict Brown, David Dobkin, Szymon Rusinkiewicz, Tim Weyrich.
Journal on Computing and Cultural Heritage 4(2), November 2011.

Multi-Feature Matching of Fresco Fragments

Multi-Feature Matching of Fresco Fragments

Corey Toler-Franklin, Benedict Brown, Tim Weyrich, Thomas Funkhouser, Szymon Rusinkiewicz.
ACM Transactions on Graphics (Proc. SIGGRAPH Asia), Seoul, Korea, December 2010.

A System for High-Volume Acquisition and Matching of Fresco Fragments: Reassembling Tehran Wall Paintings

A System for High-Volume Acquisition and Matching of Fresco Fragments: Reassembling Tehran Wall Paintings

Benedict Brown, Corey Toler-Franklin, Diego Nehab, Michael Burns, Andreas Vlachopoulos, Christos Doumas, David Dobkin, Szymon Rusinkiewicz, Tim Weyrich.

ACM Transactions on Graphics (Proc. SIGGRAPH), Los Angeles, CA, August, 2008.