Reassembling the Thera Frescoes: Automated Digitization and Matching of Fragments

Summary

The archaeological site of Akrotiri on the island of Thera (modern-day Santorini) has proven a treasure trove of information about Aegean civilization and culture. Among its most valued artifacts are wall paintings (frescoes), which have been preserved in the volcanic ash since the seventeenth century BC. The frescoes, however, are typically recovered in fragments of a few centimeters to a few tens of centimeters in length, and reconstructing complete wall sections occupies a major portion of the effort at Akrotiri.

We are engaged in a project to assist archaeologists and conservators by digitizing excavated fragments and using computer algorithms to automatically propose matches on the basis of 3D edge profile, color, and other cues. An intuitive user interface will allow conservators to see and evaluate matches on the basis of any or all of the above criteria. We hope to greatly reduce the time that is currently spent manually testing large numbers of fragments against each other in the search for matches.

Our project has three major components:

• Acquisition. Beginning with commercially-available color and 3D scanners, we built a hardware configuration for capturing the color texture of each fragment, together with its thickness and 3D shape. We have written custom software to control the scanners, ensuring that capture is easy (may be performed by non-computer-experts) and fast (no more than a few minutes per fragment). The captured data is automatically processed into a final model, and a simple interface allows the user to verify the result and quickly correct occasional errors. All captured data is stored together with semantic information about the fragment, as well as any annotations provided by archaeologists and conservators.

  The acquisition system and data processing algorithms are described in our Siggraph 2008 paper, " A System for High-Volume Acquisition and Matching of Fresco Fragments: Reassembling Theran Wall Paintings" (accompanying video).

  Our 3-D scanner setup.

  
  
  
• Matching algorithms. Building on recent research results, we are investigating a system for computing candidate matches between fragments, and proposing them to the user for verification. We will explore different cues for performing this matching, including the 3D shape of edges, fragment thickness, color and edges on the surface, any user-supplied annotations, and arbitrary combinations of different cues.

  We have started by developing a matcher based on the 3D edge information, called the "ribbon matcher." Designed to mimic a procedure used by the conservators to find and verify matches, the ribbon matcher exhaustively tests every possible configuration of a pair of fragments to find an optimal arrangement. On a modern PC, it takes about two seconds to test a pair of fragments. We achieve this performance by resampling each fragment edge in regular rows and columns (a "ribbon"). Using these ribbons, we can effectively roll one fragment around the other, updating the alignment and associated error in constant time at each step. The process is quite similar to a convolution, and is fully described in our Siggraph 2008 paper and accompanying video.

  The ribbon matcher operates by sliding a window across a pair of ribbons, incrementally updating the alignment and associated error. The process is quite similar to searching for a matching window in a pair of images.

  
  
  
• User interface. We are developing a computer program that allows users to move fragments or groups of fragments on the screen, easily evaluating the proposed matches. The software keeps track of information associated with each fragment, including excavation data, annotations, and the color and 3D scans. In addition, there is data associated with each match, including the quality of the match and any conservators' annotations. The software will also include features such as revision control and support for concurrency, allowing multiple users to work either simultaneously or serially.

Results

• Real Fragments
  As of August 2008 we have scanned 283 real fragments at Akrotiri grouped into three test sets &emdash; red (134 fragments), white (105 fragments), and spiral (44 fragments). These fragments come from the "spiral" wall painting, and were selected by the conservators for the test. Each set contains a small number of known matches, but most fragments do not match any others in the test set. The matches we have found with out ribbon matcher (and which the conservators have verified on the actual fragments) are shown below. The conservators know of two matches which we have not found, however two of the three white matches are "new," that is, the conservators were not aware of these matches until we proposed them.
  
• Test Fresco
  The conservators have also prepared a test fresco for us, in the style of the ancient frescoes. They applied an earthquake to this fresco, then mailed it to us. Although these fragments are not as old as the real ones, the large umber of matches helps us test whether we can effectively reconstruct an entire fresco. This fresco is also shown below, with all the matches than we found marked with a colored line. The different colors indicate different parameters to the ribbon matcher, which are fully described in our paper (video).
  
• Images of the results are shown here.

Press, Publications and Links

• 'Virtual archaeologist' reconnects fragments of an ancient civilization
  Princeton University press release, August 2008
• A System for High-Volume Acquisition and Matching of Fresco Fragments: Reassembling Theran Wall Paintings | [PDF] [MPEG4 Video]
  ACM Transactions on Graphics (Proc. SIGGRAPH), August 2008
  Benedict J. Brown, Corey Toler-Franklin, Diego Nehab, Michael Burns, David Dobkin, Andreas Vlachopoulos, Christos Doumas, Szymon Rusinkiewicz, and Tim Weyrich.
• Analyzing Fracture Patterns in Theran Wall Paintings | [PDF]
  The 11th International Symposium on Virtual Reality, Archaeology and Cultural Heritage (VAST), September 2010
  Hijung Shin, Christos Doumas, Thomas Funkhouser, Szymon Rusinkiewicz, Ken Steiglitz, Andreas Vlachopoulos, and Tim Weyrich.
• Multi-Feature Matching of Fresco Fragments | [PDF]
  ACM Transactions on Graphics (Proc. SIGGRAPH Asia), December 2010
  Corey Toler-Franklin, Benedict Brown, Tim Weyrich, Thomas Funkhouser, and Szymon Rusinkiewicz.
• Private Wiki