AI turns hidden Buddhist relief into 3D model using 134-year-old photo

A groundbreaking AI-driven project recently recreated hidden Buddhist relief panels at Indonesia’s Borobudur temple, a UNESCO World Heritage Site. Using artificial intelligence, a team of computer scientists reconstructed intricate 3D models from century-old grayscale photos, achieving a level of detail previously unreachable. Their work debuted as a proof-of-concept at the 32nd ACM Multimedia Conference last month.

A digital stereoscope for cultural heritage

The project’s success hinged on a neural network model designed to extract depth and detail from a single 2D image, essentially creating a “stereoscope” for the digital age. This tool allows the generation of 3D reconstructions even when only limited photographic data exists. “Our new method tackles this by enhancing depth estimation, particularly along soft edges, using a novel edge-detection approach,” explained Satoshi Tanaka, co-author and researcher at Japan’s Ritsumeikan University.

The Borobudur temple, which houses the world’s largest collection of Buddhist bas-reliefs, became an ideal testing ground for this technology. In the late 19th century, Borobudur’s lower level was enclosed, sealing 156 relief panels behind stone walls. Although photos of these panels were taken before their concealment, prior attempts to reconstruct their details failed due to compressed depth values in the 2D images. With this new AI model, the team achieved a 95% reconstruction accuracy, although some fine details still remain challenging.

Overcoming historical obstacles with technology

What sets this model apart is its approach to “soft edges” in relief carvings, where the curvature of stone surfaces represents depth. Previous models struggled with these subtleties, producing flattened images. To overcome this, researchers mapped edge curvature to better reflect 3D spatial changes. This adjustment proved essential in bringing out intricate elements in the artwork, such as human figures, symbols, and other decorations, that previous models struggled to capture.

The AI model combines two types of mapping: an “edge map” that tracks curves to add depth, and a “semantic map,” which categorizes elements of the image—like figures or background elements—by color. Together, these techniques allow the neural network to produce reconstructions that closely align with “ground truth” data.

Preserving the past for the digital age

Similar projects have been done before. They include using neural networks to reveal hidden details in Raphael’s artwork and to expand the catalog of Peru’s Nazca Lines. Tanaka believes the Borobudur reconstructions will benefit archaeologists and open new doors for virtual and augmented reality experiences, bringing ancient history to a wider audience.

“Our technology holds vast potential for preserving and sharing cultural heritage,” Tanaka noted. “It opens new opportunities not only for archaeologists but also for immersive virtual experiences through VR and metaverse technologies, preserving global heritage for future generations.”

While the AI model isn’t a substitute for the physical artifacts, it does offer a way to digitally preserve these treasures and bring them to life for viewers across the world. And if you ask me, it’s one of those useful, purposeful, and educational uses of artificial intelligence that we should strive for.

You can read more about the approach in the paper here.

[via Gizmodo]

---