This is a very cool piece of tech. Swedish developer Peder Norrby has done something pretty amazing with the iPhone X. He’s created a 3D optical illusion completely “in-camera”… well, in-phone, really, although it does use the phone’s front camera to achieve the effect. It’s very cool and doesn’t require any special viewing glasses. It holds all kinds of potential for the future as 3D cameras and 3D scanning start to become more advanced.
So, essentially, the way it works is with 3D head tracking with ARKit on the iPhone X. It uses the camera to look at the viewer and determine where their head is in 3D space. More specifically, where their eye is. Just one eye. It’s a monocular effect, so for best results, you’ll want to close one eye so the phone does a better job of tracking you, and the effect appears more realistic.
Once it knows where the eye is in 3D space, it creates a virtual camera at that same position pointed at a scene or object. This virtual camera then moves around the 3D scene following the movement of your eye. The phone then renders the view of the scene from the position of that camera and shows it on the phone’s screen. Essentially, it mimics what your eye would see at that angle and displays it to you.
Peder wrote an app that does all of this in realtime, and it’s not just portals he’s been playing with. He’s used the technique on several different models to show off the effect.
"It's bigger on the inside!" ?
— ΛLGΘMΨSΓIC (@algomystic) March 10, 2018
— ΛLGΘMΨSΓIC (@algomystic) March 23, 2018
Titled The Parallax View, Peder has uploaded the app to the App Store and you can download it completely free. He’s also written an article going into more depth on the subject, and he’s released the source code, too. But, there’s no support for the source. If you want to use it, you need to already have some idea of what you’re doing.
This isn’t the kind of thing you’re going to be able to watch with a friend. And, like I said, you’ll have to keep one eye closed for the best effect. While it does offer a 3D perspective effect, it’s designed for monocular vision.
But, it’s a very cool concept. I’m curious to see how far this technology may go.