The system, called Photo Wake-Up, creates a 3D animation from a single photo. In the paper, the researchers compare it to the moving portraits at Hogwarts, a fictitious part of the Harry Potter world that a number of tech companies have tried to recreate. Previous attempts have been mildly successful, but this system is impressive in its ability to isolate and create a pretty realistic 3D animation from a single image.
The researchers tested the system on 70 different photos they downloaded online, which included pictures of Stephen Curry, the anime character Goku, a Banksy artwork, and a Picasso painting. The team used a program called SMPL and deep learning, starting with a 2D cutout of the subject and then superimposing a 3D skeleton onto it. “Our key technical contribution, then, is a method for constructing an animatable 3D model that matches the silhouette in a single photo,” the team told MIT Technology Review.
The team reportedly used a warping algorithm to ensure the cutout and the skeleton were aligned. The team’s algorithm is also reportedly able to detect the direction a subject is looking and the way their head is angled. What’s more, in order to make sure the final animation is realistic and precise, the team used a proprietary user interface to correct for any errors and help with the animation’s texturing. An algorithm then isolates the subject from the 2D image, fills in the remaining space, and animates the subject.
Source: New Photo Wake-Up System Turns Still Images Into 3D animations
Robin Edgar
Organisational Structures | Technology and Science | Military, IT and Lifestyle consultancy | Social, Broadcast & Cross Media | Flying aircraft