Nvidia has presented its latest model of Artificial Intelligence (AI) that under the name of Instant NeRF converts a set of 2D photos into a rendered 3D scene.
The first time a snapshot was taken 75 years ago with a Polaroid camera achieved “capture the 3D world in a realistic 2D image”, as Nvidia has recalled in a statement. Now, the company claims to achieve just the opposite: “Turn a collection of photos into a 3D scene in a matter of seconds.”
Known as “reverse rendering,” the process uses AI to “approximate the way light behaves in the real world by allowing researchers to reconstruct a 3D scene from a handful of 2D images taken from different angles.”
For this approximation, Nvidia has used a technology based on NeRF (neuronal radiation fields), called Instant NeRF, which uses neural networks to represent and render realistic 3D scenes based on a collection of 2D images.
As the company has defended, it is “the fastest to date“The model requires just a few seconds to train on a few dozen still photos, plus data on the camera angles they were taken from, and can then render the resulting 3D scene in tens of milliseconds,” accelerating the process at 1000x.
The company explains about this process that “in a scene that includes people or other moving elements, the faster the photos, the betterIn this sense, if there is too much mobility during the 2D capture process, the 3D scene generated by the AI ”will be blurry”.
From there, NeRF technology fills in the blanks and reconstructs the scene by predicting the color of light radiation in any direction.
From the company they assure that their researchers have been working on Instant NeRF “for a couple of years” and that their technology “is one of the fastest” managing to reduce the rendering time from “several minutes” to occur “almost instantaneously”.
Nvidia suggests that Instant NeRF “can be used to create avatars and scenes in virtual worlds”, as well as to “capture video conference participants and their environments in 3D or to reconstruct scenes for 3D digital maps”.