As part of my work with The New York Times R&D lab, I've been exploring how to use neural radiance field technology — NeRF — for capturing moments that might help tell a story or represent a person in 3D. This year, I'm augmenting my work at the times with a role as teaching fellow in immersive journalism at the Newhouse School, where I plan to further these explorations. Collected here are a few examples of NeRF captures that have been instructive.
With NeRF primarily being output as video renders, we can easily combine multiple captures through simple video editing. All that is required is making two clean captures, syncing a virtual camera path, and blending the resulting video exports.
Because NeRF offers a higher fidelity for complex 3D objects with sparser inputs, I can scan a scene that requires the subjects hold still in under a minute, even though that scene contains multiple reflective surfaces and detailed topography. This is with a single camera input, but capture time can be decreased substantially with a multi-camera rig.
Using NeRF capture methods outdoors with a simple wide angle camera like a GoPro allows an editor to mimic the effects of a drone in post-production.
The ability to render atmospheric effects, like fog, as well as the particular way in which NeRF renders represent unresolved data similarly can be an interesting aesthetic effect.
Post production allows not only the manipulation of camera position, but also perceived focal length, showing potential for cinematic movement and manipulation.
Capturing a simple studio portrait with a three camera NeRF rig on a track.
AJ's Cool Internet Website
I try to make cool and weird things on the internet to tell stories.