Non-linear Production – Virtual lighting with photographic renderers

Virtual lighting with photographic renderers

Images in the real-world are created (broadly speaking) from two different paradigms: by painting or by photographing. Painting is a process of re-creation. We observe reality (or our own imagination) and then create that on the canvas. Photography is fundamentally different: we don't re-create reality, we capture it. Both image creation methods are valid, and both are capable of spectacular imagery. Live-action motion pictures are almost exclusively created through photography. Animation is the exception -- hand drawn 2D animation obviously uses the "painting" approach.

Okay, let's now bring in computers. Just like in the real world, there are two fundamentally different ways to create an image in the computer. The first way is analogous to painting, and is what Pixar's rendering software does. In fact, virtually all CGI used in motion pictures today is made using the "painting" approach (really good, really realistic painting, but it's still painting). Avatar's CGI portions were also created in this manner.

The second way computers create images is analogous to photography. This method is not used in 3D animation or visual effects today, but it is used to do architectural, automotive, and jewerly visualization. Why? Because these disciplines don't want an artistic re-creation -- an artist's "interpretation" if you will -- they want to know what a building, or a headlight assembly, or a piece of jewelry will look like exactly before it's made. They want absolute accuracy, especially when it comes to lighting. They want a digital photograph.

I'm proposing that live-action virtual productions use 3D rendering software from the second, photographic paradigm, and that if they do, cinematographers will be able to light virtual sets the same way they are lighting physical sets in the real world.

Here's how it would work. A cinematographer would set up "virtual" lighting equipment exactly the same way they would on a normal live-action shoot. Performance capture technology would track the location of the lighting equipment (similar to how virtual cameras are tracked today). The computer would then take a "photograph" of the virtual set from wherever you placed the virtual camera. And because the computer uses the second method to create the image, the result will be physically-correct and 100% accurate. It's as if we really built that virtual set, really did add those virtual lights and bounce cards and gels and silks and barndoors -- and took a photograph.

What's cool is that this second approach is fully interactive! You can move a "virtual bounce card" in the performance capture volume and instantly see the results in the virtual camera's viewfinder. Move a virtual light, and immediately see the changes. And these are final quality renderings -- digital photographs -- not the cheesy video game imagery we've seen in the past. A cinematographer really can light a virtual set using the same methodology and approach they use to light the equivalent live-action set. Moreover, they have to: just like real-world photographs, there are no "cheats" with these virtual photographs.

Cinematographers will be able to light just as well virtually as they do today in the real world. And they'll get the credit on the title card. No cinematographer (to my knowledge) has ever been given credit on a 3D animated film. But with my approach, on live-action style virtual productions, they will get the credit they deserve because they'll be directing "photography" once again.

Next: The future: Avatar with live-action story reels
Previous: Applying the Pixar process to live-action filmmaking
Back to index