In previous columns, I've explained how real-time animation will completely disrupt entire huge segments of the movie and television business. As evidenced by a recent article in Variety, Hollywood is finally beginning to take notice.

To recap, there are two methods for creating animated footage:

  1. Traditional animation, which came out of the film industry, and is optimized to produce the highest quality possible.
  2. Real-time animation, which came out of the video game industry, and is optimized for speed, so that the game feels interactive.

What's different today is that the quality of real-time animation is beginning to approximate the quality available with traditional animation. While this sounds like just another example of "better/cheaper/faster" it's actually a situation where technology will disrupt an entire industry.

In the film and TV industry, traditional animation is currently used for: 1) Video effects aka VFX to add animation to live-action sequences, and 2) completely animated features like Frozen.

I've written previously about how real-time is changing the way animated features are produced. This column is about how they're change VFX.

Earlier this week, I heard a presentation, hosted by the Animatic Boston community, from the Boston-based company Zero VFX, which did the video effects (like the arena crowd scenes) for Creed II, among many other high-end projects.

In that presentation, three very talented young people explained how they work with directors to add VFX during what's called "post-processing"--activities that take place after the live action footage has been filmed.

What struck me about the process they described was that the director really had no idea how the final footage would look until the VFX had been added during post-processing. As a result, there was a vast amount of confusion and back-to-the-drawing-board. This back-and-forth is very expensive, needless to say, even without the burden of using traditional animation tools which are themselves expensive and slow.

Everything changes when you start thinking about adding VFX in real-time. With real-time, the VFX would be available for the direct to view the very moment that the scene is filmed. The director would know exactly how the scene will look with the VFX, because the VFX is already there, and then make shooting decisions appropriately.

In some cases, the VFX might actually be displayed on set, so that the actors themselves can see it while they're filming a scene. Variety gives this example from the light speed jump scenes from recent Star Wars film Solo:

"The scene wasn't filmed in front of a green screen, as is typically the case with movies that rely heavily on visual effects. Instead, Lucasfilm's Industrial Light & Magic unit had built an elaborate setup of five 4K laser projectors around the Falcon's cockpit, which displayed the iconic hyperdrive animation in real time. The setup not only allowed Glover and his fellow actors to perform in less of a vacuum, but the projectors were also used as the sole source of lighting -- resulting in stunning reflections of the flashing blue lights in the actors' eyes."

Can you see how radically different this is for everyone involved? Not only is post-processing entirely eliminated but the actors and director can see and react to the VFX as if were part of the real world. (Which, in a certain way, it actually is.)

Real-time VFX thus turns a disjointed, step-by-step process into an immediate and interactive process. That's a huge change, not just in technology, but in the entire process. It's like the difference between snail mail and email; yes, they both carry messages to and fro, but email has completely altered the way that all offices operate.

This is an important change in how studios do business, according to Isabelle Riva, head of the Made with Unity team, a group that promotes the use of Unity real-time technology to studios and filmmakers:

Instant visualization will be demanded everywhere in many different media pipelines. It's really seminal, spreading like a virus because it makes filmmaking more fluid by closing the separation between directors and the final product."

In other words, real-time is driving a fundamental change in the process by which live-action movies and TV are created. It compresses a complicated step-by-step process into an interactive, what-you-see-is-what-you-get process. As with all disruptive transformations, there will undeniably be some big money to be made riding this wave of change.