A couple of months ago, I pointed out that the advent of high-quality real-time animation will disrupt the Hollywood studio system and other segments of the video media industry, thereby creating a new billion-dollar industry segment.
I received over a dozen emails asking me for more information, specifically in what areas and applications the technology would become disruptive. The question is a large one that deserves its own column. Hence this column.
Quick background: For about two decades, real-time 3-D animation was a quick and dirty way to develop computer games. "Real" CGI artists and the companies that employed them (like Disney) looked down on real-time as "that gaming crap."
Nevertheless, there was an underground creating content using real-time and some of it, like the Red vs. Blue web series and the streaming feature film Borg War became popular enough to spawn many millions of views. Still, the "look and feel" was pretty crude.
Fast-forward 15 years. As computers have gotten insanely faster, the animation techniques and algorithms that were once only possible using huge data centers and small armies of animators started becoming practical on single computers.
With game engines like Unreal and animation suites like Reallusion's iClone family, what used to be "quick and dirty" started looking a lot like "faster/smarter/better." And cheaper--as in geometrically cheaper--to get the same or similar results.
With traditional computer animation, it can take hours, even days, to "render" (i.e., build) a single frame of finished film. Since there are at least 30 frames per second in finished video, it takes a lot of time and computer horsepower to do high-quality work.
With real-time animation, however, you can get draft-quality finished video as quickly as you can display it on a screen. Higher quality takes a bit longer, but render times are calculated in seconds per frame, rather than hours per frame.
While those back-end improvements are impressive, the biggest advances have been in the creation phase, which replaces highly wonky, difficult-to-use tools with scene builders and character creators that are as easy to use as word-processing programs.
To illustrate this, I've been using myself as a bit of a guinea pig, to see whether a mere mortal can make acceptable animations that rival the products the big studios were making just a few years ago.
In the process, I've also been interviewing and getting coached by people who understand this technology, and learning about how it's being applied inside many segments of the entertainment industry.
Based on my research and experiences, here are five areas where real-time 3-D animation is disrupting established practices and establishing new markets:
1. Low-cost CGI and VFX Footage for Feature Films
Real-time is being used for more than pre-visualization. In an increasing number of cases, it's being harnessed to create actual footage.
For example, the breakout star of Rogue One: A Star Wars Story (2016) was arguably the anti-C3PO android K-2SO. Under normal circumstances, K-2SO would have been added laboriously in expensive post-processing.
In Rogue One, however, K-2SO was rendered in real-time using the Unreal Engine (developed originally for computer gaming) so that it could interact with the actors on the screen, allowing the director to make changes to the robot's performance during live filming.
The Star Wars movie production teams have continued to develop this real-time technology so that it can play a bigger role in final production, thereby vastly reducing production costs. Here's the kind of quality that's just around the corner:
Another movie that used real-time rather than traditional animation was the multiple-award-winning film Finding Dory (2016). Michael Zaman, supervisor of real-time computer graphics at Axis (probably the U.K.'s top animation studio), explains:
People will use real-time visualization and motion capture with digital doubles and characters more and more as way to take things to a sort of digital live set, where a director can be a live-action-style director without the limitations of reality.
While it might seem from the examples above that big studios are dominating the "cutting edge" application of real-time animation, the technology is so inexpensive that even rank amateurs can get pretty amazing results, as you'll see below.
2. Deep Pre-Visualization to Reduce Film Production Costs
Live filming is incredibly expensive. A studio must hire many people and keep them on set for days and do multiple shots (using different cameras and setups) to get the raw footage they need for the editing process.
The same is true for CGI that interacts with real characters. Coordinating live action with computer graphics characters can be insanely complex. Both work streams (live action and CGI) must come together perfectly. Costs explode if there's a problem and reshoots are required.
Traditionally, all the director and camera people have as a guide are the script and a set of hand-drawn cartoons (called a storyboard) showing more of less what's needed--a technology that hasn't changed since the silent film era (!!!).
Some forward-looking directors now use real-time animation to simulate every shot and create a draft film to confirm that those shots tell the story. Only then does the director do live shooting, which now takes much less time, because he or she already knows exactly what to shoot.
For example, Keanu Reeves used the iClone 3D animation software to reduce the shooting budget for his recently released film Replicas. As you can see from the trailer below, Replicas is heavy on mixed live action and effects, which would have been prohibitively expensive if shot using traditional methods:
3. Television Studios Located in Virtual Reality
Real-time animation will play a huge role in creating interactive virtual environments, according to Michael Zaman from Axis:
A larger emerging space is the location-based media, such as the Void, using real-time VR on a large scale for immersive gaming experiences. Similarly, theme parks are looking at using real-time and VR to take rides to new places.
Overall, it's the application of real-time animation in a VR environment that has the most potentially disruptive effect on Hollywood, since it can be used to create a virtual studio to create TV shows, training videos, and so forth.
Here's an example of how this would work from Flipside, a company that's pioneering in this area:
Yes, the animated character is pretty basic but creating higher resolution characters with more realism is simply a matter of adding computer processing speed. And since Moore's law doesn't seem to be grinding to a halt, that's clearly where real-time animation is headed.
4. Independently Produced Animated Features
Back in 2015, the Grammy-winning rock band Muse used Reallusion's iClone software to create eerie 3-D animated background images in the official video for their hit single "The Handler." (Worth checking out, if you like alternative rock.)
Real-time animation has come a long way since 2015, so I was interested to find out how easy it would be to develop an entire music video using real-time animation, even by somebody who lacks specific training in animation.
With this in mind, I contacted author and TV writer Martin Olson and asked him whether he would mind if I took a shot at animating "Halloween Dance," a song he wrote and produced with his daughter, actress and R&B artist Olivia Olson.
Once I got the permission of this extraordinarily talented duo, I set to work. Within a few months (working in my spare time), I created the following:
Please keep in mind that I have no formal training as an animator and that I made that video using nothing more than a midrange gaming computer in my spare time using Reallusion's iClone and Character Creator software.
While the animation obviously isn't Disney quality, "Halloween Dance" just won an award for Best Animation Short and I won an award for Best Animation Short Director at the prestigious Los Angeles Independent Film Festival. Which is pretty cool, all things considered.
Obviously, the animation quality in "Halloween Dance" isn't anywhere near the experimental Star Wars material I showed earlier in this column. But that's only because I haven't upgraded my system to that level yet. It's definitely within my reach, should I decide to reach for it.
More important, the millions of creators whose talents far exceed mine will be able to use real-time to create incredible content--content rivaling the best CGI coming out today--at a tiny fraction of the time and cost.
5. Trade Show Kickoff Videos
About six months ago, I invited fellow Inc.com columnist Tom Koulopoulos over to my home office to show him some of the real-time animation I was doing and ask him for ideas of how the technology might be used in marketing and advertising.
He suggested that companies, especially high-tech ones, might want to use real-time animation to create an arresting video to kick off a trade show or conference. His thinking was that everyone is always looking for something new and different. Real-time animation certainly qualifies.
On Tom's advice, I created a concept video using iClone, which he subsequently showed to executives at Breakaway, arguably the top branding agency in Boston. They confirmed Tom's opinion that creating such videos is a viable business concept.
The idea: create characters based upon the company's CEO and management team and them put them in a science-fiction or superhero environment (e.g., your management team as the Avengers). Something like this would be financially impossible with traditional animation, which can run as high as $10,000 a second.
Using real-time animation, however, a trade-show kick-off video of reasonably high quality could probably be created for less than $10,000 a minute. That's a whopping 6,000 percent reduction in production costs.
To test the viability of the concept, I enlisted the advice of two top experts:
- Cristian Moras, one of the top animation consultants in the country, whose experience spans the best animated films from Disney and DreamWorks over the past couple of decades.
- Darious Britt, an incredibly talented independent filmmaker whose feature film Unsound has won numerous awards and whose YouTube channel is the go-to place for aspiring independent filmmakers.
With Cris's advice on the animation and lighting and Darious's advice on the pacing and cinematography, I was able to create and redo the project to give it a much more polished look and feel:
The results are cool enough (IMHO) that if somebody asked me, I could probably put together a team that could make a killer trade show kickoff vid, like showing your management team as a group of superheroes, like the Avengers. Just sayin'.
The point isn't that making a trade show video with personalized CEO characters is so brilliant but that it shows how real-time animation makes it possible to create entertainment that would otherwise be far too expensive to otherwise contemplate.
This is especially true as real-time programs and systems proliferate and become more accessible and easier to use. Zaman cites Unreal/Unity (as in the Star Wars example) as well as CryEngine, Amazon's Lumberyard, and Eevee, a new real-time renderer in the free open-source Blender environment.
As you can see from the five examples above, this real-time animation thing will have the same effect on computer animation that desktop publishing had on printing and compositing.
The reason this presents such an opportunity is that, despite the participation of the big studios, there's an establishment of CGI professionals who haven't kept up with the trends and are defending their respective turfs. Zaman explains:
The lack of experience and exposure as well as "expected" limitations of real-time is something a lot of people use as a reason not to adopt real-time.
Another limitation to adoption in bigger studios is just that the real-time process is so comparatively simple that it eliminates entire job categories. It's turning into a classic situation where people are terrified they'll be automated out of a job.
It's going to change huge swaths of the entertainment and training industries in most major and niche markets. It's inevitable that this tectonic shift will create many new businesses, and some new billionaires as well.