Since 2019, Google searches for virtual production and related terms have seen an increase of nearly 500 percent. Major companies like Epic Games have published field guides explaining how VP works. But we still run into plenty of people who aren’t sure how to get started, or exactly what benefits a virtual production pipeline can bring. If you’re one of them, read on.

What is virtual production?
Currently, there is no one simple definition of the term. However, most agree that virtual production describes the ability to mix live footage with computer graphics, get real-time feedback, and help make decisions on set.

David Morin, Chairman of the Joint Technology Committee on Virtual Production, has said, “VP is the process of creating the digital world, beginning with the inception of the movie and ending with the final VFX, centered around the real-time interaction on set.”

Instead of shooting in real locations or in front of a green screen, actors perform in front of an LED volume with CGI backgrounds. This ideally means the crew can see the final shot on the day of shooting, rather than waiting months for post-production.

Virtual production sets and LED stages
Virtual production exploded in popularity for two main reasons: the Disney+ series The Mandalorian and the COVID-19 pandemic, which has driven adoption of these techniques at a rate no one expected. The Mandalorian, which premiered in 2019, was the first major series to combine camera tracking with game engine animation and LED panels. However, the concept was not new; other films that used LED or projection tech include Gravity and Rogue One.

Imagine a set that lets you keep the sun at the perfect angle all day, swap out backgrounds with a few clicks, or even see a CGI creature interact with the actors—in real time. With LED stages, filmmakers exchange green screens for large walls made of LED panels, which display real-time environments and visual effects directly on set.

Real-time engines and camera tracking are required for virtual production. As creators of the world’s most advanced camera tracking system, we work with teams that are shooting on these types of stages every day. And we are starting to see the barriers to entry lowering, which means VP is no longer just for blockbusters and tentpole productions—now smaller, independent studios can benefit as well. So we’re here to share 4 tips for how to do it right.

1. Don’t Slash Your Budget
Beyond obvious savings like travel and lodging, virtual production techniques can help save a great deal on both production and post-production costs. The more planning you do up front, the more you can save on the back end. You can bring locations to you, or shoot any time of the day or night—the sun never sets on an LED wall.

In other ways, it can be more expensive. LED walls, for instance, take an immense amount of electricity. You’ll need a lot of GPU power. You may also need to purchase new equipment and learn how to use new tech, or hire crew members who have these skills. So, it’s important to understand what your production will require before adjusting your budgets.

2. Use the Best Tools
We mentioned that real-time engines and camera tracking are two key elements of virtual production, allowing flexibility and faster turnaround times. But many of the VFX tools currently on the market aren’t designed to be used on set, or in real time. For instance, although camera tracking has been around for years, Ncam Reality is the only system that can be used both indoors and outdoors without swapping out equipment, whether you’re in a small, intimate studio or a vast, snowy field.

Some of the classic production tools, like high-quality cameras and lenses, will always be essential, and that’s a big reason we’ve specialized in tracking technology. An in-camera view makes a huge difference for creatives on set, but you need to align the real and the photorealistic to make that happen. Mocap will continue to be a big piece of the puzzle, as well as real-time compositing and visual effects. LED walls will be widely used over the next few years, and we expect they will continue to improve in both quality and functionality.

3. Find the Right People
Since virtual production is still in the early phases, there are a limited number of people who truly understand the complete puzzle and how all the pieces fit together. Currently there is a real need for hands-on experience, and many companies have opened virtual production facilities in the past two years (including our own locations in Eastern Europe and Latin America, built to support the growing demand for virtual production training).

We are already seeing new roles in the world of VP, including Virtual Production Manager, Virtual Production Supervisor, Virtual Production Technical Director, and so on. Some of these roles are progressions from existing roles within VFX, but it’s likely we will also see moves from other departments, including DIT and camera. There are lots of exciting opportunities for individuals and companies looking to embrace new technologies and workflows, as well as those looking to re-train and learn new skill sets.

4. Know There is a Learning Curve
If you’re just getting started with VP, you may be surprised by how different it is, requiring changes to everything from scripts to set design and blocking. The single biggest struggle many teams face with VP is the idea of changing their entire workflow. In the past, filmmakers would come to set with an idea. In a virtual production pipeline, you have to come with most of your 3D content already done. This means a great deal of the creation process has to be started weeks—or even months—earlier than many are used to.

We’ve heard from some people who think virtual production or ICVFX means always getting final imagery in-camera. But this won’t always be possible. According to ILM, the number of final shots captured in-camera on the first season of The Mandalorian was around 50 percent. Ideally, we will get to a point where everything is final pixel. For now, don’t stress if some of your shots still need a little work in post.

And while everyone is buzzing about LED walls, It’s not just about the wall. You still need to have great content to feed into the wall. So don’t forget the basics!

For a demo of Ncam’s real-time camera tracking, get in touch with our team.