Visual effects has evolved a lot, specially after the pandemic. Virtual production studios are the new toy everyone in Hollywood is watching out for!
The visual effects industry is moving towards a versatile new production technique called virtual production. This might signal the end of the ‘green screen’ era.
Hollywood has a new toy. It’s called virtual production. Although this technique has been around for quite some time, the new advances in technology have raised the bar higher than ever before.
Until recently, most visual effects shots would need a lot of green screen work. When we see the Avengers fighting in the mammoth battle sequence, the background is all green screen. Chroma keying or green screen has always been around. The green screen is then ‘keyed’ out of the shot (which means removed) and whatever the required background is, is then added in post production. And this technique is not an easy one. Many artists break their backs, sitting behind computer screens trying to perfect the keying process. There are many challenges involved in this process as well.
Imagine having to deal with masking or rotoscoping hair and other such objects. Or, having to deal with reflective surfaces that catch the ‘green spill’ from the huge green screen backgrounds. It is a lot of work. The production work also involves a lot of prep. Huge green screens have to be lit and the talent are kept as far away as possible from the green screens. This is done so that they can be lit separately and to avoid the green color spilling onto them. Even though the final output is rewarding for the vfx artists, gaffers, technicians etc. The working hours involved amount to an insane number.
Once that is done, there are many other layers to compositing. Foreground and background layers are added. Natural elements like dust, smoke etc are also added in post. As shown in the picture above, these markers are used to track camera motion and sync that with the virtual camera in compositing software like Nuke. Sometimes this data is also used in 3D softwares like Autodesk Maya or 3DS Max to create CGI characters, objects or backgrounds and sync them with the camera motion of the real production world. That technique is called motion or camera tracking.
All of this has been very effectively used over the past years and we have seen some amazing work come to life on the silver screen. Now, this has just become better to a whole new level.
Virtual production! If you have seen The Mandalorian you know what the final output looks like on screen.
In virtual production, the green screen is replaced by massive LED walls. These are LED screens that are connected together and they can display the environment that the story takes place in. This makes it seamless. There is no distinction between where the actual set ends and where the virtual set begins. Having a virtual set also helps to cast light and reflections on the actors and their surroundings. In the case of The Mandalorian, his suit is reflective and it would be painful to have to deal with green spill on his suit. With virtual production they can mimic the actual environmental conditions on his suit. From the lighting, the temperature of the light and the reflections that one can see on him are all captured in-camera. This takes it very close to a real environment. Also, control over lighting and how to shift the environment in the background is all possible. Imagine have to replace an actual set, and in this case, one can just change the environment digitally and in real time. It gives a whole new level of flexibility to filmmakers.
The LED walls are connected to the Unreal Engine 5 in the above example. It is to date the most advanced engine ever. True to its name, it is unreal! The camera parameters are fed into the unreal engine and this helps the perspective to shift along with the actual production camera. As the camera moves in the real world, the background being projected on the virtual LED wall also moves along with it. This gives it a whole new level of detail and control.
While this article doesn’t dive into the technicalities of virtual production, it shows the impact, the diverse range of options and the polished outcomes that it can produce. Virtual production is here to stay! Does this mean the end of the green screen era? Well not entirely, green screen is still a widely used technique and it will alway exist. But virtual production will now take center stage in all digital environments and set extensions. It will be exciting to see what technicians and artists come up with in the future as virtual production ramps up across the globe.
