Previous Posts:
Every day, we wake up in a world where people go to the theatre and come home with a feeling of wonder. They are immersed in a world, lost in a reality that is not their own. When the feeling of wonder wears off, they wonder how it was created. If perhaps the person is a filmmaker, he is inspired to create great things.
How does this continue? How are we still feeling the way those who watched the original Star Wars trilogy felt, when we watch the latest Star Wars show on Disney Plus? The answer is Innovation.
“Innovation is taking two things that exist and putting them together in a new way.” – Tom Freston
Virtual Production is a great example of innovation, the reason people are still at the movies every week. It combines elements of both the real and virtual worlds into something special.
A large portion of simple VFX shots follow the same blueprint.
Taking two elements, usually a background and foreground, and merging them into one cohesive image. Generally this is done in a compositing software like After Effects or Nuke.
For most shots we still have to film the characters and objects in the foreground, but what if we could film them with the background behind them on the set. Well, how do you display a background? Generally using a projector or an LED screen. This is exactly what filmmakers use in Virtual Production. Now we can have the foreground and also add any background to a scene just by placing a huge screen, like an enormous TV, behind the set.
More Technology
This is cool, but we need to record it. In order to do this, we use a camera rig similar to the ones used on a normal production. A regular camera is used, as what we are recording is in the physical world. There are, however, some added elements to the camera rig. The main one of these is a real time tracking solution.
The reason that camera tracking is needed is this thing called parallax. The background and foreground will be synced from one angle, but if you move the camera the parallax breaks.
This will happen because you can’t move around in an image, the image being projected has a fixed perspective. Remember I said you can’t move around in an image, well you can move around in a video game. Why is this? Because of the fact that a video game character is inside of a 3D world. These days video games are becoming more and more photorealistic because of photo scans and real time rendering. Each moment in a video game, you are being shown many images(frames) that are being rendered very fast. In new softwares like Unreal Engine 5, we can now render far more complex images in real time. Rendering something that even came close to photorealism used to take hours but now we can do it as it’s happening.
If you are projecting a photoreal image that is being rendered in real time on the LED wall, you now have the ability to move around.
If you have a camera that is sending its location to the computer that is displaying the 3D world, you can now have a virtual camera to see the world through that will be following the real world camera. This is a bit complicated, but the outcome will be that the parallax issue will be fixed.
We now have a complete system but why do we use this over the more traditional methods in the visual effects world. In the next section, we took a look at how virtual production adds a huge amount of realism to both the visuals, and the actors performances. We then looked at how this technology has and probably will be used in the entertainment industry. We then closed with why these advancements won’t take everyone’s jobs.
Finally
I hope that you enjoyed learning more about the technology of Virtual Production and maybe now you understand why it has been gaining so much attention in the recent years. There is so much more to learn about Virtual Production and Visual Effects, so stick around as we continue to Unmask VFX.
Keep learning, and making art!
Samuel Collett
Video to see these ideas visually
Unreal Engine are changing the game when it come to virtual production:
https://www.unrealengine.com/
Pingback:Virtual Production: Passing Filter or Permanent Fixture. - Unmasking Visual Effects