Pixotope Keyer
Introduction
Separating actors and props from a green screen scene is a critical function for any virtual-set or mixed-reality project. The aim is to:
identify and isolate the foreground elements from the background screen color
preserve:
fine edge details such as hair
transparency of translucent objects such as glass bottles
remove any spill color from the actors
recover the shadows from the actors and props
The foreground objects need to be completely solid where there is no transparency.
The quality of the produced key is highly dependent on a good keying setup.
Here are some basic rules.
The green screen
The screen should be evenly colored if painted. Ensure a proper chroma green paint is used, and that the surfaces are free from brush marks
The screen must be stretched tightly to avoid creases if it is made of fabric
Ensure the floor color is the same color as the walls, and that it is clean and without marks
The walls should be at least 3 m high to allow keying of standard-height actors - taller props or actors would require higher walls
The walls should meet the floor seamlessly without creases or shadows. Often, coving is used at the junctions to smooth the transition
The walls should be lit evenly with a wash of light and with no visible highlights or hot spots
The floor should be lit evenly
The subjects should be picked out from behind by a rim light to give them some separation from the screen
The front of the subjects should be illuminated with beauty or fill lights
The camera
Ensure that the camera has the correct lens and that it is positioned in the desired position in the set
It is normal to have the camera at the same height as the actors' heads so that it is easy to key the actors if they move some distance from the green screen towards the camera
Make sure proper white balance is set
Make sure the scene is correctly exposed
Ensure the camera is not set to a high gain, which might introduce noise
Check the camera's "clean" video signal on a monitor to ensure there is minimal noise in the image
NOTE: Noise is an inherent part of any camera video image, and minimizing this is critical for good images as well as creating a good key.
The main issue is to not pick up too much spill color from the environment onto the actors, which would make keying more difficult.
The Talent
Do not position them too close to the walls, and do not let them get closer than 1 m
Light them from behind to create good edge definition
Light them from the front to create attractive lighting, and to remove any under-illuminated hollows in skin and clothing that could show the screen color
Position them far enough from the camera to be fully framed with the choice of lens and zoom setting
Talent should not wear any clothing that includes the same color as the screen color
Video keyer
The video keyer is an easy-to-set-up real-time chroma keyer that you can use to key your video input. It is a component implemented inside the Pixotope Video Pipeline. The keyer extracts an alpha mask from the received frame, based on information provided to the algorithm, and removes specific color (despills) for the background of the scene and its reflections in the image. The user interface for the keyer is included in the Pixotope Director under Production → Adjust → Video Keyer. The UI and keyer component communicate over the network.
Initialize
The first step is to set the reference color(s) for the keyer. The colors are automatically extracted from the labels provided in the labeling tool. The labeling tool is located in Video Keyer → Initialize.
Keyer model
There are 2 keyer models available:
Chroma-PX1: legacy keyer present in Pixotope 1.5
Chroma-PX2: new keyer using a more advanced color model with more precise color extraction and sharper alpha mask
Adjustments
Refine
The Refine category remaps the alpha values so the user will be able to suppress values with low transparency and increase values with high opaqueness. The functionality is driven by two threshold values and linearly interpolated in between.
The 3 parameters help to define the area between fully transparent and fully opaque for the keying process.
The exclusion is based on the color deviation and the color saturation. Both parameters are computed independently and are added to the computed mapped alpha (in the diagrams below, Lt and Ut represent Remove Background and Adjust Foreground respectively).
Displayed functionality of Refine adjustments.
Add Luma Key
This is an additional tool to exclude the foreground pixels from the background screen based on luminance difference.
Note that the luminance reference for the backing color is controllable:
If "Automatic Clean Plate Generation" is
enabled: the reference is automatically generated from the locally closest backing color
disabled: the reference is issued from the labelling tool
Key color intensities
By default, all colors are considered equal, but as for example yellow is quite close to green mathematically, it can be useful to key certain colors harder than others (and even make colors repel keying). With these sliders you can change how aggressive the key will be towards the three color channels individually. Negative values mean that this color repels keying.
Mask Adjustments
Erode
Removes pixels on the mask boundaries, making the mask smaller.
Gain
Adjusts the transparency of the mask.
Gamma
Scales the alpha by an exponential function.
Example of gamma transformation with different exponents.
Detail Enhancements (only for Chroma - PX2)
This parameter brings more details to the mask through a local analysis of the luminance variations resulting in sharper edges.
Shadows
The Shadow controls allow the user to add shadows that exist on the evenly lit backdrop. Shadows are added based on the luma of the pixels; any pixel darker than the original pick will be included in the shadow, the alpha strength depending on just how dark the pixel is. The "Add shadows" slider controls the alpha of the shadows, while "Adjust contrast" controls the color, trying to darken the fill in the area where shadows are identified.
Despill
The Despill controls allow the user to suppress the background color removal and adjust hue for the areas containing a significant amount of spill.