Pixotope Keyer
Introduction
Separating actors and props from a green screen scene is a critical function for any virtual-set or mixed-reality project. The aim is to:
identify and isolate the foreground elements from the background screen color
preserve:
fine edge details such as hair
transparency of translucent objects such as glass bottles
remove any spill color from the actors
recover the shadows from the actors and props
The foreground objects need to be completely solid where there is no transparency.
The quality of the produced key is highly dependent on a good keying setup.
Here are some basic rules.
The green screen
The screen should be evenly colored if painted. Ensure a proper chroma green paint is used, and that the surfaces are free from brush marks or other recognisable “texture”
The screen must be stretched tightly to avoid creases if it is made of fabric
Ensure the floor color is the same color as the walls, and that it is clean and without marks
The walls should be at least 3 m high to allow keying of standard-height actors - taller props or actors would require higher walls
The walls should meet the floor seamlessly without creases or shadows. Often, coving is used at the junctions to smooth the transition
The walls should be lit evenly with a wash of light and with no visible highlights or hot spots
The floor should be lit evenly
The subjects should be picked out from behind by a rim light to give them some separation from the screen and to minimise edge spill
The front of the subjects should be illuminated with beauty (key and fill) lights
The camera
Ensure that the camera has a properly calibrated lens and that it is positioned in the desired position on the set
It is normal to have the camera at the same height as the actors' heads so that it is easy to key the actors if they move some distance from the green screen towards the camera
Make sure the lens has been back-focussed.
Make sure proper white balance is set
Make sure the scene is correctly exposed
Ensure the camera is not set to a high gain, which might introduce noise
Check the camera's "clean" video signal on a monitor to ensure there is minimal noise in the image
Do not use image sharpening on the camera if this can be avoided. It may well produce edge artefacts on the key. The image can always be sharpened after the keying process.
NOTE: Noise is an inherent part of any camera video image, and minimizing this is critical for good images as well as creating a good key.
The main issue is to not pick up too much spill color from the environment onto the actors, which would make keying more difficult.
The Talent and Lighting
Do not position them too close to the walls, ideally no closer than 2m and never closer than 1 m
Light them from behind to create good edge definition- rim lights
Light them from the front to create attractive lighting, and to remove any under-illuminated hollows in skin and clothing that could show the screen color - key and fill lights
Position them far enough from the camera to be fully framed with the choice of lens and zoom setting
Talent should not wear any clothing that includes the same color as the screen color
Also be careful of colours that are close to the background color e.g. yellow or cyan for a green screen; cyan or magenta for a blue screen. These may be problematic for de-spill.
Try to avoid transparent clothing that might pick up and reflect the background color
Talent should avoid clothing textures that might cause moiré patterns e.g. herringbone, plaid or dog-tooth patterns
Video keyer
The video keyer is an easy-to-set-up, real-time chroma keyer that you can use to key your video input. It is a component implemented inside the Pixotope Video Pipeline. The keyer algorithm extracts an alpha mask from the received frame, allowing the removal of the specifically picked color from the background of the scene and from its reflections in the foreground image. The user interface for the keyer is included in the Pixotope Director under Production → Adjust → Video Keyer.
Initialize
The first step is to set the reference color(s) for the keyer. The labelling tool is used to select the background colors to be extracted to leave just the foreground. The labelling tool is located in Video Keyer → Initialize.
Background and foreground colors can be picked and modified to define the values used to create the alpha channel to extract the foreground image.
Keyer model
There are 3 keyer models available:
Chroma - PX1 (legacy): legacy keyer present in Pixotope 1.5
Chroma - PX1 - Linear (legacy): a derivative of the legacy keyer using linear converted video instead of gamma corrected video - for use with linear colourspace
Chroma - PX2: new keyer using a more advanced color model with more precise color extraction and sharper alpha mask
Adjustments
Refine
The Refine category remaps the alpha values so the user will be able to suppress values with low transparency and increase values with high opaqueness. The functionality is driven by two threshold values and interpolated in between. The different keyer models use a different interpolation method.
The 3 parameters help to define the area between fully transparent and fully opaque for the keying process.
The exclusion is based on the color deviation and the color saturation. Both parameters are computed independently and are added to the computed mapped alpha (in the diagrams below, Lt and Ut represent Remove Background and Adjust Foreground respectively).
Displayed functionality of Refine adjustments.
Add Luma Key
This is an additional tool to exclude the foreground pixels from the background screen based on luminance difference.
Note that the luminance reference for the backing color is controllable:
If "Automatic Clean Plate Generation" is
enabled: the reference is automatically generated from the locally closest backing color
disabled: the reference is issued from the labelling tool
Key color intensities
By default, all colors are considered equal, but as for example yellow is quite close to green mathematically, it can be useful to key certain colors harder than others (and even make colors repel keying). With these sliders you can change how aggressive the key will be towards the three color channels individually for Chroma - PX1 keyer. Negative values mean that this color repels keying.
As Chroma - PX2 keyer uses a different color model, the control is concentrated on a unique slider related to global adjustment strength. This control is automatically balanced for green or blue screen usage.
Mask Adjustments
Erode
Removes pixels on the mask boundaries, making the mask smaller. Generally this should be avoided.
Gain
Adjusts the transparency of the mask. Increasing the Gain will harden up the soft edge of the mask. Use with care.
Gamma
Scales the alpha by an exponential function. This will noticeably vary the effect of low alpha values.
Example of gamma transformation with different exponents.
Detail Enhancements (only for Chroma - PX2)
This parameter brings more details to the mask through a local analysis of the luminance variations resulting in sharper edges, but it may also bring back image noise.
Shadows
The Shadow controls allow the user to add shadows that exist on the evenly lit backdrop. Shadows are added based on the luma of the pixels; any pixel darker than the original pick will be included in the shadow, the alpha strength depending on just how dark the pixel is. This is why it is important to pick a mid green rather than the darkest green when initializing the keyer.
The "Add shadows" slider controls the alpha of the shadows, while "Adjust contrast" controls the color, trying to darken the fill in the area where shadows are identified.
Despill
The Despill controls allow the user to suppress the background color removal with separate Luma and Chroma controls and to adjust the color range and hue for the areas containing a significant amount of spill.
Respill
The Respill controls allow for the replacement of of the background color with a specific custom color, or to automatically use a color from the 3D scene. Respill can really help to blend the foreground elements into the 3D scene. It is usually best to automatically select the respill color from the background.