BPE Compositing Mesh
This actor mimics the person found in the image using NVIDIA’s Body Pose Estimation.
It consists of several capsules whose size can be controlled via parameters
It has attached all the materials needed for compositing
It can cast shadows and be visible in reflections
It has physics
Radiusvalues control the shape of our Capsule-character
Body Scalescales all
Position Scaleallows moving the Capsule-character closer and further away from camera
Tick in Editorallows for in-editor updates of positions
Collision Typecontrols physics
Smoothenables a simple low pass filter
Afilter parameter (A → 0 smoother but more “lag”)
Automatic Plane Position
Body Pose Estimation can also be used to control the VS Compositing Plane position, and essentially placing the plane where the Actor is in 3D space. This means the Actor can walk towards or away from the camera, which is difficult with a fixed VS Compositing Plane position.
To use, check:
Use Body Pose Estimation for Position on the VS Compositing Plane
The VS Comp Plane will follow the Pelvis position.
Automatic Position Scale
The system infers the position in depth from the camera based on its analysis of the image, and an understanding of human morphology, including an idea of a “standard height”. From the projection of the camera, the image will look the correct height, but the skeleton position in 3D space may not be completely accurate. This is more likely to be true for large or small people i.e. a small person will be assumed to be of average height and just a bit further from camera; a tall person assumed to be closer to camera. This could become an issue with 3D interactions - moving AR objects in front of or behind the VS Comp Plane, or when the Presenter tries to interact with virtual objects.
Automatic Position Scale provides a way to scale the depth from the camera so that the Presenter is at their true height and position from camera in the 3D Scene.
The default value for
Automatic Position Scale is 0.1.
Refining the Automatic Position Scale
To calibrate this practically on set:
Position a visible marker on the studio floor and measure the distance to camera.
Place a Calibration Cone at the position of the marker in the 3D scene and check it tracks
Ensure BPE is enabled and get the chosen presenter to stand on the physical marker on set
View the scene from the left or right side with a virtual camera, to show the Skeleton and the Calibration Cone - they should both be at the same position in depth from camera
Adjust the value of the
Automatic Position Scaleso that the Skeleton lines up with the Cone