BPE Compositing Mesh

This actor mimics the person found in the image using NVIDIA’s Body Pose Estimation.

Learn more about how to Set up Body Pose Estimation (BPE)

• It consists of several capsules whose size can be controlled via parameters

• It has attached all the materials needed for compositing

• It can cast shadows and be visible in reflections

• It has physics

Parameters

• Radius values control the shape of our Capsule-character

• Body Scale scales all Radius values

• Position Scale allows moving the Capsule-character closer and further away from camera

• Tick in Editor allows for in-editor updates of positions

• Collision Type controls physics

• Smooth enables a simple low pass filter

  • A filter parameter (A → 0 smoother but more “lag”)

Automatic Plane Position

Body Pose Estimation can also be used to control the VS Compositing Plane position, and essentially placing the plane where the Actor is in 3D space. This means the Actor can walk towards or away from the camera, which is difficult with a fixed VS Compositing Plane position.

To use, check: Use Body Pose Estimation for Position on the VS Compositing Plane

The VS Comp Plane will follow the Pelvis position.

Automatic Position Scale

The system infers the position in depth from the camera based on its analysis of the image, and an understanding of human morphology, including an idea of a “standard height”. From the projection of the camera, the image will look the correct height, but the skeleton position in 3D space may not be completely accurate. This is more likely to be true for large or small people i.e. a small person will be assumed to be of average height and just a bit further from camera; a tall person assumed to be closer to camera. This could become an issue with 3D interactions - moving AR objects in front of or behind the VS Comp Plane, or when the Presenter tries to interact with virtual objects.

The Automatic Position Scale provides a way to scale the depth from the camera so that the Presenter is at their true height and position from camera in the 3D Scene.

The default value for Automatic Position Scale is 0.1.

Refining the Automatic Position Scale

To calibrate this practically on set:

1. Position a visible marker on the studio floor and measure the distance to camera.

2. Place a Calibration Cone at the position of the marker in the 3D scene and check it tracks

3. Ensure BPE is enabled and get the chosen presenter to stand on the physical marker on set

4. View the scene from the left or right side with a virtual camera, to show the Skeleton and the Calibration Cone - they should both be at the same position in depth from camera

5. Adjust the value of the Automatic Position Scale so that the Skeleton lines up with the Cone