Set up TalenTrack

TalenTrack allows an accurate live estimation of the pose and position of a human body directly from a video image, using Pixotope’s TalenTrack hardware. It provides a real-time set of 3D data points corresponding to skeletons with bones and body parts.

With it you can:

• position the VS compositing plane, allowing the talents on the green screen set to freely move towards and away from the camera

• Keep track of 1-4 target talents (Per TalenTrack machine)

• render shadows and reflections more accurately and even render footprints

• allow the talents to interact with the virtual environment using

  • physics

  • trigger boxes

  • hand actions

  • or a clicker control for pick up & place logic

• Requires a separate TalenTrack machine to run alongside a Pixotope machine

• Uses between 3 to 6 ceiling cameras to keep track of the targets. In practice this also means you won’t lose tracking if you pan the main camera away from the target actor, or if you zoom with the camera. This is in contrast to Body Pose Estimation (BPE)

• Can run alongside a Pixotope machine that’s paired with a standard tracking machine

Install TalenTrack

To run and install TalenTrack, you need an additional approved machine dedicated to running TalenTrack per your own specification. Please checkout its manual.

Example of a single machine setup of TalenTrack

Use TalenTrack in level

Before getting started, the Pixotope Editor needs to be correctly routed to the TalenTrack machine.

In Editor

1. Enable the Pixotope TalenTrack plugin and restart the Editor

   

2. Go to Pixotope World Settings and navigate to TalenTrack Sender Port

   

3. Fill in the port number from the TalenTrack machine. In this example: 60015

   

4. Go to Place Actors > Pixotope

5. Add the "TalenTrack & Plane" actor to the level

   • Dropping this actor into the scene creates:

     • a "TalenTrack Compositing Mesh"

     • and a "VS Internal Compositing Plane" which follows the "TalenTrack Compositing Mesh" in position and scale

       • Rotate with Camera is enabled

         

6. In the TalenTrack details tab: Head down to the components section.
   Set person index to reflect which tracking target should use the selected “TalenTrack & Plane” actor. Set target number between 1-4

   

7. Choose if the plane should auto-size with "Use Body Pose Estimation for scale"

   • TPose - always provide enough space for handling fast movement and hand action covering the maximum extent of the body

   • Tight - closely follow the detected body with a small margin around to ensure it doesn’t clip (for smaller green screens)

   • Off - no autosizing, manual sizing

The same setup as the TalenTrack & Plane actor can be achieved by adding

• a "TalentTrack Compositing Mesh"

• a "VS Internal Compositing Plane"

  • with "Use body pose estimation for position" enabled

• and linking "Position Scale" of both actors

Advanced

Use TalenTrack in blueprints

All TalenTrack points can be directly accessed in blueprints via the BPE TalenTrack Manager.

Get World Point	Returns Position and Orientation of a specific point in world space
Is Point Valid	Checks if the point was received and is not zero
Is Active	Checks if the TalenTrack connection is established
Start	Starts the connection to the TalenTrack backend using the port provided in World Settings. If a TalenTrackActor is present in the scene it will establish connection. Needs to be called at least once before any data can be received.
Stop	Stops the connection.

In addition, the TalenTrackCompositingMesh has multiple “attachment” points that can be used.