Skip to main content
Skip table of contents

Set up Pixotope Living Photo

  • Creating light field files using the generator, a Pixotope Living Photo Generator license is needed

  • Using the light field files in the engine, a Pixotope Living Photo Render Add-on license is needed

Pixotope Living Photo is based on lightfield technology, which allows you to create high-quality 3D images that look lifelike and blend seamlessly with any AR, XR, and virtual studio workflows.

Pixotope Living Photo compared to flat 2D image

Setting up Pixotope Living Photo is a 3 step process:

Test with example files

If you just want do do a quick test run, you can download

Record talent

To streamline the recording of multiple talents we recommend to do a test production going through all the steps! This way you can get familiar with which recording setup and talent movements work best for your intended use.

To record your talent you need

  • a green/blue screen and lighting setup

  • a way to rotate the camera to record in portrait mode (for optimized resolution)

  • a turntable painted green/blue

Check out these recommended turntable sets from our partner → Pre-Motion

Prepare your green screen

  • Use any kind of green or blue screen

    • Choose a colour which does not appear in the clothes of the talent

  • The green screen should cover the floor, but it does not have to be very wide

  • The distance between the talent and the green screen should be at least 2m to avoid green spill

Set up your lights and camera

Lights

  • Choosing an even lighting helps to avoid shadows which might not always fit the shooting/viewing angle

  • We recommend to use the same camera which you later use for the live production, but really any camera can be used

Camera

  • Rotating the camera by 90 degrees to portrait mode optimizes the available resolution

  • The position of the camera should have the same height as in the live production

    • If this is not decided yet, using the height of the talent’s upper chest is a good starting point

  • Use a horizontal position and do not tilt the camera too much towards the floor

  • Camera settings like gain, gamma, black etc. can be chosen freely

    • We recommend to use a neutral setting and add effects/colour grading/gamma in post production

Directing your talent

  • The feet of the talent should be centered on the turntable

  • To achieve the effect that the talent’s head follows the camera during the live production, the talent needs to turn their head during the recording as well. Face the camera as long as possible.

  • The talent should not blink

  • Avoid significant movements with other body parts

Make the recording

  1. Set a mark on the turntable indicating the zero point

  2. Make the mark face the camera

  3. Choose to record a single or a double turn

Double turn (recommended)

  1. Position the talent so they are seen from the front

  2. Start the recording and rotate the turntable 2x

A double turn allows for more creative flexibility

Single turn

  1. Position the talent so they are seen from the back

  2. Start the recording and rotate the turntable 1x

Create light field

Install Pixotope Living Photo Generator

  1. Pixotope Living Photo Generator is installed together with Pixotope Graphics

  2. Launch Director and log in

  3. Select the Pixotope Living Photo Generator license

  4. Launch the application via the Windows Start menu

In 24.1, it is not possible to have a Pixotope Living Photo Generator license activated simultaneously with a Live VS/AR or Live XR license.

To switch between Pixotope Living Photo Generator and Pixotope Graphics, please close one and change the license type in Director.

Prepare the video

  1. Click "Open" on the top left to import the recorded video file

    • Any format supported by ffmpeg can be used

  2. Rotate the video by adding a Transform processor on the top right

  3. Remove unwanted areas by adding one or more Crops processors

    • Choose between different shapes and adjust its corners

  4. Key the image by adding a ChromaKey processor

    • To pick a color area

      1. Hold down Shift + Right click to drag a selection rectangle around the color you want to remove

      2. Hold down Shift and drag the rectangle onto the color swatch in the processor tab on the right

  5. Remove the unwanted color spill from the talent by adding a Despill processor

    • To pick a color area, see above

Use the Position slider underneath the image to scrub through the video.

It is possible to use multiple processes at the same time and also change their order.

Canvas menu

Canvas menu

Unprocessed (Shift): Click this menu, or hold shift to view the footage unedited without processes. This is also handy to use while picking colors for ChromaKey and Despill.

Modes

There are three modes to view the footage in.

  • Composed - shows current footage and all active processes

  • Alpha - shows a black and white silhouette of the final image. This is commonly used to locate any undesired holes in the footage

  • Binarized Alpha - shows the binarized alpha image. Depending on the render engine alpha values cannot be processed. Here the binarized alpha image should be used.

Background color

Color, saturation and value may be adjusted in order to judge the processed areas better with the human eye, especially the chroma key result. A light gray is recommended, as usually the hair details and spill could be recognized more easily. Processed areas means areas that have been received treatment through cropping, chroma key and despill. The regions with the background colour will be removed entirely once the footage is ready to be exported to Pixotope.

Save this profile

In case you need to do further editing or apply the same values to new footage, save them as a profile for later use

  1. Type a new profile name into the Profile dropdown on the top right

  2. Click the Save icon next to it

The profile is now selectable using the Profile dropdown.

To remove a profile, select it and click the Remove icon.

Set light field range

  1. Select a frame where the Zero point* is in front by using the Position slider

  2. Move the white Origin Crosshair with your mouse (drag and drop) to exactly the center mark at the front of the turntable. This helps to find exactly the same point again for the Origin and Out marks.

  3. Define the starting position of the light field by clicking and dragging the image into the IN slot on the top left

  4. Use the Position slider to go to the OUT and the ORIGIN position and repeat step 2

  5. Specify the amount of turns

IN

ORIGIN

OUT

Double turn

Zero point* in front
Talent facing the camera

turntable position before the first turn

Zero point* in front
Talent the facing camera

turntable position after first and before second turn

Zero point* in front
Talent facing the camera

turntable position after the second turn

Single turn

Zero point* in front
Talent facing the other way

turntable position before the first turn

Zero point* in the back
Talent facing the camera

turntable position after 180 degrees turn

Zero point* in front
Talent facing the other way

turntable position after the first turn

* Zero point: indicated by the mark at the front of the turntable

Set the Origin point

  1. Drag and drop the Origin Crosshair exactly at the center of the turntable (between the talent’s feet) to set the Origin crosshair.

Export the light field

  1. Click the Export icon to generate the light field file

    • Save it to a location inside the project folder

    • A .lf file is created

If you are using very large footage and experience issues when exporting the light field file, please contact support.

Add to level

  1. Place the Living Photo Actor in the level by finding it in the Place Actor tab and dragging it into the level

  2. Go to the Details of the actor and scroll down to the Living Photo section

  3. Link to the exported light field file, by specifying the Light Filed File Path

  4. Select the TrackedCamera actor as the Target actor the talent will look towards

Additional settings

Light Field Index Offset

Specify which frame/position the light field should start with

Wrap Light Field Index

Automatically continue to the start/end frame when a larger/smaller index is chosen

Reverse Light Field Index

Reverse the order of frames

Skip Images

Improve performance during rapid movements at the cost of angular resolution by using/loading only every second/third image

  • 1 - use every image (nothing skipped)

  • 2 - use every other image

  • 3 - use every third image

Use unjittered projection

Move the texture past Unreal Engine TAA, giving a much sharper result

Always on top

Ensure that the Light Field Actor is always rendered on top regardless of being placed behind other actors

Pitch Sensitivity

Determine how much the Living photo adjusts its pitch angle when the Target Actor moves vertically

Reset Scale to Light Field Aspect

Reset custom move, rotation and scale position to default

Tips and tricks

Good practice

It’s considered good practice to make sure the feet are touching the floor. Be mindful that the feet will not look entirely grounded and may look better if hid behind something.

Set up viewing range

You can usually handle the light field like a regular texture; but you can adjust the starting point of the video and if the texture should also be following the camera in vertical direction; not recommended, but sometimes a valid option for small camera movements.

What to avoid?

As we are shooting the talent just from one height, we do not record information from higher angles. Therefore also during the broadcast you should avoid higher shooting angles, e.g. high crane shoots.

As a solution, you can use multiple cameras during the shoot!!! Besides the regular camera, which is positioned at the height of the breast of the talent, you can use an addtional one from below for hero shots. Also you can use one camera at a higher position (especialy while using a flying/drone camera during the broadcast). During the live broadcast you have to decide which lightfield to use. You can not blend from one file to the other one!

JavaScript errors detected

Please note, these errors can depend on your browser setup.

If this problem persists, please contact our support.