How to create Shorts, Reels, and TikToks with 3D camera motions (dolly, boom, orbit)

Overview

This guide shows how to create short videos (Shorts, Reels, TikTok) where the camera does a 3D motion like dolly, boom, or orbit. This effect is achieved by rendering a photorealistic 3D model, which is created from a video captured with a phone (or potentially any video). Volurama is an advanced tool for creating neural radiance fields (NeRFs), which are photorealistic 3D models. This tutorial will show you how to capture the necessary video with a phone, setup the 3D motion, and render a short video.

Step-by-step instructions

1. Capture an input video with a phone
   Volurama can potentially work with any type of input video, but for reliable results, we suggest starting with the capture process here. We recommend using the widest FOV setting available on the camera. For best results, use a video that is about 5 to 30 seconds long. Move the camera around in a spiral, circle, or square pattern to get each part of the scene from several different points of view. You can capture in portrait or landscape view.
2. Create a new project in Volurama
   Launch Volurama and click "Create New Project".

3. Select an input video
   Select a video file on your computer. It can be an mp4, mov, or mkv file.

4. Select a project directory
   Select a folder where files related to this project will be saved. You should create a new directory just for this. Large amounts of data may be created in this folder, and files may be automatically deleted from it as well.

5. SFM & NeRF options
   This screen allows you to configure settings which affect the quality of the results, and how long it takes to process. For a faster run, leave these at the default values. When ready, click "Start Processing".
   
   Tip: change "Structure from Motion" ▸ "# Iterations" from 40 to 25 to save time without greatly affecting quality.
   Tip: for videos which contain a lot of moving objects, try changing "Structure from Motion" ▸ "Outlier Percentile" from 0.8 to a smaller number like 0.5 or 0.25.

6. Wait for processing, visualization
   It may take a while to process, depending on your settings and computer. While you wait, there are three main visualizations to look at: keypoint tracking, structure from motion optimization, and NeRF optimization. Keypoint tracking is the first step in the computer vision pipeline for determining camera motion. Structure from motion is the part of the system which solves a math problem to determine the cameras position and orientation in every frame of the input video. NeRF optimization is the part where it uses machine learning to create a photorealistic 3D model of the scene.

7. Real-time 3D preview
   The 3D view is a quick and dirty preview of your scene, not the most photorealistic rendering possible (see the "Preview Render" window for that, and make it bigger). The 3D view only includes nearby objects, it doesn't include the background; its OK if it is missing some parts of the scene, as long as they appear in the "Preview Render" view. The spiral in the video below is the software's estimate of the 3D path the camera followed in the input video.
8. Adjust timeline duration (optional)
   Volurama renders videos as output, and the length of these videos is determined by the timeline. To change the duration of rendered videos, use the menu: Virtual Camera ▸ Change Timeline Duration. Modify the timeline duration first, before setting up virtual camera motion.

9. Virtual camera motion presets
   To create a visually pleasing result, Volurama can render the scene from the point of view of a virtual camera, which moves along any path we want. The simplest way to set up this motion is to use the motion presets, e.g. the menu Virtual Camera ▸ Dolly Forward (or any of the others). Most of these presets create two keyframes in the timeline, one at the first frame and one at the last. It is convenient to start with a preset like this, then modify the keyframes to further adjust the virtual camera motion.

10. Edit keyframes
    Use the Keyframe Editor window to adjust the position and rotation of the virtual camera. The Preview Render window shows what the final output from the virtual camera will be.

11. Orbit Editor
    To make the virtual camera move in a circle instead of a line, use the Orbit Editor. The orange dashed line shows the path that will be created, but it won't do anything until you press the "Create Keyframes" button. You may wish to adjust the Start Angle and End Angle so that the camera does not rotate too far, and see parts of the scene which were not captured by the original video.

12. Virtual camera settings - optimized for Shorts, Reels, and TikToks
    To create short content, use the following Virtual Camera Settings: the camera type should be "Rectilinear". The width and height should be 1080 x 1920. The horizontal field of view is an artistic choice.

13. Render video
    To begin rendering the final output, use the menu: File ▸ Render Video. This will open the render config screen, which has the same options as the Virtual Camera Settings, as well as options for which video compression formats to create. ProRes is near-lossless and maintains more quality (Volurama uses ProRes 422LT by default), but has larger filesize.

14. View rendered frames
    While rendering, each frame is saved as an image in the /render_frames subdirectory of the project. It is useful to look at these results before the full render completes. After all frames are finished rendering, a video .mp4 or .mov file is generated in the project directory.
15. The result: a short video with an interesting 3D camera motion