System overview

The system has three main steps:

Step 1. Make fragments: build local geometric surfaces (referred to as fragments) from short subsequences of the input RGBD sequence. This part uses RGBD odometry, Multiway registration, and RGBD integration.

Step 2. Register fragments: the fragments are aligned in a global space. This part uses Global registration, ICP registration, and Multiway registration.

Step 3. Integrate scene: integrate RGB-D images to generate a mesh model for the scene. This part uses RGBD integration.

Example dataset

We use the SceneNN dataset to demonstrate the system in this tutorial. Alternatively, there are lots of excellent RGBD datasets such as Redwood data, TUM RGBD data, ICL-NUIM data, and SUN3D data.

The tutorial uses the 016 sequence from the SceneNN dataset. The sequence is from SceneNN oni file archieve. The oni file can be extracted into color and depth image sequence using OniParser from the Redwood reconstruction system. Alternatively, any tool that can convert an .oni file into a set of synchronized RGBD images will work. This is a quick link to download the 016 sequence.

Quick start

Put all color images in the image folder, and all depth images in the depth folder. Run following commands from the root folder.

cd examples/Python/ReconstructionSystem/
python [config_file] [--make] [--register] [--integrate]

config_file has parameters and file paths. For example, ReconstructionSystem/config/redwood.json has the following script.

    "name": "Redwood Dataset:",
    "path_dataset": "/Users/jaesikpa/Downloads/011_frag/",
    "path_intrinsic": "",
    "max_depth": 4.0,
    "voxel_size": 0.05,
    "max_depth_diff": 0.07,
    "tsdf_cubic_size": 3.0,
    "icp_method": "color"

We assume the color images and the depth images are synchronized and registered. "path_intrinsic" specifies path to a json file that stores the camera intrinsic matrix (See Read camera intrinsic for details). If it is not given, the PrimeSense factory setting is used. For your own dataset, use an appropriate camera intrinsic and visualize a depth image (likewise Redwood dataset) prior to use the system.