Reconstruction system (Tensor)#

This tutorial demonstrates volumetric RGB-D reconstruction and dense RGB-D SLAM with the Open3D Tensor interface and the Open3D Hash map backend.

It is possible to run the tutorial with the minimalistic dataset SampleRedwoodRGBDImages, but it is recommended to run the tutorial with real-world datasets with longer sequences to demonstrate its capability. Please refer to RGBD images for more available datasets. The Redwood dataset can be a good starting point.

If you use any part of the tensor-based reconstruction system or the hash map backend in Open3D, please cite [Dong2021]:

@article{Dong2021,
    author    = {Wei Dong, Yixing Lao, Michael Kaess, and Vladlen Koltun}
    title     = {{ASH}: A Modern Framework for Parallel Spatial Hashing in {3D} Perception},
    journal   = {arXiv:2110.00511},
    year      = {2021},
}

Note

As of now the tutorial is only for online dense SLAM, and offline integration with provided poses. The tutorials for tensor-based offline reconstruction system, Simultaneous localization and calibration (SLAC), and shape from shading (SfS) tutorials as mentioned in [Dong2021] are still under construction. At current, please refer to Reconstruction system for the legacy versions.

Quick start#

Getting the example code

# Activate your conda environment, where you have installed open3d pip package.
# Clone the Open3D github repository and go to the example.
cd examples/python/t_reconstruction_system/

# Show CLI help for ``dense_slam_gui.py``
python dense_slam_gui.py --help

Running the example with default dataset.

# The following command, will download and use the default dataset,
# which is ``lounge`` dataset from stanford.
python dense_slam_gui.py

It is recommended to use CUDA if available.

# The following command, will download and use the default dataset,
# which is ``lounge`` dataset from stanford.
python dense_slam_gui.py --device 'cuda:0'

Changing the default dataset. One may change the default dataset to other available datasets. Currently the following datasets are available:

  1. Lounge (keyword: lounge) (Default)

  2. Bedroom (keyword: bedroom)

  3. Jack Jack (keyword: jack_jack)

# Using jack_jack as the default dataset.
python dense_slam_gui.py --default_dataset 'bedroom'

Running the example with custom dataset using config file. Manually download or store the data in a folder and store all the color images in the image sub-folder, and all the depth images in the depth sub-folder. Create a config.yml file and set the path_dataset to the data directory. Override the parameters for which you want to change the default values.

Example config file for online reconstruction system has been provided in examples/python/t_reconstruction_system/default_config.yml, which looks like the following:

 1name: Default reconstruction system config
 2fragment_size: 100
 3device: CUDA:0
 4engine: tensor
 5multiprocessing: false
 6path_dataset: ''
 7depth_folder: depth
 8color_folder: color
 9path_intrinsic: ''
10path_color_intrinsic: ''
11depth_min: 0.1
12depth_max: 3.0
13depth_scale: 1000.0
14odometry_method: hybrid
15odometry_loop_interval: 10
16odometry_loop_weight: 0.1
17odometry_distance_thr: 0.07
18icp_method: colored
19icp_voxelsize: 0.05
20icp_distance_thr: 0.07
21global_registration_method: ransac
22registration_loop_weight: 0.1
23integrate_color: true
24voxel_size: 0.0058
25trunc_voxel_multiplier: 8.0
26block_count: 40000
27est_point_count: 6000000
28surface_weight_thr: 3.0

Capture your own dataset#

This tutorial provides an example that can record synchronized and aligned RGBD images using the Intel RealSense camera. For more details, please see Capture your own dataset.

Getting started with online reconstruction system#