Reconstruction system (Tensor)

This tutorial demonstrates volumetric RGB-D reconstruction and dense RGB-D SLAM with the Open3D Tensor interface and the Open3D Hash map backend.

It is possible to run the tutorial with the minimalistic dataset SampleRedwoodRGBDImages, but it is recommended to run the tutorial with real-world datasets with longer sequences to demonstrate its capability. Please refer to RGBD images for more available datasets. The Redwood dataset can be a good starting point.

If you use any part of the tensor-based reconstruction system or the hash map backend in Open3D, please cite [Dong2021]:

@article{Dong2021,
    author    = {Wei Dong, Yixing Lao, Michael Kaess, and Vladlen Koltun}
    title     = {{ASH}: A Modern Framework for Parallel Spatial Hashing in {3D} Perception},
    journal   = {arXiv:2110.00511},
    year      = {2021},
}

Note

As of now the tutorial is only for online dense SLAM, and offline integration with provided poses. The tutorials for tensor-based offline reconstruction system, Simultaneous localization and calibration (SLAC), and shape from shading (SfS) tutorials as mentioned in [Dong2021] are still under construction. At current, please refer to Reconstruction system for the legacy versions.

Quick start

Getting the example code

# Activate your conda enviroment, where you have installed open3d pip package.
# Clone the Open3D github repository and go to the example.
cd examples/python/t_reconstruction_system/

# Show CLI help for ``dense_slam_gui.py``
python dense_slam_gui.py --help

Running the example with default dataset.

# The following command, will download and use the default dataset,
# which is ``lounge`` dataset from stanford.
python dense_slam_gui.py

It is recommended to use CUDA if avaialble.

# The following command, will download and use the default dataset,
# which is ``lounge`` dataset from stanford.
python dense_slam_gui.py --device 'cuda:0'

Changing the default dataset. One may change the default dataset to other avaialble datasets. Currently the following datasets are available:

  1. Lounge (keyword: lounge) (Default)

  2. Bedroom (keyword: bedroom)

  3. Jack Jack (keyword: jack_jack)

# Using jack_jack as the default dataset.
python dense_slam_gui.py --default_dataset 'bedroom'

Running the example with custom dataset using config file. Manually download or store the data in a folder and store all the color images in the image sub-folder, and all the depth images in the depth sub-folder. Create a config.yml file and set the path_dataset to the data directory. Override the parameters for which you want to change the default values.

Example config file for online reconstruction system has been provided in examples/python/t_reconstruction_system/default_config.yml, which looks like the following:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
name: Default reconstruction system config
fragment_size: 100
device: CUDA:0
engine: tensor
multiprocessing: false
path_dataset: ''
depth_folder: depth
color_folder: color
path_intrinsic: ''
path_color_intrinsic: ''
fragment_size: 100
depth_min: 0.1
depth_max: 3.0
depth_scale: 1000.0
odometry_method: hybrid
odometry_loop_interval: 10
odometry_loop_weight: 0.1
odometry_distance_thr: 0.07
icp_method: colored
icp_voxelsize: 0.05
icp_distance_thr: 0.07
global_registration_method: ransac
registration_loop_weight: 0.1
integration_mode: color
voxel_size: 0.0058
trunc_voxel_multiplier: 8.0
block_count: 40000
est_point_count: 6000000
surface_weight_thr: 3.0

Capture your own dataset

This tutorial provides an example that can record synchronized and aligned RGBD images using the Intel RealSense camera. For more details, please see Capture your own dataset.