Open3D 0.11.0 is full of features

Open3D 0.11.0 Release Notes

We are excited to present Open3D 0.11.0!

Open3D 0.11.0 introduces a brand new 3D Machine Learning module, nicknamed Open3D-ML. Open3D-ML is an extension of your favorite library to bring support for 3D domain-specific operators, models, algorithms, and datasets. In a nutshell, users can now create new applications combining the power of 3D data and state-of-the-art neural networks! Open3D-ML is included in all the binary releases of Open3D 0.11.0.

Open3D-ML comes with support for Pytorch +1.4 and TensorFlow +2.2, the two most popular machine learning frameworks. The first iteration of this module features a 3D semantic segmentation toolset, including training and inference capabilities for RandlaNet and KPConv. The toolset supports popular datasets such as SemanticKITTI, Semantic3D, 3D Semantic Parsing of Large-Scale Indoor Spaces S3DIS, Toronto3D, and Paris-Lille-3D. Open3D-ML also provides a new model zoo compatible with Pytorch and TensorFlow, so that users can enjoy state-of-the-art semantic segmentation models without hassles.

We have endowed the new Open3D-ML module with a new data viewer tool. Users can now inspect their datasets and model’s predictions in an intuitive and simple way. This visualization tool includes support for Pytorch and TensorFlow frameworks and is fully customizable due to its Pythonic nature.

This viewer has been built upon the new visualization API, integrating the new Rendering and GUI modules. Thanks to the new visualization API, users can perform advanced rendering, fully programmatically from Python and C++. Users can also create slick GUIs with a few lines of Python code. Check out how to do this here.

The Open3D app has also been extended to include the following features:

  • Support for FBX and glTF2 assets
  • Full support for PBR models.

Open3D 0.11 includes for the first time support for Linux ARM (64-bit) platforms. This has been a long-time requested feature that finally made it into the release. You can now enjoy all Open3D features, including our new rendering and visualization pipelines in OpenGL-enabled ARM platform.

[Breaking] Please, notice that the API and the structure of Open3D have changed considerably after an intense refactoring process. You will need to update your code to use the new namespaces. Please, check the full changelog and the documentation for further information.

We hope you find Open3D 0.11.0 exciting and useful. Happy coding!

Remember that you can reach out with questions, requests, or feedback through the following channels:

The Open3D team

Legend:

  • [Added]: Used to indicate the addition of new features
  • [Changed]: Updates of existing functionalities
  • [Deprecated]: Functionalities / features removed in future releases
  • [Removed]: Functionalities/features removed in this release
  • [Fixed]: For any bug fixes
  • [Breaking] This functionality breaks the previous API and you need to check your code

Installation and project structure

  • [Added] fetch Filament with CMake FetchContent (#2085)
  • [Added] speeds up compilation by caching 3rdparty downloads (#2155)
  • [Added] Show STATIC_WINDOWS_RUNTIME in cmake summary when configuring for Windows (#2176)
  • [Added] Move master releases to new bucket for lifecycle management (#2453)
  • [Added] added missing license header in ml module py files (#2333)
  • [Added] add vis namespace (#2394)
  • [Added] Devel wheels for users (#2429)
  • [Added] Build Filament on ARM64 (#2415)
  • [Changed] cmake: pickup python from PYTHON_EXECUTABLE or from PATH (#1923)
  • [Changed] avoid deduplication of the -gencode option (#1960)
  • [Changed] do not link main library when building the tf ops lib because of (#1981)
  • [Changed] do not use system jsoncpp (#2005)
  • [Changed] update Eigen to use the GitLab commit id (#2030)
  • [Changed] update formatter: clang-format-10, yapf 0.30.0 (#2149)
  • [Changed] disable CMP0104 warning (#2175)
  • [Changed] Build examples iteratively on Windows CI (#2199)
  • [Changed] simplify filament build-from-source (#2303)
  • [Changed] set cmake minimum to 3.15 to support generator expressions in (#2392)
  • [Changed] cmake 3.18 required for windows (#2435)
  • [Changed] ubuntu 20.04 build filament from source CI (#2438)
  • [Fixed] turobojpeg windows static runtime (#1876)
  • [Fixed] fix auto & warning (as error) for clang 10 (#1924)
  • [Fixed] Fix Eigen warning when using CUDA (#1926)
  • [Fixed] fix bug in import_3rdparty_library for paths without trailing '/' (#2084)
  • [Fixed] Fix tbb build (#2311)
  • [Fixed] Fix for cmake externalproject_add patch_command (#2431)
  • [Breaking] restructure project directory and namespace (#1970)
  • [Breaking] reorg: opend3d::gui -> open3d::visualization::gui (#1979)
  • [Breaking] change folder case (#1993)
  • [Breaking] Reorg: Added namespace 'rendering' for visualization/rendering (#2002)
  • [Breaking] remove voxel_pooling namespace (#2014)
  • [Breaking] reorg: remove hash_* namespaces (#2025)
  • [Breaking] Rename GLHelper namespace (#2024)
  • [Breaking] Removed visualization::gui::util namespace (#2013)
  • [Breaking] lower case "open3d/3rdparty" intall dir (#2083)
  • [Breaking] refactor pybind namespace (#2249)
  • [Breaking] renamed torch python namespaces (#2330)
  • [Breaking] Move geometry to open3d::t (#2403)
  • [Breaking] move io to open3d::t (#2406)

CORE features and applications

  • [Added] Add cleanup flag in TriangleMesh::select_by_index (#1883)
  • [Added] Orient normals using propagation on MST of Riemannian graph (#1895)
  • [Added] PointCloudIO: UnitTests and Benchmarks (#1891)
  • [Added] expose UniformTSDFVolume's origin in Python API (#1762)
  • [Added] open3d_show_and_abort_on_warning(Core) (#1959)
  • [Added] ml-module (#1958)
  • [Added] add make check-cpp-style, apply-cpp-style (#2016)
  • [Added] ml op test code and torch reduce_subarrays_sum op (#2050)
  • [Added] CUDA header as system header for CMake 3.16 (#2058)
  • [Added] scalar support to more binary ops (#2093)
  • [Added] Tensor api demo (#2067)
  • [Added] Initial tensor-based pointcloud (#2074)
  • [Added] support Tensor.item() in pybind (#2130)
  • [Added] MKL integration with tests (#2128)
  • [Added] Linear algebra module (#2103)
  • [Added] rpc visualization interface (#2090)
  • [Added] Pick the color for all voxels when creating a dense VoxelGrid. (#2150)
  • [Added] Assimp Base integration (#2132)
  • [Added] ISS(Intrinsic Shape Signature) Keypoint Detection Module (#1966)
  • [Added] ask assimp to build zlib (#2188)
  • [Added] initial tensor-based image class (#2161)
  • [Added] Enable CI on Ubuntu 20.04 (focal) with CUDA on GCE (#2308)
  • [Added] ARM CI (#2414)
  • [Added] initial support for tensor-based mesh (#2167)
  • [Added] pybind for tpoincloud (#2229)
  • [Added] Add pybind for Application::PostToMainThread, fix grammar error in comment (#2237)
  • [Added] Tensor for custom object data types (#2244)
  • [Added] Nearest Neighbor module (#2207)
  • [Added] torch wrapper for voxel pooling (#2256)
  • [Added] Support cuda memory cacher (#2212)
  • [Added] ml-contrib subsample library (#2254)
  • [Added] python binding of NN class (junha/nn pybind) (#2246)
  • [Added] contrib neighbor search for ML ops (#2270)
  • [Added] GCE GPU CI docker (PyTorch + cuDNN) (#2211)
  • [Added] Re-add multithreaded performance improvements to ClassIO (#2230)
  • [Added] torch continuous conv wrappers (#2287)
  • [Added] Support Float32 and Float64 neighbor search (#2241)
  • [Added] Layer interface for voxel_pooling op (#2322)
  • [Added] Fast compression mode for png writing (issue #846) (#2325)
  • [Added] Added pybinds for scene, fixed bug with Open3DScene and LOD (#2323)
  • [Added] NanoFlann Parallel Search (#2305)
  • [Added] XYZI format IO with tgeometry::PointCloud (#2356)
  • [Added] Support CPU/GPU hashmap (#2226)
  • [Added] OpenBLAS/LAPACK and support for ARM (#2205)
  • [Added] Added max error threshold (#2411)
  • [Added] Add function to compute the volume of a watertight mesh (#2407)
  • [Added] Ray/line to axis-aligned bounding box intersections (#2358)
  • [Added] IO wrapper for geometry::PointCloud -> t::geometry::PointCloud (#2462)
  • [Added] Nacho/robust kernels (#2425)
  • [Changed] test_color_map.py: adjust rtol to allow enough FP tolerance for OPENMP reordering; add .request to all import urllib (#1897)
  • [Changed] Refactor CMake buildsystem (#1782)
  • [Changed] Refactor pointcloud tests (#1925)
  • [Changed] expose poisson rec threads param (#2035)
  • [Changed] TensorList refactoring and comparison tensor ops (#2066)
  • [Changed] updated internal fields of conv layers to ease debugging (#2104)
  • [Changed] speeds up legacy pointcloud converter (#2216)
  • [Changed] Update TriangleMeshSimplification.cpp (#2192)
  • [Changed] object-oriented dtype (#2208)
  • [Changed] use pybind11's gil management (#2278)
  • [Changed] use link target torch_cpu for cpu builds (#2292)
  • [Changed] make 3rdparty_tbb depend on ext_tbb (#2297)
  • [Changed] Update camera.cpp (#2312)
  • [Changed] Delay cuSOLVER and cuBLAS init so exceptions are transferred to Python. (#2319)
  • [Changed] convert noncontiguous tensors instead of throwing an exception. (#2354)
  • [Changed] Refector some image tests failing on arm simulator (#2393)
  • [Changed] Ensure C++ and Python units tests always run (#2428)
  • [Removed] disable CreateFromPointCloudPoisson test for macos (#2054)
  • [Removed] Remove looking for X11 on macOS (#2334)
  • [Removed] Remove unused variable from SolveJacobianSystemAndObtainExtrinsicMatrix (#2398)
  • [Removed] Nacho/remove openmp guards (#2408)
  • [Fixed] fix coord frame origin bug (#2034)
  • [Fixed] fix utility::LogXX {} escape problem (#2072)
  • [Fixed] Release Python GIL for fast multithreaded IO (#1936)
  • [Fixed] PointToPlane and ColoredICP only require target normal vectors. Fix #2075 (#2118)
  • [Fixed] fixed radius search op for torch (#2101)
  • [Fixed] fix windows python dtype convert (#2277)
  • [Fixed] list pytorch device correctly for pytest (#2304)
  • [Fixed] Fix handling of 4 channel images on PNG export (#2326)
  • [Fixed] Fix for "symbol already registered" (#2324)
  • [Fixed] Slice out-of-range (#2317)
  • [Fixed] fix NanoFlannIndexHolderBase mem leak (#2340)
  • [Fixed] fix c_str() temp address warning (#2336)
  • [Fixed] fix required_gradient propagation for the offset parameter (#2350)
  • [Fixed] -bugfix for float extents for torch cconv layer (#2361)
  • [Fixed] Fix nvidia download error (#2423)
  • [Fixed] fix filament default CLANG_DEFAULT_CXX (#2424)

Rendering and visualization

  • [Added] App: add option to material combobox to restore to original values from file (#1873)
  • [Added] Add support for PNG with alpha channel (#1886)
  • [Added] GUI: implements image widget (#1881)
  • [Added] GUI: added threading and loading dialog to app (#1896)
  • [Added] Integrates point cloud I/O progress into app loading progress bar (#1913)
  • [Added] Adds menu option on macOS to make Open3D viewer default for file types (#2031)
  • [Added] Implements python bindings for gui namespace (#2042)
  • [Added] Added gui::TreeView widget (#2081)
  • [Added] Adds ability to set button padding (#2082)
  • [Added] gui::TreeView now supports arbitrary widgets for its cells (#2105)
  • [Added] Added FBX to Open3DViewer dialog (#2204)
  • [Added] GUI changes for Open3D-ML visualization (#2177)
  • [Added] Enable transparency for lit material (#2239)
  • [Added] Add shader for materials with transparency (#2258)
  • [Added] Unconditionally take base color from MATKEY_COLOR_DIFFUSE (#2265)
  • [Added] Added unlitGradient shader for colormaps and LUT (#2263)
  • [Added] Expose method to update vertex attributes (#2282)
  • [Added] Added ability to change downsample threshold in Open3DScene (#2349)
  • [Added] Faster Filament geometry creation for TPointCloud (sometimes up to 90%) (#2351)
  • [Added] Better algorithm for downsampling (#2355)
  • [Added] Add bounding-box-only mode for rotation. (#2371)
  • [Added] Add "__visualization_scalar" handling to FilamentScene::UpdateGeometry (#2376)
  • [Added] Enable python to render to image (#2413)
  • [Added] Added ability to set the preferred with of gui::NumberEdit (#2373)
  • [Added] Expose caching related functions in Python (#2409)
  • [Added] TPointCloud support for new Scene class (#2213)
  • [Added] Downsample point clouds with by using a different index array (#2318)
  • [Added] Added unlitSolidColor shader (#2352)
  • [Added] Added special name for TPointCloud rendering to create UV arrays from scalar on C++ side (#2363)
  • [Added] Add has_alpha to pybind for Material (#2383)
  • [Added Add alpha to baseColor of shader (and simplify some shader calculations) (#2396)
  • [Changed] update_progress callbacks for ReadPointCloud and WritePointCloud (#1907)
  • [Changed] only update single geometry in Visualizer::AddGeometry and Visualizer::RemoveGeometry (#1945)
  • [Changed] Updated Info.plist file for app to claim it edits and is the default type for the file associations. Also adds .pcd as a supported file type. (#2001)
  • [Changed] overload draw_geometries (#1997)
  • [Changed] Visualization refactor: Open3DViewer and rendering::Scene (#2125)
  • [Changed] Disable MTL file saving for OBJ files with no UVs and textures (issue #1974) (#2164)
  • [Changed] Upgrade Open3D to use Filament 1.8.1 (#2165)
  • [Changed] Improve UI responsiveness when viewing large models (#2384)
  • [Changed] Force scene redraw when layout changes (#2412)
  • [Fixed] Fix window showing buffer from last resize when a window is moved on macOS (#2076)
  • [Fixed] Fixed crash when create an autosizing window with no child windows (which would autosize to (0, 0), which doesn't go over well) (#2098)
  • [Fixed] Fixed hidpi on Linux (#2133)
  • [Fixed] Tell macOS that Python using GUI on macOS works like an app (#2143)
  • [Fixed] Fix GUI to build with Windows (#2153)
  • [Fixed] Model loading and rendering (#2194)
  • [Fixed] Fix rendering anomalies with ao/rough/metal texture (#2243)
  • [Fixed] Fix vis-gui.py not being able to load any geometry (#2273)
  • [Fixed] fix screen rendering in offscreen mode (#2257)
  • [Fixed] Fix 'All' option in file dialog in vis-gui.py (#2274)
  • [Fixed] Fix left side of checkboxes in a treeview on the left not being clickable (#2301)
  • [Fixed] Fix checkboxes in treeview not always redrawing when clicked (#2314)
  • [Fixed] Fix crash on Abandoned FBX model (#2339)
  • [Fixed] Fixes for UI anomalies caused by responsiveness changes (#2397)
  • [Fixed] Force redraw when menu visibility changes (#2416)
  • [Fixed] Fix Scene::SetBackgroundColor() not working if ShowSkybox() not previously called (#2452)
  • [Fixed] Caching Related Bug Fixes (#2451)
  • [Fixed] Clean up Filament to avoid crashes. (#2348)
  • [Removed] Eliminate union in MaterialParameter as it was being used incorrectly (#1879)

Documentation, tutorials, and examples

  • [Added] jupyter tutorial on compute_point_cloud_distance (#1884)
  • [Added] add cmake 3.12+ installation docs (#1914)
  • [Added] Docs updated for build with CUDA (#2055)
  • [Added] improved doc for ml layers and ops (#2296)
  • [Added] added documentation about the ml build options (#2401)
  • [Added] add docs for arm build (#2445)
  • [Changed] Update docker-gui.rst (#1868)
  • [Changed] Docs: disable multiprocessing for sphinx-build (#1951)
  • [Changed] doxygen: fix for reorg; cleanup namespaces (#1975)