open3d.ml.tf.ops.sparse_conv

open3d.ml.tf.ops.sparse_conv(filters, inp_features, inp_importance, neighbors_index, neighbors_kernel_index, neighbors_importance, neighbors_row_splits, output_type=tf.float32, normalize=False, max_temp_mem_MB=64, name=None)

General sparse convolution.

This op computes the features for the forwad pass. This example shows how to use this op:

import tensorflow as tf
import open3d.ml.tf as ml3d

# This filter has 3 "spatial" elements with 8 input and 16 output channels
filters = tf.random.normal([3,8,16])

inp_features = tf.random.normal([5,8])


out_features = ml3d.ops.sparse_conv(
    filters,
    inp_features=inp_features,
    inp_importance=[],
    neighbors_index=[0,1,2, 1,2,3, 2,3,4],
    # neighbors_kernel_index defines which of the "spatial"
    # elements of the filter to use
    neighbors_kernel_index=tf.convert_to_tensor([0,1,2, 0,1,2, 0,1,2], dtype=tf.uint8),
    neighbors_importance=[],
    neighbors_row_splits=[0,3,6,9]
)

# or with pytorch
import torch
import open3d.ml.torch as ml3d

# This filter has 3 "spatial" elements with 8 input and 16 output channels
filters = torch.randn([3,8,16])

inp_features = torch.randn([5,8])


out_features = ml3d.ops.sparse_conv(
    filters,
    inp_features=inp_features,
    inp_importance=torch.FloatTensor([]),
    neighbors_index=torch.IntTensor([0,1,2, 1,2,3, 2,3,4]),
    # neighbors_kernel_index defines which of the "spatial"
    # elements of the filter to use
    neighbors_kernel_index=torch.ByteTensor([0,1,2, 0,1,2, 0,1,2]),
    neighbors_importance=torch.FloatTensor([]),
    neighbors_row_splits=torch.LongTensor([0,3,6,9])
)

Parameters

• filters –

  A Tensor. Must be one of the following types: float32, float64, bfloat16.

  The filter parameters. The shape of the filter is [depth, height, width, in_ch, out_ch]. The dimensions ‘depth’, ‘height’, ‘width’ define the spatial resolution of the filter. The spatial size of the filter is defined by the parameter ‘extents’.

• inp_features – A Tensor. Must have the same type as filters. A 2D tensor which stores a feature vector for each input point.

• inp_importance – A Tensor. Must have the same type as filters. An optional scalar importance for each input point. The features of each point will be multiplied with the corresponding value. The shape is [num input points]. Use a zero length Tensor to disable.

• neighbors_index –

  A Tensor. Must be one of the following types: int32, int64.

  The neighbors_index stores a list of indices of neighbors for each output point as nested lists. The start and end of each list can be computed using ‘neighbors_row_splits’.

• neighbors_kernel_index –

  A Tensor. Must be one of the following types: uint8, int16.

  Defines which kernel element to use for each neighbor. This array has the same length as neighbors_index.

• neighbors_importance – A Tensor. Must have the same type as filters. Tensor of the same shape as ‘neighbors_index’ with a scalar value that is used to scale the features of each neighbor. Use a zero length Tensor to weigh each neighbor with 1.

• neighbors_row_splits – A Tensor of type int64. The exclusive prefix sum of the neighbor count for the output points including the total neighbor count as the last element. The size of this array is the number of output points + 1.

• output_type – An optional tf.DType from: tf.float32, tf.float64, tf.bfloat16. Defaults to tf.float32. The type for the output.

• normalize – An optional bool. Defaults to False. If True the output feature values will be normalized using the sum for ‘neighbors_importance’ for each output point.

• max_temp_mem_MB – An optional int. Defaults to 64. Defines the maximum temporary memory in megabytes to be used for the GPU implementation. More memory means fewer kernel invocations. Note that the a minimum amount of temp memory will always be allocated even if this variable is set to 0.

• name – A name for the operation (optional).

Returns

A Tensor of type output_type. A Tensor with the output feature vectors for each output point.