open3d.ml.torch.models.PointTransformer#

class open3d.ml.torch.models.PointTransformer(name='PointTransformer', blocks=[2, 2, 2, 2, 2], in_channels=6, num_classes=13, voxel_size=0.04, max_voxels=80000, batcher='ConcatBatcher', augment=None, **kwargs)#

Semantic Segmentation model. Based on PointTransformer architecture https://arxiv.org/pdf/2012.09164.pdf

Uses Encoder-Decoder architecture with Transformer layers.

name#: Name of model. Default to “PointTransformer”.

blocks#: Number of Bottleneck layers.

in_channels#: Number of features(default 6).

num_classes#: Number of classes.

voxel_size#: Voxel length for subsampling.

max_voxels#: Maximum number of voxels.

batcher#: Batching method for dataloader.

augment#: dictionary for augmentation.

__init__(name='PointTransformer', blocks=[2, 2, 2, 2, 2], in_channels=6, num_classes=13, voxel_size=0.04, max_voxels=80000, batcher='ConcatBatcher', augment=None, **kwargs)#

Initialize.

Parameters:

cfg (cfg object or str) – cfg object or path to cfg file
dataset_path (str) – path to the dataset
**kwargs (dict) – Dict of args

forward(batch)#

Forward pass for the model.

Parameters:: inputs – A dict object for inputs with following keys point (tf.float32): Input pointcloud (N,3) feat (tf.float32): Input features (N, 3) row_splits (tf.int64): row splits for batches (b+1,)
Returns:: Returns the probability distribution.

get_loss(sem_seg_loss, results, inputs, device)#

Calculate the loss on output of the model.

Parameters:

sem_seg_loss – Object of type SemSegLoss.
results – Output of the model.
inputs – Input of the model.
device – device(cpu or cuda).

Returns:

Returns loss, labels and scores.

get_optimizer(cfg_pipeline)#

Returns an optimizer object for the model.

Parameters:: cfg_pipeline – A Config object with the configuration of the pipeline.
Returns:: Returns a new optimizer object.

inference_begin()#

Function called right before running inference.

Parameters:: data – A data from the dataset.

inference_end(inputs, results)#

This function is called after the inference.

This function can be implemented to apply post-processing on the network outputs.

Parameters:: results – The model outputs as returned by the call() function. Post-processing is applied on this object.
Returns:: Returns True if the inference is complete and otherwise False. Returning False can be used to implement inference for large point clouds which require multiple passes.

inference_preprocess()#

This function prepares the inputs for the model.

Returns:: The inputs to be consumed by the call() function of the model.

preprocess(data, attr)#

Data preprocessing function.

This function is called before training to preprocess the data from a dataset. It consists of subsampling pointcloud with voxelization.

Parameters:

data – A sample from the dataset.
attr – The corresponding attributes.

Returns:

Returns the preprocessed data

transform(data, attr)#

Transform function for the point cloud and features.

This function is called after preprocess method. It consists of calling augmentation and normalizing the pointcloud.

Parameters:

data – A sample from the dataset.
attr – The corresponding attributes.

Returns:

Returns dictionary data with keys (point, feat, label).

update_probs(inputs, results, test_probs)#