open3d.ml.tf.models.PointRCNN#

class open3d.ml.tf.models.PointRCNN(*args, **kwargs)#

Object detection model. Based on the PoinRCNN architecture https://github.com/sshaoshuai/PointRCNN.

The network is not trainable end-to-end, it requires pre-training of the RPN module, followed by training of the RCNN module. For this the mode must be set to ‘RPN’, with this, the network only outputs intermediate results. If the RPN module is trained, the mode can be set to ‘RCNN’ (default), with this, the second module can be trained and the output are the final predictions.

For inference use the ‘RCNN’ mode.

Parameters:

  • name (string) – Name of model. Default to “PointRCNN”.

  • device (string) – ‘cuda’ or ‘cpu’. Default to ‘cuda’.

  • classes (string[]) – List of classes used for object detection: Default to [‘Car’].

  • score_thres (float) – Min confindence score for prediction. Default to 0.3.

  • npoints (int) – Number of processed input points. Default to 16384.

  • rpn (dict) – Config of RPN module. Default to {}.

  • rcnn (dict) – Config of RCNN module. Default to {}.

  • mode (string) – Execution mode, ‘RPN’ or ‘RCNN’. Default to ‘RCNN’.

__init__(name='PointRCNN', classes=['Car'], score_thres=0.3, npoints=16384, rpn={}, rcnn={}, mode='RCNN', **kwargs)#
call(inputs, training=True)#
filter_objects(bbox_objs)#

Filter objects based on classes to train.

Parameters:

bbox_objs – Bounding box objects from dataset class.

Returns:

Filtered bounding box objects.

static generate_rpn_training_labels(points, bboxes, bboxes_world, calib=None)#

Generates labels for RPN network.

Classifies each point as foreground/background based on points inside bbox. We don’t train on ambiguous points which are just outside bounding boxes(calculated by extended_boxes). Also computes regression labels for bounding box proposals(in bounding box frame).

Parameters:

  • points – Input pointcloud.

  • bboxes – bounding boxes in camera frame.

  • bboxes_world – bounding boxes in world frame.

  • calib – Calibration file for cam_to_world matrix.

Returns:

Classification and Regression labels.

get_batch_gen(dataset, steps_per_epoch=None, batch_size=1)#
get_optimizer(cfg)#

Returns an optimizer object for the model.

Parameters:

cfg_pipeline – A Config object with the configuration of the pipeline.

Returns:

Returns a new optimizer object.

inference_end(results, inputs)#

This function is called after the inference.

This function can be implemented to apply post-processing on the network outputs.

Parameters:

results – The model outputs as returned by the call() function. Post-processing is applied on this object.

Returns:

Returns True if the inference is complete and otherwise False. Returning False can be used to implement inference for large point clouds which require multiple passes.

loss(results, inputs, training=True)#

Computes the loss given the network input and outputs.

Parameters:

  • Loss – A loss object.

  • results – This is the output of the model.

  • inputs – This is the input to the model.

Returns:

Returns the loss value.

preprocess(data, attr)#

Data preprocessing function.

This function is called before training to preprocess the data from a dataset.

Parameters:

  • data – A sample from the dataset.

  • attr – The corresponding attributes.

Returns:

Returns the preprocessed data

transform(data, attr)#

Transform function for the point cloud and features.

Parameters:

args – A list of tf Tensors.