open3d.ml.torch.models.PointRCNN

class open3d.ml.torch.models.PointRCNN(name='PointRCNN', device='cuda', classes=['Car'], score_thres=0.3, npoints=16384, rpn={}, rcnn={}, mode='RCNN', **kwargs)

Object detection model. Based on the PoinRCNN architecture https://github.com/sshaoshuai/PointRCNN.

The network is not trainable end-to-end, it requires pre-training of the RPN module, followed by training of the RCNN module. For this the mode must be set to ‘RPN’, with this, the network only outputs intermediate results. If the RPN module is trained, the mode can be set to ‘RCNN’ (default), with this, the second module can be trained and the output are the final predictions.

For inference use the ‘RCNN’ mode.

Parameters

  • name (string) – Name of model. Default to “PointRCNN”.

  • device (string) – ‘cuda’ or ‘cpu’. Default to ‘cuda’.

  • classes (string[]) – List of classes used for object detection: Default to [‘Car’].

  • score_thres (float) – Min confindence score for prediction. Default to 0.3.

  • npoints (int) – Number of processed input points. Default to 16384.

  • rpn (dict) – Config of RPN module. Default to {}.

  • rcnn (dict) – Config of RCNN module. Default to {}.

  • mode (string) – Execution mode, ‘RPN’ or ‘RCNN’. Default to ‘RCNN’.

__init__(name='PointRCNN', device='cuda', classes=['Car'], score_thres=0.3, npoints=16384, rpn={}, rcnn={}, mode='RCNN', **kwargs)

Initialize.

Parameters

  • cfg (cfg object or str) – cfg object or path to cfg file

  • dataset_path (str) – Path to the dataset

  • **kwargs (dict) – Dict of args

filter_objects(bbox_objs)

Filter objects based on classes to train.

Parameters

bbox_objs – Bounding box objects from dataset class.

Returns

Filtered bounding box objects.

forward(inputs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

static generate_rpn_training_labels(points, bboxes, bboxes_world, calib=None)

Generates labels for RPN network.

Classifies each point as foreground/background based on points inside bbox. We don’t train on ambiguous points which are just outside bounding boxes(calculated by extended_boxes). Also computes regression labels for bounding box proposals(in bounding box frame).

Parameters

  • points – Input pointcloud.

  • bboxes – bounding boxes in camera frame.

  • bboxes_world – bounding boxes in world frame.

  • calib – Calibration file for cam_to_world matrix.

Returns

Classification and Regression labels.

get_loss(results, inputs)

Computes the loss given the network input and outputs.

Parameters

  • Loss – A loss object.

  • results – This is the output of the model.

  • inputs – This is the input to the model.

Returns

Returns the loss value.

get_optimizer(cfg)

Returns an optimizer object for the model.

Parameters

cfg_pipeline – A Config object with the configuration of the pipeline.

Returns

Returns a new optimizer object.

inference_end(results, inputs)

This function is called after the inference.

This function can be implemented to apply post-processing on the network outputs.

Parameters

results – The model outputs as returned by the call() function. Post-processing is applied on this object.

Returns

Returns True if the inference is complete and otherwise False. Returning False can be used to implement inference for large point clouds which require multiple passes.

preprocess(data, attr)

Data preprocessing function.

This function is called before training to preprocess the data from a dataset.

Parameters

  • data – A sample from the dataset.

  • attr – The corresponding attributes.

Returns

Returns the preprocessed data

transform(data, attr)

Transform function for the point cloud and features.

Parameters

cfg_pipeline – config file for pipeline.

training: bool