groundTruthMultisignal

Ground truth label data for multiple signals

Description

The groundTruthMultisignal object contains information about the ground truth data source, label definitions, and marked label annotations for multiple signals. The source of the signals can be a video, image sequence, lidar point cloud, or any other custom format containing multiple signals. You can export or import a groundTruthMultisignal object from the Ground Truth Labeler app.

To create training data for deep learning applications from arrays of groundTruthMultisignal objects, use the gatherLabelData function.

Creation

To export a groundTruthMultisignal object from the Ground Truth Labeler app, on the app toolstrip, select Export Labels > To Workspace. The app exports the object to the MATLAB^® workspace. To create a groundTruthMultisignal object programmatically, use the groundTruthMultisignal function (described here).

Syntax

gTruth = groundTruthMultisignal(dataSources,labelDefs,roiData,sceneData)

Description

example

gTruth = groundTruthMultisignal(dataSources,labelDefs,roiData,sceneData) returns an object containing ground truth labels that can be imported into the Ground Truth Labeler app.

dataSources specifies the sources of the ground truth data and sets the DataSource property.
labelDefs specifies the label, sublabel, and attribute definitions of the ground truth data and sets the LabelDefinitions property.
roiData specifies the identifying information, position, and timestamps for the marked region of interest (ROI) labels and sets the ROILabelData property.
sceneData specifies the identifying information and timestamps for marked scene labels and sets the SceneLabelData property.

Properties

expand all

`DataSource` — Sources of ground truth data
vector of `MultiSignalSource` objects

Sources of ground truth data, specified as a vector of MultiSignalSource objects. These objects contain information that describe the sources from which ground truth data was labeled. This table describes the type of MultiSignalSource objects that you can specify in this vector.

`MultiSignalSource` Object Type	Data Source	Class Reference
`VideoSource`	Video file	`vision.labeler.loading.VideoSource`
`ImageSequenceSource`	Image sequence folder	`vision.labeler.loading.ImageSequenceSource`
`VelodyneLidarSource`	Velodyne^® packet capture (PCAP) file	`vision.labeler.loading.VelodyneLidarSource`
`RosbagSource`	Rosbag file	`vision.labeler.loading.RosbagSource`
`PointCloudSequenceSource`	Point cloud sequence folder	`vision.labeler.loading.PointCloudSequenceSource`
`CustomImageSource`	Custom image format	`vision.labeler.loading.CustomImageSource`

To specify additional data sources, create a new type of MultiSignalSource object by using the vision.labeler.loading.MultiSignalSource class.

`LabelDefinitions` — Label definitions
table

Label definitions, specified as a table. To create this table, use one of these options.

In the Ground Truth Labeler app, create label definitions, and then export them as part of a groundTruthMultisignal object.
Use a labelDefinitionCreatorMultisignal object to generate a label definitions table. If you save this table to a MAT-file, you can then load the label definitions into a Ground Truth Labeler app session by selecting Open > Label Definitions from the app toolstrip.
Create the label definitions table at the MATLAB command line.

This table describes the required and optional columns of the table specified in the LabelDefinitions property.

Column Description Required or Optional

Name

Strings or character vectors specifying the name of each label definition.

Required

SignalType

SignalType enumerations that specify the signal type supported for each label definition. Valid values are Image for image signals such as videos or image sequences, PointCloud for lidar signals, or Time for scene label definitions.

If a label definition supports multiple signal types, then the label definition has a separate row for each signal type. For example, consider a label definition named car. In the Ground Truth Labeler app, you draw this label as a rectangle in image signals and a cuboid in lidar point cloud signals. In the LabelDefinitions table, car appears twice and has these Name, SignalType, and LabelType values.

`Name`	`SignalType`	`LabelType`
`'car'`	`Image`	`Rectangle`
`'car'`	`PointCloud`	`Cuboid`

Required

LabelType

labelType (Computer Vision Toolbox) enumerations that specify the type of each label definition.

For ROI label definitions with a SignalType of Image, valid LabelType enumerations are:

labelType.Rectangle
labelType.Line
labelType.PixelLabel
labelType.ProjectedCuboid

For ROI label definitions with a SignalType of PointCloud, the only valid LabelType enumeration is labelType.Cuboid.

For scene label definitions, the only valid LabelType enumeration is labelType.Scene.

You can also add ground truth data that is not an ROI or scene label to a groundTruthMultisignal object. In the label definitions table, specify a label definition whose labelType is Custom.

Required

Group

Strings or character vectors specifying the group to which each label definition belongs.

Optional

If you create label definitions at the MATLAB command line, you do not need to include a Group column.

If you export label definitions from the Ground Truth Labeler app or create them using a labelDefinitionCreatorMultisignal object, the label definitions table includes this column, even if you did not specify groups. The app assigns each label definition a Group value of 'None'.

Description

Strings or character vectors that describe each label definition.

Optional

If you create label definitions at the MATLAB command line, you do not need to include a Description column.

If you export label definitions from the Ground Truth Labeler app or create them using a labelDefinitionCreatorMultisignal object, the label definitions table includes this column, even if you did not specify descriptions. The Description for these label definitions is an empty character vector.

LabelColor

1-by-3 row vectors of RGB triplets that specify the colors of the label definitions. Values are in the range [0, 1]. The color yellow (RGB triplet [1 1 0]) is reserved for the color of selected labels in the Ground Truth Labeler app.

Optional

When you define labels in the Ground Truth Labeler app, you must specify a color. Therefore, an exported label definitions table always includes this column.

When you create label definitions using the labelDefinitionCreatorMultisignal object without specifying colors, the returned label definition table includes this column, but all column values are empty.

PixelLabelID

Scalars, column vectors, M-by-3 matrices of integer-valued label IDs. PixelLabelID specifies the pixel label values used to represent a label definition. Pixel label ID values must be between 0 and 255.

Optional

When you define pixel labels in the Ground Truth Labeler app or the labelDefinitionCreatorMultisignal object, the generated label definitions table includes this column.

When creating a label definitions table at the MATLAB command line, if you set LabelType to labelType.PixelLabel for any label, then this column is required.

Hierarchy

Structures containing sublabel and attribute data for each label definition. For an example of the Hierarchy format, see Export and Explore Ground Truth Labels for Multiple Signals.

Optional

When you define sublabels or attributes in the Ground Truth Labeler app or the labelDefinitionCreatorMultisignal object, the generated label definitions table includes this column.

`ROILabelData` — ROI label data
`ROILabelData` object

ROI label data across all signals, specified as an ROILabelData object.

For Rectangle, Cuboid, ProjectedCuboid, and Line label types, ground truth data that is not a floating-point array has a data type of single.

`SceneLabelData` — Scene label data
`SceneLabelData` object

Scene label data across all signals, specified as a SceneLabelData object.

Object Functions

`selectLabelsByLabelName`	Select multisignal ground truth by label name
`selectLabelsByLabelType`	Select multisignal ground truth by label type
`selectLabelsByGroupName`	Select multisignal ground truth by label group name
`selectLabelsBySignalName`	Select multisignal ground truth by signal name
`selectLabelsBySignalType`	Select multisignal ground truth labels by signal type
`gatherLabelData`	Gather synchronized label data from ground truth
`writeFrames`	Write signal frames for ground truth data to disk
`changeFilePaths`	Change file paths in multisignal ground truth data

Examples

collapse all

Create Ground Truth from Multiple Signals

Open Script

Create ground truth data for a video signal and a lidar point cloud sequence signal that captures the same driving scene. Specify the signal sources, label definitions, and ROI and scene label data.

Create the video data source from an MP4 file.

sourceName = '01_city_c2s_fcw_10s.mp4';
sourceParams = [];
vidSource = vision.labeler.loading.VideoSource;
vidSource.loadSource(sourceName,sourceParams);

Create the point cloud sequence source from a folder of point cloud data (PCD) files.

pcSeqFolder = fullfile(toolboxdir('driving'),'drivingdata','lidarSequence');
addpath(pcSeqFolder)
load timestamps.mat
rmpath(pcSeqFolder)

lidarSourceData = load(fullfile(pcSeqFolder,'timestamps.mat'));

sourceName = pcSeqFolder;
sourceParams = struct;
sourceParams.Timestamps = timestamps;

pcseqSource = vision.labeler.loading.PointCloudSequenceSource;
pcseqSource.loadSource(sourceName,sourceParams);

Combine the signal sources into an array.

dataSource = [vidSource pcseqSource]

dataSource = 

  1x2 heterogeneous MultiSignalSource (VideoSource, PointCloudSequenceSource) array with properties:

    SourceName
    SourceParams
    SignalName
    SignalType
    Timestamp
    NumSignals

Create a table of label definitions for the ground truth data by using a labelDefinitionCreatorMultisignal object.

The Car label definition appears twice. Even though Car is defined as a rectangle, you can draw rectangles only for image signals, such as videos. The labelDefinitionCreatorMultisignal object creates an additional row for lidar point cloud signals. In these signal types, you can draw Car labels as cuboids only.
The label definitions have no descriptions and no assigned colors, so the Description and LabelColor columns are empty.
The label definitions have no assigned groups, so for all label definitions, the corresponding cell in the Group column is set to 'None'.
Road is a pixel label definition, so the table includes a PixelLabelID column.
No label definitions have sublabels or attributes, so the table does not include a Hierarchy column for storing such information.

ldc = labelDefinitionCreatorMultisignal;
addLabel(ldc,'Car','Rectangle');
addLabel(ldc,'Truck','ProjectedCuboid');
addLabel(ldc,'Lane','Line');
addLabel(ldc,'Road','PixelLabel');
addLabel(ldc,'Sunny','Scene');
labelDefs = create(ldc)

labelDefs =

  6x7 table

      Name       SignalType       LabelType        Group      Description    LabelColor    PixelLabelID
    _________    __________    _______________    ________    ___________    __________    ____________

    {'Car'  }    Image         Rectangle          {'None'}       {' '}       {0x0 char}    {0x0 double}
    {'Car'  }    PointCloud    Cuboid             {'None'}       {' '}       {0x0 char}    {0x0 double}
    {'Truck'}    Image         ProjectedCuboid    {'None'}       {' '}       {0x0 char}    {0x0 double}
    {'Lane' }    Image         Line               {'None'}       {' '}       {0x0 char}    {0x0 double}
    {'Road' }    Image         PixelLabel         {'None'}       {' '}       {0x0 char}    {[       1]}
    {'Sunny'}    Time          Scene              {'None'}       {' '}       {0x0 char}    {0x0 double}

Create ROI label data for the first frame of the video.

numVideoFrames = numel(vidSource.Timestamp{1});
carData = cell(numVideoFrames,1);
laneData = cell(numVideoFrames,1);
truckData = cell(numVideoFrames,1);
carData{1} = [304 212 37 33];
laneData{1} = [70 458; 311 261];
truckData{1} = [309,215,33,24,330,211,33,24];
videoData = timetable(vidSource.Timestamp{1},carData,laneData, ...
                      'VariableNames',{'Car','Lane'});

Create ROI label data for the first point cloud in the sequence.

numPCFrames = numel(pcseqSource.Timestamp{1});
carData = cell(numPCFrames, 1);
carData{1} = [27.35 18.32 -0.11 4.25 4.75 3.45 0 0 0];
lidarData = timetable(pcseqSource.Timestamp{1},carData,'VariableNames',{'Car'});

Combine the ROI label data for both sources.

signalNames = [dataSource.SignalName];
roiData = vision.labeler.labeldata.ROILabelData(signalNames,{videoData,lidarData})

roiData = 

  ROILabelData with properties:

    video_01_city_c2s_fcw_10s: [204x2 timetable]
                lidarSequence: [34x1 timetable]

Create scene label data for the first 10 seconds of the driving scene.

sunnyData = seconds([0 10]);
labelNames = ["Sunny"];
sceneData = vision.labeler.labeldata.SceneLabelData(labelNames,{sunnyData})

sceneData = 

  SceneLabelData with properties:

    Sunny: [0 sec    10 sec]

Create a ground truth object from the signal sources, label definitions, and ROI and scene label data. You can import this object into the Ground Truth Labeler app for manual labeling or to run a labeling automation algorithm on it. You can also extract training data from this object for deep learning models by using the gatherLabelData function.

gTruth = groundTruthMultisignal(dataSource,labelDefs,roiData,sceneData)

gTruth = 

  groundTruthMultisignal with properties:

          DataSource: [1x2 vision.labeler.loading.MultiSignalSource]
    LabelDefinitions: [6x7 table]
        ROILabelData: [1x1 vision.labeler.labeldata.ROILabelData]
      SceneLabelData: [1x1 vision.labeler.labeldata.SceneLabelData]

Tips

groundTruthMultisignal objects with video-based data sources rely on the video reading capabilities of your operating system. A groundTruthMultisignal object created using video data sources remains consistent only for the same platform that was used to create it. To create a platform-specific groundTruthMultisignal object, convert the videos into sequences of images.
To create a groundTruthMultisignal object containing ROI label data but no scene label data, specify the SceneLabelData property as an empty array. To create this array, at the MATLAB command prompt, enter this code.
```
sceneData = vision.labeler.labeldata.SceneLabelData.empty
```

Documentation

groundTruthMultisignal

Description

Creation

Syntax

Description

Properties

`DataSource` — Sources of ground truth data
vector of `MultiSignalSource` objects

`LabelDefinitions` — Label definitions
table

`ROILabelData` — ROI label data
`ROILabelData` object

`SceneLabelData` — Scene label data
`SceneLabelData` object

Object Functions

Examples

Create Ground Truth from Multiple Signals

Tips

See Also

Apps

Objects

Topics

Automated Driving Toolbox Documentation

Support

Documentation

groundTruthMultisignal

Description

Creation

Syntax

Description

Properties

DataSource — Sources of ground truth data vector of MultiSignalSource objects

LabelDefinitions — Label definitions table

ROILabelData — ROI label data ROILabelData object

SceneLabelData — Scene label data SceneLabelData object

Object Functions

Examples

Create Ground Truth from Multiple Signals

Tips

See Also

Apps

Objects

Topics

Automated Driving Toolbox Documentation

Support

`DataSource` — Sources of ground truth data
vector of `MultiSignalSource` objects

`LabelDefinitions` — Label definitions
table

`ROILabelData` — ROI label data
`ROILabelData` object

`SceneLabelData` — Scene label data
`SceneLabelData` object