Detect objects and lanes from visual measurements
Automated Driving Toolbox / Driving Scenario and Sensor Modeling
The Vision Detection Generator block generates detections from camera measurements taken by a vision sensor mounted on an ego vehicle.
The block derives detections from simulated actor poses and generates these detections at intervals equal to the sensor update interval. By default, detections are referenced to the coordinate system of the ego vehicle. The block can simulate real detections with added random noise and also generate false positive detections. A statistical model generates the measurement noise, true detections, and false positives. To control the random numbers that the statistical model generates, use the random number generator settings on the Measurements tab of the block.
You can use the Vision Detection Generator to create input to a Multi-Object Tracker block. When building scenarios and sensor models using the Driving Scenario Designer app, the camera sensors exported to Simulink® are output as Vision Detection Generator blocks.
Actors
— Scenario actor posesScenario actor poses in ego vehicle coordinates, specified as a Simulink bus containing a MATLAB structure.
The structure must contain these fields.
Field | Description | Type |
---|---|---|
NumActors | Number of actors | Nonnegative integer |
Time | Current simulation time | Real-valued scalar |
Actors | Actor poses | NumActors -length array of actor pose structures |
Each actor pose structure in Actors
must have these
fields.
Field | Description |
---|---|
ActorID | Scenario-defined actor identifier, specified as a positive integer. |
Position | Position of actor, specified as a real-valued vector of the form [x, y, z]. Units are in meters. |
Velocity | Velocity (v) of actor in the x-, y-, and z-direction, specified as a real-valued vector of the form [vx, vy, vz]. Units are in meters per second. |
Roll | Roll angle of actor, specified as a real-valued scalar. Units are in degrees. |
Pitch | Pitch angle of actor, specified as a real-valued scalar. Units are in degrees. |
Yaw | Yaw angle of actor, specified as a real-valued scalar. Units are in degrees. |
AngularVelocity | Angular velocity (ω) of actor in the x-, y-, and z-direction, specified as a real-valued vector of the form [ωx, ωy, ωz]. Units are in degrees per second. |
To enable this input port, set the Types of detections
generated by sensor parameter to Objects
only
, Lanes with
occlusion
, or Lanes and
objects
.
Lane Boundaries
— Lane boundariesLane boundaries in ego vehicle coordinates, specified as a Simulink bus containing a MATLAB structure.
The structure must contain these fields.
Field | Description | Type |
---|---|---|
NumLaneBoundaries | Number of lane boundaries | Nonnegative integer |
Time | Current simulation time | Real scalar |
LaneBoundaries | Lane boundaries | NumLaneBoundaries -length array of lane boundary structures |
Each lane boundary structure in LaneBoundaries
must
have these fields.
Field | Description |
| Lane boundary coordinates, specified as a real-valued N-by-3 matrix, where N is the number of lane boundary coordinates. Lane boundary coordinates define the position of points on the boundary at specified longitudinal distances away from the ego vehicle, along the center of the road.
This matrix also includes the boundary coordinates at zero distance from the ego vehicle. These coordinates are to the left and right of the ego-vehicle origin, which is located under the center of the rear axle. Units are in meters. |
| Lane boundary curvature at each row of the Coordinates matrix, specified
as a real-valued N-by-1 vector. N is the
number of lane boundary coordinates. Units are in radians per meter. |
| Derivative of lane boundary curvature at each row of the Coordinates
matrix, specified as a real-valued N-by-1 vector.
N is the number of lane boundary coordinates. Units are
in radians per square meter. |
| Initial lane boundary heading angle, specified as a real scalar. The heading angle of the lane boundary is relative to the ego vehicle heading. Units are in degrees. |
| Distance of the lane boundary from the ego vehicle position, specified as a real scalar. An offset to a lane boundary to the left of the ego vehicle is positive. An offset to the right of the ego vehicle is negative. Units are in meters. |
| Type of lane boundary marking, specified as one of these values:
|
| Saturation strength of the lane boundary marking, specified as a real scalar from 0 to
1. A value of |
| Lane boundary width, specified as a positive real scalar. In a double-line lane marker, the same width is used for both lines and for the space between lines. Units are in meters. |
| Length of dash in dashed lines, specified as a positive real scalar. In a double-line lane marker, the same length is used for both lines. |
| Length of space between dashes in dashed lines, specified as a positive real scalar. In a dashed double-line lane marker, the same space is used for both lines. |
To enable this input port, set the Types of detections
generated by sensor parameter to Lanes
only
, Lanes only
,
Lanes with occlusion
, or
Lanes and objects
.
Object Detections
— Object detectionsObject detections, returned as a Simulink bus containing a MATLAB structure. For more details about buses, see Create Nonvirtual Buses (Simulink).
You can pass object detections from these sensors and other sensors to a tracker, such as a Multi-Object Tracker block, and generate tracks.
The detections structure has this form:
Field | Description | Type |
---|---|---|
NumDetections | Number of detections | Integer |
IsValidTime | False when updates are requested at times that are between block invocation intervals | Boolean |
Detections | Object detections | Array of object detection structures of length set by
the Maximum number of reported
detections parameter. Only
NumDetections of these detections
are actual detections. |
The object detection structure contains these properties.
Property | Definition |
---|---|
Time | Measurement time |
Measurement | Object measurements |
MeasurementNoise | Measurement noise covariance matrix |
SensorIndex | Unique ID of the sensor |
ObjectClassID | Object classification |
ObjectAttributes | Additional information passed to tracker |
MeasurementParameters | Parameters used by initialization functions of nonlinear Kalman tracking filters |
The Measurement
field reports the position and
velocity of a measurement in the coordinate system specified by
Coordinate system used to report detections.
This field is a real-valued column vector of the form
[x; y; z;
vx; vy;
vz]. Units are in meters per second.
The MeasurementNoise
field is a 6-by-6 matrix that
reports the measurement noise covariance for each coordinate in the
Measurement
field.
The MeasurementParameters
field is a structure
with these fields.
Parameter | Definition |
---|---|
Frame | Enumerated type indicating the frame used to
report measurements. The Vision Detection
Generator block reports detections in
either ego and sensor Cartesian coordinates, which
are both rectangular coordinate frames. Therefore,
for this block, Frame is always
set to 'rectangular' . |
OriginPosition | 3-D vector offset of the sensor origin from the ego vehicle origin. The vector is derived from the Sensor's (x,y) position (m) and Sensor's height (m) parameters of the block. |
Orientation | Orientation of the vision sensor coordinate system with respect to the ego vehicle coordinate system. The orientation is derived from the Yaw angle of sensor mounted on ego vehicle (deg), Pitch angle of sensor mounted on ego vehicle (deg), and Roll angle of sensor mounted on ego vehicle (deg) parameters of the block. |
HasVelocity | Indicates whether measurements contain velocity. |
The ObjectAttributes
property of each detection
is a structure with these fields.
Field | Definition |
---|---|
TargetIndex | Identifier of the actor,
ActorID , that generated the
detection. For false alarms, this value is
negative. |
To enable this output port, set the Types of detections
generated by sensor parameter to Objects
only
, Lanes with
occlusion
, or Lanes and
objects
.
Lane Detections
— Lane boundary detectionsLane boundary detections, returned as a Simulink bus containing a MATLAB structure. The structure had these fields:
Field | Description | Type |
---|---|---|
Time | Lane detection time | Real scalar |
IsValidTime | False when updates are requested at times that are between block invocation intervals | Boolean |
SensorIndex | Unique identifier of sensor | Positive integer |
NumLaneBoundaries | Number of lane boundary detections | Nonnegative integer |
LaneBoundaries | Lane boundary detections | Array of clothoidLaneBoundary objects |
To enable this output port, set the Types of detections
generated by sensor parameter to Lanes
only
, Lanes with
occlusion
, or Lanes and
objects
.
Unique identifier of sensor
— Unique sensor identifier1
(default) | positive integerUnique sensor identifier, specified as a positive integer. The sensor identifier distinguishes detections that come from different sensors in a multisensor system. If a model contains multiple sensor blocks with the same sensor identifier, the Bird's-Eye Scope displays an error.
Example: 5
Types of detections generated by sensor
— Select the types of detectionsObjects only
(default) | Lanes only
| Lanes with occlusion
| Lanes and objects
Types of detections generated by the sensor, specified as
Objects only
, Lanes
only
, Lanes with occlusion
,
or Lanes and objects
.
When set to Objects only
, no
road information is used to occlude actors.
When set to Lanes only
, no
actor information is used to detect lanes.
When set to Lanes with
occlusion
, actors in the camera field of
view can impair the sensor ability to detect lanes.
When set to Lanes and objects
,
the sensor generates object both object detections and
occluded lane detections.
Required interval between sensor updates (s)
— Required time interval0.1
(default) | positive real scalarRequired time interval between sensor updates, specified as a positive real scalar. The value of this parameter must be an integer multiple of the Actors input port data interval. Updates requested from the sensor between update intervals contain no detections. Units are in seconds.
Required interval between lane detections updates (s)
— Time interval between lane detection updatesRequired time interval between lane detection updates, specified as a positive real scalar. The vision detection generator is called at regular time intervals. The vision detector generates new lane detections at intervals defined by this parameter which must be an integer multiple of the simulation time interval. Updates requested from the sensor between update intervals contain no lane detections. Units are in seconds.
Sensor's (x,y) position (m)
— Location of the vision sensor center[3.4 0]
(default) | real-valued 1-by-2 vectorLocation of the vision sensor center, specified as a real-valued 1-by-2 vector. The Sensor's (x,y) position (m) and Sensor's height (m) parameters define the coordinates of the vision sensor with respect to the ego vehicle coordinate system. The default value corresponds to a forward-facing vision sensor mounted to a sedan dashboard. Units are in meters.
Sensor's height (m)
— Vision sensor height above the ground plane0.2
(default) | positive real scalarVision sensor height above the ground plane, specified as a positive real scalar. The height is defined with respect to the vehicle ground plane. The Sensor's (x,y) position (m) and Sensor's height (m) parameters define the coordinates of the vision sensor with respect to the ego vehicle coordinate system. The default value corresponds to a forward-facing vision sensor mounted a sedan dashboard. Units are in meters.
Example: 0.25
Yaw angle of sensor mounted on ego vehicle (deg)
— Yaw angle of sensor0
(default) | real scalarYaw angle of vision sensor, specified as a real scalar. Yaw angle is the angle between the center line of the ego vehicle and the optical axis of the camera. A positive yaw angle corresponds to a clockwise rotation when looking in the positive direction of the z-axis of the ego vehicle coordinate system. Units are in degrees.
Example: -4.0
Pitch angle of sensor mounted on ego vehicle (deg)
— Pitch angle of sensor0
(default) | real scalarPitch angle of sensor, specified as a real scalar. The pitch angle is the angle between the optical axis of the camera and the x-y plane of the ego vehicle coordinate system. A positive pitch angle corresponds to a clockwise rotation when looking in the positive direction of the y-axis of the ego vehicle coordinate system. Units are in degrees.
Example: 3.0
Roll angle of sensor mounted on ego vehicle (deg)
— Roll angle of sensor0
(default) | real scalarRoll angle of the vision sensor, specified as a real scalar. The roll angle is the angle of rotation of the optical axis of the camera around the x-axis of the ego vehicle coordinate system. A positive roll angle corresponds to a clockwise rotation when looking in the positive direction of the x-axis of the coordinate system. Units are in degrees.
Source of object bus name
— Source of object bus nameAuto
(default) | Property
Source of object bus name, specified as Auto
or
Property
. If you select Auto
,
the block automatically creates a bus name. If you select
Property
, specify the bus name using the
Specify an object bus name parameter.
Example: Property
Source of output lane bus name
— Source of lane bus nameAuto
(default) | Property
Source of output lane bus name, specified as Auto
or Property
. If you choose Auto
,
the block will automatically create a bus name. If you choose
Property
, specify the bus name using the
Specify an object bus name parameter.
Example: Property
Object bus name
— Name of object busName of object bus, specified as a valid bus name.
Example: objectbus
To enable this parameter, set the Source of object bus
name parameter to Property
.
Specify an output lane bus name
— Name of output lane busNamer of output lane bus, specified as a valid bus name.
Example: lanebus
To enable this parameter, set the Source of output lane
bus name parameter to
Property
.
Maximum number of reported detections
— Maximum number of reported detections50
(default) | positive integerMaximum number of detections reported by the sensor, specified as a positive integer. Detections are reported in order of increasing distance from the sensor until the maximum number is reached.
Example: 100
To enable this parameter, set the Types of detections
generated by sensor parameter to Objects
only
or Lanes and
objects
.
Maximum number of reported lanes
— Maximum number of reported lanes30
(default) | positive integerMaximum number of reported lanes, specified as a positive integer.
Example: 100
To enable this parameter, set the Types of detections
generated by sensor parameter to Lanes
only
, Lanes with
occlusion
, or Lanes and
objects
.
Coordinate system used to report detections
— Coordinate system of reported detectionsEgo Cartesian
(default) | Sensor Cartesian
Coordinate system of reported detections, specified as one of these values:
Ego Cartesian
—
Detections are reported in the ego vehicle Cartesian
coordinate system.
Sensor Cartesian
—
Detections are reported in the sensor Cartesian coordinate
system.
Simulate using
— Type of simulation to runInterpreted execution
(default) | Code generation
Interpreted execution
— Simulate the model using the
MATLAB interpreter. This option shortens startup time. In Interpreted
execution
mode, you can debug the source code of the block.
Code generation
— Simulate the model using generated C/C++
code. The first time you run a simulation, Simulink generates C/C++ code for the block. The C code is reused for subsequent
simulations as long as the model does not change. This option requires additional startup
time.
Maximum detection range (m)
— Maximum detection range150
(default) | positive real scalarMaximum detection range, specified as a positive real scalar. The vision sensor cannot detect objects beyond this range. Units are in meters.
Example: 250
Bounding box accuracy (pixels)
— Bounding box accuracy5
(default) | positive real scalarBounding box accuracy, specified as a positive real scalar. This quantity defines the accuracy with which the detector can match a bounding box to a target. Units are in pixels.
Example: 9
Smoothing filter noise intensity (m/s^2)
— Noise intensity used for filtering position and velocity measurements5
(default) | positive real scalarNoise intensity used for filtering position and velocity measurements, specified as a positive real scalar. Noise intensity defines the standard deviation of the process noise of the internal constant-velocity Kalman filter used in a vision sensor. The filter models the process noise using a piecewise-constant white noise acceleration model. Noise intensity is typically of the order of the maximum acceleration magnitude expected for a target. Units are in meters per second squared.
Example: 2
Maximum detectable object speed (m/s)
— Maximum detectable object speed100
(default) | nonnegative real scalarMaximum detectable object speed, specified as a nonnegative real scalar. Units are in meters per second.
Example: 20
Maximum allowed occlusion for detector
— Maximum allowed occlusion for detector0.5
(default) | real scalar in the range [0 1)Maximum allowed occlusion of an object, specified as a real scalar in the range [0 1). Occlusion is the fraction of the total surface area of an object that is not visible to the sensor. A value of 1 indicates that the object is fully occluded. Units are dimensionless.
Example: 0.2
Minimum detectable image size of an object
— Minimum height and width of an object[15,15]
(default) | 1-by-2 vector of positive valuesMinimum height and width of an object that the vision sensor detects
within an image, specified as a [minHeight,minWidth]
vector of positive values. The 2-D projected height of an object must be
greater than or equal to minHeight
. The projected
width of an object must be greater than or equal to
minWidth
. Units are in pixels.
Example: [25 20]
Probability of detecting a target
— Probability of detection0.9
(default) | positive real scalar less than or equal to 1Probability of detecting a target, specified as a positive real scalar less than or equal to 1. This quantity defines the probability that the sensor detects a detectable object. A detectable object is an object that satisfies the minimum detectable size, maximum range, maximum speed, and maximum allowed occlusion constraints.
Example: 0.95
Number of false positives per image
— Number of false detections generated by vision sensor per image0.1
(default) | nonnegative real scalarNumber of false detections generated by the vision sensor per image, specified as a nonnegative real scalar.
Example: 1.0
Minimum lane size in image (pixels)
— Maximum size of lane[20,3]
(default) | 1-by-2 real-valued vectorMinimum size of a projected lane marking in the camera image that can
be detected by the sensor after accounting for curvature, specified as a
1-by-2 real-valued vector, [minHeight minWidth]
. Lane
markings must exceed both of these values to be detected. Units are in
pixels.
To enable this parameter, set the Types of detections
generated by sensor parameter to Lanes
only
, Lanes only
, or
Lanes and objects
.
Accuracy of lane boundary (pixels)
— Accuracy of lane boundary3
(default) | positive real scalarAccuracy of lane boundaries, specified as a positive real scalar. This parameter defines the accuracy with which the lane sensor can place a lane boundary. Units are in pixels.
Example: 2.5
To enable this parameter, set the Types of detections
generated by sensor parameter to Lanes
only
, Lanes only
, or
Lanes and objects
.
Add noise to measurements
— Enable adding noise to vision sensor measurementson
(default) | off
Select this parameter to add noise to vision sensor measurements.
Otherwise, the measurements are noise-free. The
MeasurementNoise
property of each detection is
always computed and is not affected by the value you specify for the
Add noise to measurements parameter.
Select method to specify initial seed
— Method to specify random number generator seedRepeatable
(default) | Specify seed
| Not repeatable
Method to set the random number generator seed, specified as one of the options in the table.
Option | Description |
---|---|
Repeatable | The block generates a random initial seed
for the first simulation and reuses this seed for
all subsequent simulations. Select this parameter
to generate repeatable results from the
statistical sensor model. To change this initial
seed, at the MATLAB command prompt, enter:
|
Specify seed | Specify your own random initial seed for reproducible results by using the Specify seed parameter. |
Not repeatable | The block generates a new random initial seed after each simulation run. Select this parameter to generate nonrepeatable results from the statistical sensor model. |
Initial seed
— Random number generator seed0
(default) | nonnegative integer less than 232Random number generator seed, specified as a nonnegative integer less than 232.
Example: 2001
To enable this parameter, set the Random Number
Generator Settings parameter to Specify
seed
.
Select method to specify actor profiles
— Method to specify actor profilesParameters
(default) | MATLAB expression
Method to specify actor profiles, specified as
Parameters
or MATLAB
expression
. When you select
Parameters
, set the actor profiles using
the parameters in the Actor Profiles tab. When you
select MATLAB expression
, set the actor
profiles using the MATLAB expression for actor
profiles parameter.
MATLAB expression for actor profiles
— MATLAB expression for actor profilesstruct('ClassID',0,'Length',4.7,'Width',1.8,'Height',1.4,'OriginOffset',[-1.35,0,0])
(default) | MATLAB structure | MATLAB structure array | valid MATLAB expressionMATLAB expression for actor profiles, specified as a MATLAB structure, a MATLAB structure array, or a valid MATLAB expression that produces such a structure or structure array.
If your Scenario Reader block reads data from a drivingScenario
object, to obtain the actor profiles directly from this
object, set this expression to call the actorProfiles
function on the object. For example:
actorProfiles(scenario)
.
Example: struct('ClassID',5,'Length',5.0,'Width',2,'Height',2,'OriginOffset',[-1.55,0,0])
To enable this parameter, set the Select method to specify actor profiles parameter to MATLAB expression
.
Unique identifier for actors
— Scenario-defined actor identifier[]
(default) | positive integer | length-L vector of unique positive
integersScenario-defined actor identifier, specified as a positive integer or
length-L vector of unique positive integers.
L must equal the number of actors input into the
Actors input port. The vector elements must
match ActorID
values of the actors. You can specify
Unique identifier for actors as
[]
. In this case, the same actor profile
parameters apply to all actors.
Example: [1,2]
To enable this parameter, set the Select method to
specify actor profiles parameter to
Parameters
.
User-defined integer to classify actors
— User-defined classification identifier0
(default) | integer | length-L vector of integersUser-defined classification identifier, specified as an integer or
length-L vector of integers. When
Unique identifier for actors is a vector, this
parameter is a vector of the same length with elements in one-to-one
correspondence to the actors in Unique identifier for
actors. When Unique identifier for
actors is empty, []
, you must specify
this parameter as a single integer whose value applies to all
actors.
Example: 2
To enable this parameter, set the Select method to
specify actor profiles parameter to
Parameters
.
Length of actors cuboids (m)
— Length of cuboid4.7
(default) | positive real scalar | length-L vector of positive valuesLength of cuboid, specified as a positive real scalar or
length-L vector of positive values. When
Unique identifier for actors is a vector, this
parameter is a vector of the same length with elements in one-to-one
correspondence to the actors in Unique identifier for
actors. When Unique identifier for
actors is empty, []
, you must specify
this parameter as a positive real scalar whose value applies to all
actors. Units are in meters.
Example: 6.3
To enable this parameter, set the Select method to
specify actor profiles parameter to
Parameters
.
Width of actors cuboids (m)
— Width of cuboid4.7
(default) | positive real scalar | length-L vector of positive valuesWidth of cuboid, specified as a positive real scalar or
length-L vector of positive values. When
Unique identifier for actors is a vector, this
parameter is a vector of the same length with elements in one-to-one
correspondence to the actors in Unique identifier for
actors. When Unique identifier for
actors is empty, []
, you must specify
this parameter as a positive real scalar whose value applies to all
actors. Units are in meters.
Example: 4.7
To enable this parameter, set the Select method to
specify actor profiles parameter to
Parameters
.
Height of actors cuboids (m)
— Height of cuboid4.7
(default) | positive real scalar | length-L vector of positive valuesHeight of cuboid, specified as a positive real scalar or
length-L vector of positive values. When
Unique identifier for actors is a vector, this
parameter is a vector of the same length with elements in one-to-one
correspondence to the actors in Unique identifier for
actors. When Unique identifier for
actors is empty, []
, you must specify
this parameter as a positive real scalar whose value applies to all
actors. Units are in meters.
Example: 2.0
To enable this parameter, set the Select method to
specify actor profiles parameter to
Parameters
.
Rotational center of actors from bottom center (m)
— Rotational center of the actorRotational center of the actor, specified as a
length-L cell array of real-valued 1-by-3
vectors. Each vector represents the offset of the rotational center of
the actor from the bottom-center of the actor. For vehicles, the offset
corresponds to the point on the ground beneath the center of the rear
axle. When Unique identifier for actors is a
vector, this parameter is a cell array of vectors with cells in
one-to-one correspondence to the actors in Unique identifier
for actors. When Unique identifier for
actors is empty, []
, you must specify
this parameter as a cell array of one element containing the offset
vector whose values apply to all actors. Units are in meters.
Example: [ -1.35, .2, .3 ]
To enable this parameter, set the Select method to
specify actor profiles parameter to
Parameters
.
Focal length (pixels)
— Camera focal length[800,800]
(default) | two-element real-valued vectorCamera focal length, in pixels, specified as a two-element real-valued
vector. See also the FocalLength
(Computer Vision Toolbox) property of
cameraIntrinsics
.
Example: [480,320]
Optical center of the camera (pixels)
— Optical center of camera[320,240]
(default) | two-element real-valued vectorOptical center of the camera, in pixels, specified as a two-element
real-valued vector. See also the PrincipalPoint
(Computer Vision Toolbox) property of
cameraIntrinsics
.
Example: [480,320]
Image size produced by the camera (pixels)
— Image size produced by camera[480,640]
(default) | two-element vector of positive integersImage size produced by the camera, in pixels, specified as a
two-element vector of positive integers. See also the ImageSize
(Computer Vision Toolbox) property of
cameraIntrinsics
.
Example: [240,320]
Radial distortion coefficients
— Radial distortion coefficients[0,0]
(default) | two-element real-valued vector | three-element real-valued vectorRadial distortion coefficients, specified as a two-element or
three-element real-valued vector. For details on setting these
coefficients, see the RadialDistortion
(Computer Vision Toolbox) property of
cameraIntrinsics
.
Example: [1,1]
Tangential distortion coefficients
— Tangential distortion coefficients[0,0]
(default) | two-element real-valued vectorTangential distortion coefficients, specified as a two-element
real-valued vector. For details on setting these coefficients, see the
TangentialDistortion
(Computer Vision Toolbox) property of
cameraIntrinsics
.
Example: [1,1]
Skew of the camera axes
— Skew angle of camera axes0
(default) | real scalarSkew angle of the camera axes, specified as a real scalar. See also
the Skew
(Computer Vision Toolbox)
property of cameraIntrinsics
.
Example: 0.1
visionDetectionGenerator
| cameraIntrinsics
(Computer Vision Toolbox)