rlSimulinkEnv

Create reinforcement learning environment using dynamic model implemented in Simulink

collapse all in page

Syntax

env = rlSimulinkEnv(mdl,agentBlocks)
env = rlSimulinkEnv(mdl,agentBlocks,obsInfo,actInfo)
env = rlSimulinkEnv(___,'UseFastRestart',fastRestartToggle)

Description

The rlSimulinkEnv function creates a reinforcement learning environment object from a Simulink® model. The environment object acts an interface so that when you call sim or train, these functions in turn call the Simulink model to generate experiences for the agents.

example

env = rlSimulinkEnv(mdl,agentBlocks) creates the reinforcement learning environment object env for the Simulink model mdl. agentBlocks contains the paths to one or more reinforcement learning agent blocks in mdl. If you use this syntax, each agent block must reference an agent object already in the MATLAB® workspace.

example

env = rlSimulinkEnv(mdl,agentBlocks,obsInfo,actInfo) creates the reinforcement learning environment object env for the model mdl. The two cell arrays obsInfo and actInfo must contain the observation and action specifications for each agent block in mdl, in the same order as they appear in agentBlocks.

env = rlSimulinkEnv(___,'UseFastRestart',fastRestartToggle) creates a reinforcement learning environment object env and additionally enables fast restart. Use this syntax after any of the input arguments in the previous syntaxes.

Examples

collapse all

Open Live Script

Create a Simulink environment using the trained agent and corresponding Simulink® environment from the Create Simulink Environment and Train Agent example.

Load the agent in the MATLAB® workspace.

load rlWaterTankDDPGAgent

Create an environment for the rlwatertank model, which has an agent block called "RL Agent". Since the agent used by the block is already in the workspace, you do not need to pass the observation and action specifications to create the environment.

env = rlSimulinkEnv('rlwatertank','rlwatertank/RL Agent')
env = 
SimulinkEnvWithAgent with properties:

           Model : rlwatertank
      AgentBlock : rlwatertank/RL Agent
        ResetFcn : []
  UseFastRestart : on

Validate the environment by performing a short simulation for two sample times.

validateEnvironment(env)

You can now train and simulate the agent within the environment using train and sim, respectively.

This example uses:

Open Live Script

For this example, consider the rlSimplePendulumModel Simulink model. The model is a simple frictionless pendulum that is initially hanging in a downward position.

Open the model.

mdl = 'rlSimplePendulumModel';
open_system(mdl)

Create rlNumericSpec and rlFiniteSetSpec objects for the observation and action information, respectively.

obsInfo = rlNumericSpec([3 1]) % vector of 3 observations: sin(theta), cos(theta), d(theta)/dt
obsInfo = 
  rlNumericSpec with properties:

     LowerLimit: -Inf
     UpperLimit: Inf
           Name: [0x0 string]
    Description: [0x0 string]
      Dimension: [3 1]
       DataType: "double"


actInfo = rlFiniteSetSpec([-2 0 2]) % 3 possible values for torque: -2 Nm, 0 Nm and 2 Nm
actInfo = 
  rlFiniteSetSpec with properties:

       Elements: [3x1 double]
           Name: [0x0 string]
    Description: [0x0 string]
      Dimension: [1 1]
       DataType: "double"

You can use dot notation to assign property values for the rlNumericSpec and rlFiniteSetSpec objects.

obsInfo.Name = 'observations';
actInfo.Name = 'torque';

Assign the agent block path information, and create the reinforcement learning environment for the Simulink model using information extracted in the previous steps.

agentBlk = [mdl '/RL Agent'];
env = rlSimulinkEnv(mdl,agentBlk,obsInfo,actInfo)
env = 
SimulinkEnvWithAgent with properties:

           Model : rlSimplePendulumModel
      AgentBlock : rlSimplePendulumModel/RL Agent
        ResetFcn : []
  UseFastRestart : on

You can also include a reset function using dot notation. For this example, randomly initialize theta0 in the model workspace.

env.ResetFcn = @(in) setVariable(in,'theta0',randn,'Workspace',mdl)
env = 
SimulinkEnvWithAgent with properties:

           Model : rlSimplePendulumModel
      AgentBlock : rlSimplePendulumModel/RL Agent
        ResetFcn : @(in)setVariable(in,'theta0',randn,'Workspace',mdl)
  UseFastRestart : on
Open Live Script

Create an environment for the Simulink model in the example Train Multiple Agents to Perform Collaborative Task, when the agents trained in that example are already in the workspace.

Load the agents in the MATLAB® workspace.

load rlCollaborativeTaskAgents

Create an environment for the rlCollaborativeTask model, which has two agent blocks. Since the agents used by the two blocks (agentA and agentB) are already in the workspace, you do not need to pass their observation and action specifications to create the environment.

env = rlSimulinkEnv('rlCollaborativeTask',["rlCollaborativeTask/Agent A","rlCollaborativeTask/Agent B"])
env = 
SimulinkEnvWithAgent with properties:

           Model : rlCollaborativeTask
      AgentBlock : [
                     rlCollaborativeTask/Agent A
                     rlCollaborativeTask/Agent B
                   ]
        ResetFcn : []
  UseFastRestart : on

You can now simulate or train the agents within the environment using sim or train, respectively.

Input Arguments

collapse all

Simulink model name, specified as a string or character vector.

Agent block paths, specified as a string, character vector, or string array.

If mdl contains a single agent block, specify agentBlocks as a string or character vector containing the block path.

If mdl contains multiple agent blocks, specify agentBlocks as a string array, where each element contains the path of one agent block.

mdl can contain RL Agent blocks whose path is not included in agentBlocks. Such agent blocks behave as part of the environment, selecting actions based on their current policies. When you call sim or train, the experiences of these agents are not returned and their policies are not updated.

Multi-agent simulation is not supported for MATLAB environments.

The agent blocks can be inside of a model reference. For more information on configuring an agent block for reinforcement learning, see RL Agent.

Observation information, specified as an array of specification objects or a cell array.

If mdl contains a single agent block, specify obsInfo as an array of rlNumericSpec objects, an array of rlFiniteSetSpec objects or a mix of such objects.

If mdl contains multiple agent blocks, specify obsInfo as a cell array, where each cell contains an array of specification objects for the corresponding blocks in agentBlocks.

For more information, see getObservationInfo.

Action information, specified as an array of specification objects or a cell array.

If mdl contains a single agent block, specify actInfo as an array of rlNumericSpec objects, an array of rlFiniteSetSpec objects, or a mix of such objects.

If mdl contains multiple agent blocks, specify actInfo as cell array, where each cell contains an array of specification objects for the corresponding blocks in agentBlocks.

For more information, see getActionInfo.

Option to toggle fast restart, specified as either 'on' or 'off'. Fast restart allows you to perform iterative simulations without compiling a model or terminating the simulation each time.

For more information on fast restart, see How Fast Restart Improves Iterative Simulations (Simulink).

Output Arguments

collapse all

Reinforcement learning environment, returned as a SimulinkEnvWithAgent object.

For more information on reinforcement learning environments, see Create Simulink Environments for Reinforcement Learning.

See Also

| | | | | |

Topics

Introduced in R2019a

Reinforcement Learning Toolbox Documentation

Support