%% Create Stochastic Actor from Custom Basis Function
% Create an observation specification object (or alternatively use getObservationInfo to extract the specification object from an environment). For this example, define the observation space as a continuous four-dimensional space, so that a single observation is a column vector containing 2 doubles.

obsInfo = rlNumericSpec([2 1]);

%
% The stochastic actor based on a custom basis function does not support continuous action spaces. Therefore, create a discrete action space specification object (or alternatively use getActionInfo to extract the specification object from an environment with a discrete action space). For this example, define the action space as a finite set consisting of 3 possible values (named 7, 5, and 3 in this case).

actInfo = rlFiniteSetSpec([7 5 3]);

%
% Create a custom basis function. Each element is a function of the observations defined by obsInfo.

myBasisFcn = @(myobs) [myobs(2)^2; myobs(1); exp(myobs(2)); abs(myobs(1))]

%
% The output of the actor is the action, among the ones defined in actInfo, corresponding to the element of softmax(W'*myBasisFcn(myobs)) which has the highest value. W is a weight matrix, containing the learnable parameters, which must have as many rows as the length of the basis function output, and as many columns as the number of possible actions.
% Define an initial parameter matrix.

W0 = rand(4,3);

%
% Create the actor. The first argument is a two-element cell containing both the handle to the custom function and the initial parameter matrix. The second and third arguments are, respectively, the observation and action specification objects.

actor = rlStochasticActorRepresentation({myBasisFcn,W0},obsInfo,actInfo)

%
% To check your actor use the getAction function to return one of the three possible actions, depending on a given random observation and on the current parameter matrix.

v = getAction(actor,{rand(2,1)})

% You can now use the actor (along with an critic) to create a suitable discrete action space agent.
% Copyright 2019 The MathWorks, Inc.