%% Create Value Function Critic from Deep Neural Network
% Create an observation specification object (or alternatively use getObservationInfo to extract the specification object from an environment). For this example, define the observation space as a continuous four-dimensional space, so that a single observation is a column vector containing 4 doubles.

obsInfo = rlNumericSpec([4 1]);

%
% Create a deep neural network to approximate the value function within the critic. The input of the network (here called myobs) must accept a four-element vector (the observation vector defined by obsInfo), and the output must be a scalar (the value, representing the expected cumulative long-term reward when the agent starts from the given observation).

net = [featureInputLayer(4, 'Normalization','none','Name','myobs') 
       fullyConnectedLayer(1,'Name','value')];

%
% Create the critic using the network, observation specification object, and name of the network input layer.

critic = rlValueRepresentation(net,obsInfo,'Observation',{'myobs'})

%
% To check your critic, use the getValue function to return the value of a random observation, using the current network weights.

v = getValue(critic,{rand(4,1)})

% You can now use the critic (along with an actor) to create an agent relying on a value function critic (such as rlACAgent or rlPGAgent).
% Copyright 2019 The MathWorks, Inc.