Markov Decision Process

A Markov Decision Process (MDP) is a Discrete Time Stochastic Control Process characterized by a set of states, actions, and a state transition function (usually a Transition Probability Matrix for discrete state- and action-spaces). An MDP also posesses the Markov Property . This means that if the current state of the MDP at time

t

is known, transitions to a new state at time

t+1

are independent of all previous states.

MDPs are useful for studying a wide range of Optimization Problem s solved via Dynamic Programming and Reinforcement Learning . They are used in a variety of areas, including robotics, automated control, economics and in fabrication/fulfillment processes.

DEFINITION

A Markov Decision Process is a tuple

(S,A,P_\cdot(\cdot,\cdot),R(\cdot))

, where

$S$ is the State space,

$A$ is the action space,

Information About ™

Markov Decision Process

Information About ^{^™}