MultiAgentDecisionProcess  Release 0.2.1
MDPValueIteration Class Reference

MDPValueIteration implements value iteration for MDPs. More...

#include <MDPValueIteration.h>

+ Inheritance diagram for MDPValueIteration:
+ Collaboration diagram for MDPValueIteration:

Public Member Functions

double GetQ (Index time_step, Index sI, Index jaI) const
 Get Q-value for finite-horizon case.
double GetQ (Index sI, Index jaI) const
 Get Q-value for infinite-horizon case.
QTable GetQTable (Index time_step) const
QTables GetQTables () const
 MDPValueIteration ()
 (default) Constructor
 MDPValueIteration (const PlanningUnitDecPOMDPDiscrete &pu)
void Plan ()
void SetQTable (const QTable &Q, Index time_step)
void SetQTables (const QTables &Qs)
 ~MDPValueIteration ()
 Destructor.
- Public Member Functions inherited from MDPSolver
Index GetMaximizingAction (Index time_step, Index sI)
const
PlanningUnitDecPOMDPDiscrete
GetPU () const
 Returns a ref to the PlanningUnit.
virtual double GetQ (Index time_step, const JointBeliefInterface &jb, Index jaI) const
virtual double GetQ (const JointBeliefInterface &jb, Index jaI) const
QTable LoadQTable (std::string filename)
QTables LoadQTables (std::string filename, int nrTables)
 MDPSolver ()
 (default) Constructor
 MDPSolver (const PlanningUnitDecPOMDPDiscrete &pu)
void Print () const
void SetPU (const PlanningUnitDecPOMDPDiscrete &pu)
virtual ~MDPSolver ()
 Destructor.
- Public Member Functions inherited from TimedAlgorithm
void AddTimedEvent (const std::string &id, clock_t duration)
 Adds event of certain duration, e.g., an external program call.
std::vector< double > GetTimedEventDurations (const std::string &id)
 Returns all stored durations (in s) for a particular event.
void LoadTimers (const std::string &filename)
 Load timing info from file filename.
void PrintTimers () const
 Print stored timing info.
void PrintTimersSummary () const
 Sums data and prints out a summary.
void SaveTimers (const std::string &filename) const
 Save collected timing info to file filename.
void SaveTimers (std::ofstream &of) const
 Save collected timing info to ofstream of.
void StartTimer (const std::string &id) const
 Start to time an event identified by id.
void StopTimer (const std::string &id) const
 Stop to time an event identified by id.
 TimedAlgorithm ()
 (default) Constructor
virtual ~TimedAlgorithm ()
 Destructor.

Private Member Functions

void Initialize ()
template<class M >
void Plan (std::vector< const M * > T)
 Vector<const M*> T is the vector of matrices specifying the transition model (one matrix for each joint action).
void PlanSlow ()
 Uses the GetTransitionProbability() interface, which is slow.

Private Attributes

bool _m_finiteHorizon
 Are we solving a finite-horizon problem?
bool _m_initialized
 Is the MDPValueIteration object initialized?.
QTables _m_QValues
 _m_QValues represents the non-stationary MDP Q function.

Additional Inherited Members

- Static Public Member Functions inherited from MDPSolver
static QTable LoadQTable (std::string filename, unsigned int nrS, unsigned int nrA)
static QTables LoadQTables (std::string filename, unsigned int nrS, unsigned int nrA, unsigned int nrTables)
static void SaveQTable (const QTable &Q, std::string filename)
static void SaveQTables (const QTables &Q, std::string filename)

Detailed Description

MDPValueIteration implements value iteration for MDPs.

Definition at line 47 of file MDPValueIteration.h.

Constructor & Destructor Documentation

MDPValueIteration::MDPValueIteration ( )
inline

(default) Constructor

Definition at line 185 of file MDPValueIteration.h.

MDPValueIteration::MDPValueIteration ( const PlanningUnitDecPOMDPDiscrete pu)

Definition at line 35 of file MDPValueIteration.cpp.

References _m_initialized.

MDPValueIteration::~MDPValueIteration ( )

Destructor.

Definition at line 42 of file MDPValueIteration.cpp.

Member Function Documentation

double MDPValueIteration::GetQ ( Index  time_step,
Index  sI,
Index  jaI 
) const
inlinevirtual

Get Q-value for finite-horizon case.

Implements MDPSolver.

Definition at line 226 of file MDPValueIteration.h.

References _m_QValues.

double MDPValueIteration::GetQ ( Index  sI,
Index  jaI 
) const
inlinevirtual

Get Q-value for infinite-horizon case.

Implements MDPSolver.

Definition at line 230 of file MDPValueIteration.h.

References _m_QValues.

QTable MDPValueIteration::GetQTable ( Index  time_step) const
virtual

Implements MDPSolver.

Definition at line 84 of file MDPValueIteration.cpp.

References _m_QValues.

QTables MDPValueIteration::GetQTables ( ) const
virtual

Implements MDPSolver.

Definition at line 79 of file MDPValueIteration.cpp.

References _m_QValues.

template<class M >
void MDPValueIteration::Plan ( std::vector< const M * >  T)
inlineprivate
void MDPValueIteration::SetQTable ( const QTable Q,
Index  time_step 
)
virtual

Implements MDPSolver.

Definition at line 94 of file MDPValueIteration.cpp.

References _m_QValues.

void MDPValueIteration::SetQTables ( const QTables Qs)
virtual

Implements MDPSolver.

Definition at line 89 of file MDPValueIteration.cpp.

References _m_QValues.

Member Data Documentation

bool MDPValueIteration::_m_finiteHorizon
private

Are we solving a finite-horizon problem?

Definition at line 61 of file MDPValueIteration.h.

Referenced by Initialize(), Plan(), and PlanSlow().

bool MDPValueIteration::_m_initialized
private

Is the MDPValueIteration object initialized?.

Definition at line 58 of file MDPValueIteration.h.

Referenced by Initialize(), MDPValueIteration(), Plan(), and PlanSlow().

QTables MDPValueIteration::_m_QValues
private

_m_QValues represents the non-stationary MDP Q function.

I.e. _m_QValues[t][sI][jaI] gives the expected reward at time-step t (time-to-go = horizon - t).

Definition at line 55 of file MDPValueIteration.h.

Referenced by GetQ(), GetQTable(), GetQTables(), Initialize(), Plan(), PlanSlow(), SetQTable(), and SetQTables().


The documentation for this class was generated from the following files: