MultiAgentDecisionProcess
Release 0.2.1
|
MDPValueIteration implements value iteration for MDPs. More...
#include <MDPValueIteration.h>
Public Member Functions | |
double | GetQ (Index time_step, Index sI, Index jaI) const |
Get Q-value for finite-horizon case. | |
double | GetQ (Index sI, Index jaI) const |
Get Q-value for infinite-horizon case. | |
QTable | GetQTable (Index time_step) const |
QTables | GetQTables () const |
MDPValueIteration () | |
(default) Constructor | |
MDPValueIteration (const PlanningUnitDecPOMDPDiscrete &pu) | |
void | Plan () |
void | SetQTable (const QTable &Q, Index time_step) |
void | SetQTables (const QTables &Qs) |
~MDPValueIteration () | |
Destructor. | |
![]() | |
Index | GetMaximizingAction (Index time_step, Index sI) |
const PlanningUnitDecPOMDPDiscrete * | GetPU () const |
Returns a ref to the PlanningUnit. | |
virtual double | GetQ (Index time_step, const JointBeliefInterface &jb, Index jaI) const |
virtual double | GetQ (const JointBeliefInterface &jb, Index jaI) const |
QTable | LoadQTable (std::string filename) |
QTables | LoadQTables (std::string filename, int nrTables) |
MDPSolver () | |
(default) Constructor | |
MDPSolver (const PlanningUnitDecPOMDPDiscrete &pu) | |
void | Print () const |
void | SetPU (const PlanningUnitDecPOMDPDiscrete &pu) |
virtual | ~MDPSolver () |
Destructor. | |
![]() | |
void | AddTimedEvent (const std::string &id, clock_t duration) |
Adds event of certain duration, e.g., an external program call. | |
std::vector< double > | GetTimedEventDurations (const std::string &id) |
Returns all stored durations (in s) for a particular event. | |
void | LoadTimers (const std::string &filename) |
Load timing info from file filename. | |
void | PrintTimers () const |
Print stored timing info. | |
void | PrintTimersSummary () const |
Sums data and prints out a summary. | |
void | SaveTimers (const std::string &filename) const |
Save collected timing info to file filename. | |
void | SaveTimers (std::ofstream &of) const |
Save collected timing info to ofstream of. | |
void | StartTimer (const std::string &id) const |
Start to time an event identified by id. | |
void | StopTimer (const std::string &id) const |
Stop to time an event identified by id. | |
TimedAlgorithm () | |
(default) Constructor | |
virtual | ~TimedAlgorithm () |
Destructor. |
Private Member Functions | |
void | Initialize () |
template<class M > | |
void | Plan (std::vector< const M * > T) |
Vector<const M*> T is the vector of matrices specifying the transition model (one matrix for each joint action). | |
void | PlanSlow () |
Uses the GetTransitionProbability() interface, which is slow. |
Private Attributes | |
bool | _m_finiteHorizon |
Are we solving a finite-horizon problem? | |
bool | _m_initialized |
Is the MDPValueIteration object initialized?. | |
QTables | _m_QValues |
_m_QValues represents the non-stationary MDP Q function. |
Additional Inherited Members | |
![]() | |
static QTable | LoadQTable (std::string filename, unsigned int nrS, unsigned int nrA) |
static QTables | LoadQTables (std::string filename, unsigned int nrS, unsigned int nrA, unsigned int nrTables) |
static void | SaveQTable (const QTable &Q, std::string filename) |
static void | SaveQTables (const QTables &Q, std::string filename) |
MDPValueIteration implements value iteration for MDPs.
Definition at line 47 of file MDPValueIteration.h.
|
inline |
(default) Constructor
Definition at line 185 of file MDPValueIteration.h.
MDPValueIteration::MDPValueIteration | ( | const PlanningUnitDecPOMDPDiscrete & | pu | ) |
Definition at line 35 of file MDPValueIteration.cpp.
References _m_initialized.
MDPValueIteration::~MDPValueIteration | ( | ) |
Destructor.
Definition at line 42 of file MDPValueIteration.cpp.
|
inlinevirtual |
Get Q-value for finite-horizon case.
Implements MDPSolver.
Definition at line 226 of file MDPValueIteration.h.
References _m_QValues.
|
inlinevirtual |
Get Q-value for infinite-horizon case.
Implements MDPSolver.
Definition at line 230 of file MDPValueIteration.h.
References _m_QValues.
|
virtual |
|
virtual |
|
private |
Definition at line 46 of file MDPValueIteration.cpp.
References _m_finiteHorizon, _m_initialized, _m_QValues, PlanningUnit::GetHorizon(), PlanningUnitMADPDiscrete::GetNrJointActions(), PlanningUnitMADPDiscrete::GetNrStates(), MDPSolver::GetPU(), Globals::MAXHORIZON, TimedAlgorithm::StartTimer(), and TimedAlgorithm::StopTimer().
Referenced by Plan(), and PlanSlow().
|
inlineprivate |
Vector<const M*> T is the vector of matrices specifying the transition model (one matrix for each joint action).
Definition at line 68 of file MDPValueIteration.h.
References _m_finiteHorizon, _m_QValues, PlanningUnitDecPOMDPDiscrete::GetDiscount(), PlanningUnit::GetHorizon(), PlanningUnitMADPDiscrete::GetNrJointActions(), PlanningUnitMADPDiscrete::GetNrStates(), MDPSolver::GetPU(), PlanningUnitDecPOMDPDiscrete::GetReward(), TimedAlgorithm::PrintTimersSummary(), TimedAlgorithm::StartTimer(), and TimedAlgorithm::StopTimer().
|
inlinevirtual |
Implements MDPSolver.
Definition at line 191 of file MDPValueIteration.h.
References _m_initialized, TransitionModelMapping::GetMatrixPtr(), TransitionModelMappingSparse::GetMatrixPtr(), PlanningUnitMADPDiscrete::GetNrJointActions(), MDPSolver::GetPU(), PlanningUnitMADPDiscrete::GetTransitionModelDiscretePtr(), Initialize(), PlanSlow(), TimedAlgorithm::StartTimer(), and TimedAlgorithm::StopTimer().
|
private |
Uses the GetTransitionProbability() interface, which is slow.
Duplication of code from the templatized version, but well...
Definition at line 100 of file MDPValueIteration.cpp.
References _m_finiteHorizon, _m_initialized, _m_QValues, PlanningUnitDecPOMDPDiscrete::GetDiscount(), PlanningUnit::GetHorizon(), PlanningUnitMADPDiscrete::GetNrJointActions(), PlanningUnitMADPDiscrete::GetNrStates(), MDPSolver::GetPU(), PlanningUnitDecPOMDPDiscrete::GetReward(), PlanningUnitMADPDiscrete::GetTransitionProbability(), Initialize(), TimedAlgorithm::PrintTimersSummary(), TimedAlgorithm::StartTimer(), and TimedAlgorithm::StopTimer().
Referenced by Plan().
|
virtual |
|
virtual |
|
private |
Are we solving a finite-horizon problem?
Definition at line 61 of file MDPValueIteration.h.
Referenced by Initialize(), Plan(), and PlanSlow().
|
private |
Is the MDPValueIteration object initialized?.
Definition at line 58 of file MDPValueIteration.h.
Referenced by Initialize(), MDPValueIteration(), Plan(), and PlanSlow().
|
private |
_m_QValues represents the non-stationary MDP Q function.
I.e. _m_QValues[t][sI][jaI] gives the expected reward at time-step t (time-to-go = horizon - t).
Definition at line 55 of file MDPValueIteration.h.
Referenced by GetQ(), GetQTable(), GetQTables(), Initialize(), Plan(), PlanSlow(), SetQTable(), and SetQTables().