MultiAgentDecisionProcess
Release 0.2.1
|
AlphaVectorPlanning provides base functionality for alpha-vector based POMDP or BG techniques. More...
#include <AlphaVectorPlanning.h>
Public Member Functions | |
AlphaVectorPlanning (const PlanningUnitDecPOMDPDiscrete &pu) | |
(default) Constructor | |
GaoVectorSet | BackProject (const ValueFunctionPOMDPDiscrete &v) const |
Back projects a value function. | |
GaoVectorSet | BackProject (const VectorSet &v) const |
Back projects a value function, represented as a VectorSet. | |
GaoVectorSet | BackProjectMonahan (const QFunctionsDiscrete &Qs) const |
GaoVectorSet | BackProjectMonahan (const ValueFunctionPOMDPDiscrete &V) const |
VectorSet | CrossSum (const VectorSet &A, const VectorSet &B) const |
Compute the cross-sum of two vector sets. | |
void | ExportPOMDPFile (const std::string &filename) const |
Exports the POMDP to file named filename. | |
ValueFunctionPOMDPDiscrete | GetImmediateRewardValueFunction () const |
Returns the value function induced by the reward model. | |
const PlanningUnitDecPOMDPDiscrete * | GetPU () const |
Returns a ref to the PlanningUnit. | |
void | Initialize () |
void | MonahanCrossSum (const GaoVectorSet &G, ValueFunctionPOMDPDiscrete &V, Index a, bool doIncPrune) const |
ValueFunctionPOMDPDiscrete | Prune (const ValueFunctionPOMDPDiscrete &V) const |
Prune a valuefunction. | |
VectorSet | Prune (const VectorSet &V) const |
Prune a VectorSet. | |
BeliefSet | SampleBeliefs (ArgumentHandlers::Arguments args) const |
Sample a belief set according to the arguments. | |
QFunctionsDiscrete | ValueFunctionToQ (const ValueFunctionPOMDPDiscrete &V) const |
virtual | ~AlphaVectorPlanning () |
Destructor. | |
![]() | |
void | AddTimedEvent (const std::string &id, clock_t duration) |
Adds event of certain duration, e.g., an external program call. | |
std::vector< double > | GetTimedEventDurations (const std::string &id) |
Returns all stored durations (in s) for a particular event. | |
void | LoadTimers (const std::string &filename) |
Load timing info from file filename. | |
void | PrintTimers () const |
Print stored timing info. | |
void | PrintTimersSummary () const |
Sums data and prints out a summary. | |
void | SaveTimers (const std::string &filename) const |
Save collected timing info to file filename. | |
void | SaveTimers (std::ofstream &of) const |
Save collected timing info to ofstream of. | |
void | StartTimer (const std::string &id) const |
Start to time an event identified by id. | |
void | StopTimer (const std::string &id) const |
Stop to time an event identified by id. | |
TimedAlgorithm () | |
(default) Constructor | |
virtual | ~TimedAlgorithm () |
Destructor. |
Static Public Member Functions | |
static void | ExportBeliefSet (const BeliefSet &B, const std::string &filename) |
Exports a belief set to a file. | |
static void | ExportPOMDPFile (const std::string &filename, const PlanningUnitDecPOMDPDiscrete *pu) |
Exports the POMDP represented by pu to file named filename. | |
static void | ExportValueFunction (const std::string &filename, const ValueFunctionPOMDPDiscrete &V) |
Exports a value function V to file named filename. | |
static void | ExportValueFunction (const std::string &filename, const QFunctionsDiscrete &Q) |
Exports a Q functions Q to file named filename. | |
static void | ExportValueFunction (const std::string &filename, const QFunctionsDiscreteNonStationary &Q) |
Exports a Q functions Q to file named filename. | |
static std::vector< int > | GetDuplicateIndices (const VectorSet &V) |
Returns a vector<int> which for each vector k in V specifies whether it is unique, in which case vector<int>[k] is set to -1, or whether it is a duplicate of another vector l in V, in which case vector<int>[k] is set to the index of l, where 0 <= l < V.size()). | |
static ValueFunctionPOMDPDiscrete | GetImmediateRewardValueFunction (const PlanningUnitDecPOMDPDiscrete *pu) |
Returns the value function induced by the reward model of pu. | |
static ValueFunctionPOMDPDiscrete | ImportValueFunction (const std::string &filename) |
Imports a value function from a file named filename. | |
static QFunctionsDiscreteNonStationary | ImportValueFunction (const std::string &filename, size_t nr, size_t nrA, size_t nrS) |
static ValueFunctionPOMDPDiscrete | QFunctionsToValueFunction (const QFunctionsDiscrete &Q) |
static QFunctionsDiscrete | ValueFunctionToQ (const ValueFunctionPOMDPDiscrete &V, size_t nrA, size_t nrS) |
static bool | VectorIsInValueFunction (const AlphaVector &alpha, const ValueFunctionPOMDPDiscrete &V) |
Private Types | |
typedef boost::numeric::ublas::compressed_vector < double > | SparseVector |
Private Member Functions | |
GaoVectorSet | BackProjectFull (const VectorSet &v) const |
Implements equation (3.11) of PhD thesis Matthijs. | |
GaoVectorSet | BackProjectSparse (const VectorSet &v) const |
Implements equation (3.11) of PhD thesis Matthijs. | |
void | DeInitialize () |
Private Attributes | |
bool | _m_initialized |
std::vector< const ObservationModelMapping::Matrix * > | _m_O |
std::vector< const ObservationModelMappingSparse::SparseMatrix * > | _m_Os |
std::vector< std::vector < SparseVector * > > | _m_OsForBackup |
const PlanningUnitDecPOMDPDiscrete * | _m_pu |
A pointer to the PlanningUnit (which can only be a PlanningUnitDecPOMDPDiscrete or derived type). | |
std::vector< const TransitionModelMapping::Matrix * > | _m_T |
std::vector< const TransitionModelMappingSparse::SparseMatrix * > | _m_Ts |
std::vector< std::vector < SparseVector * > > | _m_TsForBackup |
std::vector< std::vector < std::vector< SparseVector * > > > | _m_TsOsForBackup |
bool | _m_useSparse |
AlphaVectorPlanning provides base functionality for alpha-vector based POMDP or BG techniques.
Definition at line 62 of file AlphaVectorPlanning.h.
|
private |
Definition at line 76 of file AlphaVectorPlanning.h.
AlphaVectorPlanning::AlphaVectorPlanning | ( | const PlanningUnitDecPOMDPDiscrete & | pu | ) |
(default) Constructor
Definition at line 63 of file AlphaVectorPlanning.cpp.
References _m_initialized, _m_useSparse, GetPU(), and PlanningUnitMADPDiscrete::GetTransitionModelDiscretePtr().
|
virtual |
GaoVectorSet AlphaVectorPlanning::BackProject | ( | const ValueFunctionPOMDPDiscrete & | v | ) | const |
Back projects a value function.
Definition at line 201 of file AlphaVectorPlanning.cpp.
References _m_useSparse, BackProjectFull(), and BackProjectSparse().
Referenced by BackProjectMonahan(), and Perseus::BackupStageLeadIn().
GaoVectorSet AlphaVectorPlanning::BackProject | ( | const VectorSet & | v | ) | const |
Back projects a value function, represented as a VectorSet.
Implements equation (3.11) of PhD thesis Matthijs.
Definition at line 230 of file AlphaVectorPlanning.cpp.
References _m_useSparse, BackProjectFull(), and BackProjectSparse().
|
private |
Implements equation (3.11) of PhD thesis Matthijs.
Definition at line 241 of file AlphaVectorPlanning.cpp.
References _m_O, _m_T, GetDuplicateIndices(), PlanningUnitMADPDiscrete::GetNrJointActions(), PlanningUnitMADPDiscrete::GetNrJointObservations(), PlanningUnitMADPDiscrete::GetNrStates(), GetPU(), TimedAlgorithm::StartTimer(), and TimedAlgorithm::StopTimer().
Referenced by BackProject().
GaoVectorSet AlphaVectorPlanning::BackProjectMonahan | ( | const QFunctionsDiscrete & | Qs | ) | const |
Definition at line 681 of file AlphaVectorPlanning.cpp.
References QFunctionsToValueFunction().
GaoVectorSet AlphaVectorPlanning::BackProjectMonahan | ( | const ValueFunctionPOMDPDiscrete & | V | ) | const |
Definition at line 688 of file AlphaVectorPlanning.cpp.
References BackProject(), PlanningUnitDecPOMDPDiscrete::GetDiscount(), PlanningUnitMADPDiscrete::GetNrJointActions(), PlanningUnitMADPDiscrete::GetNrJointObservations(), PlanningUnitMADPDiscrete::GetNrStates(), GetPU(), and PlanningUnitDecPOMDPDiscrete::GetReward().
|
private |
Implements equation (3.11) of PhD thesis Matthijs.
Definition at line 387 of file AlphaVectorPlanning.cpp.
References _m_Os, _m_Ts, _m_TsOsForBackup, GetDuplicateIndices(), PlanningUnitMADPDiscrete::GetNrJointActions(), PlanningUnitMADPDiscrete::GetNrJointObservations(), PlanningUnitMADPDiscrete::GetNrStates(), GetPU(), TimedAlgorithm::StartTimer(), and TimedAlgorithm::StopTimer().
Referenced by BackProject().
Compute the cross-sum of two vector sets.
Definition at line 611 of file AlphaVectorPlanning.cpp.
Referenced by MonahanCrossSum().
|
private |
Definition at line 166 of file AlphaVectorPlanning.cpp.
References _m_O, _m_Os, _m_OsForBackup, _m_T, _m_Ts, _m_TsForBackup, and _m_TsOsForBackup.
Referenced by Initialize(), and ~AlphaVectorPlanning().
|
static |
Exports a belief set to a file.
Definition at line 1135 of file AlphaVectorPlanning.cpp.
|
static |
Exports the POMDP represented by pu to file named filename.
Export is in Tony Cassandra's POMDP file format.
Definition at line 1051 of file AlphaVectorPlanning.cpp.
References Globals::COST, BeliefInterface::Get(), PlanningUnitDecPOMDPDiscrete::GetDiscount(), MultiAgentDecisionProcessDiscreteInterface::GetJointAction(), MultiAgentDecisionProcessDiscreteInterface::GetJointObservation(), PlanningUnitMADPDiscrete::GetNewJointBeliefFromISD(), PlanningUnitMADPDiscrete::GetNrJointActions(), PlanningUnitMADPDiscrete::GetNrJointObservations(), PlanningUnitMADPDiscrete::GetNrStates(), PlanningUnitMADPDiscrete::GetObservationProbability(), PlanningUnitDecPOMDPDiscrete::GetReferred(), PlanningUnitDecPOMDPDiscrete::GetReward(), DecPOMDPInterface::GetRewardType(), MultiAgentDecisionProcessDiscreteInterface::GetState(), PlanningUnitMADPDiscrete::GetTransitionProbability(), Globals::REWARD, JointAction::SoftPrintBrief(), and NamedDescribedEntity::SoftPrintBrief().
Referenced by ExportPOMDPFile().
void AlphaVectorPlanning::ExportPOMDPFile | ( | const std::string & | filename | ) | const |
Exports the POMDP to file named filename.
Definition at line 1045 of file AlphaVectorPlanning.cpp.
References ExportPOMDPFile(), and GetPU().
|
static |
Exports a value function V to file named filename.
Export is in a simple text format used by Tony Cassandra for .alpha files.
The format is simply:
<action>
<action>
Definition at line 877 of file AlphaVectorPlanning.cpp.
Referenced by ExportValueFunction().
|
static |
Exports a Q functions Q to file named filename.
Definition at line 833 of file AlphaVectorPlanning.cpp.
References ExportValueFunction(), and QFunctionsToValueFunction().
|
static |
Exports a Q functions Q to file named filename.
Definition at line 841 of file AlphaVectorPlanning.cpp.
References ExportValueFunction().
|
static |
Returns a vector<int> which for each vector k in V specifies whether it is unique, in which case vector<int>[k] is set to -1, or whether it is a duplicate of another vector l in V, in which case vector<int>[k] is set to the index of l, where 0 <= l < V.size()).
Definition at line 1206 of file AlphaVectorPlanning.cpp.
References PrintTools::PrintVectorCout(), and Globals::PROB_PRECISION.
Referenced by BackProjectFull(), and BackProjectSparse().
ValueFunctionPOMDPDiscrete AlphaVectorPlanning::GetImmediateRewardValueFunction | ( | ) | const |
Returns the value function induced by the reward model.
Definition at line 1008 of file AlphaVectorPlanning.cpp.
References GetPU().
Referenced by PerseusStationary::GetImmediateRewardBeliefSet().
|
static |
Returns the value function induced by the reward model of pu.
Definition at line 1015 of file AlphaVectorPlanning.cpp.
References PlanningUnitMADPDiscrete::GetNrJointActions(), PlanningUnitMADPDiscrete::GetNrStates(), PlanningUnitDecPOMDPDiscrete::GetReward(), AlphaVector::SetAction(), and AlphaVector::SetValue().
|
inline |
Returns a ref to the PlanningUnit.
Definition at line 102 of file AlphaVectorPlanning.h.
References _m_pu.
Referenced by AlphaVectorPlanning(), BackProjectFull(), BackProjectMonahan(), BackProjectSparse(), PerseusPOMDPPlanner::BackupStage(), PerseusBGPlanner::BackupStageAll(), Perseus::BackupStageLeadOut(), PerseusBGPlanner::BackupStageSampling(), PerseusBGPlanner::BackupStageSamplingAlt(), AlphaVectorPOMDP::BeliefBackup(), AlphaVectorBG::BeliefBackupBGIP_Solver(), AlphaVectorBG::BeliefBackupExhaustiveOnlyKeepMax(), AlphaVectorBG::BeliefBackupExhaustiveStoreAll(), ExportPOMDPFile(), PerseusStationary::GetImmediateRewardBeliefSet(), GetImmediateRewardValueFunction(), Perseus::GetInitialNonStationaryQFunctions(), AlphaVectorBG::GetMask(), Initialize(), MonahanCrossSum(), Perseus::PlanEndOfIteration(), Perseus::PlanLeadOut(), Prune(), SampleBeliefs(), PerseusStationary::SetValueFunction(), and ValueFunctionToQ().
|
static |
Imports a value function from a file named filename.
Function lacks error checking.
Definition at line 907 of file AlphaVectorPlanning.cpp.
References AlphaVector::SetAction(), AlphaVector::SetBetaI(), and AlphaVector::SetValues().
Referenced by ImportValueFunction(), and PerseusStationary::SetValueFunction().
|
static |
Definition at line 853 of file AlphaVectorPlanning.cpp.
References ImportValueFunction(), and ValueFunctionToQ().
void AlphaVectorPlanning::Initialize | ( | ) |
Reimplemented in Perseus.
Definition at line 90 of file AlphaVectorPlanning.cpp.
References _m_initialized, _m_O, _m_Os, _m_OsForBackup, _m_T, _m_Ts, _m_TsForBackup, _m_TsOsForBackup, _m_useSparse, DeInitialize(), TransitionModelMapping::GetMatrixPtr(), ObservationModelMapping::GetMatrixPtr(), ObservationModelMappingSparse::GetMatrixPtr(), TransitionModelMappingSparse::GetMatrixPtr(), PlanningUnitMADPDiscrete::GetNrJointActions(), PlanningUnitMADPDiscrete::GetNrJointObservations(), PlanningUnitMADPDiscrete::GetNrStates(), PlanningUnitMADPDiscrete::GetObservationModelDiscretePtr(), GetPU(), and PlanningUnitMADPDiscrete::GetTransitionModelDiscretePtr().
Referenced by PerseusBGPlanner::PerseusBGPlanner(), and PerseusPOMDPPlanner::PerseusPOMDPPlanner().
void AlphaVectorPlanning::MonahanCrossSum | ( | const GaoVectorSet & | G, |
ValueFunctionPOMDPDiscrete & | V, | ||
Index | a, | ||
bool | doIncPrune | ||
) | const |
Definition at line 637 of file AlphaVectorPlanning.cpp.
References CrossSum(), PlanningUnitMADPDiscrete::GetNrJointObservations(), PlanningUnitMADPDiscrete::GetNrStates(), GetPU(), Prune(), AlphaVector::SetAction(), and AlphaVector::SetValue().
ValueFunctionPOMDPDiscrete AlphaVectorPlanning::Prune | ( | const ValueFunctionPOMDPDiscrete & | V | ) | const |
Prune a valuefunction.
Definition at line 718 of file AlphaVectorPlanning.cpp.
References PlanningUnitMADPDiscrete::GetNrStates(), and GetPU().
Referenced by MonahanCrossSum().
Prune a VectorSet.
Definition at line 775 of file AlphaVectorPlanning.cpp.
|
static |
Definition at line 1034 of file AlphaVectorPlanning.cpp.
Referenced by BackProjectMonahan(), PerseusBGPlanner::BackupStageAll(), PerseusBGPlanner::BackupStageSampling(), PerseusBGPlanner::BackupStageSamplingAlt(), and ExportValueFunction().
BeliefSet AlphaVectorPlanning::SampleBeliefs | ( | ArgumentHandlers::Arguments | args | ) | const |
Sample a belief set according to the arguments.
Definition at line 453 of file AlphaVectorPlanning.cpp.
References _m_pu, QMDP::Compute(), DEBUG_AlphaVectorPlanning_BeliefSampling, BeliefInterface::Get(), PlanningUnit::GetHorizon(), MultiAgentDecisionProcessDiscreteInterface::GetISD(), PlanningUnitMADPDiscrete::GetNewJointBeliefInterface(), PlanningUnitMADPDiscrete::GetNrJointActions(), PlanningUnitMADPDiscrete::GetNrStates(), PlanningUnitMADPDiscrete::GetProblem(), GetPU(), QMDP::GetQ(), PlanningUnitDecPOMDPDiscrete::GetReferred(), Globals::MAXHORIZON, ArgumentHandlers::Arguments::nrBeliefs, ArgumentHandlers::Arguments::QMDPexploreProb, ArgumentHandlers::Arguments::resetAfter, MultiAgentDecisionProcessDiscreteInterface::SampleInitialState(), MultiAgentDecisionProcessDiscreteInterface::SampleJointObservation(), MultiAgentDecisionProcessDiscreteInterface::SampleSuccessorState(), BeliefInterface::SanityCheck(), BeliefInterface::Set(), TimedAlgorithm::StartTimer(), TimedAlgorithm::StopTimer(), ArgumentHandlers::Arguments::uniqueBeliefs, JointBeliefInterface::Update(), and ArgumentHandlers::Arguments::useQMDPforSamplingBeliefs.
Referenced by PerseusStationary::InitializeBeliefs().
QFunctionsDiscrete AlphaVectorPlanning::ValueFunctionToQ | ( | const ValueFunctionPOMDPDiscrete & | V | ) | const |
Definition at line 1155 of file AlphaVectorPlanning.cpp.
References GetPU().
Referenced by ImportValueFunction(), PerseusStationary::SetValueFunction(), and PerseusStationary::StoreValueFunction().
|
static |
Definition at line 1162 of file AlphaVectorPlanning.cpp.
References AlphaVector::SetAction().
|
static |
Definition at line 1241 of file AlphaVectorPlanning.cpp.
References AlphaVector::GetAction(), AlphaVector::GetNrValues(), AlphaVector::GetValue(), and Globals::PROB_PRECISION.
Referenced by PerseusBGPlanner::BackupStageAll(), PerseusBGPlanner::BackupStageSampling(), and PerseusBGPlanner::BackupStageSamplingAlt().
|
private |
Definition at line 86 of file AlphaVectorPlanning.h.
Referenced by AlphaVectorPlanning(), and Initialize().
|
private |
Definition at line 71 of file AlphaVectorPlanning.h.
Referenced by BackProjectFull(), DeInitialize(), and Initialize().
|
private |
Definition at line 74 of file AlphaVectorPlanning.h.
Referenced by BackProjectSparse(), DeInitialize(), and Initialize().
|
private |
Definition at line 78 of file AlphaVectorPlanning.h.
Referenced by DeInitialize(), and Initialize().
|
private |
A pointer to the PlanningUnit (which can only be a PlanningUnitDecPOMDPDiscrete or derived type).
Definition at line 68 of file AlphaVectorPlanning.h.
Referenced by GetPU(), and SampleBeliefs().
|
private |
Definition at line 70 of file AlphaVectorPlanning.h.
Referenced by BackProjectFull(), DeInitialize(), and Initialize().
|
private |
Definition at line 73 of file AlphaVectorPlanning.h.
Referenced by BackProjectSparse(), DeInitialize(), and Initialize().
|
private |
Definition at line 77 of file AlphaVectorPlanning.h.
Referenced by DeInitialize(), and Initialize().
|
private |
Definition at line 79 of file AlphaVectorPlanning.h.
Referenced by BackProjectSparse(), DeInitialize(), and Initialize().
|
private |
Definition at line 81 of file AlphaVectorPlanning.h.
Referenced by AlphaVectorPlanning(), BackProject(), and Initialize().