This package provides a core interface for creating MODIA objects comprising Markov decision processes (MDPs) and/or partially observable Markov decision processes (POMDPs). The package is written in the Julia Programming Language.
! TODO: Describe MODIA architecture in more detail here.
The POMDPModia.jl package is highly compatible and dependent on the other packages in the JuliaPOMDP ecosystem.
The (PO)MDP definitions used in POMDPModia.jl are instantiate through the POMDPs.jl package. Through the POMDPs.jl interface, problems are defined and solved using a large variety of tools available:
Package |
Build |
Coverage |
|---|---|---|
| POMDPs |  |
 |
| QuickPOMDPs |  |
 |
| POMDPModelTools |  |
 |
| BeliefUpdaters |  |
 |
| POMDPPolicies |  |
 |
| POMDPSimulators |  |
 |
| POMDPModels |  |
 |
| POMDPTesting |  |
 |
Also, see the supported solvers for the summary of solvers available, and the solvers documentation for examples and writing custom solvers.
In this short example, a MODIA will be created for various amounts of three different TigerPOMDP characteristics.
using POMDPModels: TigerPOMDP
using QMDP: QMDPSolver
# Define the DPs, DCs, and SSF for a MODIA object
tiger_problem1 = TigerPOMDP(-1.0, -100.0, 10.0, 0.90, 0.90);
tiger_problem2 = TigerPOMDP(-2.0, -50.0, 17.0, 0.80, 0.60);
tiger_problem3 = TigerPOMDP(-3.0, -75.0, 8.0, 0.85, 0.75);
DPs = [tiger_problem1, tiger_problem2, tiger_problem3];
DCs = [4, 1, 2];
SSF = Base.minimum;
# Create a MODIA, where all POMDP beliefs are initialized uniformly
modia = MODIA(DPs, DCs, SSF)
initialize_beliefs!(modia, BeliefUpdaters.uniform_belief);
# Compute optimal alpha vectors (policies) offline, and a define belief updater
solver = QMDPSolver();
policies = POMDPs.solve(solver, modia);
bu = BeliefUpdaters.DiscreteUpdater(modia);
# Simulate the MODIA object for 5 timesteps, retrieve discounted rewards
sim = POMDPSimulators.RolloutSimulator(max_steps=5)
initial_states = convert(Array{Bool},(rand(initialstate(modia))))
r_totals = POMDPs.simulate(sim, modia, policies, bu, modia.beliefs, initial_states)If you have found the MODIA architecture to be useful, consider citing the paper:
@inproceedings{ijcai2017-664,
author = {Kyle Hollins Wray and Stefan J. Witwicki and Shlomo Zilberstein},
title = {Online Decision-Making for Scalable Autonomous Systems},
booktitle = {Proceedings of the Twenty-Sixth International Joint Conference on
Artificial Intelligence, {IJCAI-17}},
pages = {4768-4774},
year = {2017},
doi = {10.24963/ijcai.2017/664},
url = {https://doi.org/10.24963/ijcai.2017/664},
}