/ParticleFilterTrees.jl

Sparse Particle Tree POMDP Solvers

Primary LanguageJulia

ParticleFilterTrees.jl

CIcodecov

Particle Filter Trees with Double Progressive Widening

Parameters

Parameter Type Default Description
max_depth Int 20 Maximum tree search depth
n_particles Int 100 Number of particles representing tree node belief
criterion Any PolyUCB(1.) Action selection criterion
k_o Float64 10.0 Initial observation widening parameter
alpha_o Float64 0.0 Observation progressive widening parameter
k_a Float64 5.0 Initial action widening parameter
alpha_a Float64 0.0 Action progressive widening parameter
tree_queries Int 1_000 Maximum number of tree search iterations
max_time Float64 Inf Maximum tree search time (in seconds)
rng AbstractRNG Random.default_rng() Random number generator
value_estimator Any FastRandomSolver() Belief node value estimator
check_repeat_obs Bool true Check that repeat observations do not overwrite beliefs (added dictionary overhead)
enable_action_pw Bool false Incrementally sample and add actions if true; add all actions at once if false
beliefcache_size Int 1_000 Number of particle/weight vectors to cache offline (reduces online array allocations)
treecache_size Int 1_000 Number of belief/action nodes to preallocate in tree (reduces Base._growend! calls)

Usage

using POMDPs, POMDPModels
using ParticleFilterTrees

pomdp = LightDark1D()
b0 = initialstate(pomdp)
solver = PFTDPWSolver(tree_queries=10_000, check_repeat_obs=false)
planner = solve(solver, pomdp)
a = action(planner, b0)

SparsePFT

Using sufficiently large treecache_size and beliefcache_size allows for very few online allocations.

using BenchmarkTools

pomdp = LightDark1D()
b0 = initialstate(pomdp)
solver = SparsePFTSolver(
    max_time            = 0.1, 
    tree_queries        = 100_000, 
    treecache_size      = 50_000, 
    beliefcache_size    = 50_000, 
    check_repeat_obs    = false
)
planner = solve(solver, pomdp)
action(planner, b0)
@btime action(planner, b0)
100.006 ms (0 allocations: 0 bytes)