mossr/POMDPStressTesting.jl

Adaptive stress testing of black-box systems within POMDPs.jl

POMDPStressTesting.jl

Adaptive stress testing of black-box systems, implemented within the POMDPs.jl ecosystem.

Interface

To stress test a new system, the user has to define the GrayBox and BlackBox interface outlined in src/GrayBox.jl and src/BlackBox.jl.

GrayBox Interface

The GrayBox simulator and environment interface includes:

• GrayBox.Simulation type to hold simulation variables
• GrayBox.environment(sim::Simulation) to return the collection of environment distributions
• GrayBox.transition!(sim::Simulation) to transition the simulator, returning the log-likelihood

BlackBox Interface

The BlackBox system interface includes:

• BlackBox.initialize!(sim::Simulation) to initialize/reset the system under test
• BlackBox.evaluate!(sim::Simulation) to evaluate/execute the system under test
• BlackBox.distance(sim::Simulation) to return how close we are to an event
• BlackBox.isevent(sim::Simulation) to indicate if a failure event occurred
• BlackBox.isterminal(sim::Simulation) to indicate the simulation is in a terminal state

Functions ending with ! may modify the Simulation object in place.

Solvers

Several solvers are implemented.

Reinforcement learning solvers

• MCTSPWSolver

Deep reinforcement learning solvers¹

• TRPOSolver
• PPOSolver

Stochastic optimization solvers

• CEMSolver

Baseline solvers

• RandomSearchSolver

Example

An example implementation of the AST interface is provided for the Walk1D problem:

• Julia source: test/Walk1D.jl
• Jupyter notebook: Walk1D.ipynb
• Descriptive tutorial-style write-up: walk1d.pdf (created using TeX.jl)

Installation

Install the required forked packages then the POMDPStressTesting.jl package:

using Pkg
pkg"registry add https://github.com/JuliaPOMDP/Registry"
pkg"add https://github.com/sisl/CrossEntropyMethod.jl"
pkg"add https://github.com/mossr/MCTS.jl"
pkg"add https://github.com/sisl/POMDPStressTesting.jl"

Contributing

We welcome contributions—please fork the repository and submit a new Pull Request with your changes.

---

Package maintained by Robert Moss: mossr@cs.stanford.edu

¹ TRPO and PPO thanks to Shreyas Kowshik's initial implementation.