Online algorithms for POMDPs with continuous state, action, and observation spaces
Zachary Sunberg, Mykel Kochenderfer
Code Available — Be the first to reproduce this paper.
ReproduceCode
- github.com/zsunberg/ContinuousPOMDPTreeSearchExperiments.jlOfficialIn papernone★ 0
- github.com/AdaCompNUS/magicpytorch★ 17
- github.com/sisl/PA-POMCPOW.jlnone★ 4
- github.com/JuliaPOMDP/POMCPOW.jlnone★ 0
Abstract
Online solvers for partially observable Markov decision processes have been applied to problems with large discrete state spaces, but continuous state, action, and observation spaces remain a challenge. This paper begins by investigating double progressive widening (DPW) as a solution to this challenge. However, we prove that this modification alone is not sufficient because the belief representations in the search tree collapse to a single particle causing the algorithm to converge to a policy that is suboptimal regardless of the computation time. This paper proposes and evaluates two new algorithms, POMCPOW and PFT-DPW, that overcome this deficiency by using weighted particle filtering. Simulation results show that these modifications allow the algorithms to be successful where previous approaches fail.