Details

Autor(en) / Beteiligte

• Haksar, Ravi N.
• Schwager, Mac

Titel

Constrained Control of Large Graph-based MDPs Under Measurement Uncertainty

Ist Teil von

• IEEE transactions on automatic control, 2023-11, Vol.68 (11), p.1-16

Ort / Verlag

New York: IEEE

Erscheinungsjahr

2023

Quelle

IEEE Electronic Library Online

Beschreibungen/Notizen

• We consider controlling a graph-based Markov decision process (GMDP) with a control capacity constraint given only uncertain measurements of the underlying state. We also consider two special structural properties of GMDPs, called Anonymous Influence and Symmetry. Large-scale spatial processes such as forest wildfires, disease epidemics, opinion dynamics, and robot swarms are well-modeled by GMDPs with these properties. We adopt a certainty-equivalence approach and derive efficient and scalable algorithms for estimating the GMDP state given uncertain measurements, and for computing approximately optimal control policies given a maximum-likelihood state estimate. We also derive sub-optimality bounds for our estimation and control algorithms. Unlike prior work, our methods scale to GMDPs with large state-spaces and explicitly enforce a control constraint. We demonstrate the effectiveness of our estimation and control approach in simulations of controlling a forest wildfire using a model with <inline-formula><tex-math notation="LaTeX">10^{1192}</tex-math></inline-formula> total states.