Details

Autor(en) / Beteiligte

• Li, Sarah H. Q

Titel

Scalable Coordination of Intelligent Vehicles in Shared Markovian Dynamics

Ort / Verlag

ProQuest Dissertations & Theses

Erscheinungsjahr

2023

Quelle

ProQuest Dissertations & Theses A&I

Beschreibungen/Notizen

• Driven by the growing demand for mobility and connectivity, future aerospace-based transportation systems necessitate efficient coordination of not just one, or two, but a population of intelligent vehicles that execute independent tasks in a shared operation environment. Combining techniques from game theory, optimization, and Markov decision process, the dissertation tackles three key challenges in coordinating intelligent vehicles sharing a disruption-prone environment: 1) maximizing safety in multi-vehicle trajectory planning, 2) strengthening fleet resiliency to resource disruptions, and 3) optimizing individual performance and safety when coordination is not possible. All three challenges revolve around building a coordination framework for intelligent autonomous vehicles that prioritizes each vehicle’s performance and safety. Grounded in this goal, this dissertation combines theoretical tools with data-driven verification to provably facilitate large-scale autonomy in urban air spaces and ground transportation.This dissertation uses Markov games and Markov decision processes to optimize decision-making in environments influenced by unpredictable external disruptions. These models help us understand how competitive route planning and unpredictable resource disruptions impact individual safety and the overall congestion level in the environment. For instance,how can multiple aircraft owned by different airlines collectively adjust their routes, so that each aircraft’s collision risk is minimized despite uncertain airport delays? Modeling each aircraft’s interdependent decision-making process as a coupled Markov decision process, this dissertation derives efficient algorithms for finding routes that can be simultaneously optimalfor all aircraft. Furthermore, this dissertation uses these models to derive incentives that produce fleet-level trends and investigate collision minimization techniques with and without a central coordinator.In the centralized coordination scheme, a Markov game is explicitly formulated for coordinating individual decision-makers who must operate in a shared state-action space while executing independent tasks. In Chapter 3, the Markov decision process routing game model is expanded to atomic Markov games. The Markov game model is then applied to minimizecollision risks in air traffic management and optimize warehouse path planning considering stochastic package arrival times. Multiple necessary and sufficient conditions on the player cost functions that ensure the existence of Nash equilibrium are given, as well as a first-order gradient descent method that uses iterative dynamic programming to compute the game’s Nash equilibrium of the game. In Chapter 4, the Markov decision process congestion game model is used to study the effectiveness of incentives in enforcing population constraints and demonstrated on a group of ride-hail drivers in New York City. The stability of Markov games under resource disruptions and adversarial learning dynamics are analyzed in Chapters 5 and 6.In the uncoordinated scheme, an individual decision maker who cannot explicitly coordinate with others (but nonetheless share a state-action space) is modeled by a Markov decision process with non-stationary parameter uncertainty, and the resulting non-stationary Bellman iteration is analyzed via a novel set-theoretic approach. In Chapter 7, a novel perspective on classic contraction operators used in Markov decision processes is introduced. Interaction between decision-makers is abstracted as a compact set of parameter uncertainty on an individual Markov decision process, and a set-based operator is introduced to derive convergence guarantees for dynamic programming under non-stationary parameter uncertainty.