Stochastic Methods for Multi-Level and Multi-Objective Optimization

Abstract

Multi-level optimization problems, with either single or multiple objectives at each level, have become instrumental in a variety of problems arising in machine learning, cybersecurity, and defense, including neural architecture search, continual learning, hyperparameter tuning, fairness, network interdiction, and power network defense. This thesis develops state-of-the-art stochastic methodologies and rigorous theoretical analyses to advance multi-level and multi-objective optimization in innovative directions. While stochastic gradient methods are well studied for single-level problems, recent advances have focused on bilevel or single-level multi-objective problems. However, many real-world applications involve conflicting objectives across multiple levels; scenarios that remain largely unexplored from a stochastic approximation perspective. This thesis bridges this gap by extending the algorithmic framework and convergence analysis beyond classical stochastic gradient methods. We explore several complex stochastic problems, including constrained bilevel optimization, bilevel problems with multiple objectives at one or more levels, as well as trilevel optimization.