Content area
We study structural, algebraic, and statistical aspects of learning systems, with an emphasis on understanding how structure, symmetry, and singularity influence their behavior. Part I examines models whose parameterizations or loss landscapes have polynomial or rational form. Using tools from algebraic geometry and numerical algebraic geometry, we investigate the geometry of their critical sets, the role of degeneracies and singularities, and the symmetries that arise from overparameterization or model design. This part also considers rational neural networks, for which we introduce algebraic regularization schemes aimed at improving trainability and analyze the resulting optimization landscapes through a combination of theoretical and numerical methods.
Part II focuses on learning systems motivated by applications in statistics and engineering. Here we study procedures for estimating means of bounded random variables based on betting strategies, with an emphasis on statistical guarantees and practical performance. We also explore models of resilience and recovery in artificial and biological neural networks and investigate machine learning components used in digital twins for manufacturing systems, where structural assumptions play a central role in inference and control.
The application of algebraic and numerical algebraic tools to machine learning theory is still at an early stage, and many questions remain open. A deeper understanding of how algebraic structure aligns with learning dynamics, how singularities arise in parameterized models, and how these features relate to implicit bias and other emergent phenomena may provide valuable insight. The mathematical ideas explored in this thesis reflect only a small part of what may be possible, but working with these tools has been a source of enjoyment due to the elegance they bring to complex systems. I hope that readers will find the methods and perspectives presented here accessible and motivating for future exploration.