Content area
Learning-based neural controllers have shown great promise in modern systems such as autonomous driving and robotics control. However, such controllers are hard to certify, preventing them from being widely employed in safety-critical applications.
Control barrier function (CBF) is a powerful tool to certify the safety of control systems by establishing a forward invariant set that the controller will remain within. However, traditional methods for artificial CBF design lack the expressiveness required for learning-based controllers and complex systems.
To tackle these difficulties, we propose learning-based certification tools for learning-based controllers. Specifically, we design neural control barrier functions (CBFs), barrier functions (BFs), and Lyapunov methods to certify the safety of learning-based neural controllers.
We start by certifying a given neural-network-based controller by quantifying the reachable forward invariant set using neural barrier functions (BFs) and neural network certification tools. Then, we extend the Lyapunov and barrier ideology to constrained reinforcement learning by training a constraint-aware policy, whose effectiveness is demonstrated on tasks with fairness constraints. In the context of explicitly finding safety violations, we propose a sampling-based Monte Carlo Tree Search (MCTS) framework for high-dimensional nonconvex optimization. Lastly, we complete the training and certification loop by jointly proposing an efficient and exact verification framework for ReLU-activated neural CBFs, along with a corresponding training method that regularizes the neural network’s activation patterns. The efficient certification enables the use of counterexamples to guide the training process, and the regularization improves the certification performance.
These findings assert the efficacy of learning-based certification methods for learning-based controllers and provide a principled path toward deploying safe, neural network-based control systems in real-world settings.