Metaheuristic Hyperparameter Optimization Using Optimal Latin Hypercube Sampling and Response Surface Methodology

Abstract

Hyperparameters allow metaheuristics to be tuned to a wide range of problems. However, even though formalized tuning of metaheuristic parameters can affect the quality of the solution, it is rarely performed. The empirical selection method and the trial-and-error method are the primary conventional parameter selection techniques for optimization heuristics. Both require a priori knowledge of the problem and involve multiple experiments requiring significant time and effort, yet neither guarantees the attainment of optimum parameter values. Of the studies that perform formal parameter tuning, experimental design is the most commonly used method. Although experimental design is feasible for systematic experimentation, it is also time-consuming and requires extensive effort for large optimization problems. The computational effort in this study refers to the number of experimental runs required for hyperparameter tuning, not the computational time for each run. This study proposes a simpler, faster method based on an optimized Latin hypercube sampling (OLHS) technique augmented with response surface methodology for estimating the best hyperparameter settings for a hybrid simulated annealing algorithm. The method is applied to solve the aircraft landing problem with time windows (ALPTW), a combinatorial optimization problem that seeks to determine the optimal landing sequence within a predetermined time window while maintaining minimum separation criteria. The results showed that the proposed method improves sampling efficiency, providing better coverage and higher accuracy with 70% fewer sample points and only 30% of the total runs compared to full factorial design.