ShrinkHPO: Towards Explainable Parallel Hyperparameter Optimization

In this era of exploding data volumes, more and more complex data analysis tasks are now accomplished by machine learning (ML) or deep learning (DL). Despite the powerful and flexible task processing capability, it also brings challenges such as how to reasonably design the optimal hyperparameters configuration and the huge consumption of time for a single validation. Existing Hyperrarameter Optimization (HPO) methods are gradually facing performance bottlenecks. However, in an era where progressively data-centric big data analytics methods are prevalent, such efficiency issues need to be urgently addressed in the design of intelligent DBMS. In this paper, we propose ShrinkHPO, an efficient and explainable-designed HPO approach with a major focus on (
$a$
) efficient hyperparameter configuration search strategy, (b) asynchronous executing intervention, and (c) XAI (eXplainable AI) design. ShrinkHPO employs a hyperparameter weight estimation strategy named Shrink-search together with an asynchronous execution design to improve HPO efficiency. To the best of our knowledge, ShrinkHPO is the first HPO method that introduces explainable analysis and verification to ensure the judgment of “high significance” hyperparameters. We also conduct a series of experiments on data analysis tasks in the database domain (classification, regression, time series classification, CASH [1], etc.), collecting HPO results on the usual ML or DL models in each task. Compared to SOTA asynchronous and sequential HPO baselines, ShrinkHPO achieves top performance on both accuracy (RMSE for regression tasks) and time cost, accelerating from 20% to a maximum of 2.2 x.