Dynamic Heuristic Approach to Enhance the Performance of Few-Shot Meta-Learning

Meta-learning has recently become an interesting topic for researchers, particularly in supervised learning problems with a lack of training data. Meta-learning has shown effectiveness in generalization and adapting to solve new tasks with only a few data points. To train the meta-learning model, an optimizer should be used to update parameters during each training cycle. Most of the developed meta-learning models have used one of the standard gradient-based optimizers. However, due to the special nature of meta-learning models, we propose a custom optimizer to train the meta-learning model. Our proposal is a new optimizer based on combining a metaheuristic algorithm with traditional gradient-based techniques. The algorithm initiates the searching for the optimal values of the parameters of the model (weights vector) using some random initial candidate solutions. Iteratively, according to the performance of each individual, population members might either be reproduced or eliminated from the population. The learning process will then continue using classic gradient optimization, starting with the best solution found by the heuristic algorithm. Our experimental analysis shows that this strategy could enhance the performance of training meta-learning models and enable the efficient finding of optimal parameters due to the dynamic characteristics of our proposed strategy.