A fully automatic Bayesian model for adaptive activation functions in artificial neural networks

There is a significant focus in deep neural networks on finding activation functions that can enhance network performance. Scientists are also interested in developing flexible activation functions that can adapt during training to avoid overfitting and provide reliable parameter learning.This paper presents a Bayesian algorithm that employs Markov Chain Monte Carlo (MCMC) to estimate activation function parameters and model weights. This algorithm accelerates convergence through effective sampling and evaluates its effectiveness on two datasets, achieving high accuracy with low complexity.