Unrolling Decentralized Stochastic Frank Wolfe Algorithm

Decentralized Frank-Wolfe methods are suitable for solving the decentralized constrained optimization with clients having computational limitations, as they solve a linear program to find the descent direction instead of the computationally intense projection operation. However, traditional decentralized Frank-Wolfe approaches may exhibit limitations in scenarios where convergence speed is critical. To address this challenge, we propose an unrolled decentralized Frank-Wolfe method to improve convergence. In particular, each layer learns the best descent direction by training a neural network, facilitating faster convergence through layer-wise updates. Furthermore, we introduce two weight sharing techniques, namely intra-client and inter-client weight sharing mechanisms, that can reduce the training complexity associated with a growing number of network layers and clients, respectively. These techniques ensure scalability of the model and improves its performance. We demonstrate the efficacy of the proposed unrolled model on various machine-learning tasks for both synthetic and real-world datasets.