Online Learning in Reproducing Kernel Hilbert Space With Non-IID Data

We analyze the convergence of online regularized learning algorithm based on dependent and non-stationary online data streams for the nonparametric regression problem in reproducing kernel Hilbert space (RKHS). We show that the algorithm achieves mean-square convergence if the algorithm gain and regularization parameter are chosen appropriately, the online data streams are weakly dependent and satisfy the eigenvalue-wise persistence of excitation condition. Especially, for the case with independent but non-identically distributed online data streams, we give more intuitive convergence conditions on the drifts of the probability measures induced by the data.