Low-Rank Approximation in Kernel Learning

The kernel matrix plays a pivotal role in kernel learning and spatial statistics. This paper contends that under a broad range of conditions, the kernel matrix becomes ill-conditioned as the matrix dimension significantly increases. This ill-condition can lead to computational breakdowns in some algorithms and substantial output variation in response to minor alterations of input variables. Low-rank approximation emerges as a potent solution to mitigate this ill-condition problem. Through a comprehensive numerical study, we contrast various low-rank approximations, offering insightful guidance on the selection of a suitable low-rank approximation for practical application.