Neural networks in non-Euclidean metric spaces

Multilayer perceptrons (MLPs) use scalar products to compute weighted activation of neurons providing decision borders using combinations of soft hyperplanes. The weighted fan-in activation function corresponds to Euclidean distance functions used to compute similarities between input and weight vector. Replacing the fan-in activation function by non-Euclidean distance function offers a natural generalization of the standard MLP model, providing more flexible decision borders. An alternative way leading to similar results is based on renormalization of the input vectors using non-Euclidean norms in extended feature spaces. Both approaches influence the shapes of decision borders dramatically, allowing to reduce the complexity of MLP networks.