The Effects Of Non-linear Operators In Voxel-Based Deep Neural Networks For 3D Style Reconstruction

Neural Style Transfer has been successfully applied for generating plausible novel 2D images by transferring the style of a painting to an existing content image and has been extended to image, video, motion and animation manipulation. Besides first works on 2D/3D hybrid approaches relying on 2D rendering operations, a consistent end-to-end 3D Neural Style Transfer is missing. For realizing such an end-to-end 3D Neural Style Transfer, we suggested a first 3D voxel-based neural network architecture in previous works. Here, we extend this architecture to a resolution of 256
³
voxels to process vastly finer visual features, and we include our recently introduced standardized gram matrix approach, which reduces noise and visually improves style transfer results. We furthermore show that the type of activation function used in the network is crucial for the successful application in 3D Neural Style transfer. In particular, we demonstrate that the recently proposed bipolar exponential linear unit (belu) operator achieves a consistent style reconstruction independent of filter orientation relative to the shape surface. We compare the operator to the commonly used exponential linear units (elu) and provide insights for the shown behavior. In addition to a purely visual inspection, we evaluate the constructed shapes using local 3D shape descriptors and confirm the subjective visual improvements. We conclude that the utilization of belu operators in the deep neural network architecture is a crucial component for realizing high-resolution voxel-based 3D Neural Style Transfer.