An Inference Performance Evaluation of TensorFlow and PyTorch on GPU Platform Using Image Super-Resolution Workloads

Since a growing number of parameters in deep learning model occurred, the overhead of inference performance is comparable to training, which promotes to various deep learning frameworks continually optimized for providing better inference performance. As one of the prevalent model tasks in recent years, super-resolution plays an essential role in the development of computer vision. Our study aimed to evaluate the inference performances between two most popular frameworks, TensorFlow and PyTorch. We selected eight representative super-resolution models in deep learning and calculated three metrics, inference latency, Peak Signal-to-Noise Ratio (PSNR), and Structural Similarity Index Measure (SSIM). For enhancing the reproducibility of our experiments, Open neural network exchange (ONNX) was introduced to convert trained models between TensorFlow and PyTorch. We also conducted experiments on both local and cloud platforms to ensure the reliable results. The results on local platform showed that the overall inference latency differences between two frameworks were up to $4 \%$, and the differences in PSNR and SSIM were within $1 \%$. However, noticeable differences persisted between the two frameworks in separate eight models and four datasets. The results of local platform were reproduced on cloud platform. Our findings can provide proposals for end-users when choosing frameworks between TensorFlow and PyTorch.