Tuning Performance via Metrics with Expectations

Modern server systems employ many features that are difficult to exploit by software developers. This paper calls for a new performance optimization approach that uses designated metrics with expected optimal values. A key insight is that expected values of these metrics are essential in order to verify that no performance is wasted during incremental utilization of processor features. We define sample primary metrics for modern architectures and present three distinct techniques that help to determine their optimal values. Our preliminary results successfully provide 2x-4x extra speedup during tuning of commonly-used software optimizations on the matrix-multiply kernel. Additionally, our approach helped to identify counter-intuitive causes that hurt multicore scalability of an optimized deep-learning benchmark on a Cascade Lake server.

1 Introduction

Motivation. Mainstream servers employ multiple features in order to achieve high-performance: multicore, multi-level caches, powerful vector units, and complicated microarchitectures. On one hand, it is required to utilize all these features to reach peak performance [1]. On the other hand, it is hard for developers to efficiently utilize all these features. Thus, techniques to identify potential optimizations and guide their tuning are valuable for the software community.