An Empirical Study of Highly Impactful Bugs in Mozilla Projects

Bug triaging is the process that consists in screening and prioritising bugs to allow a software organisation to focus its limited resources on bugs with high impact on software quality. In a previous work, we proposed an entropy-based crash triaging approach that can help software organisations identify crash-types that affect a large user base with high frequency. We refer to bugs associated to these crash-types as highly-impactful bugs. The proposed triaging approach can identify highly-impactful bugs only after they have led to crashes in the field for a certain period of time. Therefore, to reduce the impact of highly-impactful bugs on user perceived quality, an early identification of these bugs is necessary. In this paper, we examine the characteristics of highly-impactful bugs in Mozilla Firefox and Fennec for Android, and propose statistical models to help software organisations predict them early on before they impact a large population of users. Results show that our proposed prediction models can achieve a precision up to 64.2% (in Firefox) and a recall up to 98.3% (in Fennec). We also evaluate the benefits of our proposed models and found that, on average, they could help reduce 23.0% of Firefox' crashes and 13.4% of Fennec's crashes, while reducing 28.6% of impacted machine profiles for Firefox and 49.4% for Fennec. Software organisations could use our prediction models to catch highly-impactful bugs early during the triaging process, preventing them from impacting a larger user base.

I. Introduction

Today, many software organisations (e.g., Microsoft, Mozilla) embed automatic crash reporting tools in their software systems. Whenever the software crashes (i.e., terminates unexpectedly in the user environment), the automatic crash reporting tool collects information about the crash and sends a detailed crash report to the software vendor. A crash report usually contains a signature, the stack trace of the failing thread, runtime information, such as the crash time, and information about the user environment, e.g., the operating system, the version, and the install age. Crashes with the same signature are grouped together and filed in the same bug report. Software quality managers and developers usually judge the priority and severity of a bug by looking at the frequency of its related crashes [14].