SEthesaurus: WordNet in Software Engineering

Informal discussions on social platforms (e.g., Stack Overflow, CodeProject) have accumulated a large body of programming knowledge in the form of natural language text. Natural language process (NLP) techniques can be utilized to harvest this knowledge base for software engineering tasks. However, consistent vocabulary for a concept is essential to make an effective use of these NLP techniques. Unfortunately, the same concepts are often intentionally or accidentally mentioned in many different morphological forms (such as abbreviations, synonyms and misspellings) in informal discussions. Existing techniques to deal with such morphological forms are either designed for general English or mainly resort to domain-specific lexical rules. A thesaurus, which contains software-specific terms and commonly-used morphological forms, is desirable to perform normalization for software engineering text. However, constructing this thesaurus in a manual way is a challenge task. In this paper, we propose an automatic unsupervised approach to build such a thesaurus. In particular, we first identify software-specific terms by utilizing a software-specific corpus (e.g., Stack Overflow) and a general corpus (e.g., Wikipedia). Then we infer morphological forms of software-specific terms by combining distributed word semantics, domain-specific lexical rules and transformations. Finally, we perform graph analysis on morphological relations. We evaluate the coverage and accuracy of our constructed thesaurus against community-cumulated lists of software-specific terms, abbreviations and synonyms. We also manually examine the correctness of the identified abbreviations and synonyms in our thesaurus. We demonstrate the usefulness of our constructed thesaurus by developing three applications and also verify the generality of our approach in constructing thesauruses from data sources in other domains.

1 Introduction

Developers often discuss in the form of natural language text on social platforms (such as Stack Overflow, CodeProject) to share and acquire programming knowledge. Therefore, many natural-language-processing (NLP) based techniques have been proposed and developed to mine programming knowledge from such informal discussions. The mined knowledge can assist developers for many software engineering tasks, such as searching for documents [1], [2], categorizing software technologies [3], extracting API mentions and usage insights [4], [5], recovering traceability among informal discussions (e.g., duplicate questions [6]) or between code and informal discussions [7], linking domain-specific entities in informal discussions to official documents [8] and mining technology landscapes [9], [10]. To make an effective use of NLP techniques in these tasks, a consistently-used vocabulary of software-specific terms is essential, since NLP techniques assume that the same words are used whenever a particular concept is mentioned.