Stakeholder Preference Extraction From Scenarios

Companies use personalization to tailor user experiences. Personalization appears in search engines and online stores, which include salutations and statistically learned correlations over search-, browsing- and purchase-histories. However, users have a wider variety of substantive, domain-specific preferences that affect their choices when they use directory services, and these have largely been overlooked or ignored. The contributions of this paper include: (1) a grounded theory describing how stakeholder preferences are expressed in text scenarios; (2) an app feature survey to assess whether elicited preferences represent missing requirements in existing systems; (3) an evaluation of three classifiers to label preference words in scenarios; and (4) a linker to build preference phrases by linking labeled preference words to each other based on word position. In this study, the authors analyzed 217 elicited directory service scenarios across 12 domain categories to yield a total of 7,661 stakeholder preferences labels. The app survey yielded 43 stakeholder preferences that were missed on average 49.7% by 15 directory service websites studied. The BERT-based transformer showed the best average overall 81.1% precision, 84.4% recall and 82.6% F1-score when tested on unseen domains. Finally, the preference linker correctly links preference phrases with 90.1% accuracy. Given these results, we believe directory service developers can use this approach to automatically identify user preferences to improve service designs.