A Dataset for Analyzing Crowdsourced Feedback in Usability Testing

Conventional usability evaluation methods despite being widely used usually fall short of producing comprehensive results since they involve testing with few real users/experts in a controlled environment. One possible solution to address this issue is to involve a sufficiently large number of potential end-users to perform usability tests in a completely natural environment. However, this approach generates a massive amount of user feedback, which can be challenging for experts to analyze manually. Integrating machine learning techniques presents a promising solution to address this issue. By leveraging machine learning, the analysis of vast amounts of user feedback can be automated and expedited. However, the effective implementation of machine learning algorithms relies on the availability of relevant datasets. To address this need, we created a comprehensive dataset of 911 usability flaws. This dataset would serve as a valuable resource for researchers and practitioners seeking to train and test machine learning models in the domain of usability testing. The dataset was constructed through a user study involving 800 participants who performed usability testing after getting trained. Additionally, the dataset underwent a thorough validation process conducted by a team of three experts who were carefully chosen based on their expertise in usability evaluation.