Anatomy of a large-scale real-time peer evaluation system

In the past decades, we have seen benefits from digital education tools to increase interaction and allow for more regular assessment of students’ knowledge. The existing suggestions are however limited in how comprehensively they can assess students’ production skills based on the taught syllabus. There are some endeavours in the area of automated essay scoring, automated short answer grading or similar methods which would provide feedback without necessarily providing a grade. In terms, of rapid in class feedback of open ended questions there aren’t many helpful solution for the population we want to help. More suggestions which focus on the people either evaluating or being evaluated is necessary. Given the pressures of teaching at scale, how can human-centered technology aid in the development of comprehensive evaluation methods? In this paper, we suggest and implement a method to conduct qualitative evaluations for large scale settings in the context of undergraduate education. To do this, we use word vectors to visualize what students think in response to a prompt.