Metrics for Estimating Accuracy, Reliability, and Bias in Peer Assessment

The challenge of teaching staff being overworked and not having enough time to provide individualized feedback and guidance to students is a significant problem. Open-ended assignments foster creative thinking but add to the workload. The abundance of solutions makes individualized assessment necessary. Peer assessment, where students evaluate their peers’ assignments based on their motivation, knowledge, and resourcefulness, has proven effective in higher education courses.This paper provides an overview of existing metrics used in peer assessment, defines what constitutes correct and consistent evaluation, and suggests ways to choose appropriate metrics for learning objectives. The paper also highlights limitations of measures presented in the published papers, shows the advantages and disadvantages of different ways of calculating the selected measures through examples.