Comparing Student Results on a Programming Task During a Contest and a Practice Situation with the Help of the ProgCont System

At the Faculty of Informatics of the University of Debrecen, we have used the ProgCont system to evaluate programming tasks for over a decade. During that period, several improvements have been made to enhance the feedback provided by the system. Among others, the annotation technique has been introduced to assist in the efficient debugging of partially correct solutions. In our article, we have selected one of the tasks of the Regional Programming Team Competition organized in 2012 at the Faculty of Informatics, University of Debrecen, in the ProgCont system, and analyzed the students' results during the competition and practice. First, we compare the scores on different test cases, and then we try to find some connection on the test case level. Later, during our investigation, we took advantage of the test case annotation facility recently introduced in the system. We sought answers to whether the errors made during the solutions were related to whether they were made in the contest (in a live situation) or during practice situations. By using the annotation technique, we were able to examine the question separately for subproblems. As a result, we were able to find patterns that can represent their connections for a deeper understanding of those subproblems that can cause difficulties for our students.