Automatic Evaluation of Student Software Solutions in a Virtualized Environment

Universities are using Automated Programming Assessment Systems (APAS) to minimize problems that emerge by manually managing student software solutions: subjectivity, inefficiency in case of many solutions, and lack of fast and rich feedback for the student. Lately, handling code executions in a secure way has become a requirement. Researchers addressed this topic and proposed generic security models: user-level restrictions, process-level restrictions, and virtualization. In the case of virtualization, commonly suggested solutions are virtual machines and containers. With all the benefits of virtual machines, but less resource-demanding, containers are becoming widely used last years. This paper proposes a newly developed assessment model and an information system and proposes a solution for the automatic evaluation of student software solutions in a virtualized environment. It analyzes the key parts of the student solution source codes and commands against a combination of the test case input data, using the pattern-finding method. This method was used in the context of finding the required lexical structures. The proposed system is used during the exam evaluation of the Programming course at the College of Algebra, and the relational database courses, for evaluating SQL solutions in combination with the previously developed system at the College of Algebra.