Exploring Machine Learning Methods to Identify Patterns in Students' Solutions to Programming Assignments

Technology and automation have become increasingly critical for organizations today, and programming has become an essential skill for all STEM majors to meet this demand. Graduates are expected to possess programming skills to meet the needs of the modern workforce. Acquiring programming skills is a challenging task, and institutions often struggle to provide adequate resources to meet the industry's demand for proficient computer programmers. To take steps toward better understanding programming challenges among undergraduate students in science disciplines, this study aims to characterize patterns in students' solutions to programming assignments over the course of a semester. With this, the goal is to characterize students' most common challenges and take steps toward providing automated feedback. Specifically, this study applies machine learning (ML) classification algorithms to analyze student artifacts from a college-level introductory Python programming course for science majors, including source code from labs, homework assignments, projects, and two live-coding exams. Data was collected as part of a semester-long course and pre-processed and de-identified. The researchers labeled the data, and relevant features were selected to prepare the data for training the ML algorithms. Various classification algorithms were trained, and the resulting ML models were evaluated for their accuracy. Then, the study deployed a quantitative research method to evaluate both the effectiveness of various ML models and the quality of the feedback the model could provide, such as efficiency and accuracy. The research results are expected to inform the development of machine-learning algorithms to provide higher-quality feedback mechanisms for students in introductory programming courses. In that manner, this study contributes to improving the quality of programming education.

I. Introduction

National reports call for high-quality education in computing disciplines to secure a sustainable and equitable future. The United Nations Educational, Scientific, and Cultural Organization (UNESCO) identified the high demand for STEM education and programming skills among the next generation of professionals [1]. On a global scale, industrial development and global economies are driven by computational and technological innovation, which, in large part, is subject to the supply of a skilled workforce of programming professionals [2]–[4]. Even with advancements in artificial intelligence (AI), computing programming is at a low risk of automation and is even being increasingly distributed in other fields where scientists outside of computer science rely on coding to assist their work, research, teaching, and learning [5], [6].