Determining Student Effort in Online Programming Courses through the Analysis of Video Annotations

Video-based learning has become increasingly popular in online education, with many platforms offering video annotation features to promote student engagement. However, traditional engagement metrics may need to fully capture the complexity of learning behaviors, such as ineffective effort during the learning process. This study aims to introduce a novel "weighted effort value" indicator to measure students' learning effort in online programming courses and establish a student classification framework based on effort-performance relationships. Building upon previous qualitative analysis results, we quantify students' efforts by considering the quality and types of annotations and their relationships with learning outcomes. By examining the relationship between students' weighted effort values and standardized quiz scores, we establish a framework to identify student groups with different effort patterns. The "weighted effort value" regression model strengthens the correlation between students' annotation engagement and learning outcomes. Although other factors still influence the single-factor regression analysis results, the proposed classification framework effectively identifies student characteristics and reveals students with imbalanced effort-performance relationships. Finally, a series of personalized teaching intervention suggestions are proposed, providing practical guidance for educators to optimize instructional design. The study offers a valuable new dimension for online learning research, enriches the understanding of learning behaviors, and has the potential to support personalized learning analytics across disciplines. These aspects are of great significance for improving the overall quality of education in the era of intelligent learning.

I. Introduction

Online education's rapid development has provided learners with unprecedented flexibility and personalization. Video-based learning (VBL) environments, such as Massive Open Online Courses (MOOCs) and some flipped classroom or on-demand courses, have been widely used [1], with VBL programming courses gaining increasing popularity. However, the lack of interaction between teachers and students in VBL environments poses challenges for assessing student efforts, such as participation and classroom performance.