Different assignments as different contexts: predictors across assignments and outcome measures in CS1

This paper reports an analysis of quantitative data obtained during four weeks of a CS1 course. The data consists of programming events logged while students complete eight programming projects and include keystrokes, text pastes, task switches, and run attempts. We analyze the data to answer two related research questions. The first is which commonly studied student programming behaviors generalize well as predictors across programming assignments. The second question is which commonly studied student programming behaviors generalize well as predictors across outcome measures. We find that of the attributes we tested only a small subset are consistent predictors of success across projects, although most have some correlation in some projects. Few attributes were consistent across performance measures. Considering that many intervention strategies use small numbers of projects for student classification, our results suggest that care should be taken in drawing conclusions from data analyzed in the aggregate, both across programming projects and across performance measures.