A Replication Study: Validation of the 19-item Short Form for the MUSIC Inventory for Engineering Student Engagement

The current research follows our work to validate the original, 26-item MUSIC Model of Motivation Inventory with Engineering students (presented in our 2022 FIE paper), where we validated the MUSIC inventory except for one of the MUSIC factors (Interest) which had several items cross-load onto other factors. Since then, the original survey authors have published a 19-item short form of the MUSIC Inventory. Our work seeks to validate the MUSIC survey instrument in the first-year engineering program, Michigan Technological University. We believe the MUSIC inventory could be a valuable tool for engineering education. To date, utilization of the MUSIC inventory has included few studies in an engineering learning context. This lack of uptake may be due to a lack of validation studies for the MUSIC Inventory in engineering classrooms. We report our early steps to validate the MUSIC Inventory for engineering programs. Our sample differed from the original validation work in that our sample consisted of mostly first-year students, while Jones sampled across class standing. All of our participants were engineering majors enrolled in a common first-year program of required coursework. In contrast, Jones sampled students in general education courses from various disciplines. We followed Jones and Wilkens' methodology for validation, which included validating items, scoring, and factors using a variety of analyses. Results revealed small differences in means between the long and short-form factors, but effect sizes indicate that they are negligible. Independence between the 5-factor scores from the 19-item version were gauged by examining correlations among the factors scores. Our correlations were higher than those Jones and Wilkens reported, leading us to question the independence between the Usefulness and Interest scale scores. Finally, a confirmatory factor analysis revealed high correlations between some factor scores and a lower-than-desired GFI. Again, problems appeared to stem from the Interest factor of the MUSIC Inventory. To better understand the validity of the inventory within engineering education, we conducted a 4-factor confirmatory factor analysis removing the Interest factor items. All fit indices improved, with the GFI approaching the desired value (0.90) at 0.879, and all correlations fell below the desired r = 0.71 (except for the Empowerment/Usefulness correlation of r = 0.776). In summary, the problems we reported between the Interest and Usefulness scores when validating the 26-item MUSIC Inventory for engineering students in an earlier study continue to exist in the more recent 19-item form. Possible reasons are discussed. We recommend caution when interpreting the Interest factor using the MUSIC inventory within engineering classes or programs and suggest future research.