An empirical evaluation of a Reflective Writing Framework (RWF) for Reflective Writing in Computer Science Education

Reflective writing is vital in learning professional skills and particularly in Computer Science (CS). However, there appears to be a lack of literature relating to a reliable of the current reflective writing frameworks used for CS education. This paper describes a novel Reflective Writing Framework (RWF) that has been applied to the manual content analysis of CS students' reflective writings. The paper aims to empirically examine this RWF in terms of reflection indicators and levels that can be manually assessed by raters. The results of this manual content analysis showed empirically that the coding scheme of the RWF is valid and reliable. The results were that the inter-rater reliability (IRR) of the overall coding scheme of the RWF increased from 0.5 to 0.8 in pilot tests and ranged between 0.40 and 0.75 in terms reflection indicators and levels; substantially, this means ranged from moderate to substantial agreement. This research contributes to CS education an RWF that can be reliably annotated and was validated by CS experts.