Path Complexity Correlates with Source Code Comprehension Effort Indicators

We describe our work on the relationship between the asymptotic path complexity of a program, time-to-completion, subjective complexity, correctness performance, and levels of brain (de)activation in select Brodmann areas of the brain, as measured by fMRI, while a human subject attempts to understand the source code of a program. Asymptotic path complexity gives an asymptotic upper bound on how quickly the number of paths through a program grows with increasing execution depth. (De)activation levels in the studied Brodmann areas of the brain are known to correlate with different specific types of cognitive effort. We add the asymptotic path complexity metric to an existing fMRI-code-comprehension data set that compares common code metrics to cognitive effort. Our results show that, according to Kendall rank correlation, asymptotic path complexity has (1) better correlation than all other metrics with code comprehension task completion time, (2) better correlation than all other metrics with subjective participant complexity, (3) better correlation than all other metrics for brain areas responsible for semantic processing, (4) correlations comparable to lines of code and Halstead complexity, and better correlation than (McCabe’s) cyclomatic complexity and dependency degree for participant response correctness and for (de)activation levels in brain areas responsible for rational thought and extracting signal from noise, and, finally, (5) worse correlation than all metrics (except McCabe’s cyclomatic complexity) for brain areas responsible for motion plandisse, language and audio processing, and additional forms of semantic processing. Overall, our results indicate that path complexity is a useful metric for measuring many aspects of code comprehension effort.