Concept Extraction: A Modular Approach to Extraction of Source Code Concepts

Code examples have always been one of the most sought after pieces of information when it comes to understanding and mastering programming concepts. Research shows extracting knowledge from such examples in any online tutoring system is a challenging task. Current methods rely upon specifically formed regular expressions that must be tailor made to the input language, or generation of an AST for the given input program. In our paper, we extend upon existing implementations in code recommendation software using a novel keyword based search tree (k-BST) method. K-BST recommends relevant code fragments by extracting existing keywords, matching with relevant coding examples by k-means clustering, and recommending the relevant coding examples back to the user. K-BSTs also address several major issues which modern knowledge extraction software often run into, like ease of use, extendibility to other domains and run time. With that in mind, K-BSTs are designed to tackle ease of use with popular recognizable file formats such as CSV while keeping the run time of extracting relevant keywords to be extremely low (compared to the more popular method that uses AST).