SHIM: Semantic Hierarchical clustering with Interactive Machine learning

Clustering is widely used, and there are available techniques for using semantic interactions to incorporate human feedback. Hierarchical agglomerative clustering produces complete substructure, offering rich possibilities for user interactions. However, currently available human-in-the-loop techniques miss out on the advantages of hierarchical agglomerative clustering. In this paper, we present a technique, SHIM, for semantic, interactive hierarchical clustering that helps users understand their data through exploring and modifying clusters and cluster substructure with semantic interactions and automatic descriptive rules. The proposed solution includes a compact visualization of hierarchical clustering customized for this context and connected to a table view that shows data detail and descriptive rules simultaneously. We assess our design decision and usability with expert feedback, and use simulations to test whether the metric learning method will improve a model with limited interactions.