A Brief Survey of Offline Explainability Metrics for Conversational Recommender Systems

Conversational recommendation systems (CRS) are being embedded into search engines, streaming services, and commercial products. Explanations in conversational recommendation systems can increase user trust and satisfaction and help system designers fine-tune and improve the system. [1–2] Evaluating the quality of explanations for a recommendation system is a current challenge due to its relative newness as an attribute to optimize for, a small number of conversational recommendation datasets, and also due to a lack of enough annotated data. [1, 3–4] To facilitate the improvement of conversational recommendation systems explain-ability we survey and propose the creation of an automated metric to score how well a conversational recommender system explains itself for multiple levels of stakeholders, which evaluates the explain-ability of recommendation output and for each turn dialog output.