1. Introduction
Understanding and evaluating the Caller Experience (CE) provided by commercially-deployed automatic spoken dialog systems is crucial to measuring whether such a system is achieving its performance goals. Knowing how callers seem to experience a system can help guide business and design decisions. Specifically, a call's CE rating can indicate which interactions need to be streamlined, simplified, or made more robust. CE, though, is not a well-understood characteristic and has long been subject to speculation based on marketing survey results and anecdotal interactions with the system. As expected, these uncertain methods of gauging CE are often inaccurate and do not provide clear guidance as to what the true caller satisfaction is, or, just as importantly, what aspects of the interaction affect it.