Fundamental Advantages of Considering Quality of Experience Distributions over Mean Opinion Scores

Due to biased assumptions on the underlying ordinal rating scale in subjective Quality of Experience (QoE) studies, Mean Opinion Score (MOS)-based evaluations provide results, which are hard to interpret and can be little meaningful. This paper proposes to consider the full QoE distribution for evaluating and reporting QoE results instead of only using MOS values. The QoE distribution can be represented in a concise way by using the parameters of a multinomial distribution without losing any information about the underlying QoE ratings, and even keeps backward compatibility with previous, biased MOS-based results. Considering QoE results as a realization of a multinomial distribution allows to rely on a well-established theoretical background, which enables meaningful evaluations also for ordinal rating scales. Exemplary evaluations are described in this work, which demonstrate these fundamental advantages of considering QoE distributions over MOS-based evaluations.