The reliability analysis of rating systems in decision making: When scale meets multi-attribute additive value model

A rating system (RS) comprises a rating metric defined by a discrete set of integers contained in an interval (e.g., [0, N]), and an aggregation rule. RSs are widely used in various fields to capture and summarize individuals' opinions on alternatives. In this paper we argue that the multi-attribute additive value model (MAVM) should be used as a benchmark to analyze the reliability of RSs, and present some tight bounds on the parameter N and overall rating scores, which guarantee the consistency between the RS [0, N] and MAVM at the ranking and rating levels. Interestingly, the tight bounds at the rating level are twice as large as those at the ranking level. The results in this paper can provide new insights about the reliability analysis of RSs.