ARM: Toward Adaptive and Robust Model for Reputation Aggregation

In dynamic, open, and service-oriented computing environments, e.g., e-commerce and crowdsourcing, service consumers must choose one of the services or items to complete their tasks. Due to the scale and dynamic characteristics of these environments, service consumers may have little or no experience with the available services. To this end, reputation systems are proposed and have played a crucial role in the success of online service-oriented transactions. In this paper, we study the current reputation systems used in commercial environments. In these rating-based reputation systems, we found they are not only resilient to the changes (time lag) but also vulnerable to unfair ratings. To address the problems in parallel, we propose an adaptive reputation model (ARM). ARM can dynamically adjust its model parameters to adapt the latest changes in a service. To tackle time lag, the proposed model generalizes the fixed sliding window, used in current commercial platforms, into a dynamic sliding window mechanism. Thus, the model can completely mitigate the influence of obsolete ratings. To detect unfair ratings, our model implements a statistical strategy based on hypothesis testing after transforming the ratings in the linear window into residuals. Experiments not only validate the effectiveness of the proposed model but also show that it outperforms the existing reputation system by 45% on average based on five test cases. The results also show that the proposed model can asymptotically converge to the underlying reputation value as ratings begin to accumulate. Note to Practitioners - The reputation models adopted by current commercial platforms, such as Amazon, eBay, and Taobao, not only suffer heavily from unfair rating but also resilient to the changes in services. To address the problems simultaneously, we design and implement a hybrid model that continuously monitors received ratings and aggregates the reputation value in a self-adaptive way. Our model first fits received fair ratings using the Bayesian linear regression approach and captures the distribution of fair ratings; it then filters out unfair ratings leveraging hypothesis testing. Finally, to sensitively respond the dynamic service changes, the dynamic sliding window algorithm in our model shifts the rating collection window into a new one and discards outdated ratings, reputation value is aggregated in the new window to mitigate the influence of obsolete ratings. Extensive experiments are conducted on widely used scenarios to demonstrate the efficiency and the effectiveness of our proposed model.