How best to form a single judgment out of many is an age-old problem in decision theory. Yet, it has so far not been studied how well a group of individuals will fare when it aggregates its own judgments. To answer this question, I propose “self-aggregation”, a method which asks individuals to vote for an option and to simultaneously provide a threshold of the number of people that would convince them of the opposite.
Self-aggregation picks an option if more people vote for it than the average threshold provided in the group. I compare the performance of self-aggregation, both theoretically and experimentally, with simple and confidence-weighted majority voting, and with the Surprisingly Popular Algorithm (SPA) recently proposed by Prelec et al. (2017).
In a model in which individuals behave as approximate Bayesians whose updating can be distorted by random noise, self-aggregation is predicted to outperform alternatives if distortion from Bayesian responses is minor. In an experimental test, respondents solve a binary decision problem in a stylized urn experiment in which responses and aggregation results can be directly compared to the Bayesian prescription. In the experiment, self-aggregation compares favorably to (simple and confidence-weighted) majority voting, but does not realize its theoretical advantage over the SPA. The results show that while the meta-cognitive abilities of individuals are challenged by complex methods such as self-aggregation and the SPA, responses contain sufficient information to outperform methods based on less challenging questions.