Trade-off between quality and speed of collective discovery processes: from social simulations to empirical evidences with the Nobel Game
Date/Time
Date(s) - 06/09/2024
11:30 am
Categories No Categories
Conference on Complex Systems
Exeter, 2-7 2024
“Trade-off between quality and speed of collective discovery processes – from social simulations to empirical evidences with the Nobel Game”
In collaboration with José Segovia
Science as a social activity is prototypical of what we might call collective discovery processes: the collective production and curation of a common pool of cultural items meeting pre-defined quality standard (e.g. verified/invalidated by experiments). Both production and curation are done in a continuous and distributed way : 1) new cultural items are elaborated privately by members of the collective and shared when their authors are convinced they meet the quality standard ; 2) at any time, peers can re-examine publicly shared cultural items and reassess their level of quality. Another example of this type of situation is the development of open-source software, where the quality standard is generally the absence of bugs.
Understanding the factors that influence the quality and speed of collective discovery processes is of utmost importance for the design of efficient social organisations but the above definition hides a high social and cognitive complexity. Usually, the degree of confidence in the adequacy of a cultural elements to the quality standard depends on the time spent in the private part of the process by its author (How long did the scientist test her results? How hard did a programmer tried to debug his own code?). On the other hand, members of a collective are often incentivized for sharing their production asap or checking other’s contributions.
How will members of a collective manage the exploration/exploitation trade-off between devoting enough time to producing high-quality cultural items and publishing often enough to collect incentives? If the quality of the common pool of cultural items is guaranteed by peer review, how can we ensure that this review is carried out correctly?
The Nobel Game is a general framework based on game theory to study collective discovery processes \cite{chavalariasPopper1998}. It has been mobilized in \cite{chavalarias2017s} to study a stylized model of scientific discovery through social simulations where is has been demonstrated that the exploration/exploitation trade-off at the individual level is translated at the collective level into a trade-off between the quality of the common pool of cultural items and the speed of the discovery of the best items.
Here we present evidence from the first social experiment that exactly mirrors the mechanics behind this mathematical model of collective discovery process, confirming the first insights. Thanks to an online multi-players game, we where able to investigate the role played by the different incentives on collective discovery processes and to study the potential effect of changes in population size or difficulty of the space to be discovered. Among other things, we confirm that with human subjects, increasing publication pressure is detrimental to the average quality of the collective discovery process while at the same time increasing its speed.
