When sufficient experience to make informed decisions is unavailable, expert advice can help us navigate uncertainty. As expertise evolves, driven by continuous learning in human experts or model updates in artificial experts, it is crucial to adopt adaptive approaches. Existing methods for exploiting non-stationary experts focus on competing with the single best expert. In contrast, this work harnesses the power of collective intelligence to facilitate better decision-making in the face of evolving expertise or dynamic environments. To achieve this, we propose the novel CORVAL approach which optimally combines the insights of multiple experts. By adapting to drifts in expertise, our novel approach can surpass the performance of the single best expert as well as previous approaches. Empirical evaluations on a diverse range of non-stationary problems, including active learning applications, showcase the improved performance of our approach in collective decision-making scenarios.
Abels, A, Trianni, V, Nowé, A & Lenaerts, T 2025, 'Collective Intelligence in Decision-Making with Non-Stationary Experts', Journal of Artificial Intelligence Research, vol. 83, 9, pp. 1-27. https://doi.org/10.1613/jair.1.16228
Abels, A., Trianni, V., Nowé, A., & Lenaerts, T. (2025). Collective Intelligence in Decision-Making with Non-Stationary Experts. Journal of Artificial Intelligence Research, 83, 1-27. Article 9. https://doi.org/10.1613/jair.1.16228
@article{4bf1e8e4f67146f0927d75bfb7ca2def,
title = "Collective Intelligence in Decision-Making with Non-Stationary Experts",
abstract = "When sufficient experience to make informed decisions is unavailable, expert advice can help us navigate uncertainty. As expertise evolves, driven by continuous learning in human experts or model updates in artificial experts, it is crucial to adopt adaptive approaches. Existing methods for exploiting non-stationary experts focus on competing with the single best expert. In contrast, this work harnesses the power of collective intelligence to facilitate better decision-making in the face of evolving expertise or dynamic environments. To achieve this, we propose the novel CORVAL approach which optimally combines the insights of multiple experts. By adapting to drifts in expertise, our novel approach can surpass the performance of the single best expert as well as previous approaches. Empirical evaluations on a diverse range of non-stationary problems, including active learning applications, showcase the improved performance of our approach in collective decision-making scenarios.",
author = "Axel Abels and Vito Trianni and Ann Now{\'e} and Tom Lenaerts",
note = "Publisher Copyright: {\textcopyright} 2025 Copyright held by the owner/author(s).",
year = "2025",
doi = "10.1613/jair.1.16228",
language = "English",
volume = "83",
pages = "1--27",
journal = "Journal of Artificial Intelligence Research",
issn = "1076-9757",
publisher = "AI Access Foundation",
}