Cooperative AI Foundation – Multi-Agent Risks from Advanced AI

The rapid advancement of AI technology brings with it not only vast benefits but also new risks, particularly in the form of multi-agent systems. These systems, where AI agents interact autonomously and adapt their behavior, are becoming more prevalent due to the use of sophisticated models across various domains. While currently rare, these multi-agent systems are already being utilized in high-stakes tasks such as asset trading and military operations. As their prevalence increases, it is essential to evaluate and mitigate the unique risks they pose. The Cooperative AI Foundation highlights the need for new methods of evaluating the cooperative capabilities, biases, and vulnerabilities of these systems, as well as developing technical advances for mitigating risks. Collaboration among various stakeholders and disciplines will also be crucial in addressing these complex challenges. By identifying key risk factors and failure modes specific to multi-agent systems, the foundation aims to guide future efforts in AI safety, governance, and ethics.