Instead of having a single AI system handle a complex task, developers can distribute the work among several specialized AI agents. In software development, for example, one agent might plan, another write code, and a third review it. Such multi-agent systems are already being used for applications such as software development, planning, and office assistance. To collaborate effectively, the agents need to exchange information and be able to revise their own assessments based on the results provided by others.
But this is precisely where a potential vulnerability arises. “In an open network, for example, AI agents from different providers could collaborate with one another. One of those providers might try to gain an advantage by configuring its agent to steer the network's collective decision in a particular direction,” Abedini explains. According to Abedini, such attacks work differently from conventional cyberattacks: “The agent neither takes control of the other agents nor changes their instructions. It simply uses the regular communication channels between the agents, persistently advocates a particular position, and thereby tries to change the other agents' assessments.”
Previous research has already shown that such attacks can work in principle. “The way the problem was viewed before was: We have one answer in the first round of deliberation and another in the final round, so the attack was successful,” Abedini says. “But the important question for us was what happens in between.”
The Mathematics Behind Opinion Formation
To describe this “in between,” the researchers turned to the Friedkin-Johnsen model from the social sciences. It mathematically describes how opinions change within a social network. The model takes into account how strongly someone holds on to their initial belief, how readily that belief can be changed by others, and how much influence the participants have on one another. The researchers transferred these variables to AI agents.
They were surprised by how well the model described the dynamics between LLM agents observed in their experiments. This means they can do more than simply observe that an attack works: They can mathematically describe the conditions under which it will succeed.
When a Single Agent Can Prevail
The results show that one important factor is how strongly an agent holds on to its initial position, a property the researchers call “stubbornness.” Under certain conditions, a manipulative agent with a high degree of stubbornness can steer the network's collective decision in its preferred direction. “How pronounced this behavior is can depend on factors including the language model being used, the task, and the prompts,” Abedini explains.
Stubbornness alone, however, is not enough. The agent also needs to have sufficient influence over other agents that are willing to revise their own assessments. How much influence an agent has also depends on the network topology. In a star network, a central hub communicates with all the other agents. If the attacker occupies this position, it is particularly powerful. If the attacker is located at the periphery of the network, by contrast, its influence is considerably weaker.
In a fully connected network, where every agent communicates directly with every other agent, there is no equivalent key position. What matters most here is how much weight the other agents assign to the attacker's statements. The calculations also show that the larger the network, the more influence a single attacker needs to manipulate the collective decision. “This means that the architecture of a multi-agent system itself becomes a security consideration,” Abedini says.
Collaborating Without Blind Trust
The mathematical model also helps the researchers identify ways to make such attacks more difficult. As one countermeasure, they tested a dynamic trust mechanism. A central authority occasionally gives the agents tasks for which it already knows the correct answer. Agents that repeatedly provide incorrect answers are assigned a lower influence weight. As a result, their statements have less impact on the other agents' assessments in future interactions. In the experiments, this mechanism reduced the influence of manipulative agents. "However, there is still room to improve the defense mechanism and understanding how it will work in more realistic agentic scenarios is one of the questions I want to explore in my future work,” Abedini says.
This gets to the heart of the challenge: AI agents need to listen to one another for collaboration to work. But they should not be persuaded by every stubborn neighbor who refuses to change their mind.
