iTranslated by AI
A Simple Guide to Why Multi-Agent LLM Systems Fail
Why Multi-Agent LLM Systems Fail: Explained for Bunnies
👇️PodCast 5/7 18:00 Streaming
Introduction: Trouble in Rabbit Village
Hello! Today, let's talk about the troubles the bunnies are facing.
In Rabbit Village, bunnies with various roles live together. Everyone decided to work together to make a "Giant Carrot Cake," but for some reason, things aren't going well...
But why isn't it working? Even though they are all very smart bunnies, they often struggle when they try to cooperate... Isn't that strange?
Actually, a very similar problem is happening in the world of AI technology. Research has revealed that "Multi-Agent LLM Systems," where multiple AIs work together, should be wonderful in theory but often fail to function well in practice.
In this article, based on the research paper "Why Do Multi-Agent LLM Systems Fail?", we will explain this "mystery" in an easy-to-understand way by comparing it to the residents of Rabbit Village.
Let's look at why things go wrong and what the solutions are!
What is a Multi-Agent System?
First, let's explain "Multi-Agent Systems" using Rabbit Village as an example.
Rabbits are "Agents"
The bunnies living in Rabbit Village each have things they are good at or specific roles. A bunny who is good at cooking, a bunny who is good at calculations, a bunny who is good at gardening...
In the AI world, these AIs with specific roles are called "Agents." Each agent based on a Large Language Model (LLM) like ChatGPT can ask questions, make plans, and perform various tasks.
Collaborating Together: "Multi-Agent System"
In Rabbit Village, when there is a big job like preparing for a festival, the bunnies cooperate by using their respective areas of expertise. This is a "Multi-Agent System."
Similarly, in the AI world, a system where multiple AI agents cooperate to solve difficult tasks is called a "Multi-Agent System" (MAS). It is used in various fields, such as software development and research.
If a multi-agent system works well, the following benefits are expected:
- Efficiency of Division of Labor: Splitting up large tasks to solve them efficiently
- Expertise: Multiple agents bringing together their "areas of expertise"
- Knowledge Sharing: "Information sharing between agents"
- Flexibility: Finding solutions through trial and error in various situations
However, in reality, these benefits are often not fully utilized. Why is that?
Discovery of the Researcher Rabbits
Researcher rabbits wondered "why things aren't working" and conducted a large-scale investigation.
They looked into more than 150 tasks across five popular rabbit MAS (MAS frameworks) to carefully examine what kind of problems were occurring. The researcher rabbits then summarized 14 types of failure patterns based on their observations.
The rabbit MAS (MAS frameworks) investigated were as follows:
- MetaGPT Rabbits: Various roles of rabbits work in sequence, like an assembly line.
- ChatDev Rabbits: Tasks are divided into stages and handled hierarchically.
- HyperAgent Rabbits: A "Planning Rabbit" acts as the leader and gives instructions to other rabbits.
- AppWorld Rabbits: Each rabbit is skilled at using specific tools.
- AG2 Rabbits: Specialized conversation patterns and termination conditions are used.
As a result of this investigation, the researcher rabbits discovered 14 types of "failure patterns" (failure modes) and classified them into three major categories. Let's take a closer look at these three categories next.
Failure Type 1 in Rabbit Village: Instruction and Design Issues
The first major failure category is "Instruction and Design Issues." This happens when the bunnies' roles or task details are ambiguous from the start.
Failure to Follow Task Content (FM-1.1)
Rabbit Village Example:
Even though the village head asked for a chess game using classic piece movement, the engineer rabbit created a completely different system using coordinates like "(x1, y1), (x2, y2)."
This is a failure of "not understanding requirements." In the AI world, it is called a "failure to follow specifications."
Failure to Follow Role Definitions (FM-1.2)
Rabbit Village Example:
In a village meeting, the product planning rabbit, whose role was to generate ideas, took it upon themselves to make the final decision—a role meant for the CEO rabbit.
This is a problem of exceeding one's assigned role. In the AI world, it is called a "failure to follow role definitions."
Repeating the Same Work (FM-1.3)
Rabbit Village Example:
When plowing the field, a bunny might plow the same spot they already finished, wasting time.
This is the problem of repeating work that has already been completed. In the AI world, it is called a "step repetition" failure.
Forgetting Conversation History (FM-1.4)
Rabbit Village Example:
Even though they were asking "What's the weather like tomorrow?", they might suddenly start talking about something completely unrelated, like "Now, let me introduce some cooking recipes."
This is a problem of forgetting previous conversation content. In the AI world, it is called a "loss of conversation history" failure.
Unclear Termination Conditions (FM-1.5)
Rabbit Village Example:
When helping with cleaning, a bunny is told "Stop when it's clean," but they cannot judge "how clean is clean enough?"
This is a problem where they cannot determine when to finish the task. In the AI world, it is called a "lack of recognition of termination conditions" failure.
Failure Type 2 in Rabbit Village: Misalignment Between Agents
The second major failure category is "Misalignment Between Agents." This is a problem where the bunnies' "cooperation with each other" does not go well.
Resetting the Conversation (FM-2.1)
Rabbit Village Example:
Bunnies are having a meeting and are already getting close to a conclusion, but suddenly someone says, "Let's start with self-introductions," and they restart from the very beginning.
This is a problem where the conversation is restarted from scratch, ignoring previous progress. In the AI world, it is called a "conversation reset" failure.
Failure to Confirm (FM-2.2)
Rabbit Village Example:
The phone operator rabbit was told to "Please enter the password," but instead of asking back which password was needed, they entered one randomly, and the task failed.
This is a problem of proceeding without confirming things that are unclear. In the AI world, it is called a "lack of request for confirmation" failure.
Task Deviation (FM-2.3)
Rabbit Village Example:
Everyone was in a meeting to decide on furniture placement, but halfway through, they started talking about wallpaper colors and forgot the original purpose.
This is a problem of straying from the original objective. In the AI world, it is called a "task deviation" failure.
Information Non-sharing (FM-2.4)
Rabbit Village Example:
In a problem-solving meeting, a bunny who knew important information assumed, "I know this, so everyone else must know it too," and proceeded without sharing it.
This is a problem of not sharing important information. In the AI world, it is called an "information non-sharing" failure.
Ignoring Other Agents' Input (FM-2.5)
Rabbit Village Example:
In a joint project, someone ignored the opinion "I think this method is dangerous" and pushed through with their own plan anyway.
This is a problem of not considering input from other agents. In the AI world, it is called a "disregard for other agent input" failure.
Inconsistency Between Reasoning and Action (FM-2.6)
Rabbit Village Example:
A bunny thought, "It would be convenient to use this tool," but actually chose and used a different tool instead.
This is a problem where thoughts and actual actions do not match. In the AI world, it is called a "reasoning-action misalignment" failure.
Failure Type 3 in Rabbit Village: Verification and Termination Issues
The third major failure category is "Verification and Termination Issues." This involves problems with how bunnies check the results of tasks or judge when they should finish.
Premature Termination (FM-3.1)
Rabbit Village Example:
The bunnies preparing for the festival said "Good job everyone" and disbanded before everything was actually finished.
This is a problem where the work ends before it is truly completed. In the AI world, it is called a "premature termination" failure.
Failure to Verify (FM-3.2)
Rabbit Village Example:
The programmer rabbit made a chess game, but only checked that "the program compiled" and didn't check if it actually worked correctly.
This is a problem of not sufficiently checking results. In the AI world, it is called a "lack of verification / incomplete verification" failure.
Incorrect Verification (FM-3.3)
Rabbit Village Example:
In a complex calculation, a programmer rabbit answers, "2+2=5, that's correct."
This is a problem where verification was performed, but the verification itself was wrong. In the AI world, it is called an "inaccurate verification" failure.
Why Doesn't Rabbit Village Work Well?
Researcher rabbits analyzed these failure patterns and made several important discoveries.
Failures are Complexly Intertwined
It was found that failure categories don't just occur in isolation; various factors are complexly intertwined. In other words, the reason why the teamwork in Rabbit Village doesn't work well is not just a single cause, but a chain of various problems occurring together.
Also, the patterns of failures that tend to occur in Rabbit Village vary. This depends on differences in team composition, the bunnies' division of roles, and communication methods.
Failures Chain Together
A relationship where one failure "triggers" another was also discovered. For example, a problem with information non-sharing (FM-2.4) can later lead to inaccurate verification (FM-3.3).
Simple Solutions Don't Solve It?
Experiments showed that trying simple solutions (like improving instructions or clarifying bunny roles) does not solve the problems.
Researcher rabbits experimented with AG2 bunnies and ChatDev bunnies. When they clarified instructions and optimized the bunnies' division of roles, they saw a 14% improvement with ChatDev bunnies, but it still didn't reach a sufficient level of success.
This indicates that the problems in multi-agent systems are very deep, and superficial measures alone do not lead to a fundamental solution.
Ways to Improve Rabbit Village
The researcher rabbits classified the methods for solving problems into two types.
Immediate Methods: Tactical Approaches
-
Improvement of Instructions:
- Make instructions to the bunnies clearer
- Detailed descriptions of roles and tasks
- Addition of verification processes
-
Improvement of Conversation Patterns:
- Optimize the bunnies' "conversation order"
- Set clear termination conditions for conversations
-
Cross-verification:
- Bunnies checking each other's work
- A system where multiple bunnies perform verification
Long-term Methods: Structural Measures
-
Strong Verification Mechanisms:
- Introduce a third-party checker role
- Build verification methods corresponding to different stages
-
Standardized Communication Protocols:
- Establish rules for communication between bunnies
- Unify information and communication formats
-
Quantification of Uncertainty:
- Bunnies communicating "how confident they are"
- A system to request additional information when confidence is low
-
Memory and State Management:
- A system to reliably maintain conversation history
- A system to share progress status
It is important to combine these measures appropriately according to the various failure patterns. For example, for the failure of not following role definitions (FM-1.2), more explicit role definitions and optimized conversation patterns might be effective.
Summary: Aiming for a Smarter Rabbit Village
This study is the first to analyze in detail the reasons why teamwork in multi-agent LLM systems (Rabbit Village) fails. Fourteen types of failure patterns were identified and classified into three categories.
Crucially, these issues are not just about the capabilities of individual bunnies (agents) but involve Rabbit Village and its entire structure. These are problems that cannot be solved simply by the evolution of individual LLMs (Large Language Models).
Future research requires more detailed experiments and the development of specific solutions. It is necessary to consider ways to prevent critical failures from various perspectives.
To enable everyone in Rabbit Village to cooperate effectively, fundamental system design is required rather than mere superficial improvements. However, thanks to this research, the location of the problems has become clear.
Let's look forward to the realization of a smarter Rabbit Village (multi-agent system) that can solve various complex challenges in the future!
References
Original Paper: Why Do Multi-Agent LLM Systems Fail?
Discussion