2025: The Year of AI Agents? Navigating Responsibilities and Risks

2025 is being called the "first year of AI agents," and the movement to utilize AI agents across all kinds of companies and services is heating up.

NVIDIA CEO Jensen Huang predicts that AI agents will become rapidly adopted from 2025 onwards as a technology that revolutionizes daily life and business operations.

However, simply saying "we are introducing AI agents" brings up major issues regarding the location of responsibility and risk management.

In this article, I would like to consider what kind of risks and challenges arise from the introduction of AI agents, starting from the structure of responsibility when delegating work to machines and systems.

---

1. Automation and the Location of Responsibility

The Relationship Between Automation and "Delegation of Responsibility"

• Automation is the act of entrusting tasks to machines or systems that were originally the responsibility of humans (personnel).
  Let's consider a bakery as an example. In a privately owned bakery, a craftsman oversees everything from preparing the dough to baking, ensuring quality through direct observation. This is a "state where humans take direct responsibility for all processes."
  However, as the business grows and becomes a factory, parts of the process are automated by machines. As a result, some tasks are left to machines, and the areas directly managed by humans decrease.
  However, "leaving it to machines" does not mean "responsibility disappears"; the ultimate responsibility for quality and risk management remains with humans.

The Human Role in Automation

• As automation advances, the human role changes from "doing the work directly" to "monitoring and judging."
  A factory manager understands the entire process and has a mechanism to trace the cause back through the process if a problem occurs. Ultimately, they must check things like "where in this process did the mistake occur?" or "which machine setting was incorrect?"
  In other words, doesn't this suggest that even as mechanization and automation advance, a structure is necessary where humans can take responsibility, at what stage, and in what way?

---

2. Responsibility Structure Assumed with AI Agent Introduction

Responsibility of the Instructing Party

• AI agents propose and execute optimal actions, but humans specify the goals and how they are used.
  It is ultimately the introducing company or the person in charge who gives orders to an AI agent, such as "I want you to achieve this goal" or "I want you to complete this task." Therefore, even if an agent creates a plan and executes it automatically, the one who is ultimately held responsible is the "human who gave the instructions."
  If used at a personal level, it might be easier for the consequences of failure to be contained within one's own damages. However, when used in business, it could potentially affect customers and business partners.

The Weight of Responsibility in Business Use

• When providing services as a corporation or entity, mistakes and troubles are directly linked to corporate responsibility.
  For example, if you provide a feature where an AI agent orders products, and a massive amount of excess inventory is ordered due to some input error, who bears the loss?
  In many cases, the business operator providing the service using the AI agent will be held responsible. You will then need to investigate "why the mistake occurred" and "which process or input information was flawed." This is the same structure as when delegating tasks to humans, but unlike humans, you cannot hold an AI agent directly accountable.

---

3. Risk Assessment and Introduction Decisions

Whether to Accept Risk or Manage It

• The most important point when introducing AI agents is "how much risk can you tolerate?"
  When large enterprises adopted cloud services, there were initial concerns like "Is the security safe?" or "What will happen to data protection?" However, the benefits of the cloud (such as cost reduction and operational efficiency) were significant enough that adoption accelerated rapidly in the past.
  Similarly, for AI agent introduction, if you prepare mechanisms to minimize risk (setting up alerts, threshold monitoring, and establishing procedures for logging and analyzing issues when they occur) and if benefits outweigh the risks are expected, the pace of adoption could accelerate significantly.

Monitoring Systems and Governance for Agent Introduction

• It's not just "implement and forget"; post-introduction monitoring and analysis systems are crucial.
  While it is difficult for humans to constantly monitor AI agents as they execute automated tasks, preparing a system that sets key points and thresholds to trigger automatic alerts can help detect issues early.
  Also, if a problem occurs, it is necessary to track "when, where, and how" the AI agent made its decision. Governance aspects, such as how detailed audit logs and execution history are obtained, stored, and referenced, will also become extremely important.

---

4. Conclusion

1. Automation does not mean giving up responsibility; rather, it requires redefining the structure of responsibility.
   By introducing AI agents, you can expect improved efficiency and sophisticated analysis in processes previously handled manually. However, if you outsource the entire process to AI, there is a risk that it will become unclear who bears responsibility when problems arise.
   As a result, since the implementing company will be held accountable, it is necessary to clarify "how responsibility is shared and which steps will be monitored" before implementation.

2. The key to adoption is how much risk you can tolerate.
   While introducing AI agents carries risks, the potential benefits are also significant. It could lead to not only drastically increased business speed and efficiency, but also resolutions for labor shortages and long working hours.
   If you can determine that the benefits outweigh the risks after properly implementing risk measures (monitoring, log management, and clear assignment of responsibility), I believe adoption will accelerate rapidly.

3. Monitoring and governance practices for AI agents are the key.
   Going forward, "monitoring practices" regarding how to manage agents will likely mature.

   • Methods for obtaining auditable state logs
   • Algorithms and systems for detecting malfunctions or misuse
     As these are developed, an environment will be created where companies can adopt AI agents with peace of mind, accelerating their implementation.

---

Final Thoughts

There is no doubt that AI agents will continue to evolve and become more widespread. However, when it comes to their introduction, decision-makers might not approve them if the reasoning is simply "it's convenient."
When utilizing agents at a business level, it appears necessary to consider how to prepare for agent-specific risks, similar to the audit and governance efforts required when introducing other systems.
Just like the early days of cloud adoption, even if concerns are high initially, if more companies fully understand the benefits and implement corresponding measures, adoption may spread rapidly. Since things could change significantly depending on future technological advancements and the establishment of guidelines, I intend to continue monitoring this space.