iTranslated by AI
re:Invent 2025: Scientific Challenges and Practical Best Practices in the Convergence of Agentic AI and Responsible AI
Introduction
By transcribing various overseas lectures into Japanese articles, we aim to make hidden, high-quality information more accessible. The presentation we're focusing on for this project, driven by that concept, is this one!
For re:Invent 2025 transcript articles, information is compiled in this Spreadsheet. Please refer to it.
📖re:Invent 2025: AWS re:Invent 2025 - Agentic AI Meets responsible AI: Strategy and best practices (AIM422)
This video explains the convergence of Agentic AI and Responsible AI from both scientific challenges and practical aspects. Michael illustrates a spectrum of agent autonomy levels and explains the concept of planning agents and the significance of reasoning in embedded spaces. He particularly highlights unsolved scientific challenges such as the impact of embeddings on privacy and security, the replication of human behavior in inter-agent negotiation, and the difficulty of implementing subjective common sense. Peter emphasizes a crucial mindset shift: the difference between statistical inference and rule-based logic, and discusses the necessity of an evaluation stack and the characteristics of Responsible AI. Furthermore, he introduces practical best practices using the Well-Architected Framework, including the issue of trust between agents and infrastructure stewardship, advocating for the importance of starting with narrowly defined use cases.
- This article was automatically generated, preserving the content of the existing lecture as much as possible. Please note that it may contain typos or inaccuracies.
Main Content
Core Concepts of Agentic AI: A Spectrum from Conversational to Autonomous Agents
Hello everyone. Welcome to the session on the convergence of Agentic AI and Responsible AI: Strategy and Best Practices. Following the theme of re:Invent, where everything agentic seems to be front and center, today I will start by giving an overview, and then I'll hand it over to my colleague Peter Hallinan, who is sitting here with me. I'll not only talk about the fundamentals of Agentic AI but also touch a bit on the scientific challenges behind it and anticipated solutions, after which Peter will delve deeper into the industry's evolution and practical aspects.
Covering the agenda, I'll begin by aligning everyone's understanding of what agents are and what we mean by Agentic AI. Then, I'll spend most of my time discussing the scientific frontier, or more precisely, the challenges. After that, I'll pass the baton to Peter, who will talk about more practical topics.
To illustrate something like a spectrum where Agentic AI can be positioned, I think the word
agentic and the term Agentic AI are probably a bit too coarse a term for the vast spectrum they cover. Many of you are familiar with large language models, and probably use them regularly. We are all accustomed to the phenomenon of interacting with large language models on a conversational, turn-taking basis. The logical extreme of Agentic AI, I think, is an LLM-based system that has unlimited access to external resources and information, and can take significant, perhaps irreversible, actions in the real world. For example, think about making financial transactions on your behalf, signing contracts on your behalf, or even more significantly, autonomously performing military actions or medical decision-making.
At present, I don't know of any Agentic systems that touch or even approach that logical extreme, which is why I think we need language for what's in between. But this chart positions systems along a particular dimension: the level of autonomy the agent has. At one end, you have turn-taking dialogue with an LLM to complete a task or get information. The middle, which I think will be the most active area in the near future, is what I'm going to call a planning agent. These are agents that don't need to be hand-held and guided on a turn-by-turn conversational basis. You can give them a high-level goal, ask them to plan and execute the steps towards it, and you expect them to move forward. They may get stuck, revise their plan, and continue.
What are some examples of agents I have in mind? Well, I think one example that people often think about is an agentic executive assistant. What do I mean by an agentic executive assistant? This is someone who has access to and knows everything about your schedule, your calendar, your commitments, your travel preferences, the conditions under which you might be willing to pay to upgrade to first class on a long-haul international flight, and actually has the ability to make those transactions for you, to go and seek out the tickets you might have purchased yourself and purchase them for you.
Advanced Planning Agent Examples: Experiencing "turbo mode" in math paper writing
Advanced planning agents, as I said, are agents that can receive a high-level specification and then plan the steps to execute that task. Let me diverge here for a moment and tell you about a small anecdote that I experienced this summer. As you may have noticed on the title slide, I am in a position known as an Amazon Scholar, which means I split my time between Amazon and the Department of Computer Science at the University of Pennsylvania. I am a lifelong machine learning researcher. I started working in the field in the late 1980s, so suffice it to say, I've seen an amazing amount of change in the last 40 or so years.
This summer, a colleague of mine in the department, a visiting graduate student, and I actually wrote a roughly 50-page math paper from scratch, with the help of coding-based LLM tools. And I want to tell you a little bit about that experience. Many of you are, I'm sure, very familiar and probably use them regularly, but coding agents, where you basically say, I want you to write Python code to do this thing, and you interact with it in natural language, and it actually writes the code and runs the experiments for you. I think it's really over the last year that these systems have also gotten quite good at proving theorems.
So we wrote a paper where we described what the high-level outcome would be, what the theorem statement would be. And we didn't know all the details, because you don't know all the details until you actually prove the theorem. So things like what the precise convergence rates of the proposed algorithm would be, we didn't know, but we basically knew we wanted to prove that it converged to something good, and we wanted to quantify that speed. And we did this in an alternating fashion. We would say, here's the high-level statement of the theorem, here's what I think the steps of the proof are, for this step I think you need this tool from probability theory, go and try it. And we would interact with it, and correct it, and go back and forth.
At some point during this project, my colleague noticed that there was a little button at the bottom of the interface that said "turbo mode". And it turned out that in turbo mode, all you had to do was say, here's the theorem I want you to prove, I'm going away for a few hours, try and prove it. And as those of you who have worked with these systems know, they talk to themselves. And so suddenly, there would be a scroll of grayed-out text, and you would see the model talking to itself, saying, oh, that didn't work, I see what I did wrong, I should go back and correct it. Sometimes when we did this, we would come back and it was still mindlessly spinning, and it had gotten stuck in a very bad way. And then other times, we would come back, and it had actually done exactly what we asked it to do, 45 minutes after we had left.
I think we will see a lot of these kinds of agents in the near future, where you just specify a high-level goal, perhaps with as much detail as you can specify, and then let the model run and plan its own steps, backtrack when necessary, and move forward. So let me now talk about what I think are some interesting scientific challenges for agentic AI. As a scientist, by the way, I'm usually most comfortable talking about known science, what's out there about a particular topic. In this case, I can't do that, because despite the buzz around agentic AI, not just at re:Invent but in the scientific community, there's not a lot of literature specific to the topic of agentic AI. And that will obviously change very, very soon.
Embedding Space: The Native Language of AI Agents and Its Implications
So let's start by talking about the native language of AI agents, which is embeddings. When we interact with large language models in natural language, like English, the LLM isn't actually dealing with that English content directly. What it's actually doing, every time you give an input to an LLM, whether it's in the form of an English prompt, or an image, or pointing it to a codebase, or pointing it to all the prior documents in a legal proceeding, the first thing it does is it translates that data into a much lower dimensional space, albeit still a high dimensional space. And the primary purpose of that translation is to preserve similarity between content.
So, for example, in an image embedding model, two pictures of different family members will be placed closer to each other in embedding space than either of those family pictures would be to a picture of a landscape. Two romance novels would be placed closer to each other in language embedding space than either of them would be to the manual for a car. So the high-level idea of embedding is that you're re-representing the data, not as the raw data itself, but in a way that captures the similarity between the data. And incidentally, some of you may have had the somewhat unsettling experience of an LLM appearing to completely lose the thread of the conversation with you, or suddenly returning to a topic that you discussed many hours or days prior, completely out of context. And that's actually because there isn't a memory of prior conversations by the LLM, there are only embeddings of prior conversations, and that leads to these sort of cognitive quirks that we sometimes see. They're decreasing over time, but they're still there.
And because agentic AI is based on LLMs, we should expect that they're fundamentally going to be reasoning in embedding space, not in the surface input space of language or images or code that we give them. And I think this has several interesting implications. One of the difficult things about AI, and LLMs, and deep learning in general, is that while these re-representations of data are very powerful, we're still very poor at understanding what's actually happening and what it means. In particular, if we were to open up your cerebral cortex and point to a particular neuron in your cortex, save for very simple cases like low-level visual processing, in most cases, we wouldn't know what the particular purpose of that neuron actually is.
Likewise, we don't really know what the meaning of embedding space is. In this cartoon on the right, it sort of depicts the idea that multiple different users have eaten at multiple different restaurants and perhaps given different ratings or preferences. And you have that raw data, and you want to map it into an embedding space. The one thing that's inaccurate in this particular cartoon, for the sake of simplicity, is that the axes are labeled with particular attributes that we know about, like how's the quality of the food, how's the ambiance of the restaurant.
But in embedding space, the axes are fundamentally completely unlabeled. They have no direct semantics that we can interpret. And one of the short-term implications of this is that when we think about agents going out and consuming or finding content, or interacting with other agents, while they're very good at interacting with us in English, they probably won't be interacting with each other in English, and they won't need English to understand content.
For example, I predict that at some point in the not too distant future, every website on the internet will have a little place that basically says, if you're an agent, here's the pre-computed embeddings of all the content on this website. There's no point in every agent that accesses that website having to re-compute the embeddings for themselves. You may as well just pre-compute them and put them there. This is not viable today because these embedding models are proprietary and they're not standardized. But I predict that there will be some standardization that takes place, and this will be possible, because it's one of those things that just makes practical sense.
Implications of Embeddings for Privacy and Security
Another area where embeddings are important and we don't understand them well yet is with things like privacy and security. Given that embeddings take your original data and map it into an abstract space whose primary goal is to preserve distance, your data is going to get distorted through that translation. And we don't understand the shape or the characteristics of that distortion.
In particular, in our minds, we have divisions about different types of data. So, for example, when I think about my data, I think my financial history is generally pretty separate from my medical history. These are different types of data for different functions. There's no guarantee that when my data gets mapped into embedding space, those crisp separations that are in my mind still exist. And so there are privacy implications to this.
In particular, when your agentic executive system assistant is out looking for plane tickets for you and deciding whether or not you want to pay for a first-class upgrade, if a human executive assistant were doing that, they would not be taking into account how your stock portfolio has been doing recently. Although in principle, that is financial information that could affect your willingness to pay for a first-class ticket. But I think, without appropriate guardrails in place, there's no particular guarantee that your agentic executive assistant will recognize those same boundaries because of this fuzziness of different data types and the fuzziness of boundaries in data space. And obviously, there are security implications to this as well.
There's a very small literature; a couple of papers the last time I checked, on the very good question of, if you have privacy-sensitive data, and you compute an embedding into a lower-dimensional space, how much can you reverse engineer about the original raw data just from the embedding? The reason there are papers about this is because the answer is, you can reverse engineer it to some extent. And I think we need to start rethinking fundamental principles and models around privacy, security, encryption, and so on and so forth, in the world of agentic AI.
Inter-Agent Negotiation and the Ultimatum Game: Replicating Human Behavior and Collective Risk
I've used several examples of your agent going out and doing transactions on your behalf, but we can also envision a future where your agents go out and negotiate on your behalf, so not just engage in transactions. Much like there have been websites over the years that basically crawl and try to find the cheapest flights for you, there will be agents that try to do that for you. So you can imagine your agentic AI assistant bartering and negotiating with another agent to make your travel arrangements. What would that look like?
Well, there's a small but rapidly growing literature, not just in the AI and machine learning community, but also in the social sciences, where people are doing research into replicating or studying analogues of human behavior in agents and LLMs. Let me tell you about the first example I'm aware of, from a colleague and friend of mine about four years ago.
In behavioral game theory, there's something known as the ultimatum game. What is the ultimatum game? The ultimatum game is, first of all, a very simple mathematical problem that I'll describe to you, but more interestingly, it's also a behavioral experiment that economists have run for decades in many cultures, varying all sorts of conditions of the setting. But the basic setup is as follows: you bring two players into a lab, let's call them Alice and Bob. And you say, Alice, you get to make the first move, and you can propose any split you want of $10 with Bob. And Bob gets to respond, and Bob can either accept the split that Alice has proposed, in which case we actually pay out the two participants their respective shares. Or Bob can reject it, in which case both players get nothing.
What would game theory or microeconomics predict about this? Very, very simple. Assuming Alice and Bob are perfectly rational beings, Alice should reason as follows: Since Bob is perfectly rational, I should offer Bob the smallest non-zero amount, for example, a penny. So Alice will propose $9.99 and a penny. Bob is rational, so a penny is better than not having a penny, so Bob accepts. This isn't very interesting.
The interesting thing is that when you run this game in the lab, nothing close to this happens, and what does happen is remarkably consistent across many conditions and many cultures. What happens is almost always, like 90% of the time, Alice offers Bob something between three and five dollars. It's very rare that it's more than five dollars. There's already sort of a preconception for both players that Alice, by virtue of having the first move, might be entitled to more than half the pie. And if Alice's offer to Bob is in that three to five dollar range, Bob almost always accepts. The acceptance rate is very high. In the 10% or so of cases where Alice is more aggressive and offers Bob something less than three dollars, Bob's rejection rate jumps up to almost 100%.
You can look this up. This experiment has been run hundreds, probably thousands of times. One of the interesting things about this experiment is, first of all, it departs very dramatically from the mathematical prediction. The other thing is that it's very consistent across different subjects and cultures. So it's almost as if we all have this neurologically hardwired sense, a shared hardwired sense, about what's fair in this game, even though most of you had probably never heard of the ultimatum game until this very moment.
My colleague basically replicated this experiment using LLMs instead of Alice and Bob. And he prompted one LLM, and to match the sort of diversity of human subjects that you would get in a lab experiment, he also gave them personas. So he prompted one LLM and said, your name is Melissa, you're a 37-year-old medical technician, you live in Lexington, Kentucky. You can propose any split. And then he prompted another LLM. And guess what happened? The LLMs actually statistically matched human subject behavior that I alluded to almost perfectly.
So this is an area that is becoming an area of attention, not just in AI and machine learning, but more broadly in the social sciences. And people are starting to study what happens when agents negotiate and transact with each other, and some interesting phenomena are starting to emerge. In general, they're very good at tracking human behavior, and they're also very good at tracking human frailties. For example, a recent paper about a year ago showed that when LLMs engage in price negotiation games, they will inherently arrive at outcomes that from a regulatory perspective would be considered collusion or price fixing. This is interesting, and I think something that's worth taking into account when we think about agentic AI.
It's not enough to think about what happens with individual agents, much like we think about collective behavior and emergent phenomena in game theory and finance. We also need to think about that with agentic AI. You might wonder how we think about things like mob behavior in the stock market, or systemic risk that led to the 2008 financial crisis, if we get to a world in the not too distant future where many activities, including financial and strategic activities, are mediated by agents. These are examples of cases where the behavior of individual agents may seem perfectly natural or rational, but you get to very undesirable collective outcomes.
The Challenge of Subjective Common Sense: Difficulty in Incorporating Personalized Heuristics into AI
The last topic I want to touch on is probably the fuzziest and the one that I really don't know the current science for, but it's the challenge of what I call subjective common sense. Let's first talk about common sense, not subjective common sense. I'll use the fact that I've been in this field for a long time to tell you that for the longest time, AI and machine learning were plagued by the problem of instilling everyday common sense into models. Common sense means facts that we would all agree upon, even if we had never thought about them before.
So, for example, if I show you a series of pictures where there's a glass of water sitting on a table, on a surface, and then I move it in one direction, for example, to the right. We would all still agree that it is a glass of water sitting on that surface. The fact that I moved it somewhere else on the table doesn't change the fact that it's a glass of water sitting on the table. And we would also all know that if I were to tip it over, it's still a glass on the table, but it's no longer a glass of water.
And for a long time, since the beginning of the field, people in AI have struggled with the problem of how do we get models or AI systems to have this kind of ability to do common sense everyday reasoning that any human can do, not just the ability to do very particular tasks like facial recognition very well. And somehow, over the last 15 years or so, that problem has largely been solved. I have a colleague at Duke who, as far as I can tell, spends all of his time posting examples on social media of LLMs failing at basic common sense things, but for the most part, this has gotten dramatically better. And I think that's just due to the fact that if models see enough facts and images of glasses on tables that are either sitting upright or tipped over, they learn this.
Subjective common sense are heuristics about your own behavior that make perfect sense, that are perfectly well-justified, but are particular to you, and are not shared like our knowledge of a glass of water on the table. And I'll use my own behavior around leaving my door open or closed, locked or unlocked as an example, but I'd ask you to think about your own behavior on this front. And it's actually quite complicated. In my office, when I leave at the end of the day, I lock my office. When I'm in there during the day, sometimes I'm in my office and I leave my door open. Sometimes I'm in my office and I close my door, but I don't lock it because I'm in a meeting or I want to concentrate.
When I'm just out on the floor nearby or at the whiteboard in the hall, I leave my door open and unlocked, to signal that I'm nearby but not there. And you probably have similar heuristics that differ from mine, incrementally. And I think giving agentic AI this kind of personalized subjective common sense is going to be a really big challenge, because unlike things like facts about glasses of water on tables, this can't be learned from data. There isn't a vast trove of data about my particular habits around doors and locking doors.
And if this example seems too far afield or abstract, replace doors and locking doors with accounts and passwords. You probably choose to share the password for your Spotify account or your Netflix account with other family members or close friends, but you probably wouldn't do the same for your bank account.
And where agentic AI seems to be going is I kind of feel like we're back in the 1970s. We didn't have any idea how you could instill general common sense into models. And it turned out that machine learning on sufficiently large data was a good solution to that problem. But I don't think that solution is going to work anytime soon for agentic AI.
Industrial Evolution of Agentic AI: Three Trends from Present to Future
So with that, let me hand it over to Peter, who will talk about more practical topics and the evolution of agentic AI, particularly where it differs from LLM-based generative AI. My name is Peter Hallinan, and I'm a Director of Responsible AI at AWS. I lead a team of scientists and engineers and other folks who are interested in grappling with open science problems and translating their solutions into internal team and customer-facing best practices. So we just talked a little bit about scientific challenges. I think they're becoming clearer. So let's talk about some of the other challenges we have.
Okay, so where are agents headed in the near future? Let's take a look. There are a lot of effective agents today, or effective things that are called agents. We have a few from Amazon. We have Kiro, the SageMaker Model Customization Agent, the SageMaker Data Agent. We have, I've recently blogged about it, the Compliance Screening Agent. In all of our businesses, we have to be very careful about transactions, and so we have internal agents now that do that. But what are the key characteristics of these agents? It's that there's very close human interaction, and there's single vendor accountability. So the world that we were just talking about, a world where there's lots of critters running around doing useful work for us, is slightly different than the world that's here. The world that's here now is generating some value, quite a bit of value, but there's a journey to go.
Okay, so how will things evolve over the next year or two? Well, first obvious one is more task types. So, for example, say you live in the United States and you want to get a business class ticket to Paris in December using miles. You can do that, but the airlines release those tickets in dribs and drabs over time on their own schedule. It's a task you have to continually check again and again, and it's a bit of a pain, and you forget, and then, oh, shoot, I missed it. This is the kind of thing where an agent should be released to do something over a period of time. It's not a quick response. Another example might be, you have an interesting social media conversation, but you want to wait for some comments to accumulate before you invest your time. You don't want to check it again and again. You might call this a fast to slow trend. You could call it short term to long term, but it's just different types of tasks.
And this is the second trend. And by the way, the three trends I'm about to talk about are correlated, they're not independent. But the second type is, not all tasks need to be explicitly specified by the user. You can trigger log analysis from tickets. We have DevOps agents that do that kind of thing. Or you could have an agent listening to traffic information and, in this ideal personal assistant kind of view, knows you can't get there in time, so it updates your restaurant reservation. These are inferred tasks, and there's an implicit assumption here. It's that the agent has access to information and is tracking it, so you don't have to. You could think of this as reactive to proactive. The third is more connectivity to third-party workflows.
So you have this nice little remote restaurant here. It's on a beautiful beach. It has a phone, it's a landline, an old-fashioned one. And the cutting-edge travel agent is supporting you, and they call. And another person has a cutting-edge travel agent, and a third person has one, and then a whole lot of other people, and they essentially DDoS this poor restaurant. They can't keep up anymore. And this connectivity to third-party workflows is implicitly in all the magic and excitement of agents. When we talk about all the drudgery we can take out, we have to understand what the impact is going to be on those other systems that we're interacting with.
So what happens to the restaurant if Travel Agent One cancels? It really affects their business. There aren't that many tables, so you start to wonder, does that offline restaurant really have to implement their own Maître D' agent, or do they go out of business, or how do they deal with that? But these are business decisions they're going to have to face. And this is true for online workflows too. Going back to the airline example for a moment, some airlines have a policy where you can buy a ticket, and for 24 hours, you can change your mind. Now, this policy is created to provide a benefit to travelers. It's a good policy, but it's very carefully decided by data about what's the likelihood of rebooking, and who's going to risk having a seat now that's taken, that they can't sell, and so on.
As soon as you put agents into that, the calculus changes. It definitely changes. So how do these policies adapt? So we talk about agents, but we also have to talk about everything that agents interact with, whether they're agents or not, and how that change happens. And so we see some general trends from where we're really good today, in well-understood workflows across many domains, to this future with these sophisticated goals. On the x-axis, you have task complexity, or user request complexity, from simple to complex. On the y-axis, you have the robustness, resilience, and strength of the infrastructure to support agents.
And if you think about it, the fastest slow ones, well, they had to stay up continuously for a long period of time. That's not the exact same thing as just interacting with an LLM. It might be okay if it's down the next day, but this is really an infrastructure that has to support it. Anyway, that's the general trend that's happening now. So as this unfolds, how do we realize the benefits while minimizing the risks?
The Difference Between Statistical Inference and Rule-Based Logic: Two Mindset Shifts
Okay, so there are a couple of things. The first observation I want to make is, when Generative AI burst onto the scene a couple of years ago, it really expanded the number of people who were involved in AI, from a very small community to a very large one. And there was, dare I say, a lack of understanding about the difference between statistical inference and rule-based logic. And there's a big mindset shift that the entire industry is still in the middle of transitioning from. And it's people realizing that AI is not just clever software. It's actually a different kind of beast.
So I'll just highlight a couple of changes here. Obviously, traditional software is rule-based logic. If you have a statistical model, it's based on statistical inference. I'll get to what some of the deeper implications of that are in a moment. How you specify changes. In traditional models, you can write down in English. In AI, you use a dataset to specify. So this goes back to Michael's point earlier about, where is the dataset that encodes your full range of personal preferences, and do you even want such a dataset stored? That's a question too. But a dataset is key here.
And the third point is, in traditional software, you expect it to work out of the box, right? The developer is responsible for the quality of that product. But in statistical models, that's different.
The difference is significant. Developers predict the type of input distribution they will receive, but they cannot know it for sure due to privacy constraints. We don't see the prompts going into the Bedrock models. What you see as a result of this is that the people who deploy, and sometimes the end users, also have to test, because they're the ones who understand their particular data.
And this is the fourth point. When you release version N+1 of your software, you expect it to work better for all inputs than version N. But in statistical models, you expect it to work better on average, and that's a big difference. It might be a chosen metric, not average, but you can release a better performing model and still have some users experience a decrease in performance.
The last point I want to make about this kind of mindset shift is that in rule-based software, traditional software, there is a singular explanation for how you got a particular output from a particular input. The code can be implemented in different ways, but the chain of logic is the same. Whereas on the statistical side, it's different. In fact, when you release a model, that model has logic internally, but there are tens of thousands or millions of models that you didn't release, that perform exactly the same, and they have other chains of logic internally that might be slightly different, or might be very different. So the explainability problem becomes more complicated.
So there are a couple of implications for this. First and foremost, on the AI side, you have two stacks to worry about. If you think about an AI system, you're going to improve it version by version on the x-axis, and you're going to look at its performance along a particular metric on the y-axis, right? And you're going to test it with one dataset, and you're probably going to get this kind of performance. But if you test it with a different dataset, you're going to get a different performance trajectory, right? And if you test it with a third dataset, you might get it going down altogether. Performance is a function of the system and the evaluation dataset. It's not a function of the system alone. You may have heard us talk about this before, but we are reiterating this point again and again. But we still need to make this clear.
Because as a result of this, if you're a developer of an AI solution, whether it's generative or traditional, whatever, you're going to have a system stack and an evaluation stack. Your evaluation stack as a developer represents what you expect downstream people to use, but then downstream people must have their own stack with their own data to check what's going on. As a whole, we as an industry are moving in this direction, but this is not yet a well-established design pattern.
What other implications are there? And this is where we get into our traditional responsible AI slide. The reality is that statistical models have a variety of technical properties. It's not just overall performance. It includes issues around AI privacy, AI security, safety, fairness, accuracy, and so on. There are a lot of these things, and the model that's released has these properties regardless of whether you as the developer made decisions about those properties. And this is a very important point. So if we go back to the downstream stack here, they can measure you on properties that you may not have made explicit design decisions about.
And when we think about responsible AI, what we're really thinking about is helping people make explicit decisions about balancing the desired benefits and the potential risks of a service. That's the goal. We want these decisions to be explicit, not implicit.
And the last point is probably pretty self-evident, but from a product management perspective, life just got a little more complicated. In addition to all the things you had to worry about with traditional software, now you have these other properties. Now, that doesn't necessarily mean you have to optimize for all of them. It's perfectly fine to make a decision that you'll deal with this later or whatever, as long as you disclose it. But these are still additional considerations.
So that was the first mind shift. And we as an industry are still completing this mind shift. And now agents are coming in. Because why wait? Life is exciting. What's the second mind shift brought on by agents? There's actually one. Let's go back to our trusty science friends, Alice and Bob, as we consider these issues. Before Agent Alice interacts with Agent Bob, there are things Alice wants to know about Bob. Who are you? Who owns you? How can you help me? The reason is, Alice is out there doing real work, and there are decisions she has to make. How much information should I share with you? That's important. How do I negotiate a fair exchange of resources with you?
For example, if you take a zero-trust stance, is it like walking into a 7-Eleven and just buying and leaving? Probably quite different. How do you enforce the terms of the exchange? How much time, cost, or information is Agent Alice willing to invest in this particular transaction? What if it gets detailed, and you're going back and forth about a lot of things, and the agent's budget is ticking down there? Transactions ultimately cost money, so trust is quite important here.
Now, this is not the only situation where issues of trust will arise. And there are various mechanisms for addressing these. There are norms. As an example of a norm, on the internet, you have a robots.txt file. There's no law about robots.txt. It's a norm, and it has some problems, but it's worked relatively well. There are also standards. MCP is an emerging standard. TCP/IP in the early days of the internet, some people might regret that TCP/IP has lasted this long, because it has a doubling problem. And there can be existing AI regulations.
But the mindset shift I'm talking about here is that these are not necessarily core technical problems. These are problems about privacy, about the exchange of information, about what should happen. There can be research that supports the optimization of these decision-making processes, but people themselves are making these kinds of decisions all the time. They may not be perfectly optimal, but they're functional. We have civilization. So this is the kind of thing we have to work on, but if we go into it buttoned up with a zero-trust model for everything, we're not going to succeed. We're never going to get to the vision of productive interaction.
So what does this actually mean? If we think about a very simple agent ecosystem here, where you have four agents and two humans. The first thing you need is agent infrastructure, what we talked about in the diagram earlier. You need real support for short-running or long-running different kinds of interactions. This is the kind of thing that Agent Cop is trying to build. But you also need infrastructure to help people steward this nascent ecosystem. I use the word stewardship, not governance, because this isn't a purely regulatory challenge. It's not a challenge for which there's one person responsible.
It's a community that collaborates to make this happen. Let me give you one concrete example. Today, if you access the Amazon Nova model, the images are watermarked, and there's a service where if you send an image, it will tell you whether or not that image actually has the Nova watermark. This underlying infrastructure is indicated in blue at the top of the screen.
Now, what if every agent started checking the provenance of images that they received from humans or other agents? Because from an intellectual property perspective, they need to know where that image came from. Right now, this watermark checker infrastructure is designed for humans. It has an API, but it's not designed for billions of agents to suddenly start connecting to it. It would go down. So how do we support, develop, and fund that? You can imagine other kinds of checking infrastructures involved in each transaction involving new, or not new, content, and there may be other infrastructures that turn out to be relevant. But these are all still to be invented.
Practical Approach: Best Practice Frameworks and Key Considerations for Building Agents
So what do we do today? Let's get really practical. We are still in the middle of Mindshift one, and because the core of agents is based on foundation models, a lot of the work to build individual agents is similar to the work to build other AI solutions. We've released at this re:Invent what I call a best practices framework to help non-machine learning experts develop AI applications. It's available online through the Well-Architected tool. Anybody can go to that QR code URL and take a look at it. The concept is that it provides a set of questions to help people think about the decisions they need to make, and for each question, it provides best practices to help answer those questions.
Let's take a look at a couple of examples on the agent side. I will preface this by saying that the design of this framework is to start with narrowly defined use cases. Because if you want to really evaluate how well a particular system is doing for a use case and make data-driven decisions, the narrower the use case, the more effective your analysis will be. For example, building a facial recognition service, that's not a use case. That doesn't necessarily help you calibrate how well that system is doing. Building a system that recognizes missing children's faces, or trying to search for actors in a library of videos, these are more specific use cases. You can calibrate precision and recall, you can adapt your datasets, and so on.
We want to narrow it down. And we want to assess the benefits and risks of that particular use case, depending as little as possible on the particular AI solution. Then we translate the benefits and potential risks into release criteria. These are typically statistical measures of properties. You have to worry about confidence levels, confidence intervals, thresholds, and these properties are going to be in tension. But once you have that set of release criteria, you can then think about what datasets you need to evaluate those criteria, what datasets you need to build the system, and you can more effectively think about the architecture of your system. And then you go on to evaluation, and you proceed through that.
So, that's on benefits and risk analysis.
Now, I want to move as quickly as I can so we can get to more questions. As an example of a best practice in the framework, we ask builders to identify harmful events that could affect privacy. If we jump that to the agentic AI situation, you might think, okay, so Alice interacts with Bob and shares information appropriately with Bob, let's say a credit card number. And then Alice interacts with Chris, another agent, and shares information related to that transaction, maybe a social security number, appropriately. But then Bob and Chris somehow share the same memory store owned by the same person, and the information gets aggregated. What's the risk of that happening? Could it happen? How do we make that risk zero? These are the kinds of questions you want to ask early on.
In this case, when thinking about agents, I think this is pretty obvious, but as a builder, you would want to invest more time in issues of adversaries and resource constraints. For example, what happens if your agent runs out of money while it's running, and how does it recover, and shared components. Okay, so let's pick another focus area, release criteria.
This is one of the really hard things when building agents because you're asked to think about all possible properties and come up with metrics for the properties you're most concerned about. The scientific literature on many properties is overflowing with choices. We have the opposite problem of what Michael was talking about. There's not much research on many of these deeper problems, but there's too much research on this, so what do you do as a builder? But you have to do the work. So if your agent is narrowly focused, then you have to think about, for example, how effective your safeguards are at actually rejecting a request that asks an IT-based agent to answer something about legal issues. We don't want that to happen, so you want to measure that particular safeguard. You need metrics for that.
And essentially, you want to map all expected benefits and potential risks to one or more release criteria. This is very difficult, but if you do the work, everything else flows much more smoothly. Okay, we'll pick up another area a little bit here, and this is dataset planning around agents.
Because again, we're talking about a future where agents are interacting with different sources of information and other agents and so on. So you need simulation systems. You have to simulate the environment. In generative AI, you can take a more static approach to datasets, like in traditional AI, but you can't do that here, or at least in many cases you can't. And simulation environments are starting to emerge. The UK Safety Institute has
one, and Meta recently wrote one, so we expect to see many more of these kinds of evaluation frameworks. And then, for system planning, the main message here is,
you have to think through all kinds of data flows in your system. Flows through interactions with different agents, different humans, flows through components, and flows through logs. This is part of good engineering practice in many situations, so it shouldn't be too much of a surprise or a challenge, but it's very important in this case where privacy is a very central issue. Okay, so I'll stop there, and we'll take any questions. Michael, can you come up?
- This article is automatically generated using Amazon Bedrock, while maintaining the information from the original video as much as possible.
Discussion