Create a multi-agent system with Amazon Bedrock
In my previous blog post, I promised to implement a multi-agent system using Amazon Bedrock. Please read the other blog for an introduction to multi-agent systems. In this blog, you will read the details for creating a multi-agent system using Amazon Bedrock. The system contains four different agents. A product expert uses a knowledge base with product information. An order expert can access the order store and create new orders. A marketing agent has the power of an LLm to make up everything, a front-desk agent who talks to you to understand your question.
Reference to the other blog:
https://www.luminis.eu/blog/create-a-multi-agent-system-with-openai-agent-sdk/
The case to implement
The example is a customer support system consisting of multiple agents. The first point of contact is the front-desk agent. This agent will receive your request and then pass it on to one of the other agents who can help you.
Product Expert Agent
The product expert agent’s main task is to answer product-related questions. Information about the caller and the product for which they have questions is needed on handover. The product may not be specific yet, so asking questions about it is essential. The agent has a knowledge base to learn more about the product.
Order Support Agent
When a customer has placed an order, the order support agent is responsible for answering questions about it. These could be questions about the current status of the order. Has it been paid? Has it been sent? Before an order is dispatched, it can be cancelled. The order support agent needs to know who the customer is and have an order ID.
Marketing Agent
Responsible for everything that is not about products or orders, but related to the company. It maintains a list of frequently asked questions. It can tell you more about the company, its vision, contact options, and customer policies.
Implementing the agents
A hard thing when working in the cloud is the number of choices. This is not different when working with AI Agents running on AWS. Last year, I wrote about Amazon Bedrock and agents. In that post, I took the CDK approach. I deployed everything I needed as resources on AWS. The advantage is the complete platform you get to use. You can test the agent before putting it in production, all from the platform. It does come with a cost – a lot of work and no flexibility. That’s why the team at Amazon created the Inline Agent.
Inline agents provide the tools to change instructions, add actions, and connect knowledge bases at runtime. Another supported feature for inline agents is collaboration. You can create agents who work together, which is what we need to make our customer support team.
The example we created for this blog is unique. Big chunks of the supportive code are taken from the samples provided by AWS. This is the sample I used most for this project: https://github.com/awslabs/amazon-bedrock-agent-samples/tree/main/examples/agents/inline_agent
All the code is available in my GitHub repository: Bedrock Agent.
Implement the Marketing Agent
The marketing agent is our most basic agent. It uses an LLM to answer questions and can hallucinate as much as it wants about the company we work for. Before we can call an agent, we need to connect with AWS. In Python, you can use the boto3 client.
region = "eu-west-1"
bedrock_rt_client = boto3.client(
"bedrock-agent-runtime",
region_name=region
)
Calling the agent is as easy as this.
_agent_resp = bedrock_rt_client.invoke_inline_agent(
**request_params
)
The request_params is a more challenging part for this example. For this example, the configuration is like this.
{
"enableTrace": True,
"endSession": False,
"foundationModel": "eu.amazon.nova-lite-v1:0",
"instruction": (
"You are the Marketing Agent. Your primary goal is to provide "
"detailed, accurate, and helpful information about our "
"company. You can make up everything you want, but make sure "
"it is believable."
),
"agentCollaboration": "DISABLED",
"sessionId": str(uuid.uuid4()),
"inputText": "What is the mission of your company?"
}
Notice the LLM that we use amazon.nova-lite-v1:0. This is a special Bedrock model. It is a cross-region model. The advantage is that you can access the model from more regions. The disadvantage is that you don’t know which region you’re running in, which could result in higher latency. You must prepend it with the region ID to tell where to start. With the instruction, we tell the agent how to act. The following field is the agentCollaboration. In this case, it is DISABLED. That means this agent will not initiate a collaboration. It can collaborate with a Supervisor agent. The last field I want to mention is the inputText. This is the question from the user that the agent needs to answer.
In the codebase, you see several classes that help execute the agent.
- NonCollaboratingAgent — The superclass for the three agents that do not initiate a collaboration with another agent. This class helps construct the request_params we discussed before.
- MarketingAgent — Initialises the marketing agent.
- inline_agent_tools — Contains a function to call the agent and handle the response. This code is mainly copied from the AWS sample I discussed before.
- run_marketing_agent.ipynb — The Python Notebook that runs the agent.
If you start working with Bedrock agents, check out the response. It contains a lot of helpful information. Now run the Notebook. Below is the question and answer we ask.
What is the mission of your company?
“Our company’s mission is to provide innovative and high-quality products and services that improve the lives of our customers and contribute to a better world. We strive to achieve this by continuously improving our processes, investing in research and development, and fostering a culture of collaboration and excellence.”
This is all it takes to run an inline agent on Amazon Bedrock. The following agent adds more complexity. We provide an ActionGroup to interact with orders.
Implement the Order Support Agent
The order support agent can create new orders, retrieve information from existing orders, and update or delete orders. For the demo application, orders are stored as JSON documents in an S3 bucket. The LLM itself cannot access your S3 bucket. Therefore, it needs tools. With Amazon Bedrock, tools come as ActionGroups. One way of implementing an ActionGroup is by specifying the interface or API through an Openapi specification. Next, you create a Lambda function that receives an event with enough information to understand what is requested.
This code block demonstrates receiving an event and routing it to the correct function using the HTTP method.
def lambda_handler(event, context):
method = event.get("httpMethod")
path_params = event.get("pathParameters") or {}
order_id = path_params.get("id")
if not order_id:
parameters = event.get("parameters", [])
order_id = next((param['value'] for param in parameters if param['name'] == 'id'), None)
try:
if method == "POST":
return create_order(event)
elif method == "GET" and order_id:
return get_order(order_id, event)
elif method == "PUT" and order_id:
return update_order(order_id, event)
elif method == "DELETE" and order_id:
return delete_order(order_id, event)
else:
if not order_id and method != "POST":
return response(400, {"error": "Missing order ID"}, event)
return response(400, {"error": "Unsupported method or missing order ID"}, event)
except Exception as e:
return response(500, {"error": str(e)}, event)
Let’s have a look at the implementation for the GET request. For the agent to work with the Action Group, the format of the response is crucial. You need to return the ActionGroup name, the requested path and method. The code has a function to help you create the correct response.
def get_order(order_id, event):
key = f"orders/{order_id}.json"
try:
obj = s3.get_object(Bucket=BUCKET_NAME, Key=key)
order_data = json.loads(obj["Body"].read().decode("utf-8"))
return response(200, order_data, event)
except s3.exceptions.NoSuchKey:
return response(404, {"error": "Order not found"}, event)
def response(status_code, body, event):
return {
"response": {
"actionGroup": event['actionGroup'],
"apiPath": event['apiPath'],
"httpMethod": event.get("httpMethod", "POST"),
"httpStatusCode": status_code,
"responseBody": {
"application/json": {
"body": json.dumps(body) if body else None
}
}
}
}
Adjusting your Lambda and the OpenAPI specification to work together was the most challenging task for the example I have created. You can just configure logging to CloudWatch and use your Lambda logs to understand what’s happening.
The sample project contains a lot of code to deploy the Lambda function, configure the Action Group, and set up the correct roles and policies. The essential files for this part of the example are.
- lambda_function_order_handler.py — The Python-based Lambda function.
- payload-orders.json — The OpenAPI specification.
- lambda_creator.py — Infrastructure code to deploy the Lambda
- inline_agent_utils.py — Utility function to call the agent and parse the response if trace is enabled.
The initialisation of the inline agent order_support_agent is similar to the marketing_agent. We have to add the action group configuration.
{
"enableTrace": True,
"endSession": False,
"foundationModel": "eu.amazon.nova-lite-v1:0",
"instruction": (
"You are the Order Support Agent. Your primary goal is to provide "
"detailed, accurate, and helpful information about the orders. "
"Use only the information from the provided tools. If order is not "
"available, do not make up information, tell that you do not know."
"\nNext to fetching information, you also have the ability to "
"create, update and delete orders. "
),
"agentCollaboration": "DISABLED",
"sessionId": str(uuid.uuid4()),
"actionGroups": [
{
"actionGroupName": "HandleOrders",
"actionGroupExecutor": {
"lambda": ""
},
"apiSchema": {
"payload": ""
},
"description": "This action group handles the orders."
}
]
"inputText": "What is the mission of your company?"
}
This Python Notebook contains what you need to run the sample: run_order_support_agent.ipynb.
What is the status of my order with ID 123?
The status of your order with ID 123 is as follows: Customer: jettro, Order Lines: 2 Ice Cream, Total: 50.00.
Below is the trace of the response; this gives you an idea of the actions the Agent executes.
- - Step 1 - -
Took 1.2s, using 1516 tokens (in: 1321, out: 195) to complete prior action, observe, orchestrate.
The User's goal is to know the status of their order with ID 123.
(2) No information has been provided yet.
(3) The best action plan to fulfill the User's request is to fetch the order details using the provided order ID.
(4) All steps in the action plan are complete.
(5) The available action to execute the next step is GET__HandleOrders__getOrder.
(6) The information required for this action is the order ID, which is provided by the User.
(7) I have everything I need to proceed.
{'actionGroupInvocationInput': {'actionGroupName': 'HandleOrders', 'apiPath': '/orders/{id}', 'executionType': 'LAMBDA', 'parameters': [{'name': 'id', 'type': 'string', 'value': '123'}], 'verb': 'get'}, 'invocationType': 'ACTION_GROUP', 'traceId': '24854811–6d9b-44d5-a813–2cfcb6c0d27b-0'}
Using tool: /orders/{id} with these inputs:
[{'name': 'id', 'type': 'string', 'value': '123'}]
- tool outputs:
{"order_id": "123", "customer": "jettro", "order_lines": [{"product": "ice cream", "qty": 2}], "total": "50.00"}…
- - Step 2 - -
Took 1.9s, using 1691 tokens (in: 1550, out: 141) to complete prior action, observe, orchestrate.
The User's goal was to know the status of their order with ID 123.
(2) The tool result has provided the details of the order.
(3) The best action plan to fulfill the User's request is to provide the order details.
(4) All steps in the action plan are complete.
(5) No further action is needed.
(6) No additional information is required.
(7) I have everything I need to proceed.
Final response:
The status of your order with ID 123 is as follows: Customer: jettro, Order Lines: 2 Ice Cream, Total: 50.00. …
Agent made a total of 2 LLM calls, using 3207 tokens (in: 2871, out: 336), and took 3.3 total seconds
Create a new order with id 423. The customer is Roy. He wants to order 12 kipnuggets, total for the order is 5.
The resulting file 423.json in the S3 Bucket
{
"orderLines": [
{"product": "kipnuggets", "qty": 12}
],
"orderId": "423",
"customer": "Roy",
"total": 5.0
}
Implement the Product Support Agent
The third agent, the product-support-agent, uses a knowledge base to answer questions about products that we sell. A knowledge base is another component that Amazon Bedrock provides. A knowledge base can be a structured data source. Another option is a semantic data source, often a vector data source. Multiple implementations for a vector store are available. I like the serverless variants like OpenSearch and Postgres. For a cost perspective, I decided to configure the Postgresql variant. To make my life easier, I created it in the console. You need to choose an embedded and a chunking strategy.
I have 5 product sheets with some generic information and frequently asked questions. I used the Titan Text Embeddings v2 to create the embeddings.
The agent has a similar configuration to the other two agents. We need to add a block for the knowledge base. All it needs is the ID of the knowledge base.
{
"enableTrace": True,
"endSession": False,
"foundationModel": "eu.amazon.nova-lite-v1:0",
"instruction": (
"You are the Product Support Agent. Your primary goal is to provide "
"detailed, accurate, and helpful information about the products. Use "
"only the information from the provided AWS knowledge base. If the "
"information is not available, do not make up information, tell that "
"you do not know."
),
"agentCollaboration": "DISABLED",
"sessionId": str(uuid.uuid4()),
"knowledgeBases": [
{
"knowledgeBaseId": "",
"description": "Knowledge base for product support, contains information about products that we sell."
}
]
"inputText": "Do you have information about a headphone with support for Bluetooth?"
}
This Python Notebook contains the code to use the Product Support Agent: run_product_support_agent.ipynb.
Do you have information about a headphone with support for Bluetooth?
The Sony WH-1000XM5 Wireless Noise Cancelling Headphones support Bluetooth. You can pair them with your device via Bluetooth settings and download the Sony Headphones Connect app to personalize your sound settings, activate adaptive noise control, and update firmware.
Beware, sometimes the knowledge base fails. It shows a message that it went into hibernation mode due to inactivity. Just rerun the command.
We have all three agents available. Next, we create the front desk agent who collaborates with these three agents to support the customer.
Implement the Front Desk Agent.
The front-desk-agent differs from the other agents. This agent is of type SUPERVISOR_ROUTER. That means this agent talks to the user and then routes them to one of the other agents for help. The configuration of this agent is not that different. It needs a configuration for the other agents. Through the collaboratorConfigurations, the agent receives information about the agents they can collaborate with. The information about the other agents is passed through the field collaborators.
We reuse the configurations from the agents we created in the previous sections. We do have to make some adjustments. We need to remove the fields: enableTrace, endSession, and sessionId. We need to add a name to the agentName field.
{
"enableTrace": True,
"endSession": False,
"foundationModel": "eu.amazon.nova-lite-v1:0",
"instruction": (
"You are the Front Desk Agent. Your primary goal is to provide detailed, accurate, and helpful information "
"about our company and orders. Your responsibility is to talk to the user to understand their needs and "
"then relay the information to the Order Support Agent and Marketing Agent. "
),
"agentCollaboration": "SUPERVISOR_ROUTER",
"sessionId": self.session_id,
"collaboratorConfigurations": [
{
"collaboratorInstruction": "Use to answer questions about the company.",
"collaboratorName": "marketing_agent",
"relayConversationHistory": "TO_COLLABORATOR"
},
{
"collaboratorInstruction": "Use to answer questions about the company.",
"collaboratorName": "order_support_agent",
"relayConversationHistory": "TO_COLLABORATOR"
},
{
"collaboratorInstruction": "Use to answer questions about products the company is selling.",
"collaboratorName": "product_support_agent",
"relayConversationHistory": "TO_COLLABORATOR"
}
],
"collaborators": [
self.order_support_agent.agent_request_params(as_collaborator=True),
self.marketing_agent.agent_request_params(as_collaborator=True),
self.product_support_agent.agent_request_params(as_collaborator=True)
],
}
Use the Python Notebook run_frontdesk_agent.ipynb to run the agent. The following sentences show my input and the agent’s response.
I want to order a device to monitor my heart rate
To place an order for a heart rate monitoring device, please provide the following details: the total value of the order, the list of items in the order, the order ID, and the customer’s name.
Can you first suggest a product for my needs?
The Fitbit Charge 6 Fitness Tracker is a great option for monitoring your heartrate. It offers continuous heart rate monitoring, built-in GPS, stress management tools, and Google services integration.
I like that, can I order it for 99 with ID 500, I am Jettro?
Your order for the Fitbit Charge 6 Fitness Tracker has been successfully created with order ID 500. Thank you for choosing our service, Jettro!
Concluding
Amazon Bedrock agents are versatile. Integrating your functions through OpenAPI specifications and ActionGroups enables interaction with the outside world. KnowledgeBase integration gives the power of RAG using your content. The foundational model amazon.nova-lite-v1:0 works well for agents.
Making the multi-agent system work on Amazon was harder than the OpenAI Agent from the previous blog. It does come with a more production-like system.
References
Want to know more about what we do?
We are your dedicated partner. Reach out to us.