As generative AI transforms business operations, enterprises face a critical challenge: how can they ensure large language models (LLMs) deliver accurate and trustworthy responses? Without reliable data foundations, these AI models can generate misleading or inaccurate responses, potentially undermining user trust and organizational credibility.
Retrieval-Augmented Generation (RAG) tackles this key generative AI challenge, with retrieval being its most complex part due to the challenge of finding precise, relevant data from scattered sources. At Coveo, we’ve reimagined RAG with an intelligent retrieval pipeline powered by a unified hybrid index that links structured, unstructured, and permission-sensitive data. This foundation, combined with real-time machine learning-driven tuning, ensures that every response delivered by an LLM is not only relevant, but also securely grounded in the appropriate context and tailored to complex enterprise content.
We offer enterprise-grade RAG via two options: either fully managed via the Coveo Relevance Generative Answering solution, or for custom applications through our Passage Retrieval and Answer APIs — both options powered by the Coveo AI-Relevance™ Platform.
In this post, we will show how to deploy Coveo’s Passage Retrieval API as an Amazon Bedrock Agent Action Group to enhance response accuracy, enabling Coveo users to leverage their current index to rapidly deploy new generative experiences across their organization.
Coveo’s Passage Retrieval API
The Coveo Passage Retrieval API enhances LLM applications by passing ranked text passages (or chunks) retrieved from the unified index, along with appropriate metadata such as source URLs for citations, so that generated answers are traceable for end-users back to the proprietary organizational knowledge it was grounded in. This reduces the organization’s risk and builds user trust. Built on Coveo’s unified hybrid index, the Passage Retrieval API applies a two-stage retrieval process to extract the most relevant passages from structured and unstructured content sources, ensuring accuracy, security, and real-time relevance.
How the Passage Retrieval API works
- Connect content across sources with a unified index: With a hybrid index, Coveo centralizes access to structured and unstructured content from across all enterprise sources. A unified index provides better multi-source relevancy than a federated search approach by applying the ranking function across all sources. Coveo also provides a robust library of pre-built connectors to maintain seamless integration with any third-party platform (e.g., Salesforce, SharePoint, Google Drive, etc.), ensuring data and permissions stays fresh with automatic updates for real-time retrieval
- Maintain enterprise-grade security: Coveo provides the native permission model of each connected content source by importing item‑level permissions at crawl time through an early‑binding approach. Resolving access rights before indexing prevents any data leakage and boosts search performance by filtering out items a user cannot see before a query is run.
- Enhance search results with hybrid ranking: Combining semantic (vector) search, and lexical (keyword) matching ensures Coveo retrieves the right information in the right context.
- Use machine learning relevancy: AI that continuously learns from user interactions, tailoring retrieval to each user’s journey, behavior, and profile for context-aware responses.
- Relevant passages extraction via two-stage retrieval: In the first retrieval stage, Coveo’s hybrid search system is used to identify the most relevant documents. Then, it extracts the most relevant text passages from these documents, along with ranking scores, citation links, and other key metadata. This two-stage approach allows Coveo to identify accessible and relevant documents from all sources and then more precisely identify the most relevant passages
- Analytics & insights for performance tracking: With event tracking, using the Administration Console and the Knowledge Hub, admins can see exactly how your generated answers perform, where information is missing or underused, and which content needs tuning. With those insights, you can boost answer quality and drive measurable business impact.
Through native access control enforcement, behavior-based relevance adjustment, and deep analytics into content usage and performance, Coveo empowers organizations to continuously improve their GenAI experiences. Backed by consistent recognition from leading analyst firms such as Gartner, Forrester, and IDC, Coveo delivers a reliable, secure, and scalable foundation for enterprise-grade generative AI.
This structured approach helps to ensure LLM-powered applications can generate reliable, contextually relevant responses, which will build and sustain user trust and adoption of GenAI initiatives.
Solution Overview
This solution demonstrates how Coveo’s Passage Retrieval API can be integrated with Amazon Bedrock Agents to enable LLM-powered assistants that deliver accurate, context-aware, and grounded responses. It applies broadly across use cases where agents need to access proprietary knowledge, whether to support customers, assist employees, or empower sales and service teams. Whether your content is simple or complex, structured or unstructured, this approach ensures that generative responses are informed by the most relevant and trusted information available.
The following diagram illustrates the solution architecture.
Key AWS services and Coveo solutions used include:
- Amazon Bedrock Agents: Amazon Bedrock Agents act as the intelligent backbone of this solution, interpreting natural language queries and orchestrating the retrieval process to deliver grounded, contextually relevant insights. In this use case, the agent is designed to fetch precise passages from enterprise content in response to user questions, enabling applications such as virtual assistants, support copilots, or internal knowledge bots. By using structured definitions and instructions, the agent understands when and how to trigger Coveo’s Passage Retrieval API, ensuring that LLM-generated responses are backed by accurate and trusted content.
- Coveo Passage Retrieval Action Group: The Action Group defines the structured API operations that the Amazon Bedrock agent can invoke. Using OpenAPI specifications, it defines the interface between the agent and AWS Lambda functions. In this use case, fetching relevant passages based on the user’s search intent is the only available operation.
For this demo, instructions are given to Amazon Bedrock Agents on when and how to use the Passage Retrieval Action Group tool:
You will act as an expert on Coveo documentation, platform, APIs, analytics, and integration guides. Use the Coveo PR API Action Group to retrieve relevant information on Coveo documentation, platform, APIs, analytics, and integration guides. Summarize the information retrieved from the Action Group response clearly and accurately. Output Formatting guidelines:Provide clear, direct answers without introductory phrases such as “As an expert,” “Sure,” or “Here is...” When appropriate, organize content using:Numbered or bulleted listsStructured sections (e.g., features, steps, key points) Keep answers concise, informative, and free of conversational filler.
The Action Group is defined with an OpenAPI schema that specifies the Passage Retrieval API’s request and response structures.
OpenAPI Schema Example:
openapi: 3.0.0
info:
title: Coveo Passage Retrieval API
version: 1.0.0
description: >
This plugin is the main source of information for a user to quickly access
detailed information about Coveo documentation, platform, APIs, analytics, and integration guides. Use only this source and nothing else to write an answer.
servers:
- url: https://{CoveoOrgID}.org.coveo.com
description: Coveo Cloud Platform
paths:
/rest/search/v3/passages/retrieve:
post:
summary: Retrieve passages based on a user query about detailed information about Coveo documentation, platform, APIs, analytics, and integration guides.
description: >
API route that helps users quickly access detailed information about Coveo documentation, platform, APIs, analytics, and integration guides.
operationId: retrievePassages
requestBody:
required: true
content:
application/json:
schema:
type: object
properties:
query:
type: string
description: The search query entered by the user.
required:
- query
responses:
"200":
description: A list of passages related to the user query.
content:
application/json:
schema:
type: object
properties:
items:
type: object
properties:
text:
type: string
description: The text of the passage retrieved.
relevanceScore:
type: number
format: float
description: The relevance score of the passage.
document:
type: object
description: This is the document that you can use for ciatations
properties:
title:
type: string
description: Title of the document.
primaryid:
type: string
description: Primary ID of the document.
clickableuri:
type: string
description: URL of the document.
"400":
description: Invalid request, such as missing required parameters.
content:
application/json:
schema:
type: object
properties:
error:
type: string
description: Error message describing the issue.
"401":
description: Unauthorized, such as missing or invalid authorization token.
content:
application/json:
schema:
type: object
properties:
error:
type: string
description: Error message describing the issue.
"500":
description: Internal server error, indicating an issue with the platform.
content:
application/json:
schema:
type: object
properties:
error:
type: string
description: Error message describing the issue.
- Lambda Function
The lambda function defined in the Action Group is essential for enabling the Bedrock Agent to call Coveo Passage Retrieval API. The Lambda function performs the following tasks:- Receives incoming requests from the Bedrock Agent
- Queries the Coveo Passage Retrieval API using the user’s input
- Returns the relevant search results back to the Bedrock Agent
Code Sample:
import json
import http.client
def lambda_handler(event, context):
#if you want to see the input parameters
# print(event)
# print(context)
apiPath = event["apiPath"]
server = f"{COVEO_ORG_ID}.org.coveo.com"
api_key = {COVEO_API_KEY}
query = event["inputText"]
searchHub = {COVEO_SEARCH_HUB}
org_id = {COVEO_ORG_ID}
#if you want to use the rephrase query from bedrock
# for prop in event['requestBody']['content']['application/json']['properties']:
#if prop['name'] == 'query':
# query = prop['value']
headers = {
'Content-Type': 'application/json',
'Authorization': f'Bearer {api_key}'
}
request_body = {
'localization': {
"locale": "en-CA",
"timezone": "America/Montreal"
},
'additionalFields':['clickableuri'],
'query': query,
'searchHub': searchHub,
'maxPassages': 5
}
json_body = json.dumps(request_body)
try:
connection = http.client.HTTPSConnection(server)
connection.request("POST", apiPath, body=json_body, headers=headers)
response = connection.getresponse()
response_body = response.read().decode('utf-8')
print(f'HTTP Response Status: {response.status}, Reason: {response.reason}')
if 200 <= response.status < 300:
response_data = json.loads(response_body)['items']
# if you need to log the passages from the Coveo PRAPI
for i, items in enumerate(response_data):
print(f"Chunk {i+1} : {items}")
if 'requestBody' in event:
action_response = {
"actionGroup": event["actionGroup"],
"apiPath": event["apiPath"],
"httpMethod": event["httpMethod"],
"parameters": event["parameters"],
"httpStatusCode": response.status,
"responseBody": response_data,
}
session_attributes = event.get("sessionAttributes", {})
prompt_session_attributes = event.get("promptSessionAttributes", {})
return {
"messageVersion": "1.0",
"response": action_response,
"sessionAttributes": session_attributes,
"promptSessionAttributes": prompt_session_attributes,
}
else:
return ", ".join(item['text'] for item in response_data)
else:
raise Exception(f'Failed to send request to Coveo API: {response.status}, {response.reason}')
except Exception as e:
print(f'Exception occurred during HTTP callout: {str(e)}')
raise Exception(f'Failed to send request to Coveo API: {str(e)}')
finally:
connection.close()
- Coveo AI-Relevance™ Platform
Provides indexed and structured search results through the Passage Retrieval API.
Solution Walkthrough
Prerequisites
Before you begin, you will need the following prerequisites:
- An AWS account with appropriate IAM permissions
- Enabled model access for Amazon Bedrock
- A Coveo index created and ready to use
- The following Coveo information
- Coveo Organization Id
- Coveo Search Hub name
- Coveo API Key
- A Coveo Passage Retrieval Model deployed in your pipeline
Step 1: Deploy the solution with CloudFormation
To deploy your agent, follow these steps
- Use the following link, you will be routed to the Amazon CloudFormation console
- In the deployment form, specify the stack details, input a Stack name, AgentModelID, AgentName, CoveoApiKey, CoveoOrgID, CoveoSearchHub
- In the same page, under the Capabilities section, check the acknowledge checkbox.
- Click Create stack.
- Wait for the stack creation to complete.
- Verify all resources are created on the stack details page.
Step 2: Test the solution
To test the solution, complete the following steps:
- On the Amazon Bedrock console, choose Agents in the navigation pane.
- Choose the agent created by the CloudFormation stack.
- Under Test, choose the alias with the name Version 1 and expand the Test
- For this demo, Coveo technical documentation (from the website) was ingested in Coveo Search Platform index. Let’s start with the query “what is the difference between Coveo atomic and headless?”
The screenshot above shows the agent’s response to the question “What is the difference between Coveo Atomic and Headless?” in the Test Agent console with Anthropic claude 3 Haiku v1 as LLM.
- Click on Show Trace section in the right panel and expand Trace Steps to see the agent’s rationale.
The screenshot above demonstrates how Bedrock Agents process and answer the question ‘What is the difference between Coveo Atomic and Headless?’. The agent orchestrates its response by:
- First forming a rationale: To understand the difference between Coveo Atomic and Headless, I will need to retrieve relevant information from the Coveo technical documentation using the CoveoPRAPI Action Group.
- Then it invokes the CoveoPRAPI action group, which is specifically designed to retrieve relevant passages, through a Lambda function to make an API call to ‘/rest/search/v3/passages/retrieve’.
The image illustrates the agent’s systematic approach to planning, executing necessary actions through the CoveoPRAPI action group, and retrieving relevant document chunks before formulating its final response.
The Lambda function code includes a debugging feature that logs each retrieved passage from Coveo’s Passage Retrieval API. This logging mechanism iterates through all returned chunks, numbering them sequentially for easy reference. These logs are available in CloudWatch allowing you to see exactly which passages were retrieved for each user query, and how they contributed to the final response. To visualize the logs, go to the Amazon CloudWatch service console, then in the Log groups navigation panel and then find the Lambda function name.
The screenshot above shows the agent detailed logs in Amazon CloudWatch. In the logs, Coveo Passage Retrieval API returns the 5 most relevant chunks to the LLM.
Clean up
To avoid ongoing charges, delete all other resources deployed as part of the AWS CloudFormation stack. Do this by completing the following steps:
- On the AWS CloudFormation console, choose Stacks in the navigation pane.
- Choose the stack you created, then choose Delete.
- Choose Delete stack when prompted.
For more information, refer to Deleting a stack on the AWS CloudFormation console
Conclusion
By integrating Bedrock Agents capabilities with Coveo’s AI-driven retrieval, organizations can develop highly accurate, context-aware AI applications. This approach minimizes hallucinations while delivering real-time, secure responses.
We encourage you to explore pre-built examples in this repository to help you get started with Amazon Bedrock.
Learn more about Coveo AI-Relevance™ Platform and how to implement Coveo’s Passage Retrieval API to your organization.