Hello, everyone, and welcome to today's CRM Magazine web event brought to you by Coveo. I'm Bob Fiernikes. I'm the publisher of CRM Magazine, and I'll be the moderator for today's broadcast. Our presentation today is titled generating self-service success. But before we start, I just want to, tell you how you can participate in this broadcast. We will have a question and answer session at the end. So if you have any questions during the broadcast, just click on the submit button. And if we cannot get to them, we'll get to them within a few days by email. Plus, if you'd like a copy of the presentation, you can download a PDF from, the handouts tab on the console once the event is archived. And just for participating in today's event, you could win a one hundred dollar Amazon gift card. So now to introduce our speaker for today, we have, Devin Poole, senior product, market manager at Coveo. Take it away, Devin. Yeah. Thanks so much, Bob, and welcome, everyone. Really excited for you all to join us today. We've got a great session lined up to discuss arguably the hottest topic and one that you all might be sick of hearing about by now, but then again, you did decide to join this webinar of talking, of course, about generative AI. And, specifically, what we're gonna share today is how you can start to leverage this capability to drive key business results around self-service. Now I've been working in the customer support world for about the past fifteen or so years. And almost every year, getting our customers to self serve ranks among the top three priorities in some way, shape, or form. And in fact, twenty twenty four is no exception. It is the number one priority on list after list of service leader surveys. So what we're gonna talk today isn't necessarily completely new ground. Right? Getting customers to self serve is something that we've long been after. But, of course, some things have changed, namely the technology capabilities. We've significantly expanded our opportunity to engage our customers digitally over the last decade, and AI now sits at the foundation of good service strategy these days. But, of course, not all AI is generative AI, and AI alone is never gonna solve all of your self-service challenges. You know? And while we have a a lot of these advances in technology, they're also fueling new customer behaviors. I mean, think about how more digitally dexterous you are than just a couple of years ago. But there is one thing that's remained the same, and that's good news for us. Customers still want the same thing that they've always wanted when they contact customer service, and that is resolution. Right? Just fix my problem. Make the pain go away in the fast, easy, and convenient way, and I'll be happy. So with that in mind, let's start our session here by taking a look at some of the ways that customer behavior and mindset has swung decidedly toward self-service. So here we're looking at just three stats. Could have easily put a hundred boxes on this page showing you how much more likely customers are to prefer and to try self-service today. So starting on the left from the folks at Gartner, you can see that seventy percent of customers today. Right? Seven out of ten say that they prefer self-service over contacting a CSR. Now this doesn't mean that they won't ever speak to CSR, don't ever wanna contact us in person. Your phone and chat volumes, definitely support that. But what this is ultimately telling us is that customers today, they're digital first. This is where I'm gonna start. And in the middle of the page, we're feeling this as service leaders as well. Right? Customers are becoming more accustomed to self-service. Nearly three quarters of executives cite and an increased demand for self-service across the past three years. Right? Customers, were driven initially by the pandemic. You had no choice but to become digital. And for the large majority of customers, they're not going back. They said, man, this is great. This is convenient. I value my own time more than anything else. So what's happening, right, is that customers are, saying, I'm gonna try to solve it on my own, and this is only gonna get worse. Take a look at the stat on the right hand side. Gen zed are saying that they would give up on an issue entirely if they can't self serve. Well, at least thirty nine percent of them are. Right? These are the folks that were, you know, born as, you know, not just digital natives. Right? Younger millennials were born as digital natives, but these are the folks that are coming of age in a more modern, more sophisticated digital engagement. So that in and of itself, is driving us to invest a lot more in self-service. And how exactly are service leaders preparing to respond to this challenge? How are they going to invest? Well, take a look at the next slide here. What you see is that everyone's favorite new tech capability, of course, generative AI. Right. Three more stats on the page here. BCG on the left. Ninety five percent of service leaders expect to be their customers to be served by an AI bot at some point in the service journey. Doesn't mean AI bots are replacing everything, certainly not the case. But at some point in the journey, we expect that our customers are gonna interact with some sort of conversational AI or generative AI. In the middle of the page from CMP Research, eighty two percent of companies expect to be offering generative AI in customer facing self-service by twenty twenty five. And over on the right, also from the folks at CMP, right, eighty six percent of companies are planning to use generative AI in their knowledge bases by twenty twenty five. Alright. So in many ways, I'm calling twenty twenty four the year of GenAI operationalization. We spent last year learning about this capability, exploring, testing, piloting. Well, support leaders now have a plan to take decisive action to integrate this capability into their customer service tech stack. And for many organizations, the outcome that we're all hoping for here is productivity. We want our employees to be more productive, and we want our customers to be more productive in the way that they solve problems. And, hopefully, the phone volumes will finally start to to go down in meaningful ways for us. Now, of course, if it were this easy, just push a button, get Gen AI installed, and we'd be able to end our conversation here, you know, nine minutes in. But the reality of the landscape is that the waters of GenAI solutions are extremely muddy. Let me show you. Right? Here, we're looking at, you know, four large categories of generative AI moving from left to right, things like content creation. Right? Creating images or videos or Coveo. Content discovery through search analysis, knowledge management. There's conversational AI. Right? Summarization and translation and things like that. And then there are AI simulations. Right? Synthetic data, digital twins, these cuts of worlds. And each of these, that they have their uses. Right? But going back to to something we talked about at the beginning of the way, right, that there's lots of forms of generative AI. But when it comes to what customers want, well, customers just want a solution to their problem. Right? They want fast and easy answers. And this is where a capability called generative answering thrives. Generative answering is a subset capability of GenAI that combines the art of semantic search with the science of relevance in prompt engineering. If you're you're thinking, woah. Slow down, egghead. Give it to me in layman terms, please. Well, think about it like this. Right? When customers have questions, but the answers live in multiple documents or scattered across bits and pieces of other content, or you have one three hundred page document explaining everything your product does. Not that any of us have super long documentation. Right? You only need chunks and pieces of that. Well, that's where generative answering comes in. You use generative answering to ground, the the customer question in a secure company specific fact base. Right, we call that a unified index. I'll go into all of that. But you combine that, with the power of something that we all know and love, search. Right? When I go to interact with any information, I go to search for it these days. And generative AI can start to instantly return accurate answers that your customers can trust. Here's the thing. The majority of your customers today, based on serving, well, they couldn't tell you the difference between the types of generative AI or any other form of AI that you see on the page here, especially not when they're using it. Right? For most of us, we use AI every single day in our life. Right? We use simple forms of AI like spell check. Right? That's rules based AI. That's just checking against the dictionary. Or, we're using type ahead when you're sending out a text message and it's suggesting the next word. That's AI at work. The folks over at CMP Research, they did a large customer study. And, again, they found that nearly eighty percent of customers, they they don't know the difference between conversational AI and generative AI. But, again, at the end of the day, the the point of all of this is customers just want something that is gonna make their problem go away. Right? That same study from CMP Research showed that nearly three quarters of customers said they wanna spend less time solving customer problems. So just give me the answers. Right? And and that's one of the most powerful use cases for generative AI today. Help customers to quickly and easily answer their questions. So let's take a look at some of the the different AI models that are being used to dramatically increase, self-service success. Alright. Here, you can see where, you know, AI is gonna be engaging with your customers along the top of the page from public facing websites through authentication and portals, digital self-service channels like chatbots, all the way through to the the contact center. Right? What we're gonna see is that in any given interaction, customers, they are likely to be engaging with more than one channel. And so you wanna limit that. You certainly wanna try to keep that that interaction within a single, self-service channel. And leading organizations are powering that with different AI models. Right? Things like semantic search or query suggestion. That's the type ahead that I talked about earlier, or personalized content recommendations or or product recommendations or something called smart snippets when you, you know, search for an answer and you just get a little box that tells you the answer to the question that you're looking for, tells you what other people have also asked around that. That that's smart snippets. Alright? Things like case assist or case classification or, you know, in the contact center, things like knowledge recommendations or user actions, actions, showing the, agent what's happened along this journey. Those are all things that can be powered by AI. And as you see in the highlights, generative answering can be a part of each of these. It's not gonna be the only thing, but it's gonna be part of it. You need these interactions to be connected. They need them to be relevant. Let's take a look at the green bar along the bottom here. Right? As you see customers moving through a journey like this, you need to keep the context of these digital interactions so that each layer builds upon the last, and you continue to get more and more relevant answers. So, again, generative answering, it's a tool in your toolbox. Right? It's an opportunity, but a very powerful one. A great capability that can help to quickly find relevant information and help to thread together a lot of these pieces. But, of course, he probably knew there'd be a but. Right? There there's a lot of potential risk at hand here. Right. Of course, hallucinations is one that we've all heard about. It seems every other day there's some company that's being thrown, over the coals for a hallucination that they're bought in. But security is another. Right? We don't wanna expose answers. There's one risk in my conversations with leaders around the world that's often being overlooked, and it's a risk of our own making. Take a look at the next page here. Generative AI cannot become a separate siloed interaction. Right? That's the reality that we're facing. If your company or your organization is investing in generative AI as a, you know, separate silo, as one thing, one, you know, disconnected point solution, well, it's gonna fail. It's never gonna go right because we've seen this story before. In fact, we've seen this story play out in lots of different digital investments that we've made as companies over the past decade. Right? And this is part of the reason that self-service has continued to remain at the top of the priority list. We make different investments in different silos. So let me show you two realities, two potential things, and you tell me as you think about this, which one feels better to you as a customer? And, frankly, as a leader, which one feels more, like, likely that you can manage this world? So here's the first world. Right? And this is, sadly the the way that many of our digital initiatives have been deployed to date. Right? This is what, I'd call the fractured digital experience. When we invested in one chat provider, and then we separately had an asynchronous messaging provider, then some team or another put some time into, UX design. And then the marketing team came in, and they redesigned our website, but they didn't design it for service. So we had to redesign our website for service, and, of course, it's all very easily navigable. And then we bought at least one, if not several different chatbots, and we tweak and deploy those things depending on what page you're on. And then you've got search queries. Right? Those queries can be spread across many different sites with many different search engines running it. It could be your community site, your docs site, your public facing website, your help and support site. There could be multiple search boxes from multiple providers. And so what's happened here, right, is that in the past, service organizations have been great at deploying self-service channels. But we've been pretty bad at unifying those experiences on the back end and unifying, that digital, journey that our customers go on. And, of course, none of these things were intended to intentionally harm the the digital customer experience. In fact, I I I would bet anything, all of these investments were made with the best intentions in mind. We wanna give the customers what they want. We wanna give them any sort of option that they've asked us for. They say they want these things. We wanna allow them to serve themselves in their own way. Well, this isn't Burger King. You don't get to have it your way all the time. So you as leaders need to take a step back and think we are the experts. Right? We are the experts in the easiest way to achieve resolution for the problems. You handle these problems from your customers every single day. For your customers, well, they they might only handle it once, hopefully. And that's the idea. Lean on your expertise. Something we found, in a past life of mine when I was researching for the effortless Experience book, in case you've heard of that. Right? But we we talked a lot about this is that customers say they want choice. If you ask them, they'll tell you they want choice. But what they really need and what they really mean by that is they want guidance. They want you to help them navigate this very tricky landscape and to make it as simple and easy as possible for them to do that. So let me show you a different scenario. Right? This is a world that we envision, that we would call the unified AI experience. Right? This is a world about guided resolution, about simplifying the customer journey by ultimately giving them less to look at, less to do. I don't need to know, is it contact us, or is it get help, or is it customer support, or is it this search box or that search botch? No. Just give customers a single unified digital touch point, and that's what we're looking at in the middle of the page here. This is what we call the intent box. Give customers a space to tell you what they're trying to accomplish, and then guide them along the path to help them get the right answer. Right? Think of all the things you could do with a single intent box. Tell me what you're trying to do. In fact, one of our clients here at Coveo, they said this initially about the search box, and it applies, just as well to to the intent box, which is, do you wanna know anything about your customers? Do you wanna know something that they're trying to accomplish? Put a box in front of them. They'll tell you. Right? People don't lie to the search box. They're not gonna lie to the intent box. Tell me what you're trying to do, and I will help you. If you think about this, right, all of these worlds converging into one, think about how this lowers the cognitive and mental burden on your customers. Instead of trying to weave and navigate their way through your sites, they've got one place to go. Get help here. We converge all of these different disconnected, disparate types of communication. We backstop that experience with powerful AI driven personalization and contextual relevance. Your site goes from where do I look to a one stop shop for customers to get answers. Right? Both of the questions that they know to ask and the ones that they've not yet thought of. Right? You can help to guide them through the journey. Now this is the vision of what the future is gonna look like, and it's getting there very quickly, very rapidly. Right? That this is something that we're starting to see happen. So how do you start to make this happen? Right? What do you need in order to drive these types of experiences? Because, certainly, generative AI is very, very powerful. As we said earlier, it's not the only thing that you're gonna use to get your customers to self serve, but it's gonna be one of the primary tools. Right? This is what we wanna start to do. So So you've got to figure out how do we create relevant digital experiences, and and how do we create the generative variety of those. Right. Great answers that will not only solve a customer's problem, but will come across as trustworthy. Well, what differentiates that fractured experience from a unified one are are five different things, and that's what we're gonna walk through here next. Right? The first is the the connectivity layer. Right? And that's where all of this starts, allowing you to get your content wherever it resides and bring it into one central repository. Right? We we like to think about this like the plumbing. Right? You've gotta get the plumbing in your house all connected here. So getting your content, that can be YouTube videos that that you might have. It can be marketing content. It can be service content. It can be product information. Think about that. Think about where all of your content that can be used to solve customer problems lives. And you've got to securely connect it into one place, one single repository. Right? So it's not just about the content, but it's about the security that comes with it. Things like permissions that should follow a document. Right? In other words, not all customers can see the same content. Some customers are going to have access to different things in your library. And when you do this, and you start to secure the content, you you start to use, you know, both structured and unstructured content, it allows you to get the the right answer. Right? That allows you to start to, eliminate those things like hallucinations and exposures. And as you do that, right, you structure the documents in a way that's gonna allow you to, first and foremost, drive search results, and second, generate answers on the back end of those results. You start to break your documents down into smaller chunks. And that way, you can better allow the AI to better understand the meaning of those documents. And that's what we mean here by, you know, document chunking and vector generation, understanding the relationships between those different components. What's the right thing to solve this problem given what we know about this customer? The second thing here is the centralized repository itself. We call this a unified hybrid index, right, where you don't just need to host the the data or the content, but you also need to host the embeddings, the the chunks of those documents, and and the vectors. That's the numerical value of a chunk, right, that allows it to be read by AI and machine learning, and that ultimately represents the meaning here. Right? You can't have a separate document database and a vector database. Those two things need to be unified into one index. Because what that does, right, is it allows the the customer as they're searching, as they're looking, as they're clicking, as they're reading, you start to track that. You start to look at what they're doing, and that will allow you to drive greater layers of relevance. Right? That relevance layer here is critical to this because certainly lots of different systems can retrieve information. But are they retrieving the right information? The information that takes into account the context of who that customer is. Right? Two customers can come to an airline, and they can both search for what's my baggage allowance. Right? Well, one of those customers is a top tier status holder, at that airline, and they're gonna have a different allowance than, someone who's not a regular flyer. Right? Your top tier can generally check three bags per person on their itinerary, whereas, an average user probably gonna have to pay for a checked bag. Right? You've gotta retrieve that information. You retrieve it through, you know, relevance augmented retrieval and relevance augmented generation on the back end of that. Right? This allows you to understand the intent of the customer themselves, go through the entirety of your enterprise documentation, identify the most relevant snippets of those documents, the most relevant chunks there, retrieve them, and then throw them through the these relevance AI models in order to ground them. Right? We wanna ground any prompt, whether it's a search result that comes back or it's a generative answer. And very, very critical when we're talking about a generative answer that you are grounding it in a fact base, and that fact base has to come from your company. Right? It can't be using, you know, a chat GPT in order on its own, in order to do that. You have to have this relevance layer, this buffer that says, here is my company's information. Right? That is what sent to a large language model in order to generate an answer. In order to use something like ChatGPT is the world's greatest English professor. Here's this chunk and snippets of information that we have based on the prompt that we're getting from the customer. Let's formulate this into an answer. And that this is critical for driving successful answers for customers, grounding it, in the context of who the person is, grounding it in a fact base of your company's content so then hallucinations become next to impossible. Right? The the last one here is where we start to get into this single unified relevance intent box. Right? And you use a combination of the different capabilities that you see here. Things like generative answering. When that's the right thing, that that's what you will, see. Sometimes at search, sometimes the customer doesn't know which question to ask, so you need to recommend something for them. And, of course, you unify that all, with with, personalization on the back end. You've gotta, do all of this in one box, leverage all these capabilities to bring that unified experience, together for the customer, one that has relevance at the core of it. Of course, as you do this, you need to create a continuous improvement loop. Every single interaction, every click, every search, every input, every document that that a customer engages with, can be stored in usage analytics, understanding what works, what doesn't. You know, all these interactions and the interaction data continue to, close that loop and go back and feed the, relevance and personalization layer. Right? That allows you to just get better every single time at tailoring the responses that you have to your customer's needs. So how might this look for a customer? Right? A simplified page, something here in practice. Right? This is the very new, like, digital experience paradigm. Right? This one unified intent box. Tell me what you're trying to do. And regardless of the digital property that you're on, it could be your website. It could be your customer support site. So behind the an authenticated login, that's where personalization really starts to take off. Could be something like your customer community or in a chatbot. Ultimately, what you're doing is you're enabling customers to ask questions in their own language. Don't try to outguess our company's keywords. Right? That that's a behavior in customers we're gonna have to change as leaders. Just tell me what you're trying to do. Don't try to, you know, do what we all do in Google now, which is match up as many of those keywords. No. No. Tell me in your own language what you need to do, and then you match those questions with the right resources for a solution. Ultimately, what you're doing here is augmenting your customer's journey with AI, and you're guiding them through the resolution. So for some things, it could be ranked relevant search results. Right? Customer asks a question, there's an answer to it. We have a short concise document or video. Well, that's what we're gonna do. We're gonna show you the the search results. Some scenarios, might require a generated answer. So you have that capability in your pocket as well. Well, this one's when I've got a very long document or there's multiple documents that the answer lies within. We can formulate an answer from our fact base and show that where that's appropriate. Right? You build trust, by showing the customers the sources of that information, right, through citations and content lineage. This is where it came from. This will allow for additional exploration from your customer base as well. Certainly, you wanna allow customers to, you know, filter answers. Oh, only show me videos. Only show me, you know, things that come from your product documentation site. Right. This is what we call dynamic navigation and faceting here. Right? Those the check boxes that you see on the the side of a search thing today. Second, you wanna think ahead. What might come next in this customer's journey? Whether that's using something like smart snippets here and the people also ask. Because you asked us this, well, people who come back often ask us this other thing. You know, this is where you wanna start to show that. Proactively get ahead of that. It's what, in the effortless experience, we call next issue avoidance. Right? That machine learning models can start to figure that out really quickly what people are asking. Right? Or in a generated world, ask a follow-up question. What's the next best question that this customer might ask? How can we prompt them to ask that, into the the generative answering, model? And that's gonna allow, you know, contextualized conversation to take place here. Right? Sometimes it it's going to have to flow all the way to a human, and that's fine. We're never gonna live in a world where one hundred percent of our things are going to be solvable in self-service. So you wanna flow all of this information through. When it's the right time, here's a human, and that human can pick up the thread of that conversation. Right? That human can just plug right in and move along with that customer, as if that they were all on the same page the entire time. And then, you know, when that user comes back or when that customer logs in again the next time they authenticate, well, you wanna start to recommend the the next thing for them. Hey. Based on what you've done before in the past, here's something else. Right? This is something we're all used to with, you know, platforms like Netflix these days. Right? Wow. You've just finished this show. Because you watched this show and that other show, you're probably gonna wanna watch the the this third one as well. Right? This is what we wanna start to do. This is what a, you know, modern, unified, connected experience is gonna look like for our customers. Right? This is what customers are gonna start to expect and demand from companies in the near future here. Give me a hub where I can converse with your company's data, where I can converse with your company's content, where I can, move that interaction across, modalities if I need to and bring a human in, and then move back to, solving problems on my own when and where I need, where every time I log in, it learns a little bit more about me and continues to recommend things for me as a customer. This is the world that we're moving to. Right? And that world can only be powered by unified AI platforms that allow you to, jump in and and say, man, we we know along this entire journey what your customer has been looking for. So how's this gonna start to to look in practice? Right? Here's some practical examples and practical steps that you can start to take with your customers to figure out when and and where we wanna interject generative answering. Right? So while that single unified hub, it's gonna take some time to build that. There are, things you can start to do today. Right? There there are ways that you can start to deliver, in a outpaced results back into your organization. And it starts by thinking, you know, when do customers know what they're looking for? Right? If they know the thing they're looking for, search and recommendations are perfect. Right? And you take a step back, and you start to group your issues. Group the questions that you're solving for your customers. And these things that are more like level one questions. Right? Things like, what time is the the Logan Square branch on Milwaukee Avenue open on Saturdays? Right? This is a question with a named and known answer. Right? Well, you would just return a search result. In this case, you know, what what I'm showing you here is, what would look like a smart snippet. Right. Well, the branch address is here, and here are the hours by each day of the week. You pop that up to a customer in a box. Cool. I've got my answer. Right? This customer is going to the branch. They're not just looking this up for fun. Right. They've got something that they feel needs to go to the branch. So I've got to go in. I need to know when that thing is open. But smart organizations that they don't just stop there. Right? They start to, make recommendations to their customers right in the moment based on the thing that the things that they know about that customer, based on the inputs that they've got, in that exact search that the customer performed, that input the customer gave, the intent that they're showing you there, and the other contextual things. Who is this customer? What have they done with us in the past? And so you don't stop there. You start to make, you know, guided recommendations. You know, this is something we we showed, which will all eventually, you know, be in one single unified screen. But here, right, if someone's saying, hey. How do I save for college? Well, of course, you're gonna show them a five two nine plan first. But the next thing you would show them, this is the question they haven't thought yet to ask is, how do I set up a home budget in order to save and fund that five two nine plan? Right. And so those are the things that you wanna start to recommend. Again, doesn't necessarily need a generative answer yet. But those more complex questions from customers. Right? The these are the questions where it's multipart. It's multistep. There are several things that a customer may have to look through or one very, very long document, right, where it would take a customer a lot of time in order to find that answer. Those are the things right now that are perfect for generative answering. I I like to think of these as sort of level two questions. Like, hey. What's the difference between a personal loan and a commercial loan? Right? This is where a generative answer would be perfect. Well, yeah, here the difference I'm drawing from, as you can see on the example on the page, these three different documents. Right? One, two, three along the bottom. Oh, okay. So this is grounded in your fact based. As a customer, I can start to trust where this answer is coming from. Right? And start to look at, additional information if I need. And then just right below that, hey. What what's the follow-up questions? Right? You can prompt the customer, like, things like, how do I apply or what's the rate for each of these? Or could you have someone contact me? Right? You allow that conversation to continue, with something that will look at the context. Well, what's the best rate for each? Well, the the, generative model here would know that each means that, personal loan and the commercial loan. Right? You don't have to go back and type those things in because it's following along with the thread of that conversation. It's allowing you to ask that those logical follow-up questions. So generative AI is fantastic when you don't know exactly what you're looking for, but you know that an answer should exist as a customer. Right? This company's got the answer to this question somewhere. Let me start to suss that out on my own. So let let me show you here an example of a company that's starting to put this all together, before we move into our q and a. So if you got questions and you haven't been typing them into the the chat box already, now is the perfect time to to get those fingers warmed up and and start typing them in. But let me share with you an example here of a a customer of ours called Xero. Right? Xero's kind of put all of this together. They're an accounting software company. They're based out of, New Zealand, and they've been using AI powered relevance for some time. So if you think back to those five steps, they have connected their single repository. They are enriching that and using, relevance AI models to get solutions to their customers. First, it was in the form of search and recommendations, and now they've added on the generative AI capability. So that's where we're gonna double down here. Starting to look at some of the results that they've seen as they've shifted their strategy over time to include more and more self-service resolution, and then adding generative answering on top of that. So that's what we're highlighting on the page, the impact of generative answering. Right? Six weeks after they launched this, they, saw a twenty percent increase in self-service resolution, right, as measured by fewer cases coming in. These were questions that they knew used to generate cases. What happened, and why that they attribute, the this success or the three things that you see, highlighted in or, titled in blue on the left hand side. Right? They're able to provide faster answers. Customers weren't giving up on this because I had to read through two or three different documents. Now that they were giving that instantaneous answer, and customers were watching the answer become generated right in front of their eyes. Yep. Okay. Great. This thing this generative solution's doing the reading for me as a customer. The ease that comes with that. And, you know, it was reading their extensive content library and giving an answer right away fast, quickly, easily. Using the the, retrieval augmented generation or RAG, approach. Right? They were protecting the company information. Right. So this was a win for the folks at Xero. We know we put our content in a box, and it doesn't get shipped out to an LLM as itself. Right. We are securing our content. We are grounding any answer, in our own fact base that allows you to trust the accuracy of this. Right? And it follows the permissioning of a document. As you can imagine, in the world of accounting, you might need to be licensed to see certain things, or, you might not be able to access all documents. So that is gonna follow along with that user because they have that centralized unified index. And finally, on the bottom, right, how they boosted click in, as I might need to do this. And so again, in just six weeks, that they saw the ROI of, you know, not just twenty percent self-service resolution increasing, but a forty percent reduction in time to resolution. Right? Customers were getting answers much, much faster. That allowed customers to get on with their day. That allowed these CPAs who have other things to do, right, to get back to those other things, working with their clients or or whatever problem they were working on solving before They needed to engage with the knowledge at Xero. And so a fantastic discovery. We've got more information on this, if you wanna dig deeper into what they did and how they did it. But for now, let's take a step back. Let let's start to see the the questions presentation. I personally learned a lot myself. But let's let's reach back a little bit. I know you covered this. How are some of other organizations testing the accuracy of generated answer? That seems to be that seems to be the point where that you people could have problems. How do you do that? Yeah. That that that's such a good question. Right. And a company like Xero, they didn't just all of a sudden put this out to their customer base and hope that, that things were returned. What we've typically seen, are a couple of paths to to start generating customer facing answers, and both of them involve your people. Right? You create a subset of, subject matter experts within your company, whether they be from your products team, whether they be, your top tier CSRs. Right? The people that think of it, like, these are a great this is a great way to engage those folks who are saying, man, I've been here a few years. Maybe I need to move on. Maybe it's hey. We we wanna envelope you as a tester, as a subject matter expert. Right? And so you're running, you know, a b test. You're running, user testing, through your own internal people first. And typically, you're seeing that take somewhere, between, you know, six to to twelve weeks, just pounding question after question after question that you're seeing using live questions that you're getting from your search results. What are customers currently searching? Let's put that into the generative model, and let's see how that would spit out an answer. And then having that reviewed by humans. Right? Human in the loop. That is a fantastic way to start to test, you know, the accuracy of those answers. What we're seeing is very, very high accuracy when, things are grounded in, your own company's fact base as they are with Coveo. Right? You're you're seeing ninety percent plus, accuracy. Right? Far more than you would even see from, you know, a rep who's six weeks on the job. Right. Fantastic. Okay. So would a single, intent box, as you guys call it, be able to decide when to generate an answer, versus return a search result or some sort of summary? Or how does that work? Yeah. That that that's where this is all going right now, and a lot of that is driven by business rules. Right? You would put certain content, certain, documentation into what we call an index, pipeline, query pipeline in order to return generated results. So if it matches with that, it can return a generated result. If not, you'll see a search result. And that's becoming more sophisticated over time. But, where we see this thing eventually going, if you wanna take sort of a long tail to it is not only could it determine, whether it's a search result or a generated answer, but it could determine the next question and and start to recommend things there or guide you straight to a a human. Hey. This is a question you're never gonna be able to answer on your own, so let's collect some information from you on your way in and pass that information over to someone who either can contact you at your leisure, like an asynchronous style, or can hop on with you right now if this is urgent. So, yes, that that that's where, this thing is going. And I said, it'll take a little bit of time to get there. I don't have a good timeline for it, but, that is absolutely, you know, something that that comes in the future. Okay. It's it seems like you guys have really kind of changed the paradigm of how people are gonna be interacting with, you know, customer support, digital customer support. So beyond pro productivity, what other customer support KPIs will be impacted by generative answering? I mean, you've kinda changed the the landscape. So how are you gonna measure things now? Yeah. That that that's a really good question. Right? And, of course, productivity, is often the easiest way to make a business case. Right? It's dollars and cents. Sadly, for many service organizations, we're still fighting the fight of, are we a call center or a value center? Right. But, you're going to see, my thoughts here, in, measures like customer effort score. Right, you're gonna see that one impacted in the right ways. Right? So effort will go down if and remember, it sounds like a golf metric. The lower, the better. So we we will drive lower customer effort, because customers will be able to, at their leisure, interact with our content in their own natural language. You'll see, you know, things like, CSAT and NPS, other CX measures. They will start to trend positively as well as customers say, wow. Right? Especially with things like contextualized personalization. Right? We already see that with with our customers today. It's like, wow. That company gets it. They know me. They understand who I am and the questions that I'm asking. You already see measures like CSAT, and NPS go in the right direction. So it's definitely a a best of both worlds situation. Great. How long does it take somebody to get going, you know, in a typical midsize comp company? Nothing too strange. It it's gonna depend, but it's gonna depend on a couple of of variables. Right? One is building that unified hybrid index. Right? And if you've got decent content, you don't even need it doesn't need to be perfect. Right? And that's often, a hindrance to a lot of organizations. Oh, we can't do anything until our content is perfect. It's never gonna be perfect. Right. And so you would typically see, an onboarding period, of about three months to get it up and running and to have, you know, the the index getting built again. That that's widely variant, you know, depending on the the company's legacy systems and a number of variables there. But, you know, take a few months to get going. And then from there, to start generating answers, it doesn't take very long at all. Right? Like, the results we saw from Xero, that was six weeks, post taking, you know, actually, they they took our beta product into production, because they Coveo the the answers they were getting so much, and they're starting to continue to refine that. So, you know, from, creating the the index and once you start to have your your search pipeline generated well, generative answers doesn't take very long from there. Like, it can take you about ninety minutes to spin up a a generative model and then, measure in a matter of weeks until we're starting to get that thing humming for you to get the answers that you want to the questions your customers are asking. Well, that's not too much time at all jumping into the brave new world. So yeah. When we build the foundations, it's easier than you think it might be. Yeah. Absolutely. Hey, Devin. Thanks so much for joining us today. I wanna thank everybody else that joined us and and asked questions. If we haven't got to them, we'll get to them, by email. Don't worry. And if you'd like a copy of the presentation, you can use the same URL that you used for today's presentation. But once it gets posted tomorrow, we will send everyone an URL so you can use that as well. Also, if the presentation will be, kept for ninety days. And if you are a winner, of our one hundred dollar Amazon gift card, you will be notified at the end of the month. You don't have to do a thing. So that concludes our broad cast for today. Really like to thank, Devin and Coveo, for presenting, and, that's it for us today.