Hey, everyone. Welcome to this first, this first edition of the, or rather first presentation first session, I should say, of the partner borrower hours, the JNI series that we're doing in August. My name is Alex Moreau. I am a senior partner solution architect at Coveo. I've been at Coveo for over eight years and helped, or couple hundreds at this point, implementations at Coveo, making sure they were able to get up and running. I'll be your host for today. I'll also be the host for next week, when we have the workshop. And, so my the the goal of this presentation is really just to give you an introduction to what is GenAI, you know, what is the art of the possible, what value can get out of this out of out of it, you know, presenting what's the purpose, why it was built, etcetera. So I'll get started right away. Just wanna just wanna make sure, you know, that we that we get into it. There we go. I do have a disclaimer. I'm not gonna spend too much time on it, but, we are a publicly credit company. This presentation includes forward, looking statements. So, essentially, don't make, don't make decisions based on what you see in this presentation. So that's that's essentially what it means. You know? So as as I was saying, you know, the partner power hours, there's there's effectively three, presentations. So, technically, it's four, and presentations for, for for this this series. The first one of today is the art of the possible as you can see, and we're just doing an introduction, the the audience for everyone. Next week, on the thirteenth and the fifteenth, we'll be doing workshop. So on that one, you you'll be provided a conveyor organization. You will be provided a, you you'll be provided essentially a site that you can, crawl, and then we'll be building our own RGA, search interface so that you can actually, leverage it and see how easy it is to to set up, but also what you can do with it directly with your own hands. And, finally, on the week of the twentieth on the twentieth itself, I should say, there's going to be a, essentially, a a a more seller enablement presentation that will be given by Liz, if I recall correctly. That one is intended for a commercial, team. So, you know, if you're looking at, how do I pitch Coveo properly, how do I understand the value out of Coveo, How do I convince, people that that the RGA is worth it? How can I how can I prove that it does bring value? This is where you're gonna this is the one session you're gonna wanna attend where where we really go in-depth into these topics. Do yeah. I think, one, small point just one point I wanted to make as well before we actually start. You, are welcome to ask your questions in the chat, or in the in the q and a of the webinar, but please be aware, that I will not be able to get to those questions until the end of the presentation. That being said, I do have dedicated time to answer questions. So as I'm talking about the the as I'm talking about RGA, please, enter your questions in the in the q and a section of the, of the webinar, and I'll get to them at the end of the presentation. Just don't be surprised if it takes a while before I, before I get to your question. I'll I'll reserve them all for the end of the presentation. So without further ado, I wanna get into the a little bit of the why why we're doing, RGA, why why we're we're getting to that. So, you know, at this point, it's over a year ago. ChargeGPD came out and became publicly available to, to to the public essentially. And and, you know, we saw a world where, you know, search discovery, recommendations, gender events during conversations, chat personalization, it all started to converge in a single point of, point of reference, a single place where people started expecting, a a an experience out of, you know, out of how they're touching base with with, with the Internet effectively or the rest of the world. And so, and so, well, let's sort of jump into that. Right? I'm gonna talk mostly about website here, but it's mostly website and service, if you're familiar with the with the the LOBs of Coveo, the lines of the businesses. But, but I think that that there's a lot of parallels to be made there. So, you know, I'm sure we've all been on websites before in the past where there's effectively two search boxes. Right? You have the search box at the top in the header, sometimes hidden, hopefully not. And then you have it sometimes a little sort of chatbot that pops up in the in the bottom right side of your screen, the the one that you're least likely to look at. And and one is asking you to search for a query and the other is asking you to ask a question. Well, at Coveo, you know, at at the end of the day, this is all the same thing. Right? At at the end of the day, the user, when they're selecting either boxes, they're looking for something. They're looking for information. And and at the end of the day, it's always what is your intent. So what we realize is that, you know, by having all of these different experiences sort of scattered across your site, you're sort of diluting the message of what is the point, what is the point of of like, where do I go if I need to find the information? Do I should I go to the search box to get a link, or should I go to the chatbot to get a link after at the end after or get an answer that's based on that link? And so, you know, what we what we believe at Coveo was that, you know, we're able to merge these two together. So keeping this example, if we go into how that works. Right? So we have the the search box and the header and say I type in the bank. Right? This is the classic sort of Coveo, the non RGA side of Coveo. We have our at the bottom, our secure connectors are indexing content into your Coveo index. Right? It it's it's making its way to the Coveo index, and then make when you make a request in the search box, you know, we're we're giving you the results and using AI relevance to make sure that we're giving you the right result at the right time. And and, you know, that's the that's the the classic Coveo formula that we've that we've refined over over decades at this point, to to really give you a good experience, on that side. And then for the the box at the bottom right, you know, it it's using l an LLM. So it's extracting and embedding content from your knowledge corpus that is then creating vectors, calling an LLM, and getting an answer to your question. Well, one thing we that that, you know, if you if you look at that, it becomes painfully obvious. These are very much related topics. So, you know, there's a big there there are issues, right, by having these these things separate. You know, you get different search box, different intent boxes. You have different yeah. Yeah. You have to duplicate the content, the data infrastructure. You may have it in Coveo or or whichever other search provider you're using. It always works the same manner. You they have an index. So you're duplicating content in two different places. You know, the there's separate infrastructure to to handle, separate admin security, and and even sometimes different set of facts depending on on which, corpus you're you're indexing. And that can bring a lot of headaches. So what Coveo does is we merge these two together. Right? Realize that that we're able to do this extract we're already extracting and embedding content when we're using our secure connectors. Adding the vectors is something that we can just do directly from in in the index and then calling the the ELM is something we can do in parallel to getting the results. And that's something we're realizing that is really helpful because all of a sudden, you don't have to worry about is my content up you know, well, is my content updated, but is is the content that I'm managing? Is the secure the same way? It's the same content. So we're the whole architecture by bringing RGA into the search box, by bringing the answer into the search box directly. And that can really help speed up the implementation of both. So in, you know, in the back end, you know, we have the depth, the breadth of content as I was talking, the freshness of content as well. You know, you don't have to wait for a couple weeks before the the index, the index on your LLM side of things is starting generating new vectors. You know, it's it's always fresh in the index. You only have the security and governance I was talking about, and, also, you're optimizing for scale and cost. You only have one infrastructure to scale up. It's much easier than to have to manage two places at once. And in terms of results and relevance, you have a unified search box for the end user that's they're able to search what they're looking for directly in those in the same search box. It's less confusing. They can just go in there and and type what they want and get the results and an answer or both depending on what they're looking for. You get the personalization from Coveo. We make sure that we that that we're able to leverage the answers so that the answer is personalized to who the user is based on what Coveo has already learned from them, and, you know, get protection against illusionist hallucinations. You know, this is these are our our quick, points. I will get deeper into each of them, as we move forward just explaining exactly how that how that functions. It's always important to talk about security, and and this is really the way that you know, whenever I explain and I've explained RGA to a lot of our different, partners in our partner ecosystems as well as customers, right, is is if I get a little technical. RGA works from a a rack standpoint, a retrieval augmented generation standpoint. And so what this means is that we are we are leveraging the index as the single source of truth to determine what the answer to your question is. So we're not trying to learn from the whole Internet and try to find you know, we realize that the answer to your question is on your competitor's website, and then it gets confused. And and it's starting to answer that your, say, your your product does things a certain way, but that's not how you do things. That's just how your competitor is doing things because it's on their Internet profile. And so what we do is we make sure we retain the information. We we are only learning from the information you provide us so that we're only giving, answers that are accurate and and most importantly, demonstrably accurate. So we do secure content retrieval. This is our our bread and butter. It's been our bread and butter for years now. We we make sure that you when you're indexing content, you can have permissions on it. We respect those permissions even in the reg approach. In the RGA, it does respect. It will if you if some people at your company and say you're in a workplace scenario, some people at your company have access to content that some others don't. And and you start asking a question like, when is my colleague going to get fired? This information may be available somewhere in a Google Doc that's indexed by Quavail. But if you don't have access to that information, you will never get an answer to that. You will never you will never be able to to it will never be retrieved. It will never be served. And because you don't have access to it, so Coveo doesn't learn or doesn't doesn't give you an answer based on it. And so we make sure we respect those securities. Something is, I think, vitally important in this, in in this world. The second one is dynamic grounding. This is the thing that I miss the most when I use channel GPT. I wish it did. It does not. But, the the way we do dynamic grounding is that we, you know, we learn from your content. And when we provide an answer, we prove it. We say this here's the information where we got this the the the story, the sites, the links where we got the answer from. So it's we have the citations. There's a reason why in academia, we make sure to cite everything. We make sure that that, you know, that whenever you're you're bringing something to the table, you're you're not just saying something, you're proving it. You say, you know, I got this information from this place, and this is something that our our GDA model does. And I think that is vitally important. Third one, and this is something specifically for legal, I find, but, super important as well, audible auditable prompts and responses. We, we keep track of the the prompts and responses in a secure database that only admins have access to so that if you need to do, an audit on on what your r your RGML has provided, you're able to go and have a look at it. You can you can deep dive in you can dive deep in or deep dive into it rather. Sorry. And really get it get into understanding exactly what's been generated and why has it been generated that way. Importantly, there's zero retention. You know, we talk about secure content. We wanna make sure. So we've we've made sure that with our LLM provider, nothing is being contained, in in their information. We send a prompt, and and it we would get a response, and they keep nothing of that interaction. So we wanna make sure that that we really we we're not, leaking that information to a third party. We're we're keeping it, in house in that case. And finally, the last one is that I'm asking us for the future. But, you know, with tokenizing PII, that that's something that we're working on as well, but I wanna talk about it in terms of security. Now this is what it looks like. Right? If you've never seen RGA in action, this is sort of a a a standard approach to RGA that you see on the right where you you ask a question in the search box, and you get an answer that's generated that is is generating you know, you see the citations at the bottom and the rephrase button. So this this is the way that it works. So we use a grounded LLM. As I said, you know, the scale is up to ten million vectors. I think it's even going to thirty soon. We we get answers answers generated from multiple docs and sources. We support PDFs and HTML pages. We have an improved citations for verifiability of the answer. And then you were, we're in I think last part, we're available. This is available in all of the Coveo regions as well as our HIPAA compliant platform. So you can have access to that even if you're in Europe, if you're in in the Canadian region in Australia, or if you're if you're in a HIPAA compliant platform, you get you you still have access to RGA, and you can play with that. So no no headaches of, you know, sometimes some some features can only be available in certain regions. This one is available everywhere. So how do we do that? This is a little bit more complex. We'll get deeper into the architecture next week. So next week, the the the workshop more for, technical, technical audience will get deeper into into how that functions. But just a a bit of a glance at it. So we do use a security the the we use security. So So let's start over. We use secure connectors rather to to in in index the permissions in the security model, so that we're able to get the right the the right content for the right people. Then we create a, at on the index level, we create a vector, vector, by the way, if you're if you're wondering what that is. It's it's a a multidimensional representation of the terms in your index. So it's it's a a way to understand the semantics of of the content so that we're able to more easily retrieve it, at large. So we we create those vectors. It's it's a mathematical model behind the scenes, but we we have that so that we're able to more easily re retrieve that information later. Then, you know, we do the retrieval of relevance. I think something that's important to understand is that we still use the Coveo. Right? RGA still uses the Coveo recipe. The Coveo machine learning, the one that we've been building for ten years, we still have that available. We we still make sure that we use, models like ART, DNE, you know, the the normal featured results as well, the business rules that you have in place, that's being leveraged so that the more the more relevant content that's that's at the top of the results page is going to be more important to generate an answer so that we are able to give you an answer that is accurate and the most accurate based on what you have in in your index and how you've designed the architecture, the how machine learning is viewing this. And and so that's that's directly in there. Important to understand at the bottom right, behavioral user interaction data that keep in mind. That is just something. How are people interacting with RGA? How is that going to be, used to to to make the the answer, but also the result list more more relevant to that user? We use that personalization in place so that we know the user has a specific product in place. We give an insert specific to that product. And so that that's something that we keep in mind as well. And then, of course, we, yeah, and that's that's one in ten box for everything. And then we do closed loop learning, obviously, to make sure that we have, that we keep the content. Oh, yeah. No. I I forgot. There was a little animation there. This is all good. This is just a a quick, a quick intro to what RGA is, how it works. Let's talk about live implementations, because Coveo has multiple customers now that are live with, with RGA. We'll start with by talking about Xero. Xero has been live for almost a year now. So, they went live in on in September on September twenty eighth of last year before it even was released as a GA product. They were one of our beta customers. They they really were excited to to try RGA, and they've been live since then. They they've they've had a lot of success with it. You can try it directly on central dot zero dot com if you're interested in understanding that. But they they've been live with it, and they're they're very satisfied with the answers that we're able to provide there. Another one that I like to showcase because we're talking about multiple different regions, a huge amount of of of customers, and that is the Dell support portal. So, on on on the Dell support portal, we have RGA enabled as well since February. So it's been quite a while now, that they've been running with it, and they're, again, seeing a lot of results in there. And I'll talk about the results later in this presentation. But but this is already live. You know, we're, for example, in this case, preselecting English as a language to make sure that we're providing the answer in English and coming from English content, but you could you could go deeper in that. And so I just wanna showcase Dell as as a very useful, and and very high profile, customer of Kavero using RGA. And and, the last one I wanna talk about, it's not these are not the only three customers we have, but there are the three that I wanna talk about is United, United Airlines. So if you go on United dot com, you can, this one I I like to showcase this one because we're talking about a very high profile one. A lot of people travel, and and a lot of people have questions about traveling. So what can I bring in my carry on? We're able to that question. But, also, we're able to not answer questions when, you're asking, questions that should not reason be reasonable. Questions like if you go on the united dot com site and you ask, can I bring my child in my carry on luggage? Coveo will not answer because we this is not something that we want to answer. And that's fine that we don't answer. Right? We this is a very high profile one, and so I wanted to showcase it. They've been live with Coveo, for a couple months now, and they've been seeing a a huge amount of success. Let's talk about how, how you connect to RGA. How are you able to to leverage it? So for, the general availability of of, RGA, you know, there are a lot of LOBs that Coveo, are connected in, but the the one that it that RGA is finding the most success right now in, it's the service side of, side of things. So service can be a little broad. That can be an in product experience, and we do have that available in in in the in Coveo as well, in the community. So a a more public forum play a place where you can go ask questions, but also you can go in the search box and ask a question right there, get an answer directly. So a lot of self-service success right there. You can have it on the support portal as you're creating a case. How do you deflect that case? How do you answer the case before you even submit it? Right? You're saving a lot of money when you're doing that. A support agent does not need to even interact with you when when you find your answer as you're typing the search, the the as you're typing the search as you're typing your case. And finally, the last place is sort of at the end of the of the service, and that is directly in the in the agent panel. So, if you have maybe a more, internal content you don't want to put on your on your public site, then, your your support agent have access to internal KVs, and and they're able to, you know, they're able to have RGA directly in their case, panel so that they can answer more quickly, and then you're submitting a lot of time in terms of of employee time. And, you know, getting cases resolved much faster, which I think everyone agrees is a good thing at this point. So, you know, this is what I've been talking about. So this is the the Coveo Connect community. We do leverage, RGA there. In fact, we are trying to leverage RGA everywhere, in our port of contacts, including our Internet, which I will not be able to showcase. But we, we have access to the, to RGA in our own Internet, to to get answers to the questions. It's very useful. I love it. But this is a community support portal, so this is an example. You can see it, showcased right there. In for the in product experience, in the, in the box at the top, so the little question mark, yeah, you we do have RG as well if you start asking questions about how to do things directly from the Coveo platform. This, by the way, behind what you see here is the Coveo platform. So we have the in product experience right there, that you can also, that you can also see in action right there. In terms of the inside panel, we have it for our our own support agents. Right? They they are able to use RGA to to solve cases faster. It's not a a a replacement for the support agents. It's a tool to help them answer questions. And I think this is something important to really understand. It's is that we're there to help you surface content, find answers, and and find answers more quickly. And and, yeah, that's pretty much it. This is sort of a a quick overview for tomorrow. Right? How do you get that to work? And I wrote ninety minutes here, and that's because that's how quick it is to get machine learning or, sorry, to get RGA working. You don't need to spend weeks, on on getting a prototype to work. You the POC can be done in ninety minutes. Of course, the the reason I say the POC is because, when you're implementing RGA, you're spending a lot of time in the QA phase, making sure that the answers are accurate, making sure that the content that you've indexed is accurate, is giving you the right answer, and, you know, flagging if there are issues. But to get the the RGA working, ninety minutes. And that's why next week, the workshop is separated into two two one hour sessions, is because the a lot of that ninety minutes is waiting for the CRJ model to build. And and so that's why we're we're having two sessions next week just to make sure we have that time where RJ is able to build, and then we're able to go and test it the next day. But this is I just wanted to showcase this how quickly it is to set that up. It is, I think, our more our easiest solution to implement, and you get the results right away compared to, for example, query suggestions or ART. For those of you that have implemented that in the past, it always takes a bit of of, of of data and lakes data before it starts, being able to to give you interesting results. With with RGA, it the second the model is built, you all you directly already have answers available to be generated. So let's talk about how you would do that. It's very easy. You've we've if you've ever created a a machine learning model at Coveo, you know you know how to do that. It's the same principle. It's you go in the into the onion console. You you create a new machine learning model. You'll have a generative answer model right there. And then the important part is that you need to select the filter. So we see in the second panel in the middle, where you need to select the filter. So which content do you want to learn from? And then and then we build the model, and and we already have answers. So that's that's very quick to, to value. In order to enable it in the search interface, there's multi it's it's available out of the box with with Atomic as well as with headless. I like to showcase Atomic, more than headless just because it's it's so easy. It's literally a single HTML tag, and and it it handles everything that you see. In terms of in this what the the what we see here in in the screenshot, it's using the, the hosted search page builder. And that one, it's it's a click of a button. You see the right there in the in the settings, you just say, yes. I do want relevance gen relevance generative answering, and it then will automatically enable it directly on the UI. Of course, assuming you're going through the the pipeline that has RJ enabled on it, we we'll talk we'll do that next week. If you're interested in how to enable that, we'll we'll cover that base, next week. But if you've done Coveo implementation in the past, you're familiar with that already. So it it's very simple. Same thing within product experience. It's a it's a similar manner. It's just a click of a button. And this and the same thing with the Salesforce agent inside panel. So that we we've we if you've ever done that in the past, it's it's the same principle. Let's talk about the supporting tools and features. So, you know, now I've implemented it. What do I do? How do I how do I know that it's giving me value? How do I how do I check that it's that that it's working? What do I do with this? Well, there's a lot of ways that you can that you can verify that. So this is a report template that exists directly in the admin console when you create wait. So if you're familiar with the with the admin console already, there is an analytics section where you can go and report on how your search is doing. You can go and report how is Coveo helping me with everything, or is it helping me? And and and or maybe there's content gaps and all these kinds of things. So, this is already out of the box, but we created a specific a a specific template, for RGA. This is all customizable, by the way, but but there is one out of the box where you can really see a lot of information and really dig deep into what it is, but you can even dig deeper if you if you would like. But, you know, I think I think for now, this is giving a lot of value. So we have the for in this in this panel, I'll just talk about it. Right? We see the answer shown ratio. So I mentioned it quickly earlier, when talking about United, but, I think it's important to understand, you don't always want to give an answer. You know, one of the reasons, ChattGPT is is known to to hallucinate so much is because it it does not accept not giving an answer. At Caveo, we think that, you know, in terms of a business value, it's it's it's actually preferable to not answer some questions. And and some questions, you know, don't need to have an to have an answer, directly generated. So this is what we see here, for example, that there's a a only or a forty percent, or about forty percent of the time where we get if we do generate the answer, that's because that's forty percent of the time that we think an answer is valuable to the end user. That's a threshold that we're able to do. This those are thresholds that are that are possible to edit, if you if you need to. But but this is an an example. So we can see, for example, the search events, the click events, and things are going there and how many answers are shown when. Exactly as I was I was saying here. If we scroll down in in that, we have a little bit more information. How many users saw an answer in total? How many what's the what's the the search event click through? And you can compare it with, when there is an answer shown when there isn't. So there there's a lot of information in there. Obviously, you can also do an AB test. Most of our customers, not all of them, but most of our customers before going live with with Coveo RGA, they went live with a, with an AB test. So they would send, say, a half well, half is a lot. And most of the time, people start with ten percent, maybe twenty five percent of their traffic getting an RGA response and then comparing it with, the people not not seeing those answers and they're comparing, okay, is what's the success rate? Are they still searching? How much time are they spending? And and so we're you know, that we can get deeper and, actually, in fact, I will get deeper into that, in a second, in this presentation. So this is directly, you only have access to this dashboard to be able to track those numbers. And then if we scroll down, right, we have the fit the feedback you might have seen in the in the screenshots that I had. There's a a thumbs up, thumbs down. So what are the thumbs up, the thumbs down? When they do give a thumbs down, why did they give a thumbs down? It was and and, you know, we we've improved that panel. In fact, recently, we're we're we're letting people that give a thumbs down gives give more feedback. Like, was it a thumbs down because the answer is outdated? Was it because the answer is wrong? Or is it because they're they're the answer is not related to what you were looking for? So you're able to track that and be able to see what are people doing. You might see these stats you see here are are a little bit, on on the low side, and that's because we we we've been actively and, again, I think it's expected. Right? Not often do people really click on the on the thumbs button when the answers answered their question. They just take the answer and then leave. But it does happen that that, you know, when you have a bad experience, you're more likely to give bad feedback. And, yes, the answer is in Snowflake. So I mentioned it quickly earlier, but, if you wanna do an audit, all of the answers are available in the Snowflake environment that's associated to your organization. This is something only admins have access to, but you you you have access to that for auditing purposes, and I think that's important to mention. Oh, yes. I thought I forgot to even talk about it, but there you know, RGA works, as I said, with the semantic search. Like, we're creating those vectors, as I said earlier, but that changes the way that relevance works. It does add a certain boost to certain, documents. And so we've made sure that in the relevance inspector, you're able to see exactly how much semantic search is, is adding to that that document. Why is the document being, being promoted so high? And semantic search will will tell you that's because, it's semantically as close to what the user is searching. So you you have that information directly in there. The reason I'm bringing this up is because the semantic search model is directly is directly linked to the RGA model, and you you effectively need. If you if you want good results, you need semantic search in order for RGA to be to give you the right answers. Alright. That's all good. That's all now now you know a little bit more about that, but let's stop numbers. Let's let's talk, business value. You know, why why do I go with RGA? What do I expect out of this? And and and, you know, you might be imagining the the value it might bring, but let's talk about real customers and real numbers and what it gave them. So in terms just quickly for our business value framework, we're that we're looking at specifically, you know, the case submission reduced and case inflection increased. We're looking as well as fewer searches per visits and and a shorter session duration. That's because if you get the answer you're looking for directly, if you're with Genii, you're less likely to start searching, you know, searching by narrowing down your search, adding more words, clicking on facets. If you get the answer you're looking for directly, so we're expecting fewer searches and also a shorter session duration because you don't have to to spend more time trying to understand what you're looking for. And then for agent proficiency, we are looking at a decreased average handle time, a decreased escalation, and a decreased, average response time. So that that's that's what we have in the, in the terms of numbers, what we're looking at for the KPIs specifically. So let's talk about SAP Concur. See, a a customer I haven't even talked about. But they they went live with RGA, or rather they did they did an AB test with RGA to see to see what it brings in terms of value. And, they they were able to see a five percent reduction in cases. Now five percent when you're when you're Concur, you're dealing with with hundreds, if not thousands, if not I don't know exactly how many customers they have. But when you're dealing with that volume of customers, how many cases do they have open every week? Five percent is a lot. Five percent can save you a lot of money. And and I think that's something that's extremely valuable. You know, when we're talking in commerce of increasing sales by zero point five percent, that's a lot. That's that's a lot of real dollars that are that are generated there. So a five percent reduction in cases, I think, is important to note. They saw an eighty percent reduction in number of searches per visit. So that's also very good. The people tended to find their answers directly in the in RGA as opposed to having to search multiple times to try to find multiple pages to get to the page that they want, or the information that they want that might be across multiple pages. So that's also very good. And the sixty four percent decrease in content gaps. So there were a lot fewer times when people search for something and got no result. And no result when they're searching for something can be very disheartening. And so we we reduced that chunk. Then that's that's purely semantic search kicking in and and really doing wonders on that side. And that was, of course, done in four weeks. Let's talk about Forcepoint as well. Forcepoint and other of our customers that is using, RGA. I wanna talk about them because, they they they had good comments to make, and I really wanted to highlight those. So, you know, they went live again with an AB test. You can see the exact numbers in here. But in in terms of what changed, over thirty days, they saw a thirteen point four, reduction in or sorry, a thirteen point four increase in the success rate for self-service. And after three months, at fourteen percent. And I think, you know, this is fourteen percent is closer to what we're actually expecting. And that's because, of the machine learning kicking in. Right? The the yes. As I as I mentioned, RGA works doesn't need anything to work. It just works straight away. You create the model, and it it learns and it works. But if you're able to pair it with things like ART, with things like DNE, with things like Coveo suggestions, it's able to it's able to to to learn over time, who the user is, what they want, what they're looking for. And and with that information, we're able to to make a big change in a three month period. We have more data. We're able to understand more of what the user's looking for based on their navigation, for example. And that gives us a lot of of confidence that we're able to to improve that. So I think, you know, this is a this is a a fourteen percent improvement in self success in self-service success rate is is something that's extremely valuable. And they also saw a fifty three percent, feed a positive feedback received, regarding the UI. So I think that or rather from the UI directly. So that I think is very important to understand. A little bit more information. So compared to pre CRGA, their CSAT score was improved ninety seven percent. That's massive. That's that's that's amazing. You want your customers to be satisfied, and and RGA is a good way to make sure that you get there. We they saw a two hundred percent case deflection. So that's, again, massive in how much money that is saving Forcepoint, how many cases that's reducing, how many how much more time their employees have to solve more complex cases that that require more abstract thinking. And and and, you know, that I think that is very that that is very useful. And they saw twenty five percent time decrease in time to resolution. That's, again, that's a lot of time saved for those for those employees so that they can focus on maybe creating new new knowledge bases with the with with what they've learned. Or, you know, you can you can just imagine how much you how much time you're able to save there. A few comments that that Forcepoint has given us as well. I just wanna to to highlight a few of them. It saves so much time for customers and for employees. That is, I think, extremely valuable. I wanna provide them with the resolution to give them what they're looking for. It's a lifeline. Coveo brings all the information together, and RGA creates a very fast action plan. So coming coming back to what I was talking about earlier about, you know, getting the information in Coveo and then generating an answer from that source of truth. And and we're able to really get connected with all of those different touch points, be be so whether the information is on your your public website and and the SharePoint internally, and it's available on the forum that you have, we're able to to to really gather everything and and put it together. And then the highest with Coveo, you have a higher standard of self-service success in case deflection, and, you know, that I think is extremely valuable. So I wanted to to talk about that as well. Now comes a time where I know a lot of y'all are going to be very interested in, the road map. So what do we have planned for the future? Now you saw what we do have right now, the what you see what we have in terms of results right now, what we we currently have. Let's talk about what else can you expect from Coveo. What else can you expect from from RGA? And then I'm sure a few of you are are interested in that. Let's start let's start by something that's a little bit, that I've already seen in action, just not in production yet, and that is the follow-up questions. So you you ask a question. It's the part of the bottom, and we're we're suggesting what other questions you might be, interested in based off of the question that you've asked. This is something that's already available with with the smart snippets model. If you're familiar with that, that's a Coveo model that is also leveraging LLMs, but not, leveraging, GPT, but it is still leveraging an LLM to give an answer. And and so that one, we're bringing that experience to our GA by by allowing these answers to be suggested directly. Yeah. Right there. Similar to that, conversational experiences. So you'll see the the little GIF right there or GIF if you prefer. Just, typing the answer. And then at the bottom, we have a we have a ask a follow-up, search box. So, yes, giving those suggestions, but also allowing you to ask a follow-up question, keeping the context of the previous answer that that you were asking. So this is something that, again, I've seen, I've seen in demos internally. So we're expecting it, we're expecting it very, very soon. It is an early access call. So if you have customers or if if you're yourself a customer that's looking, to add that appearance, apologies. Do get in touch with us. Do get in touch with us, and and and we can we can help you, get that running. You know, that we can get the conversation started to get that running on your side. Multilingual support for RGA. This is something that we've been asked a lot. We have a lot of customers. Currently, RGA technically only works with English. And by that, what I mean is it's it's officially supported only in English. And but we are, actively in q three looking to support new languages. The the two big ones that that we're focusing on, are French and German. That's where we have the most demand for right now. But we're also looking at at Spanish and Italian, of course. And and I know that, quite a few Asian languages like Chinese and Japanese are are are being looked at actively. So, again, this is also an early access call. If you have customers that that have, let's say, a a that they have a a site in in Germany and they really wanna have RGA enabled in their German site, then, you know, get in touch with us. We can we can we can get the discussion started. We can make sure you can you can get access to that in an early stage. So please please let us know, if if that is something that you need right now. But if not, be aware this is something that's coming very soon this year. Another one that I think is very, useful. This is something we're we're, actively testing with United, and that is personalized answers using the using, user context. So things like how many points do they have. And and then, so this is, I think, I'm just I'll just let the screenshot talk for itself. So if you ask which status do I qualify for, Coveo would be able to leverage the information, how many points do you have, and then generate an answer based on that. So because you have thirteen PQFs, you do have the you are part of the pre the premier silver status, and, therefore, you have access to this specific information. So this is, I think, I think very valuable. You know, you can you can see a lot of use cases where, what product do you have? We're able to feed that directly in Coveo so we don't even have to select it in the in the in the in the facet or to specify how do I change my battery on my Dyson, v six. You could just say, how do I change my battery? And it would understand from your from your your product that you have, how how to change that. So I think this is this is something that it would add a lot of value, and this is something we're actively, working with United on on a POC on that side. So you can expect that, you can expect that in the future as well. And, I think this is the last one that I that I have in terms of RoamNet, the passage retrieval, API. So the passage retrieval API, is a way to essentially bring your own LLM. And I'll I'll just add it right there, passage retrieval text chunks. So what it's a a new API from Coveo, so that you're able to build your own RGA. So we you'd be using the Coveo side of things for the indexing, the secure connectivity, the the unified index. You know, we we have all of these, you know, the the aspect of it using the machine learning to be and and using semantic search to be able to retrieve the important information. Then we give you the text chunks. And and text chunks is just your we take those documents and we chunk them in little parts or passages, if you might wanna call them. And then we're we're giving you access to those, and then you can use that information called your own LLM and then generate an answer, based on that. That is something that a lot of our our large customers have have been asking for. They already are paying, say, for an l an external LLM, and and they are using it in some use cases, and they want us to leverage that, with Coveo. Right now, Coveo Coveo does not allow you to bring your own LLM. You we we provide the answer. It's a sort of a a a a built in, part of the the product. But with the passage retrieval API, that's you would be able to to to create your own model based on that. And so you have more control over it. That's some that's something that some customers have been asking for, and that's something that some of our partners as well have been asking for to to create their own, their their own, RGA that leverages Coveo for the for the behind the scenes aspect, but is able to generate their own answer based on that. And then it also means they have full control over the prompting, which is something that right now with Coveo, you only have control over what the query and the prompt is, not for the full full prompt. And so that's an example of why you would wanna do that. And I think that is it for, my presentation. I did wanna keep, quite a bit of time, for for the q and a section. And now what I'll do is I I will go and have a look at the, q and a aspect, see if there are any questions. So now's the perfect time. If any of you have any questions, please, you can you can feel free to ask them in the, in the q and a chat. I'll keep an eye on it. One thing that I do wanna mention, again, as I mentioned is is, next week, we're gonna be we'll be doing a workshop. So if you do wanna get your hands dirty into the into the Coveo product, please come next week. It's in two sessions, one on Tuesday, one on Thursday. We will be providing you a Coveo organization. We and you will be asked to create the full RJ model. I will be there to help you out if you have questions. But I just wanna make sure that that, you know, you you get the opportunity to play with RGA directly to get your hands dirty, if you will, and and really get a a working, prototype at the end of the at at the end the the second session. You have something that's fully working. You're able to showcase it to a potential customer. You're able to showcase it to your boss if you want RGA at your organization. So I think that's something that that, you know, you really wanna to to get your hands dirty with. I'll see. I think I have some questions, coming in. A few questions. Are the ten text chunks based on hybrid search? The text chunks are based on so if you're by hybrid search, you mean a mix of, machine learning sort of, what I would not legacy but like the foundational aspect of Coveo, with the ART, with everything and with semantic search, then, yes, it uses every it uses it's it's it's a way to get, to the part in the formula of Coveo where your you have all of your chunks that are generated, and then we send those to the LLM to with with our prompt and the query to get an answer. We sort of cut it there. We just give you the chunks that that are generated from your documents. So, yes, using semantic search. Yes. Using all of your rules, your business rules, your machine learning. So, yes, in that sense, it is, it is based on that hybrid search if that's what you mean. So I'll type it as in for live. Second question, how are LLM responses compared with ranked results in AB testing? So in in AB testing, we typically have both. So we have the the answers, at the top of the page. So if you go on, let let's say on docs of dot com is is is our documentation site and you go on the search box and you type a question, you'll get a generated answer, but you'll also get the the results at the bottom. We do find that people still click on those results at the bottom. They don't, you know, there are cases when people will just look at the answer and be fine with it, but there are still quite a lot of people still, clicking on those search results at the bottom. So if you're asking, is that good, is that you know, we're seeing in the AB testing, we're seeing fewer searches. So, fewer times do we see search events. And so by by by, by continuation, fewer clicks. But we still see we still see a lot of action on the search results side of things. But that's fine for us. We do expect if and if our LLM is is doing a good job, we expect fewer people to click on those search, on the search results, but we still see a lot of those clicks happening. So it's not it's not replacing. And it we're still able to learn with with our machinery models. And, I have another question. That's that's, I think, very big one. And the, and and I I I'm glad that you brought it up. Does RGA require data cleansing? Yes. In yes. So any search provider works in a a sort of garbage in garbage out kind of manner. If you start indexing weird content or inaccurate content or or, jumbled content, you can expect the results to be as weird, as jumbled, as as as, you know, in in the same vein. With RGA, it's even more so. So cleansing is maybe a strong term. I would say curation is a better term. And by what I mean by that is that, yes, you you need to clean the content, but it's more important to make sure that RGA learns only from the data that is accurate. And so that is something we'll do on the first session next week of the workshop is looking at how do we make sure we get to the content that is accurate? How do we make sure we get that content clean? And how do we make sure they get all of the information that we need? And and so most definitely, we do need that, that data to to be clean. We do find that's where we spend quite a bit of time in the implementation. So we get a POC working, and then we start doing tests and we realize, oh, this information is actually a little bit outdated. Maybe that source should not have been indexed in in or not have been indexed in RGA. But I think that's something to import to to to understand that's very important. You don't need to remove it from the Coveo index if you don't want RGA to learn from it. You just need to remove it from the filter in the RGA model. So and that's something we'll it's it's you'll see it's in the UI. It's very simple to do. But there you can just say you could have a metadata on the on the the data that is clean and that that says, is content r g ready, and then it's set to true. And then in the model, we say only learn from content that that have the value true on is content RTO ready. And so there there is more than one way. Right? There are use cases where you want to return content in the search results, but you don't want to have RTO learn from it. And that's still something we support. That's still definitely doable. So so thank you for that question. I think that was a that was a very important one. Again, this is a major part of the, first workshop we're gonna have next week. And, and and so we do want to get to that. So thank you for that. I'll, answer that. That was all the questions I have in the q and a. If you have more questions, more follow ups based on what I just said, please don't hesitate to ask them. I'm looking at that. Apart from that, I'll go into my next slide. There we go. I wanted to to give a quick shout out to our September fifth, event. It's an online event. You can see the time, right there in, Eastern Time and Pacific Time, where we will be having a a a a partner preview, with our CEO, our CTO, and our VP of alliances, where you'll be able to understand, you know, the ways to collaborate with Coveo to build AI practices, you know, real shareable examples, resources you can share with customers, guidance on on how to do that, and and the full schedule of events coming with Coveo, in person for the rest of twenty twenty four. So if you're interested in building an AI practice with Coveo, if you're just curious about how that works, please do join us there. You will get the best people to answer these strategic task of questions, how to do these kinds of things. So it's sort of a a follow-up to our our our partner series that we have this month. But this month, we're going a little bit deeper into the nitty gritty. And and before I leave you all, with ten minutes to spare, I'd still want to remind you, of the agenda for and I'll go back directly to it. If I go right here, and I think, yeah, let's just present from term slide. There we go. Reminds you of the, of of the next workshop. So, next week, August thirteenth and fifteenth, we have the builder workshop that I will be hosting, for technical teams. And then on August twentieth for commercial teams, how to pitch Coveo, how to how to get value out of Coveo. So you saw a little bit of an overview of that today. But if you wanna dig deeper, how do you build that that BVA? This is something that you're gonna wanna be interested in. So on that note, thank you very much for coming to this webinar. I hope that I will, I hope that I will see you all next week for our builder workshop. Otherwise, I hope that I'll see you all in the cellular enablement side. So thank you everyone, and, and have a good rest of your day.