Hello, everyone, and welcome to today's webinar, how to avoid Gen AI pitfalls, brought to you by TSIA and sponsored by Coveo. My name is Vanessa Lucero, and I'll be your moderator for today. I would now like to introduce our presenters today. John Ragsdale, distinguished researcher, vice president, technology ecosystems, TSIA. Bonnie Chase, senior director of service marketing, Coveo. And Vincent Bernard, director of Kaveo Labs, Kaveo r and t. As with all of our TSIA webinars, we do have a lot of exciting content to cover in the next forty five minutes. So let's jump right in and get started. John, over to you. Well, thank you, Vanessa. Hello, everyone, and welcome to today's webinar. Obviously, generative AI or Gen AI has just been the hottest topic, in tech and particularly in technical support. And I think it's really interesting to look at the maturity curve. I mean, Bonnie, we were together a year ago just explaining what this technology is and how you could use it. But Coveo was very early to market with, some GenAI capabilities, And they're here today to talk about, some of their big customer implementations in full development, employee facing, customer facing, and really what they are learning, what they are seeing. So for those of you who are not quite as far along on your journey, this is going to really help you answer some of those questions that you're struggling with. So what are those questions? Because I'm sure talking to a lot of companies who are, I would say there's a healthy dose of paranoia at some companies around Gen AI, and we're gonna be discussing these today. Some companies still are in that I should build it all myself, which I can guarantee unless this is a core competency of your company is absolutely going to delay, the value that you receive from the technology. The best use cases, if you're watching our webinar series, other parts of the organization are still struggling on figuring out how they can use Gen AI. But in support, those use cases are are pretty obvious, and we'll get a a good list of those today. You know, when we talk about paranoia, I hallucinations is a bit overblown. Definitely something we need to think about, but I am seeing some IT departments kind of seizing on this, as a way to sort of slow progress. And that that's a a bit worrisome when you see, some companies really far behind on evaluating this because of, what may be a little too much concern, about those two risks. And, of course, the big question is ROI. This is a big commitment of, funds for technology and resources and a lot of change management issues. So what does the ROI look like? Because for those of you who may be struggling a bit on budget, you're gonna hear some amazing ROI stories today, and that certainly is gonna help you push forward, by allowing your executives to understand where the return on investment is really coming from. So as I said, this is one of the rare occasions in my very long career that I have seen support really out in front with the adoption of cool new technology. But just the sheer volume of support interactions, and the repetitive nature of a lot of support questions means that AI, in particular, GenAI, is really the perfect tool for support. And companies have always struggled with their knowledge management programs, not enough resources, not enough knowledge workers, one size fits all content that isn't personalized by the individual account. And today, we're hearing that, you know, there's new capabilities being added into cloud products so quickly that customers Gen AI can really have an impact on all of these challenges. And when we survey our members, you see that, sixty four percent know that GenAI is gonna have an impact, twenty seven percent, I think it's going to have a big impact on knowledge management. And when we, look at what, support is doing with GenAI and what they're planning to do, seventy two percent of companies say that they are starting GenAI, projects within support. And, again, that's because support just has such the perfect use case for this, and it's really easy to understand. And beyond just, some of the the slight automation, getting into some more, productivity impacting use cases like root cause analysis. About a quarter of companies are already experimenting with this. Two thirds are, planning on looking at Gen AI for root cause analysis in the next six months. So some pretty aggressive schedules here on how to bring this technology in and and start leveraging it. So, of course, a question that everybody asks is where should I be buying this technology? You can buy, you know, your own LLMs and bring them in and implement them yourselves. But, you know, as I said, unless this is really a core competency of your company, trying to learn how to do that and do all of that training on the large language model, internally is really going to lengthen the time it takes you, to get value. And when we survey our members about where they are planning to buy Gen AI, the mass majority say that they're looking to their incumbent vendor, for intelligent search, unified search, cognitive search, whatever you call it, such as Coveo. Because this analytics platform that you all have been using for years is a very well adopted technology, has been analyzing which content is most appropriate for which question, what the most popular content is. And all of those analytics that you've been collecting for years is going to allow that GenAI engine to really hit the ground running. So it's a lot less time for it to learn, what it should recommend and identify the right content it should be using, and it's really gonna shortcut that process, to getting some real value out of these tools. And the last slide I wanted to show, I mean, we all understand it's a tough economy right now. But when we surveyed our members at the end of last year to see where they were planning technology investments, support has a lot of planned spending coming. And these are some of the areas that we see high planned spending more than half of companies. AI based based technical support technology, seventy percent, sixty five percent on self-service portal, sixty one percent knowledge management, sixty four percent on intelligent search, customer experience, etcetera, and all of these have an AI or a Gen AI component to them. So I know you all have an MBO this year to have some, AI, technology that you can leverage and show off, and your executives are really dying for some great examples of how they're being pacesetters with the new technology. So I'd like to turn things over now to our first guest speaker. Bonnie Chase is senior director of service marketing for Coveo. Bonnie, you've done so many webinars with us, and I really appreciate you coming back to talk about this really hot topic. Yeah. Absolutely. Thank you so much, John. And and, you know, one of my goals for today really is to make sure that all of you watching have some tools to help you sift through the noise. You know, there's so much noise happening around GenAI, so many different options, so many different things to consider. So, today, I'll be covering six pitfalls. This is a disclaimer slide that I wanted to include because we are showing some product today. But, really, I'll cover some some pitfalls that we have, you know, we have seen, whether it's with our our own customer base or with prospects we see in the market or even with ourselves, I've pulled together some learnings for you. There are a lot of slides that I'll go through, but my my goal really is to give you a takeaway that you can then refer back to as you're going through this discovery journey. So the first pitfall that I really wanted to call out is an inconsistent test plan. And what I mean by that is, you know, as you're evaluating different vendors, obviously, you wanna do a POC. You wanna test it out and make sure that it's working for you. But it's hard to get a sense for which vendor, you know, is more accurate or is better for what you're trying to achieve if your test plan is inconsistent. So I wanted to include some testing best practices for you. And this is really you know, this these are some of the the questions that came up in our cycle, that I wanted to share with you as well. So, you know, really taking into consideration who's providing feedback for that experience. You know, you want to have some subject matter experts who are testing it because they'll be the ones to be able to tell right off hand, you know, is this a a good answer or not? We recommend at least four different subject matter experts to get a variety there. Certain things to test would be factual accuracy of the answer, the quality of the answer, and then, you know, whether or not there's a better answer or better document. As far as testing goes, we do recommend starting with at least fifty to a hundred questions that you're capturing and tracking. So I wanna give you an example of one of the ways that we did our testing with some of our customers. We did, kind of an an AB testing side by side. And the way that we did this is we had one model. So one implementation of our generative answering solution, and then we had two experiences. One was for anyone internally in that organization's company. And one was for those subject matter experts in particular. And why we tested with both is those subject matter experts know how to search. They know the keywords to ask for. They know the answers that they need to get. On the other hand, you know, if you have an an open page with with open questions, anyone in the organization can go in and ask any question, get any answer. You can start to see the types of questions that you would get from somebody who maybe isn't a subject matter expert and the type of answers that you get from both sides. And it really gives you this this better view of how this is going to work within, a live implementation. Another thing I would like to share with you is kind of an evaluation methodology. So, again, when you're standardizing your testing across these vendors, have a single way that you're testing. Because if you're testing them differently on different data points, you won't really get a good, a good representation of what's really, the the best. So here's some examples of things that you can use to evaluate as you're testing these vendors. You know, the question category, for example, this can be around the product taxonomy. The question type, is it a how to? Is it a concept? Is it troubleshooting? You know, the difficulty of the question, the accuracy and completeness. So, you know, this list at the bottom just kind of shows the things that we're capturing. That example at the top, this is what it can look like if you're tracking something in Excel. So wanted to do something very simple so that you can see, you know, different ways to track the accuracy there. So that's the first pitfall is, you know, really think about this testing strategy in a standardized way so that you can get the most accurate representation of what you're testing. The second pitfall is really around ineffective questions. And we get this a lot, right? You know, when we have, if you're implementing a generative answering solution, one of the questions we get is how will our customers know how to ask questions? And that's a great question because, you know, the way that you phrase it matters. And it can be due to a lot of different reasons. The context and the way that you're framing the question, whether it's a simple question or an open ended question, the prompt engineering and training that's happening behind the scenes and the ambiguity of the question and how it's interpreted. So, you know, as you're testing, you know, we do recommend to make sure that you're covering a wide range of queries and questions along with multi terms. So So let me give you an example of that. Here's some best practices for questions. You know, if you want to get, you know, need further information, you know, ask for clarification. Could you clarify the main points of? If you want amounts or quantities, using the appropriate quantifying words. You know, getting a more comprehensive answer and combining you to get better answers there. And this is something that, you know, not only to be used with testing, but to to to share with your internal teams or customers, to to help them with with their questions as well. The third pitfall is not tracking value and ROI. And this is a big one. And and for us at Coveo, you know, this is a big one for us because why would you spend money on a platform if you're not getting the return that you're seeking? So this was very important for us from day one to make sure that we were able to track ROI with those customers of ours who are implementing our solution. So I'll share a little bit about the methodology that, that we follow. So this is taken from Coveo's business value team and modified just a bit to share with you today. So really kind of four steps when you're when you're evaluating a solution and and you're wanting to make sure that you're measuring things correctly, that first one is establishing those value drivers of importance. So which parts of the organization you wanna impact? How do you wanna impact those? And then what are those value drivers that meet those strategic objectives? The second step is really establishing a baseline. Now, obviously, GenAI is new. So, you know, what we do is we leverage publicly available information to develop baseline data. And this is, you know, research databases, you know, and and really to see what is standard in the market, and then, you know, establishing your own as well. Step three is really defining a hypothesis. You know, we believe that, you know, by leveraging this, we're going to reduce cases by x and improve experience by x, just as an example. And then step four is quantifying those benefits. So you have your baseline. You're tracking how you're performing with that baseline. And then at the end, it's like, okay. Does this give us a return on the investment? Are we seeing those benefits realized with this implementation? So for us, just as an example, we started looking, at implicit and explicit case deflection. Now we don't like to focus on case deflection too much in general, but as a as a a metric that's easier to measure and capture, that's something that we started with. From a qualitative perspective, also looking at, you know, the query term length, time spent searching, time consuming knowledge, accuracy, and then still looking at the case volume and and return visits. And what that can look like from a value perspective is, you know, ultimately, we're looking to lower that, cost to serve. And then from a qualitative perspective, you know, we're looking at usage patterns, the user experience, and continuous improvement. From a self-service side, here's just an example of what that looks like. You know, when we take that four stage approach, ideally, what you would see is you're tracking your key metrics. You've identified what those metrics are. These are what we've kind of captured in the beginning. And then, you know, you can see your uplift and what you're actually, seeing, and then that could bring you to the value that you're saving there. And then another example for assisted service, examples of those metrics here. And so one of our customers that, implemented this is Xero. So Xero, participated in our beta program last year. Today, they are are live with their customers, so it is a customer facing experience. And here are some examples of, you know, what they went through with their testing and and how they were able to get some results. So they did AB testing where some of their users had Gen AI and some of them didn't. And then they compared, you know, the success of those two different parties. And what they found is they really were able to increase implicit case deflection and decrease the time spent searching. So pitfall four, this is a big one that we get. And it's, you know, not completing proper content scoping. There's a lot of fear around making content, making generative answering available to your customers with available or not publicly available, but your customer facing content. But, again, you know, it doesn't have to be as big of a fear if you're doing the proper content scoping. So the importance of content scoping, of course, risk of information leakage. You wanna make sure, you know, if if you're, if you're bringing together internal and external content and that support agent that's doing the search gets an answer that's combining both, you don't necessarily want that answer going to a customer because even though the agent had access to see that, maybe that customer doesn't. So you want to make sure you know what type of content your agent is having access to that they are sharing with customers. And then the second piece is really about the precision and relevance. So the more targeted the content scope, the more accurate and relevant the generated answer. So looking at a few use case, you know, obviously, some support and community self-service. You know, you really wanna look at, you know, the content that's appropriate for that. You know, if they're there to to seek question or seek answers to common product related questions, and you'll probably have more product related content, FAQs, troubleshooting guides, things like that. From a workplace and intranet perspective, again, this will be more in-depth, maybe containing proprietary information, really focus on, those employees. And then for the support agents, again, making sure that if they're getting generated answers that they're sharing with customers, that those are the validated answers that are customer, safe. Right? So, again, don't just throw a bunch of stuff into this experience hoping that it will create the best answer. Really think through every stage that of of, every piece of this so that you make sure that it it's successful. Here are some best practices for scoping. So for us, you know, we like to segment our pipelines based on the intended audience. So making sure that, you know, we're not overlapping, you know, that customers and agents are gonna get access to the same pipeline, for example. Doing regular audits, having a feedback loop, obviously, you know, adding a top toggle button, to your experience because that may not be something you know, not everybody needs a generated answer. And then, tracking the content that's used to create those generated answers. And then finally, on this point, it's about optimizing content. You know, we get a lot of questions. You know, how do we make sure our content is ready for this? Well, the first thing is avoiding very long documents that cover multiple topics. This can this can you know, you know, text may be similar even if the topic is different and you may not get the answer that you want. And if it's really long, you know, the system that you're using may not actually read through the entire document. They may actually stop at a certain point. So when deciding on the content to use, there are a few, ways to optimize it. You know, prioritize the content that's designed to answer questions. Prioritizing content that's written in a conversational tone. Making sure they're short and focused on a single topic. You know, each document should be a sing single language. You know, and then choosing a reasonable size dataset is key as well. Another thing I wanted to call out is as you're getting your content ready for this, you wanna make sure that your boilerplate content, like the headers and footers and navigation elements, are not part of the body when, when your items are indexed. And and the reason I say this because as knowledge managers, you know, we get so used to putting in all the metadata and and all of those details in addition to the the content. But those large language models are looking at the paragraphs from the body of your content. So you need to make sure that it's clean. The body is what's important, and those keywords and key answers are in the body, not necessarily the the meta metadata or headings. So pitfall five is underestimating deployment time and resources. And this is really what John was talking about, you know, in regards to build versus buy. We get this all the time. I know, you know, Vincent has gotten questions like, well, can't I do that myself by doing x y z? And it's like, I mean, you could, but we're here to help you achieve outcomes, not just check a box. Right? So, you know, in thinking through the total cost the resources that you need to hire to do these different things. You know, considering the security, considering the connectors, considering all of the data that you need to to to have to run this, plus the experience that you need to provide, plus the maintenance, plus, plus, plus. You know, this is a technology that's going to continue to evolve. If you think you can set it and forget it, you're going to hit a roadblock, and it's you're not going to be getting the experience that you want. So, again, this is where we we really try to emphasize that, you know, if you're in it to build it yourself, you're going to have more dependencies on IT, on developer resources. You're going to have to pay your employees to maintain, develop, and continue to optimize. And at the end of the day, you're not going to be able to catch up to those vendors who who do this as their as what they do. Right? So that's just something to keep in mind. And then, you know, just quickly, this is just a summarization of how we view it. Right? You've got the total cost of ownership, delayed time to value, and then the limited time to value versus having a platform similar to Coveo where it's cost effective at scale. It can help you accelerate time to value, and it's a subscription to innovation. So as more things come out and it's continuing to evolve, you're getting that automatically, and you don't have to worry about it. Right? And so this kind of ties in nicely to, pitfall six, which is not leveraging a search platform. Now, obviously, Coveo has search at its Coveo, but we do really believe strongly in, you know, having a search platform like Coveo to help you with implementing an experience like generative answering. Because something that we need to remember is people search in different ways for different reasons. You know, a keyword search is good when, you know, you need to find a keyword. Semantic search is good when you're wanting to to get, you know, contextual information. So people want these different ways of searching and getting answers. They want the recommendations. They want to be able to search, and they want generative answering. These should not be completely different systems and completely different experiences because the goal really is we still want that effortless customer experience. Right? We don't want to make things confusing and difficult. So taking a holistic approach with a broad set of capabilities is really what will help comprehensively transform the digital experience. And so, you know, Gen AI with RAG, which is the Retrieval Augmented Generation, is just one of many techniques. And so with a search platform, you're able to bring it all together. I wanna take a moment to talk about RAG because this is something that, you know, is is key with generative answering. Again, it's retrieval augmented generation. And this is where, you know, the system goes and retrieves the documents and the paragraphs that are important to answer the question and sends it to the L. L. M. To generate the best answer. Now many of them miss on crucial aspects for the enterprise. And this is, you know, that whole infrastructure to manage, standardize, tune, control, and self optimize the ranking. And they often reduce that retrieval part to semantic similarity. But semantic similarity is not enough because it it won't provide the most relevant information. Now John mentioned that, you know, if if your search platform is already capturing all of these analytics and data points about your customers, that context can be layered into providing the answer. So instead of, you know, finding them, like, something similar to what I'm searching for, it's finding something similar to what I'm searching for, what's relevant to me and my experience with your system, who I am and the data that I have. So it's more personalized to me. So when we talk about that general search experience, leaving GenAI out of it, I mean, that is what a search platform does. We wanna make sure that it's the most relevant information is at the top. And so when it comes to generative answering, you don't just wanna send all the info and generate answers. You wanna make sure that the most relevant information is sent to the LLM. So that's the key difference and and one of the the things that makes a search platform, beneficial and more, makes the experience better and more accurate. So when we talk about search relevance, you know, obviously, there are a lot of things that can be included in that. But, again, these are the things that a search platform can bring to the equation that all help complement generative answering and make it more successful. So, you know, things like the unified index, things like the analytics and insights, the personalization, And then at the end of the day, making sure that generative answering is is threaded into your entire experience, not a bolt on on the side. So that being said, I wanna invite Vincent to go ahead and do a demo kind of showcasing how we've been able to bring it all together, after going through rigorous testing with some of our customers. Thanks, Martin. I'm quite excited. I have a lot of content to share today, so let's get started. I will use Coveo as a client to showcase what happened. So behind the scene when we were building the feature, I've been involved heavily with r and d in the in the whole development process of that feature. And then we decided to deploy it at Coveo as a client because we needed a first a first client to try it with us. So we converted the Coveo documentation first and then all the other properties after, into a, what we call a customer zero. So we tested our feature with our own stuff. The first one that we've done, and we've done it that way because this is a simple deployment. It is a public website. This is not a community. This is not something gated. It's not Salesforce or anything based like that. It is a very simple website. So the content is easy to grab with our connector, and it's easy to build. So we decided to start with this. Since we know our product, we were able, like Bonnie said, to evaluate the answers as well as subject matter experts. So that was the first story here. Let let's go and try it out, explain how Cavill, uses permission. So the way it works, if you haven't seen the feature yet, is that we are using at the top here of the, interface. We're streaming the answer that is coming back from the element. So it is a classic, I'd say, pattern that we see now in a lot of different search vendors, but that's the one we think is right for that kind of experience. So if you look at the bottom, you'll find all the classic, actually, documents that are returned. And this is why bolting an LLM on top of a a relevant search platform is so important. See the quality of the results below here, like source level permission, permission sets. They're all highly related to what I'm looking for. Even PDFs are brought back. They are coming from a variety of platforms. So documentation, they can come from our level up learning platform. They can be coming from YouTube. So those are all, like, we're grabbing public content that belongs to Coveo, and we're bringing it together. An important part as well here is to see that in the citation of that answers, you can see that we're using multiple documents to merge them together and give the best answer to the user. So that was the first deployment. That was the phase one, in our journey. And then since we, decided to make Coveo customer zero, we said, let's go through the full journey of having every touchpoint available by our clients to be interacting with that feature. Properties just to show you the range. Because when you're talking about build versus buy, you can build this experience here. But then after while you're doing it, I'll be deploying on other touchpoints while you're still building it. So time to value is extremely important. The second deployment we've done is on our Salesforce community, so Coveo partner community in this case. Again, very important deployment here because we have it is a gated community. So there is a public part, but there is also some internal content that is hidden and only available for our our implementers or partners. So here, a little bit more, complexity. So let's go with another query, how to, install Kaville for Salesforce. So a few things that are different here. This is using the native Salesforce, UI. So we're not it's Coveo components, but in the build that we call Quantic, which is actually our integration in lightning components in Salesforce. So native connectivity here, we also brought in some knowledge articles from Salesforce and something behind logins. So at this point, the level was a little bit, the I'd say the difficulty was a little bit higher. But behind the scene, we're reusing the same search platform, the same document. We just started to pile a little bit more here and add security on top of it in another UI. So, again, you'll see here, for instance, we're gonna have an event that is coming, or or that just passed a few a few a year ago regarding installing and configuring, Kaviyo for Salesforce. So this is kind of a on-site training or something like that where you're gonna find a lot of information. You have support articles in there, deployment that is similar to the documentation website, but with a little bit more. Once you get your documents in the platform, it's pretty easy to do. Bonnie touched upon an interesting point, which was scoping your content correctly for the given interface. So I'll show you how you can do it if you jump in the Coveo cloud platform. So this is the org behind, this is the Coveo organization that is powering the different interfaces that you saw. I'm in sandbox, obviously, because I don't wanna mess up with with the others. And when you're building an experience like, the one you saw, you're actually stacking machine learning models. So, basically, the storyline here is simple. You can have semantic encoding, and you can also have, relevance generative answering models if you want. But we're strong believing, we're strong believing in the fact that relevant search is at the core of a good experience. So a classic deployment will start by our machine learning algorithm, like, ART or query suggestion. But then when you reach that that that stage of having relevant generative entering on your website, if you click on it, you'll see the full guided tutorial on how to do it. It's actually very simple. But the key here is that you can go and select the part of the content you need. So you can see here in this example, I have a lot of content coming. So this is for the Internet. You know? It's our confluence internal proof reporting. I won't put that on my documentation website. But then turbo component, oh, that's cool. Those were quick components for customization. Sitecore certified, developers, also cool, Klaviyo and YouTube. So you can just go there and select a few of the, sources that you think are interesting. And then you're gonna have here at, live kind of an evaluation of the content, which content is in English. We're a French company here in in Canada, so we have a lot of French content. So it's just gonna give you, for now, with the given limit of the product, what's the outcome of the documents that we're gonna slice and serve. So very, we have good tools to help you and guide you in your deployment, making sure that the content you select is relevant and also to make sure that you're doing the right thing. Back to my deployment on Coveo customer zero, once we were done with the community, we said, why not put it directly in the product? So this is our platform. We got here at the top a little sign called, help. It has been released fresh off the press today officially, but we now have, Kaviyo relevance generative answering directly here in, in our end product. Well, first off, you you'll see it's relevant. When you open it, I'm on the machine learning section. Right off the bat, I get recommendation for things like, hey. Do you wanna know how to inspect or or check your advanced config? So it's really already ready. But in this case, I I I think Bonnie is now very good with the technology, and I like Bonnie to come and help me building some of those models. So I'll ask, can I get Bonnie with me here, or can I remove someone or add someone for the organization, for instance? So if you fire that query here, right in your help section, you're gonna have to generate an answer that's gonna guide you in your product directly. So at this point, we have Coveo on our public website. We have it on our community. We have it in our product, and each deployment comes with benefits. So what I mean is that every time we're deploying it, we see less call. We see less people asking questions. I I guess less involved with complex client asking questions because they can get served themselves with the place they are, you know, automatically. But we decided to go further. Since we have the license, we have the organization, we have all that content that we brought together for these experiences, why not put it in the hands of the employees as well? So at this point, we decided to go with Coveo at Coveo, which is here an internal, search engine for our employees. I can go and search for product documentation. I'm gonna find even, like, the Confluence articles that are for for r and d when we're working on new features. But most of the questions in these environments are HR related and and kind of, you know, behind login, obviously. So if you go and do a search like how to onboard a new employee sorry. I'm gonna I need to log back here. How to onboard a new employee, then you'll see at this point that we're we're able to extract these, very cool answers from whatever systems they are. In our case, we're using Workday internally, but also Confluence and other systems to give you something like that. So, it's also working for product questions. So explain how does the way you use permissions. You're gonna have the same answers as you had on the other end with even better content because because here you're gonna have the full internal stuff, as well. So that's what we've done internally. We think we're we're very proud as Cavill customer zero and obviously proud of this feature. But now let's go and jump a little bit on what the clients are doing with it. The clients that deployed it, one of the favorite deployments we have in the first one is Xero. So Bonnie touched upon it, regarding the return on investment they bought. You see here, it's a very cool, community. It it looks great. It's simple. It's easy to use, and the more people uses the community with Coveo RGE, the better they become at it. So it's not just, in a, ROI that is done one time. It's actually coming better and better because people starts to understand the potential. So you see if you click on the query suggestion, right off the bat, the queries are getting a little bit more complex than before. Two years ago, you only had, like, single keyword or two keywords. Now you start to get full sentences in there. And when you click on these full sentences, you're gonna get redirected to the search page, and you're gonna have a generated answer that's gonna help you. Instead of having to read these articles or to go in another page to find your content, you're gonna find right, here on the home page the generated answer. And you see it's blazing fast, super accurate. Accuracy is reached because we're grounding it to the content they are creating, and this is how you get a very good experience. Bonnie give guidelines regarding, like, indexing documents and asking questions. We're also now releasing guidelines on on the best way to write your content to make sure that the content reacts well with that feature so you can have a good experience like this one. Also wanna thank Xero for the very punchy colors they put there right in your face. So they're they're very proud of their feature and and so are we. Another very cool cool deployment I'd like to, look at is Dell, Dell dot com, so Dell support. Same experience here now that we have. And Dell is complex. Look. Zero is a b two c. Dell is b two b, very complex product. And even here, if you start typing how, for instance, you're gonna get very good queries that are now human, driven. So this is the human that are generating these queries, but the human is being, conditioned to a UI that is more natural now and and giving better answer. Dell is very interesting because they have a UI packed with feature, no surprise. So they have the creative generating at the top here, but they also used here, some people also asked. So this is a feature from smart snippet. I'll go a little bit later on how we're seeing it in the road map, but right now, this is what they have and also some badges on, like, this is a recommended article from machine learning. So they are packed with feature, and they get quite good deflection out of it. Last stretch on my end, you've seen clients. You've seen how Kuvay is using it. Now I'll just want to introduce the next steps for us. Because while we are scaling it with clients and we're deploying it with, across the world with different of our, largest client, we're also building a new generation of it. So that's Bonnie's sentence about innovation. You're subscribing with Coveo to innovation, not just having a future today. While we are deploying this, there's a team behind the scene working on the next generation. The interesting one here would be, this one. So how to add a facet to a search interface? This UI here is a little bit different. It is what we call conversational. So conversational has, two main features that I wanna share. The first one here is the rich markup. So it's, it formats the content better. It's gonna have, like, encode. It's gonna have tables. It's gonna have, bold fonts and tiles. It's even even more polite because we're in a conversational mode. So I asked here how to add a facet to a search interface, and it's gonna give me actually the right thing to do. But in my case, I don't wanna do it with the search page editor. I am a developer. I wanna do it with, Atomic, Atomic, for instance. So here, using, let's do here and do using Atomic, for instance. So you can, you know, continue your no. Let's try it here using real atomic just to give it a little bit more context. So what's gonna happen here is that you're gonna be able to fulfill the blanks. So it's gonna say how to add a facet in a search interface using Kavi Watonics. So it's gonna give you actually a conversational approach where you're gonna be able to go deeper in a topic without having to write back in. The other interesting part so you saw here, I used the ask follow-up question, and I wrote a question myself. My question wasn't complete enough, so I got a no answer. To avoid that and to avoid no answer and to make sure we're increasing, the amount of answers provided by the system, we can actually provide questions that are already written so the user has a a way to make sure that the next question will bring results because it's actually formatted the right way. Last one here, number seven here, you can also ask some more complex things like, table formatting if you wanna compare two different topics here. So in this case, sitemap or web connector in a table is gonna give you the whole, breakdown for, the two different components that you had to show. So that was it for me. I've shown you the journey of Cavill customer zero, our clients, and what's coming up. So this is obviously a better feature. It's not, in production yet. We're aiming to make it a productized, feature in this quarter, actually. So it's gonna come, accessible for you in the during this q two twenty twenty four. Bonnie, back to you. Awesome. Thank you so much, Vincent. So that concludes our presentation for today. We hope you got some really good takeaways to help you on your journey and and, are excited about some of the the things to come. Okay. Thank you so much, Bonnie. We are, unfortunately, basically, out of time we have a lot of questions. And because of that, I'm actually gonna shoot you a question. I think this I'm not sure if it'd be best answered by Vincent or Bonnie, but if we could make it as quick as possible. We have Cheryl asking, does the model provide an answer if the confidence is low? So quick answer is no. So we have thresholds, multiple thresholds in the pipeline. And if the confidence is too low, we're gonna give you search results, but we prefer to stay silent instead of giving, an incorrect answer. That's how we prevent hallucination. Okay. Thank you so much, Vincent. I know, like I mentioned, a lot of questions we couldn't answer here live, but don't worry. We will follow-up with you. We haven't forgotten about you. And since we have come to the conclusion of today's webinar, I now really like to just take this time to thank our presenters, John, Bonnie, and Vincent for delivering an outstanding session. And thank you to everyone for taking the time out of your busy schedules to join us for how to avoid Gen AI pitfalls brought to you by TSIA and sponsored by Kaveo. We look forward to seeing you at our next webinar. Take care, everyone.