Hello, everyone. Thank you so much for joining our webinar today. My name is Bonnie Chase, Senior Director product marketing for Coveo service line of business, and I'll be your moderator today. I'm joined today by our director of R and D, Vincent Bernard. Now before we get started, I have a couple of housekeeping items to cover. First, everyone is in listen only mode. However, we do want to hear from you during today's presentation. So we'll be answering questions at the end of the session. Feel free to send along any questions using the q and a section on your screen. Today's webinar is being recorded, and you'll receive the presentation within twenty four hours of the conclusion of the event. Now, this is the disclaimer to do your due diligence prior to making any purchases, and we show this slide prior to any demo of our product. Now for those of you who may not be familiar with us, Coveo has been building enterprise ready AI for over ten years. And today, we deliver that at scale for around seven hundred leading brands across the world and with the most trusted and highest level of compliance and security. We offer one single platform that connect and enterprises experiences across commerce, service, website, and workplace creating more positive customer and employee experiences, and in turn, creating transformational business outcomes. And we do this through semantic search, AI recommendations, generative answering, and unified personalization. As you can see over the years, we've continued to innovate and build upon our platform with generative answering being the latest addition. Our perspective is that you shouldn't be thinking about generative answering in a silo. Rather, we have to remember that it's part of the overall experience of your customers and or your employees. We know that experience is today's competitive front line. We've seen all the data points indicate why we need to improve our digital experience. And, you know, we we are in a lot of discussions with our customers, you know, around how to implement generative answering, you know, is this something we can build? Do we really need to buy a solution? So we want to remind you not to sacrifice the experience for the sake of checking a box. There's no magic bullet and it needs to be taken into consideration alongside the other elements of your digital experience. You know, we made this this mistake with chat bots. We've been there before where, you know, we we kind of thought that chat bots would be the solution that would start deflecting all of our cases. We started implementing it. And what we found is that it didn't really work. Right? It didn't work the way that we wanted it to because it didn't deliver on the outcomes. And part of that is because the experience wasn't there. So it was kind of a disjointed experience from the rest of the channels. You weren't really able to access all of the information and get all of the answers that you need. So again, you know, when you're thinking about generative answering, we we don't want to sacrifice the experience because that's what really delivers on the outcomes. And depending on how you implement generative answering, that's going to impact accuracy. It's going to impact personalization. And we know ultimately bad answers lead to poor experiences which can result in higher costs than customer churn. So you're looking at, you know, churn risk where, you know, customers will abandon a company if they don't have good interactions. US companies lose seventy five billion dollars a year due to poor customer service, and people are three times more likely to spread news for customer service than than positive experiences. So it's really critical for organizations to be thinking about the experience when they're thinking about generative answering. Now the key elements of a good experience, you know, are are all of these things. It's personalization that's tailored to each individual, it's ease of use, making sure that it's an effortless experience. Consistency across touch points. You don't want to have, you know, one one experience in your generative answering solution versus, you know, a different answer in your support portal as an example. Efficiency and speed as they're interacting with your digital channels, problem problem resolution, you know, being able to answer their questions wherever they are, however they interact with you, and then empathy and understanding. You know, do you know who your customers are and what their preferences are and are you helping them solve their issues without effort? So ultimately when we think about this, you know, leading up to our demo, keep in mind, we're not thinking about GenA as a separate and siloed interaction channel. It's not going to replace search. And ultimately, it's using the same data, the same users, and the same contact context as your search experience. So we see this new more powerful unified digital engagement paradigm powered by semantic search and generative experiences. So this is one single unified intent and content interaction mechanism that's guided and gives you that single source of truth. Now with Coveo, we bring it all together. You know, as I said earlier, semantic search, AI recommendations, generative answering, and unified personalization that can happen across any of your app application sites and technologies, whether it's an employee experience, a customer experience, or you're really looking behind the scenes from an IT perspective. We also extend into any UI or channel. So wherever you're having that digital experience hosted, we can push content there And then finally, you know, bringing in all of your enterprise wide content, we index all of these securely and bring that into the generative answering experience as well. Now what makes it even better is our search relevance and our foundation of search that makes this more possible with from a generative answering effective. Because when you think about generative answering, you know, you're taking information, and you're retrieving content from your sources passing it to the large language model to generate a result, you wanna make sure that you're sending the best content and the best information to that large language model, and that's what we help you do. Now I do wanna, you know, summarize before we pass it over to Vincent for the demo, you know, we do have customers live with our generative answering solution. So we see a lot of, a lot of demos that are maybe partially complete. Maybe there's some hard coding, but our solution is GA. And we do customers who are live with it and already seeing, great results. So zero is an example of a company who has been customer of Coveo since twenty seventeen, and they leverage us across their in product experience, their agent console, and their support site. And, you know, by leveraging generative answering within their customer facing experience, they were able to see an increase in case deflection by twenty percent and that search effort going down by forty percent. So, you know, again, real result we're seeing by thinking about generative answering in the context of the overall experience, not in its own silo. So to show you how this works and what this looks like, I'll go ahead and pass it over to Vincent for the demo. Thanks, Bonnie. Let's get started. I'm excited to be here today. I've been doing quite a lot of these webinars in the past years, especially in the year for RG specifically to show everything we had and the new stuff. Now today, is the time for us to showcase where we are with the product. We have been releasing it at the end of twenty twenty three. We are now one quarter in in twenty twenty four, and there's a lot of new things that I wanna share with you. As long as new, implications, I'd say, or new deployments that we're using. We'll mostly use Coville, our own company as a guinea pig for these experimentation and demos because we are a positioning ourselves as customers zero of our own products. So the first thing I wanna share with you today is, the new features that we're releasing that are coming soon. So you get a a in everyone premiere everything that's gonna be released in during the next month. So let's get started here. What I have is the classic Coveo search interface. This interface here is looking at the Cavell documentation. So, basically, if you've been implementing Cavell, you'll probably recognize what you see here. If you don't, don't worry I'll go slow and explain what's happening. So the first thing that we've released, or we're working on right now as a prototype part is a way for us to get better answers, formatting. So this is what I wanna showcase here. How to do localization in atomic? This query is from a developer. It's asking how can I add multiple languages in a search interface for instance? So if you are in a software company, you may have these technical questions that are coming on your portals. So when you start querying, you'll see that Kaville will generate the answer. But if you're not aware, we're relying on GPT three dot five turbo, with the available relevance behind the scene that is actually pulling the information from your content. What I wanna showcase here is mostly the new rich formatting. So you can see that we have very have very clear that are numbered. So we have clear headers and titles. There is also now new code formatting for easy reading for developers and these highlights are scattered all across the description, making it very easy to read. So if you have code snippets or if you have rich markup, we're using a markdown syntax here format it and make sure that the answer is is even better than before. So this is the new, answer type as we call them. We have different answer type with Cravail, and this one is the newest we're releasing. A second feature that is packed in this interface is a way for a user to continue a conversation with RGA. So what you see here is that I've asked a question how to do localization in atomic. Which is pretty cool. But now let's say I wanna continue this discussion. Right? I wanna continue kind of having a conversation and asking a little bit more. So here we added what we call ask a follow-up. In this case, we're gonna keep the current state of the page and the current question that the user have. But we'll just go a little bit deeper. So in Coveo, you can deploy your search interface in your system. Say you have a support portal. You're going to put our search interface over there, but you can also use what we call a hosted page, which is another technology we have So here, I'll continue the trend. So you see at the top my question was how to do localization in atomic, but let's say I'm in a hosted context I'll ask in, in hosted page. Sorry. And you say, oh, wow, even if you if you got a chart still able to understand or at least deflect what you want, but let me just go here and continue and host the page and give you a little bit more context. At this point, it's gonna still understand that I was in the context of doing localization but I asked how to do localization in a hosted page. So you don't need to type again. You don't need to be very verbose. It's actually just picking up what you already told to the search UI and giving you a little bit more, I'd say precise answer. Again, here back to the rich formatting I was talking about. You can see we have like these blue highlights that are just styling. They're personal and you can personalize them based on your brand. And then you also have these code snippets that are super easy to follow. So now that I ask how to do localization in atomic, and and then I I went a little bit deeper asking for a specific thing. The other feature we're looking for is to suggest some some next actions or next questions. Sometimes the user don't wanna type anything. You don't wanna type long sentences, even if the interface is letting you do it, people may be on their mobile phone or they're generally just, not entitled to to write very long sentences. What we can do here is use previous intelligence from other users and also the LLM and your content to try to find the next best best action. So here, how can I access and modify I eighteen next instance in atomic library super complex and and and detail question, but at this point, you'll see that it's just gonna understand and feel, feed the response based on the whole conversation you have? So you can go back in the previous sensor. These are all, closing themselves up. And if you click on one of them, you're gonna find the previous sensor. So very neat interface here of stacking conversations one after the other. If you want to start all over again, you can just write another basic query. It's gonna start from scratch here. Just simply giving you an answer, for your next piece of conversation you have. So those components as you see them are a part of our atomic framework, which means that if you already have COVID or GA, you're gonna be able to leverage these without any updates whatsoever. Those are the capabilities we're shipping as we go for this new feature. So it's it's kind of fun to see that Cavell even if you are implementing a solution like a generative answering solution The time that you're doing the implementation, our R and D focusing is still on new features, and you're getting quite a lot of, innovation in a short amount of time. Which means at the time that you take to test, and we've been doing that for a lot of clients. We're we're helping them validate the answers, testing the quality the the responses, And this takes a lot of time and that's time that is well invested. We need to make sure that your content is good and the answers are good. But while doing so, you're not innovating and building new stuff. So this is why we think that pairing up with a ready to go solution like Coveo will accelerate your development, in a certain Now I'll jump in Coveo cloud. Most of these webinars we've been doing, we've been showcasing how to query and see the end result that a user would see. But let me show you how would you configure that. So in a Coveo cloud organization like this one, The first thing you have to do is to build your content. So if you're not familiar with Kaville, we have a ton of different, connectors that are out of the box. So you simply just press out of source. You go cloud section and then you can select no matter if you're looking for it to index service now tickets or Salesforce knowledge base or even your Slack channel. Any content you may have in your business, you can put it in Kaville cloud. And once your content is index and searchable, you simply have to go and build a machine learning model. For a generative answering, you have two separate models that we're putting together relevant generative interviewing, which is really the snippets and the answers that you've seen in my previous demonstration, and the semantic encoder, which is semantic search to make sure that we're not just matching on lexical search, but also looking at the concepts and making sure we're able to handle these very long sentences. So I'll I'll build them. You'll see. One of the beauty of the system we've built in our opinion is the fact that it's quite easy to employees. So here once you got your content, like I've shared, you, go here, create your model. You're gonna have a little illustration that show you what's gonna happen. If you click next, we're simply gonna ask you which content should you be part of the experience. So here, I'm gonna add a documentation website, an official the website. You're gonna have the blog here and another thing. So the beauty of the system is that live you're gonna see all the content and then the different filters. So in our case, we need to have unique identifiers on the documents and also the documents needs to be in English We are working on multilingual for the end of the year, but right now it is English and you get right off the bat here a little illustration showing, okay, I down like four thousand documents that already be chunked in snippets, and we can have up to a million different documents, for a given model. If you have more detail information, you can also use the filters here and select the name metadata that you may have on these items. So let's say I only wanted you r g e on document that I have been written by Bonnie because I think Bonnie is a super good technical writer so I can scope it down to a single author or use these kinds of fields. So in this case, we'll just continue to the next phase. We'll name it, webinar. And then you fire the build process and that's it. The model will start being building. It's gonna take several hours. In my case, I don't have a lot of contents. It should be tens of minutes and that's it. And you do the same thing with the other, model, which is the cement taking code. So, basically, you see the illustration a little bit different here. What we have on is an example mustard boat. So this is a yellow kayak So you see that we're able to find kind of the links between the words and their meaning. At this point, same process. You select the sources, ideally the same one that you wanna have. And then you see if they are, if they are accepted by the model. And once you're done, you create it, you name the webinar. To and you start building. Once you have your two model built, that's it. You associate them with a given deployment with a given experience and you're gonna see these answers coming right off. So that's the two minute walk through on how to configure it. Now my last part of this demo is basically, showing you where we have it deployed because one of the main advantage of doing it with the Coveo form like ours is that once you got your content and your model, you can reuse this intelligence across multiple touch points. Here at Coveo so far, we got it in our agent panel. We got it in Salesforce. We got them on their community website. We had it on any internet We even got it straight in the platform. So those are the kind of experiences you can do with what we call in product experiences. You see the little sign here. This is to ask questions. It is a small Cavell search interface. And then if you ask something like how to report on Coveo RGA for instance. If you fire that query, you'll see that you gotta have a full experience, whoops, out you. Out to report on Kaview RG, then you're gonna have a full snippet that will give you the full answer, with the different links that have been done to do it. So you can even see the citations. You can ask for a rephrase, for instance, asking it to be to respond in a more subcent way using bullet points. So this here is the same deployment as the one you'll see on our documentation website. We just we've been able to embed it, somewhere else. And depending on the quantity of documents available in each of these search interface, you're gonna have slightly different, responses or personalize experiences based on the same content based on the same models. So a very easy way to add it to a website is to use IPX like we've done it here in the platform. Another good deployment that we've been doing is, on a, our own intranet. So you see here, this is the Coveo at Kavil portal. This is a work place environment. This is where Bonnie and I go in the morning to see what's up inside Kavil. So, how to report on Kavil RG same answer, same question as I've been asking on the platform. If an employee internally ask, hey, how to a report on Coveo RGA, you can see we're providing answers in all the different touch points and this is what Bonnie was talking about regarding consistency. No matter where people are hitting us, from a computer perspective, you're using the community portal because you're trying to open a support ticket. You're on the documentation or you are on the platform or you're an employee and you're trying to find a question, we're gonna give consistent answer across all these different touch points using the same deployment the one that I've been showing you in the admin UI in our administration council. Fun fact here, obviously, this is an intranet it's also gonna be responding to very internal HR questions. So I just wanna give you a little glimpse of what it looks like you are deploying it in the, workplace demo, for instance. So here, out to a board and employee, you'll find, if you are a manager and you're getting someone new in your team, gonna have super well detailed breakthrough of what you should do to onboard a new employee, with all the right information in the bottom. So very, very accurate, very Here, the model is using information. It's the same model. It's just using more information because this interface has access to more document. You don't need to rebuild a new model. You don't need to rebuild a new, a new, semantic search model. You just need to add more documents in your experience and while it's gonna unlock. And this is also based on permissions since I'm logged in as Vincent here. I get access to the different things I can see for instance, support cases, which some other might not depending on your internal, rights. Lastly, we'll go back to our official Kaville documentation website, which is the first one that got Kaville RG. This was the first of all, when we went with the feature live, we deployed it here first. What I wanna share with you here is a little bit different, but it's the impact of having RG if you are a Coveo client. So Kavil, we've been building ML and AI for the past ten years. It wasn't based on LLM. It was you mostly based on user behavior. And what we see now is that if you use Coveo RG in a interface like this one, we have a positive side effect of RG on the machine learning models we got deployed in the past. So one of these machine learning models is query suggestion. When you go on the search like Coveo docs and you click on the search box, you'll see some suggestions. And what happened with r g e and the fact that we're giving some neat answers and we're providing, answers is that you'll see that the customer's behavior behaviors is changing. And now we have more and more of these long and complex questions that are appearing, which is great, because we have an answer and it also, and I say highlight how the, how to interact with the system. It gives you a hint of how, the system will react. It invites you to ask these kinds of questions. So if you'd be able to hear and click on how to install Kavil for Salesforce, you're gonna be redirected on the search page, and then you're gonna have again this generative answer solution. Gonna appear over here. So it's super interesting to see that we have these kinds of not side effects, but I'd say positive outcomes on the classical machine learning model we had based on this new technology. So it's not just adding something new. It's also reusing what you already have and piggybacking on the technology we had in the previous years. That's it for my demo. Hope you liked it. We went fast, but we've seen the new markup. We've seen conversational and also, how to deploy it and the new, use cases. So back to you, Bonnie. Thanks so much, Vincent, for that great demo. Now before we move on to questions. I do want to let everyone know that we are having a virtual event next month called Coveo Relevance three sixty. And we'll be joined by Zach from Open AI and Rowan from Forrester to really talk through what a real AI strategy should look like. So this will be March twenty seventh at eleven eastern. With that, we'll go ahead and take some questions. So we've got about five minutes left, Vincent, and couple of questions are related to hallucinations. So can you talk a little bit about how we avoid hallucinations, and and an example, you know, one of the searches you did was how do I onboard an employee? So, you know, in that example, how would you know that it doesn't contain a hallucination? That's a good question. And thanks Joel for asking it. I'll mark it as answered live. So what we are doing is actually called grounding. We are taking the in the prompt we're using to query that that that LMM, we're asking specifically please do not rely on what you've learned in the past. You need solely to rely on the information that we will provide. And this is a classic rag pattern, so a retrieval invented system where Cavell is used to fetch the information and only the information that is belonging to your company, for instance. So everything you've indexed, we're using that information slicing it, dicing it, and sending the relevant chunks alongside the user question. So basically, if Bonnie asked how to onboard an employee, I'll go inside my own personal documents, the provided documents of the company I'm working for. We'll look at all the possible, documentation that matches that question, and then this is what we'll send to the LLM. Bonnie's question, some proprietary instructions that we've built, and also the fact that all the documents that we found in the search. So it's not preventing one hundred percent hallucination, but it's it's reducing the amount of them like drastically where we're talking we've seen very, very little as hallucinations so far. What we can see for hallucinations is mostly if you are indexing two documents that are contradicting each other. So let's say you have an onboarding process that is official by HR and someone else wrote a confluence regarding an informal onboarding process. Then you're gonna see maybe some some some differences in in the answer that might not be totally aligned with the official one. But at this point, it's not a nation. It's a content management problem you have internally where some content you've index for your search hasn't been approved. So long story short, we are grounding the model using only search of NS search, results that are powered by Coveo, that are secured, and then they're validated by business users. And then if ever something goes wrong, you have all the little buttons on the UI to report something wrong and correct it if ever it goes wrong. And is it, you know, I I believe there are kind of standards in place. So if a query isn't complete, it won't just spit up an answer. I mean, you did show a little preview. You just said how to Coveo, and that wasn't really a real question. It was like we can't give you an answer. Right. Not nice way to use my phone as an example, if you're right. If you if it's not a question, the model will just ignore it. And we also have another mechanism which is actually preventing. So let's say we you ask a question, we find documents but the question and the documents are you're not perfectly aligned. So we have kind of a three stage gate set up. The first one is did I find documents? The second one is in these documents, do we have relevant snippets to send to or to to to to the LLM? And the last one, the final stage is the LLM itself saying like I gave you information. This is the question. Does it make sense? Do you want to respond or not? What's your level of confidence? And this is the last gate I'd say to make sure that we're able to give a a good answer. So the system has multiple check or gates where he can choke or or simply if he's not confident enough avoid to get two of them. Amazing. And, you know, I talked a little bit about our our foundations in search Can you talk a little bit about how search impacts the relevance and and what's being, retrieved and sent to the large language model? Yeah. And and this is, we'll we'll take at the same time the question. How is this difference from RAG? Coveo is basically an enhanced rag system. And what do we mean by that is we've been building this search platform for relevance for the best twenty years. So in in a Rack system, you're gonna use a basic elastic thing to do a query it doesn't have the intelligence that you have. It doesn't have the connectivity. It doesn't have all the machine learning, and it doesn't have all the business tools make sure relevance is properly managed at scale. So that's the difference, actually. Coville is a rag, but better because our platform, it's not just a basic index. It's a full relevance platform. And and this year for the seven year again, we won the best form from Forrester in terms of inside engines. So we are far in that category. We think the best platform to provide relevant answers. And provide relevant document for further processing by LMS. Okay. Great. And, let's see. We've got a few more questions. What is the embedding model you are using? That's a good one. For embeddings, we tried a few ones, initially in the prototype phase of the product, we use ADA from OpenAI because it was for us very easy to use it as a SaaS service, but it didn't scale well with the quantity of documents and clients we have endured. So we switch to a mini LM, which is the embedding model we are using from hugging face It is a small encoder, semantic encoder. So basically when you're indexing your documents, we're chunking them and indexing them. I mean embedding them. And then when you're using a query, we're comparing the words with the embeddings to find the distance and and find some semantic meanings between them. Okay. And, we are at time. I do wanna take one more question though because this is a good one. You know, the the question is, are you sending data to open AI endpoint? You know, are the models hosted on your environment? Can you speak a little bit about safety on the data? Yep. So we have a, an agreement with Azure, Microsoft Azure to use open AI models through Asia. So it is our own, subscription over there. We are sending data to, the model not to open AI to Azure the model that self, and we're only using this technology in transit, which means that there is no logging activated. There is nothing that they can learn from and they're just like responding to a question and that's it. The the feed shut down after. So there is no persistency. It's on the fly. And and this is the stack. It's using three point five turbo GPT. And we turn down everything monitoring and logging to make sure that that Microsoft doesn't have access to anything. Even if they say they don't in the contract, we we made an extra step turning off everything. And, yeah, so that's that's how it works. Okay. Awesome. So we are at time, for any questions that we weren't able answer, we will get back to you. I invite you to, once again, join us on our webinar on March twenty seventh. You can go to Coveo dot com to learn more about our relevance generative answering solution. Thank you so much. Thanks, Vincent. Thanks, everyone. Thanks, Bonnie. Have a good day. You too.