Hi, everyone. Welcome to this month's learning series webinar. My name is Claudine Ting, and I work on the global marketing team here at Coveo. For those who are attending a learning series webinar for the first time, this monthly webinar program is where we get more hands on with how to enable Coveo's features that can help you create more relevant experiences. So today is our last session in our three part series on machine learning. And today, we'll focus on the dynamic navigation experience model, otherwise known as the DNE. And we're also gonna do a review of the machine learning models that we've discussed in the past couple sessions. Just before we get started, just wanted to, run a few housekeeping items quickly. For this webinar, we encourage you to type your questions on the q and a box and the chat box as we go along. Did we lose Claudine? It appears we may have. Well, I'm still here. Yeah. You're you're back now. You were off for a for a moment. That that must have been my Internet. I'm so sorry. But, yeah, quick recap, for this webinar. Please feel free to type your questions in the q and a box in the chat box. We'll try to answer them as we go along, or we'll provide a recap of questions during the halfway point and and at the end of the presentation. And then lastly, this webinar is being recorded, so don't you worry if you miss on miss out on anything. It will go straight to your inbox within the next twenty four to forty eight hours. So I'd like to pass the mic to Jason Mel, Melnick, director for customer onboarding and Lipika Brahma, customer success architect. They've been with us for the past couple webinars, so we're wrapping this quarter up with them again. I hope you don't miss them too much. They're very accessible to you whenever you need help from them. But I'm just gonna kick off the webinar and, take it away, Jason. Thanks a lot, Claudine. I appreciate it. Thanks for the the great intro here. As Claudine mentioned, it is gonna be my pleasure to kick this session off, and we're gonna talk about Coveo's dynamic navigation experience. Now this is a machine learning model that actually adjusts the user interface to conform to user intent. After that, Lipica is gonna tie this a bow around this entire three part series with a segment that we call putting it all together, where we're gonna piece together each of Cobayo's machine learning models on a search interface so you can see how they kinda complement one another and also learn about how and when to use each model to be most effective with Coveo machine learning. So we're gonna start with dynamic navigation experience, otherwise known as DNE as you saw on the title slide. So before we talk about DNE, let's examine the typical faceted search experience. So an end user might search for something that's very simple, maybe using only a couple of keywords. And in this case, it could have a potentially very wide range of results. Not only are they gonna get back a huge list of results, but they First one is First one is which filter options will have the most impact on relevance if I see a lot of filter facet options? Secondly is, you know, how can that search experience become more predictive, and and help narrow me down to my results faster? Because that's ultimately what I wanna do is find the right result as fast as possible. Introducing Coveo's dynamic navigation experience. Now this is a machine learning model that leverages usage analytics to interpret end user intent, and it dynamically controls the user interface to adapt to the situation. DNE actually has a few different variations. So first, it could control the facet border on a page. So if you think about maybe you have, like, a source facet or language or pricing or whatever the case may be, that can help kinda reorder those facets depending on the on the situation. Secondly, within those facets, you're gonna have different values. It can actually control the ordering of those values. Now by default, the value that has the highest quantity of associated results is gonna be at the top of that list, but that's not necessarily the most popular or the most useful, option within that facet. So DNE can also apply filters to the page automatically to narrow down results. Now this actually leverages usage analytics data to boost content within specific facet values that are selected. Now in a few moments, we're gonna talk through, like, how the model actually works and demonstrate to you how to configure it and deploy it. Alright. So how does the model work? Well, it actually is very similar to automatic relevance tuning and query suggestion. So if you were around for the earlier, part parts one and part two, we talked about some of those other models. This model is similar to that. It uses usage analytics data to learn from. If you were around for last time when we talked about case assist and smart snippets, you would probably remember that those particular models actually learn from the data, the content itself. So this is more of a behavioral based, model. So when a user is, like, selecting a facet filter to filter down the results of the given query, that facet selection is appended to the advanced query expression that comes along with Coveo search API call. Now the model is connecting the query along with any context like the search hub or tab or any other user context that comes along. And it also includes that advanced query expression, and uses that as a subdivision for the model when it builds. So there are some prerequisites to, ensuring that you're getting your DND set up correctly. So first and foremost, you require usage analytics events to train your model. So if your implementation is not using, Coveo Analytics component, you're not sending analytics to our analytics database, you're not gonna have any of that raw material needed to train the model. This is table stakes really for pretty much all of our machine learning models with the exception of smart snippets and cases. That's what we talked about. From a JS UI component perspective, you need to have Coveo JavaScript UI version two point six, zero six three or a newer that was deployed back in May twenty nineteen. So if it's a newer implementation, we should be good to go in here. And it also is compatible with Coveo atomic headless as well. We do have controllers for this. Those components that you'll need are the dynamic facet. So So when you're building out the interface, you'll have to use the dynamic facet component, which is available there. And you need the dynamic facet manager component, which controls both dynamic facets and kinda acts as a, you know, as a as a class above the facets themselves. Finally, from a licensing perspective, not every customer is gonna be able to get dynamic navigation experience. You do have to have a pro license with us. And I mentioned earlier that another option that DND has is the auto select a facet. This is only for enterprise customers. So you have to look at your license to make sure that you're licensed for the particular model variant that you wanna use. To actually enable DNE. So first things first, you're just gonna build a model in the administrative console interface. Then you're gonna determine, like, what facets and what fields you wanna use for that auto select feature if it's applicable. So you can actually pick which ones you'd like to take into consideration for this. Then you'll associate that model to a query pipeline similar like you would with any other of our machine learning models. And then finally, making sure that you're integrating the dynamic facet components into the the the the the front end interface. So that is it for prerequisites. I'm gonna get right into, actually first, let's talk a little bit about, like, what it does so you can see it in action. So I have a a site here where the Coveo dynamic navigation experience is working. So this is, Dell site. What if I wanted to search for something like a mouse? I suspect that there's gonna be a lot of mice available. I do the search. And one thing that you'll notice right here, this is actually DNE in action. So you can see that there was a category facet filter applied. I didn't apply this manually. It just came on, because it's assuming that I'm looking for a mouse. So I'm probably gonna wanna narrow down that category to accessories, keyboards, and mice. So this is the category selection that it shows for me. So that's one part of DNA doing its thing. The next thing is take a look at the facets that you'll see. So, the brand facet is the first one that's really prevalent here, of course, beyond category itself. Then I have a price facet, rating, and color. So these are the facets that are most applicable. So they must have been the ones that were most often used when searchers were looking for for different mice. If I look at my results here just to show you the impact of this, there's two hundred and twenty nine results here. If I remove that facet, it will if I then I did the same search without DNE, I would have gotten six hundred and one results. So maybe that's okay, but chances are you're gonna get a lot of results for stuff that's, you know, maybe accessories or, like, a, like, a package deal where you get a mouse with a computer or something like that. Maybe not exactly what I'm looking for. So that's one way that you can help narrow down the results and improve relevance at the same time. Let's try another one. Popular query on this site. I'm gonna look for laptop. Same kind of situation. You'll see the category was automatically applied. So this is the auto select of the category. It has chosen computers, tablets, and laptops for me. This is a good way to ensure that things like a laptop bag or like a power cord or something like that that would go as an accessory to a laptop wouldn't be included necessarily in my initial results just because the word laptop is in there. If I look at the facets below, remember before I saw brand facet and then I saw pricing facet. There are some different facets that are available here now. So I have display size as my first real big one here, and then processor, and then so on and so forth, memory storage. Some of these other facets weren't even displayed, in the other searches that I did. This is because DNE has done the research and figured out that when people are searching for laptop as an example, it is like these types of values that are most popular, the most clicked on facets, and so they're rising to the top. So that's kinda what the model does in action. Now let's talk a little bit about how simple it is actually to enable this and get it going. So I'm in a Coveo demo org right now. I haven't built a, DNE model in here yet, but I'm gonna build them really fast just to show you how quick it is. When you're in your organization, you just wanna go under the machine learning section here, on the left hand side. You do have to be an administrator in your org to do this. And I'm gonna just click add a new model right here, add model. We'll give it a name. Just gonna call it DNE test. Simple enough. And then you're gonna see over here the available options for machine learning models that you can choose from. So in this case, I'm just gonna choose dynamic navigation experience. Let's move on. So it's gonna give you the option right now to select your learning interval. So we've talked about this before, but just a quick little update on there. There's there's two, like, attributes to this. So you have your frequency and your data period. So the data period, let's talk about that one first even though it's second on the list. This is actually looking at how much data you wanna learn from, how far back do you wanna go. So it's gonna start at three months. So that means it's gonna take the last three months worth of click search data, all that analytics that we have. You can go as far as up to six months, using this interface here. You have other options actually if you get into the JSON. We did talk about that a few sessions ago as well, and you can actually override some of these UI based options if you wanted to get really technical about it. Anyway, it's it's telling me right now it's gonna look at three months worth of data. And then you have your frequency. So the frequency is how often does the model refresh. So in this case, it's refreshing weekly. So if it's looking at three months of data every week and on like, right now, Thursday of every week or whenever I set it up for, it's going to move in advance one one week rolling and continual. If you set that to daily and then every day, it's gonna refresh, and then it's gonna, like, roll one day ahead of time, but always keeping the last three months involved. This is great. So couple examples of when you'd wanna use different scenarios here. If you're dealing with content where you think end user behaviors are gonna need to change very frequently, maybe it's like a commerce site and, it's during the peak, you know, like, holiday season or whatnot, and and and there's ebbs and flows. You might wanna use daily as your refresh rate because now you're gonna always have the most top trending stuff coming up towards the top. If you have more static content that is tried and true and you wanna use as long of a data cycle as possible, you can max these things out. That's great for content that doesn't change very often, and you get a lot of, like, consistency day in and day out, with that approach. Anyway, enough about the intervals. I'm gonna choose what Coveo suggest I choose because it's the optimal solution for this and move on. Now I get to choose what I wanted to learn from. So, if you wanna add an event filter in here, you can. Basically, you have this option to choose, like, if you wanna learn from one specific hub, maybe a couple of hubs, maybe a couple of tabs. So this gives you that option. It's it's only optional, but you can actually narrow down, like, what kind of activity and where that activity is happening that you want the model to learn from. I'm gonna leave it blank and move forward. Facet auto select. So here is what I was saying before. If you want to leverage the auto select facet. So thinking about this, you wanna use, like like, I would say, a higher, like, more overarching kind of category. So, like, if you have a category facet, for example, or a product facet, these kinds of things would be pretty, useful to use here because you're gonna have probably a lot of different content under each one of those things. But the content within each of those facet values is gonna be relatively specialized. So something like, you know, category, again, would be there. I don't know offhand what kind of great facets I have here to use this option in my in my in my org, so I'm gonna bypass it for now, but this is what you would do. And then when I was all done with this configuration, I simply hit add model, and that will begin the model building process, and it goes from there. Once your model is built like any other model, what you're gonna do is finally go into your query pipeline where you want to leverage it. And you go into the machine learning section of your pipeline and click associate model, and then you're gonna have a drop down list of all of the models that are available, and you would see your DNE test. I didn't actually create it, so you don't see it in this list. If that I created it, it would have been in that list. I would add it, associate the model, and you're good to go at that point as long as your front end has, again, those dynamic facet and the dynamic facet manager components built into it, you should be good to go, moving forward with dynamic navigation experience. Just going to bring back up my presentation really quick. If you guys have any questions, probably now is the time to ask. If you don't, and you wanna learn more about it, we're gonna send this along afterward, and you can find all this content actually available on Connect, our our Coveo community, right now. If you go to connect dot coveo dot com, you will find a course around pipelines and machine learning, which will delve into a lot of the tops topics we've talked about in this session, but around machine learning specifically and DNE as part of this. There's also documentation here about, dynamic navigation experience as well as, you know, how to create and navigate or how to create and manage the model, like we just went through in a very quick demo. So if there is no questions on DNE, I can, turn things over now to my colleague, Lipika, who's gonna walk us through putting everything all together. Thank you, Jason. So, hey, everyone. So what I really wanted to do is from for the past few weeks, we have been talking about, you know, various different machine learning models. So what I really wanted to do was, you know, put it all together because we've been talking about, you know, like Jason just spoke about the DNE model, and then we have been talking about, you know, the ART model and all of these different kind of models. What I wanted to do is to actually put together everything so you understand, you know, hey. I have a community. What what are the different kind of models, the machine learning models that I can use to make sure that my community is optimized according to, you know, everything that Coveo has to offer. So let's start exactly with that. So we're gonna start off by looking at, you know, if you have a customer community, what are the different ML models that you could deploy in your customer community? So to to even start thinking about that, I would really, ask you to think about the goals that you're trying to achieve with your customer community. Most of our customers, you know, majority of customers, when they have a customer community, the real goal is to make sure, that the customers are satisfied. Their NPS is improving, making sure that the customer effort is reducing, which means that they are getting to information quickly and they're able to self-service. That's the whole reason you have a community. You know? How are we gonna do that? And I'm gonna go here one by one is you could have a query suggestions model and this is all of these models are very easy to set up and if you haven't watched our videos, our webinars before, I would suggest to go back and look at them because we've discussed all of these models that are on the screen right here. So let's start off with the query suggestions model. So what I really wanted to do is to show it to you in action. So the query suggestion model really is what we have on our community. So this is Coveo Connect, our own community. We have the query suggestions model, which learns from user behavior. It learns from all the queries that we are doing, all the searches, and it looks at what are what are successful searches. So in terms of successful searches, it looks at what are the queries that actually got a click later on and then picks up on those. So if a customer, you know, was searching for machine learning, a lot of customers are also looking for Coveo machine learning, machine learning models, how to configure machine learning, machine learning models and languages. So, and if you look at analytics, really machine learning is one of our top queries that comes on our site. So it definitely has a lot of information to learn with. Why is this important? What we have seen, because we do capture, you know, all of the analytics from the usage of these query suggestions, we have seen that customers that use query suggestions typically get, you know, a ten to twenty percent, you know, rise in the click through and the engagement on their website. That is because customers are not having to type out that entire query and they're also getting helped by queries that other customers might have made before they came on to your website. So that's the query suggestions model. So once I do do the query, as as soon as I do that, you will see the search results. And once you see the search results, you'll see some of the results have these recommended badge on it. So that is what we call as our, our ART model. So that's that's the ART model that we are looking at. That is the automatic relevance tuning model. The, the aim of the automatic relevance tuning model is to make sure that, again, we're learning from user behavior and improving the relevancy of of your community without you having to enter hundreds of manual rules to to make sure that your relevant items are are coming up on on the top. So, Jason, do you wanna add something there? Well, I wanted to add something about this because I think, like, the recommended badge is a really cool idea. It's a great way to differentiate what content has been boosted by the automatic relevance tuning model. It's not something that comes, like, out of the box by default. There's a real simple, like, component you can add into your result templates, though, to bring that badge in there. And you can configure, like, what color and what you want you want it to be and all that kind of stuff. And, it's kinda nice. So just wanted to throw that out there. If you have more questions about how to implement something like that, let us or your or your CSM know, and we can, like, provide some guidance there. Yeah. And thank you for that, Jason. We actually did in in one of our previous learning series, we actually went through how to add the recommended badge. So we will make sure to, you know, tag that into the material that we send you after after the webinar. So that's our ART model. Now if we go back and look at any of the other models that we have, so the other models that we have that we can have on a customer community is the DNE model. And, really, the goal of the DNE model, as Jason explained, is to make sure that we're reducing customer effort. We're not asking customers to, you know, keep looking at, you know, pressing the same facets over and over again. So in this situation, if, if the customer was, you know, looking for, looking for documentation and every time they look for machine learning, the first thing that they do is they they refine their search by the product that they own for Coveo, whether it could be Coveo for Salesforce, Coveo for ServiceNow, the product that they're using, which is why this particular facet is right on top to make sure that the customers are are are getting the most benefit and not having to search for the facet to filter down their results and getting to the relevant results quicker. I'm just gonna pull pull this out from here because it's not letting me go on the full screen if I have this on. So now if we go back to looking at all the other machine learning models that you could have, on on a community. You could have DNE. You could have recommendations which is, you know, people who viewed this also viewed this. It's it's a it's a very important factor that you have so much usage on your community. You have so many queries, so many clicks, you know, customers taking different journeys and trying to find solutions to whatever problem they might be having. So why are we not learning from that? The ART model is doing that but the recommendation model actually learns from page views. So it doesn't have to be, a content that is indexed by Coveo. It could be a page on your on your on your community. It could be, a page, you know, from your training site that we are not even indexing, but we are sending those page views to Coveo to learn that, you know, when customers are usually searching for machine learning, the next thing they do is they go and they look at the machine learning course. So that's what recommendation is all about. And I know when we talk about recommendation, the the thought process that goes into it is that people who look for this also look for this. But recommendations can have many different kinds of, users. They could be used to, you know, display trainings based on what you have searched for. They could be used to display products that customers can buy based on what you have searched for. There are many, many applications of recommendations. So, don't just always think about it as people who viewed this also viewed this. The next, ML model which is, you know, goes better NLP model is the smart snippets. This is also something that we showed in our previous session is when you're seeing the search results, you're actually looking at the the portion of the search result that'll give you the answer. So the customer doesn't even have to click on the result. They can just see the answer right away. Now in a customer community, we, you know, we have a stand alone page where customers can search per, but we also provide the customers a way to log a case if if they need if they still need additional help. In that scenario, you could use a couple different machine learning models that Coveo has. One is the ITD model, which is the intelligent term detection model, which works on really larger queries. So queries, you know, like the case subject, description, typically, they are longer because customers are trying to explain what they're, you know, the problem. They're they're including log files, whatnot. What ITD does is looks at those large scripts, and it takes out refined keywords, sends it to the index, and bring brings back results based on that. The other model is case assist. So if you look at any case creation page, the first thing, you'll have to do is other than entering case subject description, you might also have to enter, you know, what product this, that you own, that you are having a problem with, or it could have multiple levels of things that you need to enter before you can log a case. Case assist actually helps customers, based on the case subject and description that they've written. It helps customer actually classify their cases as as fast as possible. So these are a couple of models, not couple, many many models that you can use to improve your customer experience on your customer community or on your website. And the cool thing is that they all can fire pretty much at the same time. Like, in one query, you can use query suggestions, and then you get on a page and DNE is gonna be changing around your facets. Art's gonna be boosting up your results. You can open up and see a smart snippet on that top result. There's just a and recommendations can couldn't be right on there, or you can click into that result and then, boom, you're gonna get recommendations there. So they all can be leveraged in unique, ways that really complement one another and are seamless to the end user. They just know they're seeing a good experience, but there's a lot of different ML stuff going on behind the scenes. Thank you, Jason. So let's look at our next use case. So our next use case is if you're using Kovir on your insight panel, which is your agents are using conveyor to find relevant information. Again, couple of different models that you can use here. We have already discussed these models, so I will I will not go through them in detail. But query suggestions model, same thing that it does in the community. It'll suggest queries based on user behavior, and it's going to, it's going to learn from what other agents are typing in and what they're what they're searching for. So obviously for any new agent onboarding, when they're trying to search for something, they're already seeing suggestions on, you know, these are what my other colleagues are typing in that have been here for so long. So they definitely must know what they're searching for. Then comes the ART model doing, you know, helping them reach their relevant results sooner by understanding that for each of the cases, if you're having your agents attach the solution to the case, we send that signal back to our ML model and we say that, you know what? This this knowledge article was attached, which means this is the solution for this problem. And the next time a case similar to that comes in, we're gonna bubble it to the top. So that's what our ART model does. And the ITD model, just like in the case creation page, it will look at your case subject, your case description, any other field that you want, and it will suggest, you know, refined keywords up to five. You can increase that to five. And these five keywords then go into the index because what we are trying to do is we're trying to plow through that really block a big block of text that, you know, that you have there and really understand what the exact problem is. And all of this learns the more usage you have, the more searches you have, the more clicks you have, the better it becomes. And the machinery models, and I know this is Jason's favorite topic, is, be it it can learn from across different, you know, different use cases. So anything that you're learning from your community, all the searches and clicks happening in your community, you can pass that learning on to your contact center. So what we call as the unified relevance because machine learning models don't act alone. They can still act alone, But when they start talking to each other, they produce the best possible outcomes. The next model, if you have a workplace and, you know, it's your employees using, Coveo to for to get to relevant results quicker. The most important thing for any, any any any organization is to make sure their employees are becoming proficient, that they are getting getting the resources they need to be successful in the role that they are in. Right? So to enable all of that, we have query suggestions, ART, and I'm not gonna repeat that all again, DNE and we spoke about this already. You might have a lot of different facets, filters, we're helping employees you know, especially newer employees who don't yet know what they're looking for, what should they be selecting, what is it what is what is something categorized in, Things like DNE, ART, query suggestions, it really helps their understanding of your your organization and where things are, where to go for what, and who to go for what help. The other other other really good model that you can use here is content recommendations. Very, very important because, a lot of times, just like any other organization, there is a there's a lot of content around, and it's not always, you know, out there which one is the most relevant one, which one is the most updated one, which one is actually going to help me in my task today. So having recommendations also tells that employee who's looking for something, who's looking for a problem to solve or a task to do, a story or a journey that someone who was doing something similar to what they're trying to do now has gone through already. The pages that they have seen, the content that they have viewed, all of that can be exposed using content recommendations. And when they all work together, they're really there to help employees get to relevant content as fast as possible. And last but not the least, ML models for your ecommerce site. So if you have an ecommerce site, you know, the main goal of any organization that has an e commerce application is to make sure that your conversion is increasing, to make sure that your average order value is increasing, to make sure that your revenue is growing. And to enable and to make all of that happen, you need to make sure your customers are getting to your products quickly, to the relevant products quickly. And also you're recommending products to them because, you know, a lot of customers are are browsing on your site and the more products you show the higher chances there are for them to add more things to their cut and increase your average order value. So the query suggestions, the ART model, they're all there to bring up the products to the top. And our ART models, you know, learn from user behavior. So they're gonna recommend things that are very popular. So will the product recommendations. Now the product recommendation model is something that we haven't spoken about yet and it's a little different from the content recommendations. The product recommendation model, you can deploy different strategies. You can have a recommendation right on your home page that shows you the most frequently bought products, the most frequently viewed products, you know, the the the products that are, you know, in in trend right now, in season right now, it's there are so many applications of product recommendations, and there are many, many strategies that are available within product recommendations. So you're not just this, you know, adding recommendation on a product detail page where the customer is looking at the product and you're seeing people who bought this also bought this. But you're trying to improve your chances of increasing that AOV by adding recommendations to the detail page, the home page, the cart page. So even when they are, you know, almost about to buy things, you you can say that people like you who had a cart very similar to the products that you've bought they also bought these products so why don't you consider. So these are just there to improve your chances of you know, improving your average order value. Then comes content recommendations which is, so I'll give you a couple of examples of how you can use content recommendations. So if you look at, the screenshot that I have over here in the bottom, So the the demo side that I have here or the screenshots that I have here are that of a bicycle shop. So they're selling bicycles. They're selling bikes, cycles, and they're showing these are the related products. These are the, you know, people like you also viewed these cycles and you like that but they're also recommending content people like you are asking these questions popular articles for bike stores like yours. So you know what's what's the warranty on my bike? How how how do I you know take this? How do I change a tire? Things things like those videos that'll tell them how to, you know, assemble their bike whatnot. These are content. These are not products. But at the end of the day, the more you're helping your users, the more accessible you make your products, the better your chances are to sell more and the better your chances are to make your customers happy. So these are all the different kinds of, you know, machine learning models that you can have if you have, an ecommerce site. So with that, those are all the different machine learning models that you can have for different use cases. So I wanna quickly, go back and see Claudine if there are any questions that come back, but that was all the content I had for you guys today. Alright. Thank you so much for that, Lipika. And, also, thank you, Jason, for, for the presentation earlier as well. So, I guess, first question that we have is what let's let's go back to DNE. What if we want to implement DNE, but we have an older solution that doesn't use dynamic facets? Well, that's actually it's a really good question. It's not not not the easiest answer either, but, basically, this is a relatively new model. I mean, from May twenty nineteen is when we've kind of updated the the the late latest framework. So if you have an older solution than that, then you would require some rework. You can always update to the latest Coveo JavaScript version. That's the first start. And once you have it the latest Coveo JavaScript version installed, then those components will be readily available, and you can add them from there. Alright. Thanks for that, Jason. Another one that we have here. How do we measure the effectiveness of these machine learning models? So the measuring effectiveness for machine learning models and I will give you a broader answer and then a more strategic answer and a more tactical answer. So the more strategic answer is really focus on your goals, on your business outcome. So if your business outcome is to improve case deflection and to make sure that your customers are being, you know, the self-service, your self-service success is improving. After you've switched on your machine learning models, do you see that number going high? So that's the more strategic answer. But if you if you go down to the level of how can I see that in my dashboard or my report because I'll I'll have to wait at least six months to figure out if this is working well for me or not? So when we look at in the dashboard or report, our machine learning models, the ART model actually has a marker on it. So any content or any result that was recommended by the ART model will have a marker. So anytime a customer clicks on it, we track that. And we can tell you that, you know, out of all the clicks that came on your site, sixty percent of that came due to the machine learning model. And, typically, I tend I tell customers is that you should see that trend going up. You know, at some point, it will stabilize, but you should definitely see that that trend going up and up as you as as your machine learning model learns and improves as it goes. Okay. Thank you for that, Latika. I'm gonna throw, throw it back to the audience. Do you have any other, follow-up questions before we wrap up? K. I I don't see any any questions so far. But I guess before we wrap up, something to remind our audience, Jason and Lipika. Does any of the machine learning models that we've discussed require an additional investment? In fact, some of them do. I guess it depends on where you are, what license you have right now. So there are some models that are license dependent. So for example, smart snippets and case assist models do require a enterprise license to enable. Dynamic navigation experience, does require and product recommendations and some of those, commerce recommend commerce models require a pro license or above to enable. But if you have a base license, you should be able to access, like, previous suggestions, automatic relevance tuning, those, like and recommendations. Those will be available. Okay. But, otherwise, yeah, you would have to make a a little bit of an investment on your license, to to get those higher level models. That's good to know. Okay. So I think, that's it for today. Thank you so much everyone for joining us for the last webinar of the machine learning series. As mentioned earlier, we will be sending a recording of this so that you could review and share with your teammates at work. And, other than that, we also have a feedback survey that will pop up at the end of this webinar. Please feel free to give us feedback and share share some ideas of what you want us to cover for, our future learning series webinars. So I just thank you, everybody. Have a great rest of your day. Bye for now. Thank you. Bye.