Righty. Looks like we're great to go. Hello, everybody, and thank you so much for joining us for our employee experience roundtable, redefining the future of work, secure gen AI strategies for innovation. My name is Patricia Patsy Liang, and I lead product marketing management for Workplace here at Coveo, and I'm going to be your host for today's webinar. We're so thrilled to have you join us as we explore how emerging AI technologies, particularly GenAI, are really shaping the modern workplace. In this session, we're going to explore how intelligent search, generative answering, and other technologies help the world's leading companies enhance employee productivity and promote continuous innovation. And today, we are so honored to have this wonderful dynamic lineup of speakers with you today. We're going to have Patrick Martin, our EVP of global customer experience here at Coveo, to interview Leishan Gatson, senior technical product manager of search experiences at Vanguard, and Mike Marnoka, senior director of product development and AI at RightPoint. Each of our panelists bring a huge wealth of experience today in enterprise search, knowledge management, and AI innovation. And, collectively, they have about sixty years of enterprise experience to share with you all today. And whether you're just at the beginning of your AI journey or looking to scale existing initiatives, our goal today is to help you, come away with some actionable strategies and advice to help you drive agility and growth across your organization. And so just to quickly go through the agenda for today, we're gonna be doing introductions, an interview, and a q and a. And so do note that you are currently in listen only mode, but you can ask questions at any point during today's webinar, and we'll answer them during the q and a section or email you afterwards. And you will be getting a recording of this webinar as well in the next twenty four hours. With that out of the way, Patrick Martin, would you like to take it away? Sure. Thanks so much, Patricia, and thanks everybody for joining us today. Pretty excited to share the time with all of you and share the time with, Mike and Leishan. So, we're gonna do some some quick, quick introduction. So let's start with, you, Leishan. Tell us a little bit about yourself and your team. Yeah. So hi, everyone. I'm Leishan Gatson. I am a senior technical product manager here at Vanguard. I've been at Vanguard for over twenty five years. I've been working in the digital space for probably about fifteen years, and I've been working with search for over a decade. We have a whole fully stacked team of search developers and engineers, and we, I would say, we, support probably about twenty five applications here at Vanguard, both client experiences and employee experiences. In years of search, what excites you the most about search these days? Obviously I mean, the obvious is the Gen AI and the evolution of search, going from strictly a a list of search results to now being able to give summarized answers and being a a much greater tool for our users to get to the information they need quickly and efficiently. Love that. Yeah. Definitely. And when we think about, you know, the upcoming year, you know, what would you say are are your priorities when you're thinking about, you know, that that technology and, you know, the outcomes that you're trying to accomplish? Yeah. For twenty twenty four, we introduced GenAI to our employee base. So it would be to expand that. We just turned it on for all of our, employees for our Internet. So that's exciting for us for twenty twenty five. And, also, we are concentrating on our client experiences externally. So we're really looking to expand that to our external clients. With us being a financial company, we do have some challenges with now going to Gen AI and the FINRA rules and things like that. So we're really excited and looking into exploring that for our client base. And then I think the last thing is we're it's we're expanding internationally. So we just turned on search for our Canada site, Mexico, multilingual searches. So those are some of the things we're doing twenty twenty five. Oh, definitely some some, exciting projects ahead. And, you know, definitely, you know, a lot of a lot of folks out there are focused on, you know, improving their employee experience and their customer experience, and these technologies definitely enable that. So you've been on a journey so far. If you just reflect a little bit on the previous year on twenty twenty four, what are what are the biggest takeaways or the lessons learned you've had so far in your journey? For me, it's still that content is king. I would say regardless of what search engine, how much we use Gen AI, it's really based on the content that we are, training these models on and the content that we have in in the back. So we've learned that it's really key for us to get some content governance, do some things. I would love to explore GenAI for actually content creation and content curation and things like that. So not just about finding the content, but also about creating the content and making it more digestible for GenAI. Yeah. Definitely. I mean, you know, a a lot of people have, you know, kind of put knowledge management on the side for a long time, and now they're waking up to that exact conclusion where, you know, we've been talking about garbage in, garbage out for years, but it's taking a whole new significance and a whole new level with with Jenny I for sure because, you know, your your answers are only gonna be as good as the content that you feed it with. So definitely definitely a good good, good lesson there. So definitely, we're we're gonna talk about Gen AI for sure. But before we dive into that, we all know that, you know, great search is kind of the the foundation and lays the groundwork for great AI. So for both of you, Leishan and and Mike, what what challenges prompted you or your customers to explore an an AI powered search solution? I'll let Mike go since and maybe he can introduce himself. Yeah. Tell us a bit more too about you too and Mike at RightPoint. Yeah. Sure. Hi. Mike Maranoka, with RightPoint. I'm, the practice lead for product development and AI. I've been here about twelve years. And like Leishan, long time background in enterprise search. Right point, we're a digital agency, consulting agency that have been implementing Coveo for over eight years, and some of our partnerships are with Microsoft, Adobe, Sitecore, and, Optimizely. So trying to optimize, solutions in in that area. Wonderful. So, yeah, going to the question as far as, challenges for, you know, our customers, I I think it it comes down to, you know, there's certain hype around generative AI and, you know, expectations. Right? But it really goes down to, the challenges that they come with is finding, you know, the right use cases and experience to to implement, something. Right? And knowing where to start. So those are, you know, content is king that resonates well. Also, right, content being being a problem. So, you know, I think the the solution to that is is to start small, and incremental, work with some stakeholders, really get to know their pain points and, you know, where generative AI can be, an assistant and, you know, start working from there, gain some experience, and, and build upon that. Sorry. I'm in a moo a room that I'm not moving. Oh, no. The light went out. I have to keep moving. Go ahead. Big snowstorms. Yeah. Well, that's that's fantastic, Mike. And what what trends are you starting to see now within the market? Where are customers, most excited about when they're coming to right point for implementations? Yeah. I think, you know, some of the trends are, especially this year, I think it's all about the being more of an agent or agentic AI, right, experience and, moving from just that, generating content or answering a question to actually getting, you know, something done. So whether it's integrating with, you know, whatever platform the customer service representative is using or the sales team is using, really going deeper into the ecosystems, more connectivity, is is a theme as well. Right? I think a lot of, of the enterprise, you know, software companies are are pushing their own, you know, product, but, they kind of focus on their individual ecosystem. So, really, you know, having that connectivity across, you know, multiple systems, most companies that we, talk with don't, you know, just use Microsoft or don't use, you know, Salesforce. Right? They they have a combination of all these things, and it's really the connectivity where you unlock a a lot of value. That's an interesting point, because, you know, we we know that, you know, Microsoft and Salesforce are definitely out there with copilots and and agent force. And, you know, definitely, for these these technologies to work well, the data all needs to be within their ecosystem. Right? So how do you see how do you see that play in the future? Like, the companies that you work with, is that part of their strategy? Is that something that they're looking to do, or they it's gonna be way too complex to bring everything in just because of the nature of of, you know, their their setup or their implementation or how where their data is current currently. Yeah. I I definitely think it's, it's a challenge. Right? But, you know, even Microsoft, I I believe, said, you know, a AI is is the UI, and, you know, someone made the comment. I I don't remember who who it was, but, you know, an industry leader said, you know, SaaS platforms, you know, will eventually die, right, because of the the UI and generative AI, that's able to go talk to all these systems. You don't really need, you know, that interface to, enter in the sales information or, you know, create that ticket. Right? It's just, the the the business intelligence and knowledge is is kinda at the the AI layer. So it'll be interesting to to see. I you know, that being said, I don't think anybody's, you know, abandoning those systems at this point because they have so much, you know, data, you know, stored in those systems, and that's where the knowledge is. Right? So I think, you know, that's that's one of the things I love about Coveo is it fits well in with the other systems, right, and, can easily integrate and get up and running. And you don't have to migrate the content all into one system or, worry about the connectivity between the two. You know, Coveo kinda handles that being the the central hub where you can get that information. Definitely definitely, something to think about. Right. So, like, while while we're a little bit on on a similar topic, yeah, how do you typically guide the the organizations that you work with in in prioritizing these types of initiatives across their companies? You know, how how are they thinking about that unification of the content layer and and the experience layer and, you know, the employee slash customer journey. Right? Because to in today's world, it all needs to be unified. Right. I think, you know, when we when we look at, like, you know, implementation and and how we tackle those, there's an there's a number of ways that we we get into it. But, you know, we always start with use cases and personas. I think each company is unique and has different, you know, personas and roles. So we like to focus on the pain points or friction areas that map to those, that AI can solve, and also, you know, dive into that ROI, and the experience impact. Right? At RightPoint, we talk a lot of experience. Sometimes it isn't just the ROI or the the the bottom line impact. It's a human impact and unlocking the potential of how a company can evolve into this, you know, new generative AI powered world, and supporting employees and customers. And, you know, kinda last point is, I think, having a road map and adoption plan. Right? Whatever, you know, challenges you have, kinda sketching out what the future could look like is a good exercise. You know, figuring out what technology is involved, between the systems and how long it's gonna take you to get there and what that, you know, impact is, from a change management. And, ultimately, you know, the goal is to align that to your your leadership goals and company goals that, you know, support those those things. And once you have that in place, then I think you are in the driver's seat to, really, you know, figure out your path forward. Yeah. That makes that makes total sense here. So, Leishan, let me turn to you. I mean, we're talking about, you know, the impact of AI and AI solutions. You know, from from your perspective, was this something that you were considering, like, right from the outset, or, you know, did it become, you know, apparent for you, that you would need these types of solutions as you were looking at the different options? Yeah. So we we had recently just, onboarded Caveo, and we were just moving our normal search platforms over to Caveo when Gen AI came, like, full force. And we jumped on it immediately to be a part of the beta. So we really worked closely with Coveo to say, like, we wanted to be a part of shaping what this product actually became. So it was important for us to have a seat at the table to just understand, again, from a financial standpoint for a finance company, how does generative AI answers work for us? So we did not initially were thinking about it, until, you know, Coveo introduced it, and immediately, we jumped onto the the bandwagon to be on beta and test it out, and it's proven well for us. Yeah. And if I remember, because I I was I was part of that program and pretty involved, you know, as as a financial services company, definitely, you you wanted to go employees first. Right? Because you wanted to have those that human in the loop and making sure that there was there was some some oversight from someone and not go full out self-service with the customers without having the, you know, the the the confidence that what was being generated was, you know, fully accurate and didn't bring, you know, its share of liabilities. Like, some of the the stories we've seen, like, early on when people just went, like, that that route. Tell us about that that, that evolution or that that, how you guys thought this through? Because you did mention earlier on that you are looking now to go customer facing. So, you know, how did you how did you work through all that? Yes. We definitely did employee first. So we started with we have a IT portal that's for our IT professionals. We turned it on for them knowing that they are usually early adopters and usually have, very loud voices on what they like and what they don't like. So we put it on for what we call search it, and we've seen that they really have been using the, the generative answers and engaged. It's interesting. We really thought that, initially, that people would just get the answers and leave, but we're actually seeing that they're more engaged in clicking through the citations and clicking on the links. And it's almost like a preview of what they're looking for, so it's getting them to the information quicker and faster. And then what we did was a slower rollout for our Internet. We gave it to about fifty percent of our employees, which is we have about thirty thousand employees and contractors. So about half of that had, generative answers for a a quite a for quite some time as we looked through the content, made sure the policies and the information is correct and accurate. And like I said, we recently just expanded that to all of our thirty thousand employees. And we've only seen less than one percent feedback in the thumbs up, thumbs down, which Mhmm. To us is an indicator that people don't hate it. Like, we you know, to be honest, if they really did not like it, we would we would hear it in, many different ways. So it's almost like no news is good news. They don't give too much feedback, and usually, it's, the typical this information might be outdated or things like that that it's more around the content and not necessarily the feature of the Gen AI. Yeah. I mean, you know, you you mentioned content is king, so that's that's where it all starts. Yeah. Right? That that's definitely where where the it all starts. Definitely interesting interesting story. And we know that, you know, Gen AI is not a it's not a one size fits all. Right? There is there's a multitude of use cases, and some of them are still need to be well defined. So maybe that question goes to to both of you because, you know, we have the the customer perspective and we have, you know, the the the partner perspective. But where do you personally see the most value in in GenAI, and and how do you go about defining what success looks like? Because, you know, there's there's tons of ways to measure ROI, Mike. You touched on it, and it it can be a dollar value. It can be a human aspect to it. I'm curious to hear you both on that on that piece. I guess I'll go quick. And then, like I said, we are looking at the feedback. So we're constantly looking at the thumbs up, thumbs down. So that's an indicator. We also with our IT professionals, we, have a survey that goes out when a generated answer is generated for a user, and we've received feedback. For the most part, very positive users are saying that it's, helping them. So it's also quantitative and qualitative information. Back to, like, what Mike said, it's about the experience. A lot of our users are saying that they find it, useful for them to get them again to the information that they need quickly. I really like what you said. It's it's about the experience, and and that's that's key. Right? Definitely key. Yeah. I mean, I think on your on your side. Yeah. I'd love to add a little bit more. So so I agree. I I also think, you know, the value in that single plane of glass. Right? A lot of people talk about that as a one stop shop assistant, or experience that, you know, breaks down those technical silos. And, you know, just allowing me to get that work done right, to this. You know, we we built our own, AI assistant agent on top of the Coveo platform. We could talk about that a little bit later. But, you know, some of the KPIs that, I think are important too, and and Coveo does a a good job at they have a a generative answering performance report. Right? And some of the the KPIs there are click through rates, percentage of questions with a generative answer. You know, and and so kind of the basic things where you can measure how effective your generative AI and search is. But, you know, like, as Sean said, getting that feedback from your users is probably the most important. Right? Whether it's a survey, you know, in making sure that it's it it's being used, kinda watching token consumption on the generative AI side if you have access to that and and the usage trends, right, to make sure that, you know, it wasn't just rolled out and people try it and and left it, but actually continually, improving it along the way. Yeah. And and one of the trends that, you know, we were surprised, was what Leishan talked about is how people engage more with the content. We were actually expecting that as we would generate answers, click through rate would go down because they'd have the answer. But the reality is and and, Leishan, you Vanguard is not the only one seeing this. Click through is actually going up, which was surprising to us because we really thought it would be the other way around. So, that is definitely a demonstration that, people, you know, want to know more. You're kinda getting them engaged with your content because they are going through, you know, the, the documents and the citations, which is is pretty pretty interesting. Mike, while we're on while we're on, part of the topic here, you know, from from your perspective, how do you help your customers build strong business cases for AI adoption? Right? Because, you know, we all know when GenAI came out, like, this was a almost a boardroom discussion. Everybody wanted GenAI. But the reality is, they didn't really know where to start. And I've talked to companies who said, we have hundreds of Gen AI pilots going on right now, but none were prime time and ready for production. So how do you work with your customers to say, like, no. This is what you need to focus on, and this is where you're gonna get the best, you know, ROI? Yeah. Great question. So, again, kind of use cases and personas is where we where we start. Right? I think you really need to, you you know, get out of the trap of, like, okay. Let's try this product, you know, just for technology's sake. You know, turn on Copilot or whatever, you know, the the case may be and and really get down into what, you know, people, are doing their work on and take that through, you know, deeper than just, you know, the the surface level generative AI things that that you can do, right, to to really, unlock, you know, the value there. And, you know, it it's easy to get into the ROI, you know, impact of it. But, you know, again, I think it's more of looking at the experience of how people work and really, you know, giving them the freedom to to do more of what they're good at in their job and be more creative and impactful to the company versus, you know, just saving them some time. Right? So freeing them up, obviously, from those more redundant, you know, tasks or finding information that's buried in in different systems. But, you know, go to the higher value impact that that you can get. So we kind of work in a discovery, you know, type project that has a combination of, you know, researchers or content strategist, technologists, you know, all combined in kind of a a think tank, right, to to get you to, you know, what those prioritize use cases and, you know, figure out a plan to to move forward. I I think that's, you know, the key where you have to start. Yeah. Bringing bringing it in the flow of work. Right? Yep. Yeah. And I'll just add also one of the big things that helped us is that we had benchmarks of knowing how long it took users to do certain things before. So that helped us to also say, like, here's the areas that we know that we wanna concentrate on. It's taking our phone groups, you know, twenty minutes to search for something. Or our IT professional said it used to take them anywhere between five to twenty five minutes to find information. And they're saying generated answers are actually helping reduce reduce that and increase their productivity. So we kinda looked at those use cases as well where areas of bottlenecks, for finding information were high. I love that. Yeah. Removing removing friction. Right? Yes. Absolutely. And for sure too, I feel like getting those early stats in even before the implementation will do such a huge lift for them demonstrating that return on investment from any new technology after. So that that's really exciting to hear about, Leishan. Thank you. And I I and I'm I'm I'm assuming here, Leishan, but, you know, with those with that type of benchmark and, you know, potential impact, was it somewhat challenging to convince senior leadership to move forward with implementing these technologies for, for your employees? I think I mean, some of our IT leaders were definitely on board. I would say, probably, you know, our counterparts in, like, HR, they had some always have concerns about the accuracy. But with Coveo and the, how they mitigate for hallucinations and all those things that we were able to kinda show how sound of a product we were able to move forward. So those things showed that the accuracy and relevancy were really still a part of the generated answer. So that helped us. That's and that's definitely interesting, which leads me to, you know, my next question and really interested to hear, to hear your thoughts on this. But we all we've all seen it in the media where, you know, AI and AI agents are gonna replace the workforce, and, you know, the the the jury's still out on that. But, did you feel any type of resistance or have to manage some type of culture shift from your employees around, you know, this this change, that could be, you know, disrupting? Or did they embrace it and say, no. We we see the value because and what both of you touched on, it's gonna make my life easier. So what what reaction did you guys get? Because, you know, when we talk about introducing AI now, there's a lot of people who you know I was on I was on a call with a a customer just this week, and they're saying that there's a lot of reluctance from their agents to embrace AI because they they don't wanna share knowledge because that's that's their that's the value that they bring. Right? They don't wanna be replaced by AI. So I'm curious to hear the both of you on that. I think for us, we we actually hear that, they want it to do more. They think it's a they want it to be, do agent assessed and do do actually more transactionals so that they can concentrate on the real complex phone calls and real complex issues. So we're hearing that for some of the really minor things like a change in address or, you know, things that are kind of routine, they would love that the Gen AI took it a step further and actually went in and made those changes so that they can concentrate and give real financial advice and give real information to our users and give them a more robust, complex, situations instead of the the day to day, if that makes sense. Sure does. Yeah. Yeah. I would say, it it's definitely the, you know, human persona to, you know, fight back, have skepticism, you know, doubt, all those, you know, emotions around it. I I think where you kinda work through it is in the experimentation and, you know, slowly drip feeding it, you know, to to people, getting them using different tools, even if it's just, you know, chat g p t and and some of the other public ones to get a sense of the power. You know, but that tipping point is when you make their job easier and, you know, give them tools to make them be more impactful in their role, that's when, you know, the light bulb goes off and they say, I want more of it. Right? So Well, speaking of AI tools that might make the lives of employees a little bit easier, I think, Mike, would it be alright if I were to share my screen and maybe we talk about some of the things that RightPoint is currently doing to help facilitate that? Sure. Yeah. Yeah. So, as a, a partner of Coveo, we've had access to the passage retrieval API that just went, in public. And so we started off with kind of creating a a a bit of a demo, and it's evolved into, kind of this product. We call it Barca OmniOne. And it's it's more of an agent right on top of the Coveo, platform. So kind of a take on generative AI and search results. You know, at the bottom half of the screen is more of your search experience, but the top half is is an agent. It's led by, you know, prompt templates or samples and, what we call quick actions. But it's also, you know, how is it different than all the other agents out there? It's it's also a bit more of a personalized work app. Coveo has always been a leader in personalized relevance. So we're taking advantage of that to deliver, you know, your company news that's targeted to you if you're in a sales role or, some other job. I have a mix of your tasks in here, and onboarding, you know, adoption training material also built into it. So kind of mixing all these into, a user interface and and, experience that, leads you to answers, you know, more quickly, but also becomes your one stop shop for, you know, getting work done. So it's it's powered by Coveo, and, we have a demo video at Barca omni one dot a I if you wanna, take a look at it. And, yeah, we're pretty excited about it. It's a really beautiful interface. Yeah. It looks great. Right. And so is this meant to replace an entire Internet portal or to be, like, the standalone portal, like, that one stop shop experience? So I kind of view it as a little bit of a companion app to maybe your Internet and portal. You know, it it pulls in content and ties nicely into it. So if you need to get into those, you know, things that your Internet and portal does, it could be a mobile application as well, giving you a mobile experience. But it it also could be that, you know, Internet for you, depending on what you have today and, you know, be that entry point to to get in there. I love that. And I'm especially enjoying the calendar. This is reminding me of your Comcast implementation too. Just so polished, so sleek. Thank you for those. Alright. I'm gonna let you get back to the interview. I mean no. That that's that's the type of, you know, thing that is really when we talk about how these technologies can transform the experience, this is a great this is a great example. Right? And and how you can you can leverage the, you know, the power, excuse me, of search, your content, GenAI, and really, you know, build tools that are personalized and really add value to, to the users. So that only that's that's pretty it's a pretty slick, slick, product Barca you got there. So I wanna take us a little step back around and and talk about implementation. Right? If you if you listen to some, you know, some of the the folks or the claims out there, it seems like these things are easy to implement. But the reality is there you've got some some companies that have been trying to to go live with their their pilots going down the build routes. They're two years in, and they're still not at the point where they can they can, you know, really go prime time with it. So I'd like to get a a good understanding around, if you can walk us through the different stages of implementation when you're looking at these types of technologies, you know, search combined with generative answering. You know, what did you learn along the way? What was the surprises, either from your own experience, Leshawn or, Mike, from customer implementations that you've done? Yeah. I would say, you know, our process, it it seemed like it was it went fast, but it was took some time. Here at Vanguard, we they created a Gen AI steering committee, that is looking at all Gen AI products. So we had to go through the process. We went through security checks, vulnerabilities. I'm sorry. I keep, I'm not moving enough in this room. Hold on. Sorry. Smart rooms. Use tech techno technology today. Right? It's it saves energy. So you have to if you're not moving, the lights go on. So we go through a council for all of our GenAI, products. I would say, you know, make sure that it's secure. All the things that we're checking for, the accuracy, the human in the loop, all of those things play a part. And then, obviously, just looking across back to, like, Mike said, looking across the multiple use cases, not every use case needs a GenAI. Some of our groups were doing great with smart snippets, which is an abstraction that gives you verbatim, and they don't wanna generate an answer. So just kinda understanding each, business group and what's important to them. We have probably, like, ten different knowledge hubs here at Vanguard for different groups. Our general counsel so, again, really just understanding their their use case and what's important to them. Important to them. Yeah. You bring up the concept of smart snippets, which is, you know, an interesting one to explore. Right? Because when you're in a highly regulated, excuse me, area like like Vanguard is, you might not feel comfortable generating something new because it's not it's not gonna be vetted by, you know, legal and all of that. So actually being able to extract a snippet from an approved document, which looks like a generated answer in the end, is is, you know, kind of another way to go. So the user, you know, believes that they get some type of generative answer, but it's really just an excerpt from your documents. So definitely something interesting to consider as well. Yeah. Yep. So, Mike, turning to you. Yeah. So data. Right? And when you when I talk implementation, data is the first place to start. Right? It's it's usually the friction point. You likely have too much data, or out of date information. I was talking to a customer the other day, and they were hesitant on generative AI and their their website putting, you know, generative AI search on it wouldn't be a great experience because they didn't have the content to drive it. Right? And, I think the thing that people don't talk about is the opportunity cost of not doing it. Right? Search and generative AI puts a mega spotlight, you know, effect to point out issues or gaps in content. And if you don't implement it, you know, you have this you're missing those insights and incentives to change, you know, and it I kinda remind reminded them that your website is already being crawled and used by generative AI. Right? Whether it's JetBluePBT, Perplexity, Copilot Ving, Google. Right? They they already have your content. Right? And so I think you you really need to pivot and push harder into that. You know, so that that's always the the first, you know, hurdle is content. I do think, you know, you can start small and filter out, you know, the bad content and and only, use the good content, if you will, that that is ready for generative AI. So that may be one strategy there while you're, doing some cleanup. But, after that, it you know, kinda connecting to all the right information sources, refining, you know, that, that content or the experience, potentially, you know, customizing, to the, you know, brand, or, to the the role that users are in, and then, you know, kinda go through that rollout period. You know, we we always talk, crawl, walk, run, as kind of the model. You gotta start small, you know, roll it out to some some friendly users who can give you honest, you know, feedback and, work through some of the kinks, but also build, you know, champions along the way that become your, you know, greatest asset to convincing other people that, you know, they should also, you know, take take part in the solution. So those are some of the the highlights for implementation for me. Yeah. You bring up a oh, sorry. No. I was just gonna say I agree one hundred percent with everything Mike just said. That's exactly how we kind of looked at it as well. The subject matter experts. Yeah. Mhmm. Yeah. And and, you know, on the on the data piece, you know, such it is such a critical aspect of your implementation. And, you know, we're we're we're two years into this Jenny and I journey here at at Coveo, but we we've seen it all. And, you know, we've seen customers say we're not gonna do it until we can, you know, leverage our millions and millions and millions of documents because that we wanna go all in. And we've had customers who started with a few thousand documents because those are where the most impactful. They were the most used, and they say, that's what we're gonna start with. And as we analyze how people interact, how they engage, what are the gaps, we're gonna start introducing more content, and we're gonna build it from scratch. And to be quite honest, those who started small and built on top are getting better results than those who went all in because you don't know what you don't know in terms of your the quality or the readiness of your content if you're trying to leverage the whole thing. So you're better off saying, hey. Even if we generate answers twenty percent of the time, the other eighty percent of the time, we're gonna look at what people need, what they're looking for, and then we're gonna build on top of that. And we're gradually gonna get to thirty, forty, fifty, sixty percent of generative answers, and and that's fine. So definitely something to to think about, as you're you're going through this thought process, your implementation is start small. And, yeah, start with a smaller group. Right? You wanna you wanna test you wanna test it. You wanna get the feedback before you just go prime time. Great. I know we touched on this a little bit, but I think it's an it's an interesting topic to to visit around, you know, companies who are highly regulated. You know? Because that's that's the situation here with with Vanguard. And you did touch on it, like, with your your GenAI compliance, committee here. But how do you factor all this in when you're thinking especially on the customer side, right, because that's the next that's the next step for you, is is how do you take all that in, and and how do you set up strategies to make sure that, you know, you you are going to be able to operate in that highly regulated space and remove the whole liability factor? Well, I won't say we're there yet. We're currently in the process of getting approval through our GenAI steering committee. I will say we have everyone involved. We have lawyers involved. We have, compliance review. We're thinking about what does human in the loop mean, what does our retention of generated answers, how, so for FINRA and SEC, we have three year retention. The regulations right now do not call out AI in any particular way. They it doesn't it's saying it doesn't matter what the medium is that you're giving information. We're also looking that it doesn't give advice. So we have tons of people here involved in this process right now as we look to see if we get the green light. Obviously, we're also looking that it's only public content. We're looking you know, to Mike's point, some of this content is already in somebody's chat GPT probably. So we're looking at our public content. We're also looking at maybe, like, support type of information first. So those are the things that we're doing to try and see if this would be available for our our external clients. But we're definitely looking at all the rules and trying to see if there's a space for AI for, financial companies. Yeah. Because you're you're you probably you're probably gonna expect that your customers are gonna expect that type of experience at some point in time, and you're gonna have to deliver it to them. Right? Uh-huh. Yeah. We're we're seeing that some financial companies are diving into it a little bit, but they're not going full Gen AI. They're probably doing a a little bit more of a, like, a snippet type of interaction, but it gives the illusion that it's a generate generated answer. So I think we're just trying to kind of, explore those things, what makes sense for us. Yeah. We just, double down on Nishan's comment about, having a a a committee, right, or task force, you know, whatever you wanna call it, right, on the AI side that involves, you know, legal security, right, all those players to get them onboard and and up to speed, in their input, right, of what the challenges are so that you can address them and create a plan to to move move forward around it. Yeah. I would love to know if any financial company has gone live with GenAI, for their search, so we can get some more insight from them as well. We might have some customers we could put you in touch with. That's what I was gonna say. We can take we can take that offline. But, speaking of, just a quick, the radio host and investigative journalist in me is crying because this is such a great conversation, and I wanna keep it going. I do wanna ring us in for time, though. Patrick, I would recommend asking questions twenty and twenty three, just for the sake of time because we're a very engaged audience. Thank you. That's exactly where I was going. My next question was question twenty, so we're fully aligned. So, yeah, great conversation so far. And, you know, I'd like to know, how has it been, you know, working with with Kaveo? And I'm gonna I'm gonna put two folds on that, you know, just because I have a vested interest in it, you know, leading all the sales teams. There's there's the technology aspect. So how's it been working with the technology so far? And and are you reaching the outcomes and the results that, that you're expecting? And how has it been working with us as a company from the customer success standpoint, from the support standpoint? Because, you know, we're we're definitely, you know, our CEO says that we do outcomes as a service. So we're definitely looking, and we have a vested interest in your success and in your customer success for for you, Mike. So how has it been working with us both from a technology and a, company perspective? Is it for me or Mike? Yeah. I'll start with you as a customer. Okay. So it has I will say it's been great. Like we said, we started with the beta. And I would say, like, we both didn't wasn't sure where this was going to go, but we've been able to really give our input. I think Kaveo has been very responsive to us, allowing us to test things out, being the first to test different technologies. So that has been big for us. And from a technology standpoint, the fact that we can stand up a hosted page, like, within, you know, minutes is is awesome for us. We're constantly like I said, we have so many different use cases, twenty five applications. We're constantly just like, let's just see what this looks like before we even put it out to anyone. So that has been quick for us to not have to create a whole, like, test environment or, you know, go through those things. Being able to test quickly, within the hosted technology has been has been great for us. And the analytics are also great. Yeah. Good good to know. Mike. Yeah. I I would say, you know, the acceleration to value is, you know, if I were to sum it up in kind of a concept is where Coveo is the best. Right? You know, they have top notch learning and training, UI accelerators. Right? So you you really can get to a solution very quickly. On top of that, you know, customer services, they're super smart and responsive. You know, and I always see that with, you know, companies. And, you know, the product continues to evolve. I love how, you know, tagging and filtering and and some of the tools that, just make it easier to to do that. And, you know, you've always been an AI, you know, driven company. Right? So, having AI at the core of of all the products, even before generative AI kind of, you know, got really straight and and became hype and mainstream. So I think that's, what, to me, is is the the biggest value around Covea. Okay. Great. Great to hear. Thanks for that. And, I'll finish with this last question before we open up for for q and a. But, Mike, since you you, you just finished, what would you say are some of the the quick wins that, you know, organizations can can implement to demonstrate that value, build early momentum with their AI driven search or Gen AI or, you know, their overall AI strategy? Yeah. You know, keep it simple, but looking for high friction, you know, high value, where where data is locked in a system seems to be the the sweet spot, whether it's, you know, sales, marketing, you know, knowledge workers, that made information may be in in Salesforce, SharePoint, ServiceNow, but really to connect them to, you know, the data that's that's been locked away and take away that friction, are the, you know, biggest quick wins. Right? So even if it's just starting with with search, that's gonna be such a strong foundation for you for the the generative AI era. I love the fact that you bring in that friction again because that's really where, you know, we see the value of AI is removing removing the friction from the experience. So totally agree with that. I should from your from your perspective, you you've gone you've gone through this from a a customer standpoint. What what are the quick wins? Yeah. I would say the same is being able to aggregate information from multiple different places into one area has been the largest, game changer for our employees. And, again, getting them to the information quicker, it has has just been what we've seen, and they've been able to engage into all those documents. Great. Great. Well, thank you for that, and thank you for for your time. Patricia, I'll turn it to you for for q and a. Thanks, Scott. Amazing. And, also, shout out to Renee Gagnon, the CSM for Vanguard. What an amazing She she's one of my favorite people at the company. Love this. So alright. Let's go through the questions. We have some great questions that are left. Leishan, we've answered this earlier, but we'd love to know. What has been your biggest hurdle since the introduction with generative AI? Is it keeping up with adoption requests, security, considerations, or something else? Adoption request is interesting. No. I would say our largest hurdle has been just content. Like, really, how, like, how do we manage all this? I mean, across of all of our applications, we're doing, like, fourteen million documents. Like, it's, it's kind of, crazy how much knowledge and information is here at Vanguard. So that has been our probably our biggest hurdle, and those are things that we plan to tackle this year and think about, like, how do we really, crack down on content governance and, outdated content, things like that. Love that. Yeah. And we have a lot of exciting, product innovations in our pipeline and roadmap as well to help address some of those content concerns. So excited to announce those at Nu and Coveo. Mike, what would you say are some of, some lasting thoughts on some challenges and hurdles for maybe even for twenty twenty five? Yeah. I mean so just maybe few a few tips. Right? I'd I'd emphasize the, you know, taxonomy structure of your content and tagging. Right? So that the, generative AI in search underneath that has the right context to filter the content to what you need. You know, maybe putting in FAQ pages, right, so you can kinda get some quick wins of, just content that is is formatted that way, and putting in a continuous improvement framework. Right? Rather than a mega cleanup project. Right? Prune out bad content, fill in the content gaps, and really start, you know, listening to what people are searching for and try to fit that rather than take it on as this, you know, giant project that you have to solve everything, at the beginning. Love this. Thank you. And we have time for one more question. Let me see. Okay. This is a good one. So Mhmm. As product managers, what advice do you have for product managers going from search to AI, in these trying times? What what piece of advice would you give them as they explore these new technologies? Don't be afraid. Don't be afraid of it. That would be my advice. Like, embrace the change. It's a evolution. Think about it as just another it's just an enhancement and a feature to search. So I think people get a little bit overwhelmed that, like, and back to Mike's point, start small. Start something try it. Try something small. See if you could get some quick wins. Great. Mike? Yeah. I I mean, I just one thing that came to mind, I was on a project and working with a project manager, and she decided to just ask generative AI based on a PowerPoint. Like, hey. Can we create a project plan for me? Right? And it and it didn't. It did it. Right? And it wasn't a use case that I was, like, thinking of. But, you know, just kind of ask it questions that you would want it to do, right, and see how well it does and, you know, experiment with it. Right? And I think through those learnings, then you kinda understand it more and and can, get a sense of of what it can do and, you know, how that project, or implementation may may work out. Yeah. I think one of the the great topics that has come up a lot throughout this conversation too is this element of trailblazing and experimentation. I think everyone's expecting the future of GenAI to be today, and it's still an area that is involving a lot of exploration. So sounds like you need to get the foundations right and have the right partners to help you move forward. Well, everyone, thank you so much for joining us for our employee experience roundtable, redefining the future of work, secure Gen AI strategies for innovation. We'd love to share a few resources with you before we leave. So please don't forget, if you're interested in interested in, RightPoint's wonderful implementations, please check out Spark OmniOne. You can book a demo at this website over here. And we also have an on demand webinar as well, five strategies to enhance Gen AI, with data and content management that, I hosted at Kilometers World with a few of our wonderful product managers who are in charge of relevance generative answering here at our company. Finally, as well, Ahmed has asked me to present this to we have relevance three sixty taking place, in Barca, March sixth. So please feel free to sign up using the QR code here. And a huge thanks to Leishan from Vanguard and Mike from Roypoint for joining us today. This has been so wonderful. Thank you all so much, and we hope you have a great Thursday. You too. And thank you, Patrick. Thank you.