Alright. Well, an official welcome here. Everyone. Got a a great group joining. Still see that, some folks are still logging in That's fantastic. Also see that that a lot of you are introducing yourselves, introducing where you're calling in from. Here in the the webinar chat, that is fantastic to see. Glad that we're, getting folks from all across the world, and folks who are eager to to join today's conversation. Let me start by introducing myself. I'm Devin Pool. I'm a senior product marketing manager in our service practice here at Coveo. Prior to that, I've spent about fifteen years researching and advising leaders in the customer support and customer experience space, at a company called C EB. Some of you may be familiar with with C EB. It was a best practices research firm. Was acquired by Gartner in twenty seventeen. So I I spent few years as a Gartner analyst, there as well. Don't hate me for that if you're a technology provider and, didn't get where you wanted it to match quadrant. I had nothing to do with that side of things, but, and I I've been, looking at the the customer service space for a long time. And I'm really excited for today's discussion. Right? And I say discussion, purposely because I could sit up here and talk for the next, you know, forty five to to fifty five minutes. My voice will allow for that, but You all are aren't here. You're just here from me. Right? A lot of folks that we we talked to, prior to joining the the session said, very interested in hearing the the thoughts, the, insights, the inputs from my peers. And and that's what today's session is designed to be. Right. It is designed to be a roundtable purposely. So, we will be able to you know, be, off of, mute. You guys are able to hop on. And what I'd appreciate, you know, we'll do with, like, a rolling introduction. First time you jump in, you know, hey, it's Devin from Coveo. I look after our, you know, customer support function, whatever, you know, your title is, you don't need to, give a whole backstory. But that that's the idea. Right? Just introduce yourself briefly. And speak up, please. Like, today is a day for you to to share some insights with your peers. To learn from your peers, about arguably one of the the hottest topics that has ever hit the the business world. Right? I've been looking at, customer support technology, as I said, for about fifteen years. And You've not seen a rush of, you know, skeptical enthusiasm as I have when it comes to today's topic. Generative AI. And specifically, how when where is generative AI gonna fit in the customer support landscape. Right? What types of generative AI, are we going to be using where is your organization currently? Right? That that's something that, what we all wanna know. What are others doing? How are others approaching this topic because, you know, two things that that that I've known to be true through Micon sations with, you know, our customers, with the market, with, research experts that that, our former colleagues of mine This capability holds a tremendous amount of of potential and power in the impact it will have on the way that our customers deal with our support. It also holds a lot of potential risk for organizations as well. Hence, the healthy amount skepticism that that you hear from a lot of organizations. Some folks are are in a, you know, wait and see approach some folks are diving in ahead first. You know, and others are in between there. So let's start off. I'm gonna just share a a couple of stats, a couple of, you know, things that we've learned from the market. And then I'm gonna wanna get your thoughts. On these. Right? So this is where folks are gonna be able to chime in, you know, where you're gonna be able to hop on the line and give your opinions because that's what we wanna see here. Starting off, you know, first, just by looking at, you know, some polling from the the customer management practice. CMP research. They're part of the the CCW group. If anyone's heard of them, right? They went out. They talked to contact center leaders, you know, and they looked at what your priorities are for twenty twenty four. Right? And you can see on the x axis running horizontally here, the level of importance that people are putting on each priority. And on the y axis or the vertical, the level of difficulty. Right? And what's interesting is that often, you know, the things that are most important to us as leaders, we're we're starting to see as the most difficult. Right? Back dropped in that gray oval there in in order. Right? What do folks say they need to do in twenty twenty four? Create a frictionless experience, integrate generative AI technologies, and increase adoption of self-service. Right? So those three things. Two of them are perennial top three items for us. Right? Creating a frictionless or effortless experience. That's something that the service leaders have long sought after have made great strides against. You know, I've been researching effortless and nope to to work out and allow the research that went into. The book, the effortless experience, and things afterwards, that that is absolutely been a top priority. How do we make sure that we create, you know, seamlessness across channels? How do we make it easier for customers to find answers? That's always up there. And it's always up there because customers, you know, level of expectations always going if they see something new or cool, they want more of it. And of course, increasing adoption of self-service. That's a perennial top three priority for service and support leaders as well. Though one thing I'll I'll caveat that with is at least in my experience, I've seen it's not necessarily getting customers to adopt self-service. They've adopted it. Maybe not with your company. That that would be an interesting conversation, but they've definitely adopted it in their own lives with lots of other you know, lots of other experiences and interactions that they have. Right? The problem's never been about getting people to try it. It's about getting them to stick. Right? It's about getting them to resolve their issue fully there. And that's where I think the term adopts tends to come in from a lot of organizations. It's we want people to get in and and adopt and do it. Now, if you read that and you think Well, what we mean by a doctor is that customers only ever use self-service. Well, that that that's not ever gonna happen. You know, that is was at one point the goal for a lot of organizations What I would encourage us all to think here is not get them to adopt self-service fully. They will never contact us or use anything else. But self-service because that's not ever gonna happen. There's gonna be a need for customers to interact with our humans. But to get certain issue types, certain questions, certain queries, certain, you know, situations, that's what you wanna get customers to adopt. Fully or adopt, you know, ninety plus percent of the time. And when that happens, by the way, you should all feel free to ultimately start saying no to customers about, you know, we were calling in for something that ninety five percent of other customers can do online. I number of years ago. As a side note, worked on a best practice with the Australian taxation office, where they had adopted the system similar to this, and they felt free to start saying no to customers who were calling in for issues that they could solve. And again, if a government taxation organization can do it well, well, think there's hope for all of us. So looking at this, right, where we see, again, top three priorities, and going even further, right, into, the generative AI component of this. Right? And we're gonna, again, open it up to folks. We're gonna open up poll here in a second, and then open up the line for y'all to to talk as well. But, you know, when we look at this, It's happening and it's happening now. Right? As a priority for organizations, it's not just integrating Gen AI doing it within twenty twenty four. If you look at the three stats going left to right on the page, starting on the left hand side, ninety five percent of service leaders, according to some research that BCG has done said they expect customers to be served by an AI bought at some point in the service journey by twenty twenty five. Again, that's not at all points. That's not for every issue or every situation, but It's going to be a core thread of the way that we interact with our customers going forward. And you see that born out in the next two stats. In the middle, eighty two percent of companies, in the CMP research survey expect to offer generative AI and customer facing self-service by twenty twenty five, so by the end of this calendar year. And on the right, eighty six percent of companies say that they're planning to use JNAI in their knowledge bases by twenty twenty five. So that, as a backdrop, we'd love to know, you know, your thoughts and answers on this question. I'm gonna launch a poll for you. All of you should be able to to see that poll. You should be able to to see it up here. We'll broadcast that. Thank you in advance for your votes. Right? But what we'd love to know, and those of you that that are voting, Again, I'd love to hear from you as well, like, how far has your organization progressed, with JNAI capabilities. Right? Exploring use cases. You're testing the value. You're building a business case. You're actually in testing or piloting. You know, where are you? What what's gotten in your way? What has been successful? Like, let's start there. Right? We're all on this train. We wanna see exactly where others are. Right now. You can see, on the polling, you know, little under half of you. That keeps changing going up and down, say that we're we're in the testing and piloting phase. Right? So let's start there. Folks that that are testing. Right? If you have, answered that, we'd love to hear from you. How long have you been doing this? Where and and what are you you testing? Exactly? Yep. And by the way, you can hop on to the chat or or the Q and A section. If you're all muted and you need to let us know, hey, I wanna chime in on this. Just just hop on to the chat. Section here, let us know, hey, I got a thought, and then, my colleague, AA will, unmute you and and let you get into the discussion. Alright, Eli. You are the lucky first. We've got a door prize for you. It's an actual door. Now, Hey. Hey. If you could, unmute Eli. Welcome to the conversation. Thank you so much for being the first brave soul to to chime in. Oh, thanks. With a great introduction. It's actually early. I'm with Connexus. That's fine. I get that all the time. I'm with Connexus. The program manager for, Casey adoption program and, the senior content, specialist for our community as well. I did wanna chime in because I had actually picked the fact that we, in the touch and the pilot interface. So we did build an antenna AI bought for our customers, but then it very much centered on our product help and documentation. Right? And it's gonna be coming in an upcoming release to customers. And the idea there is that the information that we have in our product documentation is trusted. Right? So it has to come from a place where, to prevent hallucination, to prevent they bought from trying to make a balance. We wanna make sure that the information that it's given to a customer is, is accurate, and it it would actually generate that value that we want them to be able to have within the system. So the idea is it's an extension of our community. If they do want more information, definitely it does point them into our communicate community for them to be able to, for the read up on it, but little questions, like, how do you do this? It gives you that immediately based on what our health documentation is giving them, which, is our first step into that foray. Okay? Yeah. And thank you. Sharing that, Ellie. I really appreciate it. Let me ask you this. You I I know you mentioned that you practice KCS. Your organization. Right? And how much do you think that helped you to get to this phase faster? And what advice would you give to organizations that might not practice KCS right now to, you know, start to get their knowledge house in order? I think, the consort construction service innovation has the it's got a lot of resources. So that's the first plate I would depart. But also, make sure you have an article life cycle management. Right? Because, that is how you're gonna keep your, dataset clean. That's how you're gonna keep that knowledge fresh, and that's how you're gonna keep that knowledge, accurate for people. You wanna pay attention to flag it and fix it. Ask PCS demands, the adequate uncollectively owned. It means that anyone can do an update to it, but you have to and show that the integration into your case system is very well, it's seamless because agents it's a change in behavior of rage And, you'd be surprised that it, actually, a more of a barrier for managers than the agent, even though they both equally require change in behavior. But it's, it's something that requires content, repetition of what the what's an input that missed. As well as in showing that you do have, a second of sponsor because that's the one of the key things to really be able to drive it across. And I would even push that button to say that, KCS as much as great as it is. It's a different methodology, and it under the realm of knowledge management. So it might not apply to every scenario, but you can still pick up the principles of knowledge management to ensure that, what you have or what you wanna build, your LMM on really is accurate because that I think that's the, the foundation of ensuring that your your bot is not as highly stated. Because that's one of the biggest years or also. And I would say, security binding, which I think Kuberia addresses with the search. Right? But you wanna make sure that people do have access to the information that they allowed to have access to. Yeah. Just because, it's great to have the AI. It's great for it to be able to provide guidance. But there is still that element of security that we can't forget about. Right? And it, with a lot of the companies in this space now, that's one of the biggest challenges. Right? So do wanna keep an eye out on security. Yeah. Right. And security comes in two form, but there's, you know, security against, like, cyber attacks and and accidentally exposing something out to an LLM that will be used, you know, to train that. People are worried about their confidential information getting out, but the other one you bring up is the other side of security rate of, we don't wanna accidentally expose something to a customer that they're not allowed to see. Right? That could end up. I've obviously there's liability and you know, security issues there. But, like, what if we expose a wrong answer? Because, well, some, you know, the example I always run to is like an airline. If someone were to ask about, what's my baggage allowance on the plane? Well, that answer is gonna depend on whether or not you're a status holder with that airline. And so if you accidentally say, well, yeah, you're allowed up to three bags and you give the answer that is for the top tier of status holders, and I am not that. Well, I'm gonna show up to the airport and go, no no. Your chatbot told me that I could have three bags. Right? So I'm checking those for free. It just causes more frustration and headache than than it's worth by, you know, having, permissions security not follow. And and that's, as you you noted, a core tenant of our approach to this, right, that it's not just have a generic bot. It is have that bot as an extension of the capabilities, that come through a unified hybrid index that has those secure connectors and follows the right permissioning, for that documentation. I did have a question. Yes. As you. As she is speaking about that, and that would be, though, those fours is bringing up with, Einstein. Which, and now they are allowing people to be able to index all the sources that you can search through Salesforce. How with Coveo, thinking about that with regards to the fact that it is competitive. Another company that doing exactly the same thing that you do. Mhmm. And what differentiates you from what Einstein of what Salesforce is doing? Yeah. You're right. At the surface, it can certainly look like they are are very, very similar to each other, especially through something like the the UI. You know, where I know that Salesforce is going is you know, utilizing connectors to try to pull more information from outside of Salesforce in. You know, where I think we're doing it differently and better in some ways is, that the ability to put that, you know, generative AI box and actually some I'll get into a little in the middle here, like what we call the intent box. Right, putting that into different channels along the the journey. Right, because customers are gonna interact with you in different channels in different ways. And Salesforce will only own part of that. The the other thing that I think sets us apart here and not like I'm trying to turn this into a sales meeting, certainly not what it's gonna be, but the the idea is, that we have such, you know, longstanding mature relevance models and that's what's gonna help to really we think set Coveo generative answering apart. Right, the ability that we can, you know, find chunk vectorize the the right portion and most relevant portion of that document or documents. That's what's gonna be, than different in this world, and that's what's gonna become even more important. It is, did you get the answer right? And can you keep the context of that thread going with that customer? And that's exactly the vision that we are seeing as well. Does that answer your question, Ellie? Yeah. I think it does not convince silly, but close enough. Yeah. It's Believe it. I I can get you on the phone with someone much, much smarter and more technical than me in order to, dig into that. And it sounds like we should you know, hop on and and get deeper into that. But happy to to do it after this call. We'll we'll follow-up. We already we already do have a call plan someone like Antonio. And so I just wanted to pull and see if there was something I could get before then. Yeah. Well, with that, I know we we've got a lot of other folks who are are joining in here. You know, Andrew, thanks so much for for hopping on, Andrew O'keefe. I know that you said in the chat, you guys have been mostly testing, you know, internal developed solutions. I'd love to to get your thought on that, I know you're exploring it in pilot and Coveo, solution for your website as well. Do you mind providing a little more color when you say internal developed solutions. Is that something you're building on your own, or is that something for an internal use case or something else? Think is gonna be unmuting you here in in just a second. Andrew, welcome to the conversation. Hi. Thanks. Yeah, we had they've been mostly, like, company wide kind of, generative AI, and it's it's basically kind of on a pilot or testing kind of scenario or phase right now. And then as far as, Coveo and the generative AI, we've been my my upper management been looking to get us to do a pilot for that. And so, we've been we've had kind of reviews, initial reviews were from our legal department, had some initial conversations with Coveo with our legal department about this. And That's our biggest the biggest concern, and I think you've already touched on it, which is what I was I was mentioning is that the the concern that that everybody has, because we have very sensitive, highly sensitive data, and information, being in healthcare. Is that, we're not gonna serve up, the generative AI is not gonna serve up exactly or it's gonna serve up answers that aren't gonna that that that are gonna be incorrect. And it it kinda is like your example about the baggage. Right? And is show up at the airport with and it said, well, you said you I could get three free bags, right? It's it's like that. You know, it's like if something comes up in a recommends the wrong device or recommends the wrong information, or implies the wrong information, then then our company could get could be liable or or could, you know, be in trouble for, for that. You know? So it that's why we're kind of, like, Well, internally, I think we're we're we're exploring it for, like, looking at looking for internal doc and looking for, answers to things, that employees can see, only employees. But external, we're still trying to explore and figure out, you know, can we use this, without, without too much, without without causing any problems? Yeah. Yeah. Thank thank you for sharing that. And it's a valid concern. Right? It seems like every time we get a little reassurance that this is getting better. You see an example like the Air Canada a few weeks ago happen, and we all get a little skittish and take two steps backwards. Which is understandable, right, especially in your space in health care. That is I mean, you're literally dealing with life and death at times. Right, and not giving medical advice, that way, but it is something that we worry about I don't know if anyone follows like the the Gartner predictions, that they had a a Gartner prediction that said, you know, by twenty twenty seven, at least one customer will die as a result of advice given from a generative AI bot. Twenty the guy who wrote that's an old friend of mine. So I had to chuckle with him and said, what what are you doing? Right? But that that's the world. Some people think and see that we might be living in. And what you're talking about, like, if if it helps to give you some solace for for you in in others on the phone, right, is we've been here before when it comes to testing things and learning internally before exposing it. Right? This was the way most knowledge bases work. We had a knowledge base. It was for our customer or for our agents right? And they drew on that knowledge base. And once we kept seeing the same rep, repetition in questions or we could talk about what they We we knew what, documents would answer what questions. Well, then we could expose that knowledge base to our customers' drip feed slowly, but surely exposing a little more and a little more and a little more. Right? And what generative AI is changing in that is ultimately the findability and discoverability of it, at least the way that we are doing it. Right, at Coveo, our focus is on grounding that answer in your company's fact base. Right? So it is putting the the things that you are already exposing the customers, and I'm you have things that are available for your customers to see, online. Right? It's taking that documentation and putting it into a form that that's a lot easier to digest and find for a a customer. Right? The way that I think of it is you're you're looking at these issues and questions that customers are asking where the answer is available, but it would be a pretty massive Peta for the customer to go out and find that answer on their own. They might have to look through one really, really long document and find just a couple of relevant paragraph or they might have to look through two or three documents, and stitch together that answer in that their brain. And what happens at that point and is gonna continue to happen and only get worse is that customers are gonna get frustrated they're gonna lose confidence in themselves and their ability to understand this, and they're gonna throw their hands up and just pick up the phone and call or chat or email or open a case. They're they're gonna go give me a human at some point in there. Because what it comes down to ultimately, right, is uncertainty. A number of years ago, I worked on a deep quantitative study where we talk to thousands of customers about why they abandoned self-service. Right? And it's things like transparency, credibility, trustworthiness, and it came down ultimately to the idea that, you know, customers hold a lot of uncertainty in themselves. And do I really understand this? Is this really the thing and, generative answering has a very good way of explaining something that we wrote in our documentation that we thought was rock solid. Of course, if our engineers can understand this, Why can't you? Right? Every company in the world is guilty of it. We know too much about ourselves. And we use, you know, the the generative AI in many ways, like, the world's best English professor. Hey, make this sound simple for me in a way that I can digest it. Right? And there was a Anyone's a nerd like me and wants to read old psychology journals, look up something called the uncertainty reduction theory, it it was written by two guys at Northwestern in like the nineteen seventies eighties, but some of its most famous use cases are really interesting. One is around eBay. Right? In the early days of eBay, right, the idea of buying something directly from a stranger on the internet in an auction, that was just wild. Like, why would you ever do that? The amount of risk that happens there, and what they found, right, was that the company or the sellers that shared more information, shared more pictures, about it, gave more detail about that. Right? People that use communication to reduce that uncertainty. They were not just more likely to have their auction the auction items sold. They were getting more bids and they were selling for higher prices, even for the exact same item than than other people. Right? So that's where generative AI holds them a tremendous amount of power. It takes that cognitive burden off of the the customer to say Okay. I can get this in a more simplistic form than than it might have been before. Carrie, let me bring you in here. I know you've been waiting patiently on the line and have some thoughts as to where you are as well. And then we'll start to to move on, you know, out of where we are to how exactly we're we're starting to deploy this and how these decisions are being made at our organizations. But, Carrie, welcome to the conversation. Where where are you guys in your organization? Thanks, Devin. Yeah. So I'm a program manager for unified search at my company. We're UKG. We make HR payroll tech software. And the manager number of use cases for unified search so that we have our customer facing self-service piece, of course, case deflection, and then internally for our agents. So we're really just looking into GenAI overall. We're testing, you know, Coveo functionality as part of that, and then also homegrown and things that we're building internally. And, really, we we knew the test was gonna be about seeing how it worked with our content, knowing we have all kinds of content, you know, knowledge articles. We have sixty thousand knowledge articles. We have really official product documentation, case details, information like that. But what we're finding is that the, you know, specifically the RGA functionality by Coveo is performing really well with generating answers, and they're really helpful. But anything that's incorrect or where there are gaps has to do with gaps in our actual content. So Yeah. We're trying to figure out what the balance is with k. We really wanna put this functionality in place. It'd be really helpful for our users, but how do we close all these content gaps, that we really didn't even know existed prior to trying on something like this? Yeah. You know, and that's something we're talking about and thinking about as well. Right, which is if you don't have an answer, don't generate something for generative sake. Right? You can return to search result or say, you know, we don't have made on this. And, we will get you to a person to solve that. I'll show you a vision of what we see that starting to to be. Some of you probably gonna ask me for a timeline on that. I don't have a timeline on that, but, you know, like, this is where we see the world going eventually. Right? And not surprised that you're seeing great results and better than than others in your content. And that ultimately, it comes down to relevance. Right? The ability to find the most relevant piece of content, and the chunk within that piece of content you know, what what I'd say, Carrie, and for other folks on the line, I know we we have a lot of folks who are, you know, from the the web and search and digital product management side who who joined us today. I come from a contact center background. Right? And if we can get our contact center agents, anywhere north of, like, eighty percent getting the the right answer, then we're pretty happy. Afraid it and some of you are thinking like, oh god, that sounds terrible, but like humans are fallible, right? And so you wanna think about this with a tried and true process, you know, quality assurance. QA is a very well known function within the contact center world. It's a very important function for, you know, two purposes. One is auditing the the answers that are coming out and that's where it's gonna be, really helpful with GenAI. Like, you're gonna put some of that same rigor in process that you would use in a QA process. To your generative answering. Right? It's like you would be crazy not to do that because you wanna make sure that what's going out is good and know that we're never gonna get to a hundred percent, at least in in the the very near future. Right? Because things can happen. Humans are fallible. No human in the world is ever gonna get to a hundred percent, but the the machines are certainly a lot more active. Grit that than humans are, these days, and that that's only getting better. So with that, I I appreciate everything that that joining in. I know we're we're looking at, you know, generative, gene AI here. A lot of you have been talking about our generative answering capability. But, of course, you know, let me show you a couple more things. And then I wanna get into, like, how how is your organization making the the case and what's happening in your world you know, of gen AI, because this is just customer service, gen AI for the the most part. Right? There are lots of forms of generative AI. Right? There's content creation. There's content discovery. There's conversational AI. You know, there there's co pilots. I don't know if anyone's got a thought on this new term of co pilots, it seems to be like an amalgam people are throwing lots of different, capabilities into right now. Right, there there's AI simulations as well. Right? All of these different things that exist, all of these capabilities that you could bring to bear. You know, what we've seen in our position at Coveo, right, is, the the same as it's always been by the way that this framework comes from the folks at Gartner, if you're wondering, but you know, my my position has been at the end of the day customers just want answers. Right? Like, they they Coveo contacted us. They got into our knowledge base. They got onto our self help site. They submitted a case that they did something because there is some sort of pain. And the faster and easier that we can make that pain go away, the happier that customer is gonna be. And that that was the whole foundation of this idea, that we call the effortless experience in my time at CEP, right, that customers that they just want you to make the the pain go away. And they don't necessarily care how and where that happens. And they are leaning on companies that they do business with to start to guide them through that process. Right? So all of this to say, generative AI is a powerful tool in your toolbox. It's not always the right tool for the job right? It is not always the thing that that you need to use for every single situation, but it it is a very powerful tool that is gonna help us along the way. You know, and it's not the only form of AI. I've often talked to people, you know, some very, very smart people were like, oh, yeah, we're using AI. We're building a, you know, chat GPT chatbot. And you're like, what other forms of AI are you using there? What do you mean? What other forms of AI? Right? And so, there there's this, I think a little bit of like confusing conflation in the market right now that says all AI is generative AI, and I think all most of you on the line probably know that to not be true, but let me state it for the record. Right? There's lots of types of AI. And you're gonna deploy that AI at various stages along the customer service journey. Right? From public facing websites on the left through authenticated portals, digital service, self-service channels like bots and, you know, search pages and things like that. To all all the way through the contact center. You know, highlighting Genai as one of the tools that you can start to use here. And that's the the way to start to think about it. It's not the only one that that you wanna use, though. So with that, let's start to, you know, talk about this. I'm gonna throw a another poll up on the screen for us as well. Right. How is your organization making decisions about GenAI across the enterprise. And, let me move this down so we can see the question on the screen as well. Right. Do do you have a centralized generic committee? And if you answered that, I I see, you know, number of you have answered that way already. Fair warning. I'm gonna wanna know who's on that committee because that's an important to discussion to to talk about, right? And how are is that committee being formed? Are they making all the decisions? Are they giving use cases submitted to them? Etcetera. Interesting. You've got, you know, a pretty even split here in the group, right? Almost twenty five percent of us say, we've got a centralized committee, thirty one percent say we've got individual business functions that that are identifying solutions. That's pretty interesting. Another twenty five percent saying, all decisions are made within technology. And then three of you said something else. Man, I'd love to hear about that as well. Let let's start with some of the folks that are, using a centralized Genai Committee. Right? We've got a centralized team. You know, who founded that? Who's on that team? How is that team functioning at your organization right now. Who's got a thought on that one? We had such great discussion on the first question as, we'll open it up. Other folks that that are on here as well. If you've got, you know, something else going on or we've got individually business units, answering in, like, how is that working for you right now? Anyone got a thought? This is Andrea. I said something else, but we do have, like, a centralized committee, as well that's been kind of studying, like, it from an internal perspective, kind of what I was mentioning before. But then you have what something else meant in my take is that We have people like myself that are the, you know, product Coveo product manager, search project, product manager, that are looking into, other forms or other ways of doing GenAI, kind of like I mentioned before, So that's kind of where I came from from somewhat something else is that there is, like, there is, like, a centralized committee here that's looking at it and kind of determining, hey, is this okay for us to use? But then you have people like me that are kind of exploring it, for other cases beyond what they're looking at. And then that in combination with, like, our legal, department, kind of like I mentioned before. Yeah. And so that committee, you know, there's it sounds like they're setting some governance. They're they're setting some processes. Are they also deciding on use cases? Like, do they have some sort of final authority? Or if you decide in your area, this is what I wanna do, then, you know, your team can make that decision. I don't know that they've put down any sort of authority per se saying you know, like, Andrew, this is what we found and you can't do this. I think that more that that answer more comes from kind of our legal department and will come with reviews of, okay, what are we seeing with this? Yeah. And and is this what we expect. But, I think from the standpoint of what they're reviewing, you know, they do kinda have their own rules and regulations as of right now, I haven't been asked or subjected to anything as of yet, but then again, I haven't started our pilot yet. So, you know, I might I might still, that that still might be TBD. Yeah. And, you know, I mean, that there are some, guidelines. It sounds are great. Well, we we gotta make sure, of course, legally that this is available when You know, that's a, an evaluation threshold that you're gonna have to cross. I think as we would with nearly any new technology that comes in or any existing tried and true technology that comes in, house. And that that's interesting because I I've heard more often than not. Yeah. We've got a centralized committee, where different business units are gonna, you know, submit use cases, because they're looking for a couple of things. Right? Well, we don't wanna be redundant. We know we we don't wanna invest in the same capability in different areas. And we wanna have more of a overarching architecture plan for for how our tech stack is gonna run. Right? And it sounds like you guys might be things sort of closer toward that world. I really appreciate you sharing there. Other folks, other thoughts you know, where where you guys are on this. If you've got, individual units or you do have a central committee or something else, where is your organization? Hey, Devin. This is Carrie again. I will just share. We've got a little bit of both. You know, unmuted on the line here. Yeah. Still here. Still here. But no. So my answer was we have a little bit of both. We do have the centralized committee and then We still have, you know, folks in different parts of the organization kinda working on their own things. It's difficult with a large organization and everything moving so quickly. Like this, you know, this technology showed up, and then everyone wants to figure out what to do with it. It I'm sure it's it's difficult to rein in Yeah. I I agree. You know, it is, and we wanna make sure that we're smart, about the the way that, you know, we deploy and invest in these capabilities because we we've been there before. I think for a lot of companies where decisions have been made in silos and that can be, that that can be dangerous to us as a an organization so, Carrie, who's on the the centralized committee? Are there, you know, heads of business unit business functions? Is it mostly tech legal operations folks. I I've heard of wide arrays. I think most companies are trying to figure this out right now. But do you mind sharing, like, who's on your committee? What types of roles and people? Yeah. So, of course, we have we have our tech folks who would be owning the the building piece of our of whatever the solution is. And then we we do have representation from the different business units. So all across the business and, bringing their use cases and saying, hey, these are our use cases. These are things we'd like to solve with GenAI. And then the tech team responsible for building it. In the Coveo example, we're in a unique place where Of course, majority of things can be configured within Coveo. We don't need to build something separately. And we're kinda saying, hey, well, let's let's see how Coveo performs and, and then we're in the unique position where we can compare side by side with homegrown, functionality and then with Cavellos. So Yeah. We are we are the shadow organization kinda doing a little bit of our own thing. Yeah. I mean, that that's it. Right? It is known entities. Well, hey, we we're already getting, you know, good results out of, these folks in, in other areas. Right? So that's why we're seeing quite a lot of, you know, track shit with people. We're like, well, we're using this for this and the the way I see it again is this is a very logical extension of the capabilities that you already have. And you wanna make sure that, you know, we're we're building that world where AI and Gen AI are are not seen as separate things or anything like that. In fact, you know, let me jump ahead here to, oh, and and Ellie. Sorry. I know you wanna join in here. Let's, bring you on, as we're we're we're going forward. What what's your thoughts at your organization? Oh, sorry. I was still I'll put a mute back. I I really just had a question more so, and back. It looks like a lot of people do to say that they have a lot of, shadow GPT, and, I guess, that struck me at interest in it that because there is a trust with what they're already, is it because of the security aspect of what wide badly had that so many shadow GPT? Like, I mean, We also have ours, and I think, Afbo Ray into it was because we it was probably the fastest way that we could get in there without having the cost implications. So I'm I'm just curious to know why, so many shadow GPTs out there. Yeah. And and, Warren, I I'm gonna bring you in here because I saw they said, hey, there you are. Same page club. I know that you're looking at this. So, hey, if we could unmute Warren Cook, love to get your thoughts you know, why so many shadow GPTs in your organization? I think it's the nature part of it's the nature of the type of company. We are. We're a, a security company. So we are heavily focused on, on R and D in terms of the split of our head count. So If you put something like GenAI and the toolkits that are available in front of an audience that that likes to do things like this, is just naturally starts to create itself. So it becomes a a little bit of a race and, the wild west is sorting starting to calm down a little bit, but it requires corporate governance to make sure that everyone understands what the the implications are. And and training of which there can always be more. And not only of the dangers, but then how to use it correctly and what the right ways of doing it and then hopefully a centralized committee, which we struggle with. We don't have that. I just think that everyone's It's such a huge opportunity for people to feel like they can add value to a company and the tools are readily available. It's hard to suppress that. Yeah. Yep. You know, I agree with you on the the wild west kind of dying down a little bit, right, for us it's gone from twenty twenty three being the year of Genai exploration. Right? Hey, everybody get out into the mountains. They're golden them hills if you listen to the Matthew McConahan commercials enough. Right? But it is, now that the year twenty twenty for is the year of GenAI operationalization. We've got what we've learned. We've tested in a lot of cases. Some people have gone. Out ahead of us. We see it out in the world. So let's start to to make sure we're putting our hand up and getting in the game there as well. So really appreciate you you sharing that there. You know, something I I've seen. Right? There's a there's risks with a any new technology and generative AI right, is a a new technology that will mature pretty quickly given the amount of investment in the engine that it's getting right? So of course, there's security risks, there's exposure risks, there's hallucination risks, but, you know, one of the, I think risks that is becoming very under, appreciated for a lot of organizations. It is a risk of our own making as companies. Right? And that is the risk that GenAI is gonna be put in as a new silo into our organization. It's gonna be yet another place that customers end up going. Right? And let me show you what I mean. Right. This is sadly, for a lot of companies that the kind of, digital experience that has been created. And one of the main reasons that customers still, flame out in self-service, right, is I I couldn't find what I was looking for. Or I didn't really understand what what I was looking at. It wasn't relevant to me. Right. That was from Gartner serving about a year and a half ago. They said, yeah, that that fifty percent of customers who fail in self-service are still for those main reasons. And it's often because we've invested in a fractured variance. Right? We've got one, two, maybe up to four or five different search boxes or search engines that are running across our different digital properties. And they may give different answers to customers who ask the same question into it. That's a problem in and of itself. But then we've got, you know, communities in question space. We've got, you know, chat bots and chat windows popping up, and we've got UX design and navigation features, and all of these things that were all put in with the best intention in in mind. Right? We never put anything in thinking this is gonna harm the experience, but it put it been put in it and fractured and siloed ways. And that in and of itself, right? Hurts that the customer experience because they can have a much more inconsistent experience. I can go to two different parts of the website and get a totally different answer, and that's not good for me. That leads to me going just gonna have a human explain this to me. Submit a case, start a chat, send an email, pick the phone and call, or in some cases, do all of the above. Right? Like who amongst us as customers has never been chatting with someone while also waiting in the queue online after you've sent out an email to that company. I've done it. I think I might know too much about how companies, and and customer support functions work. It makes me the worst customer in the world in a lot of cases, but you know, that happens all the time that that customers are using and engaging with multiple channels, simultaneously. Because again, their goal is I just want an answer. Get me an answer as quickly as you can. I want this pain to go away so I can continue using your product or get on to the next thing on my you know, daily list of of things I've gotta do. Right? So this fractured experience, if we're looking at NII as a siloed that's gonna be just put on and it's its own thing stand alone, it is gonna be doomed to fail. Right? The the way that we are thinking about this right? It is much more as a unified AI experience. Right? Something that is powered by AI where the worlds of search and chat and chat box and all of that rolls into one intent box. Right? Tell us what you're trying to do. Tell us the thing that you need, and we're gonna help to get you there. As an organization. Right? We're gonna help to guide you along this path. You know, based on whatever you put in this intent box. And something that, one of our clients, a large telecom firm said to us, you know, that customers, it was about search. Listen, the customers don't lie to the search box. And and that's gonna be true here. Customers aren't gonna lie. Like, if put a box in front of them and let them tell you what they're trying to do. So with that, right? I'll just give you that there's, you know, vision of the future where one intent box, at at the top of the page. I know we're coming up to the end of our time here. Quickly an hour goes. When we have great conversation. You know, one intent box with whatever that the customer tells us, it could yield any of these different, results, right, where it could be, you know, and the way we're doing it now is a combination of you know, generative AI with citations and sources at the top as well as relevant search results, below those. Right? So it's like, here's the the general answer, and here's some search results. If you wanna continue to go further, And the citations are gonna link you directly into the documents where that was pulled from, where the the generated answer was pulled from. Right? But it needs to be more conversational. Right? So leveraging, you know, things like smart snippets or follow-up questions in order that's the people are also asking to ask a follow-up. Right? I'm like, this is what the next question could and should look like. Right. And so that continues the conversation and the context of the thread. So if the question is you know, what's the difference between a personal loan and a commercial loan? And the follow-up question is, what are the best rates for each one the the model knows what each one means because it has the context of the prior question. That's what I mean by conversational. But it could be you know, contact us. Hey, this isn't something we have an answer for, online. This is something you need to contact the human form. We recommend, you know, this chatting or submitting a case or picking up the phone and calling us. This is what's gonna be the best way to solve this problem with us. And then, you know, recommendations at the bottom. So the next time that person logs in the context of what they've done with you before, is sitting there. And that context says, hey, here's how you continue that journey. Sort of, you know, the version of Netflix where it is. Hey, based on these shows that you watch, these are the other shows that you are, you're gonna wanna watch. And then, of course, along the left hand side, you know, dynamic navigation and faceting that is sort of like a comfort blanket to customers. Right? They know this, and they will sort and fast it and, you know, filter things on their own that way. People are very used to that. So that's not going away at any time. With that, I know we're right at the top of the hour. I I certainly don't wanna make anyone leap for their next session. Really, really appreciate you all joining us today. Right? If you wanna, like, nerd out and talk about any of the this data, things that we've shared today, happy to hop on the phone with you at any time. And, you know, those of you that have ongoing conversations with us look forward to, you know, joining those as well. Thank you so much and enjoy the rest of your day. Yeah, Warren, thank you for that kind comment. I agree. We could've gone for hours on this one, and maybe we will someday. Thank you. Alright. Thank you. I'll enjoy the rest of your day.