Hello, everyone, and welcome to today's live webinar, bridging the AI Wow trust gap in enterprise service, brought to you by TSIA and sponsored by Coveo. I would now like to introduce our speakers for today. Dave Barca, senior director support services research for TSIA, and Danny Rajan, lead product marketing manager for Coveo. As with all of our TSA webinars, we do have a lot of exciting content to cover in the next forty five minutes. So let's jump right in and get started. Dave, over to you. Great. Thanks, Vanessa. Always a pleasure to work with our friends and partners at Coveo. Danny, a real pleasure and delight to get reunited and to to do this again for our customers, members, and and prospects. So as we work through this topic today, it's it's it's truly more important today than ever as we talk about the the trust gap in enterprise service. The the topics that we're gonna cover today include a market overview from a TSI vantage point, from a Coveo perspective. Danny's gonna cover and walk us through AI challenges in production, know, truly why search and retrieval continues to be foundational within the support industry and the technology space as a whole. Then we'll wrap up with some practical AI design principles for service. From a TSIA perspective, we are currently undergoing a massive and very architectural shift within the support industry as well as the technology space as a whole. From a support perspective, the goal is no longer just ticket ticket deflection. Don't get me wrong. Ticket deflection is still crucial and and highly necessary. But for my Agentic AI discovery research that I'm conducting currently, I'm seeing how leading organizations are moving towards that autonomous resolution, you know, where Agentic AI just doesn't surface a knowledge article. It's actually executing, complex tasks across systems to solve customer problems without human intervention. That's creating an awful lot of tension within the technology space, and we're all trapped and caught up in this for sure. Coveo is gonna talk about why enterprise AI initiatives look great in a demo. You know, they but they tend to lose that wow factor whenever they get transitioned from the demonstration to the testing down into production. And my TSIA research definitely validates this completely. I'm seeing a very strong pilot phase dominance where AI is stalling out and not moving into production. So part of this is really tied to what I refer to as the small start small paradox. I found from the Agentic AI research that I'm conducting that if you deploy an AI agent that is too basic, one, it provides no resonating value to your support agents. And when that happens, they will stop using it. More importantly, though, if it provides no resonating value to your customers, they will step away from it very quickly and ignore it. And you cannot afford that after the investment that you make in the Agentic AI deployment. So to help break out of this project ceilings, the AI must be capable of complex executions. They must be able to think through complex situations and processes. But also, you know, in order to measure how your Agentic AI is performing, there is a considerable difference in the KPIs that you need to start measuring to ensure that your maturity gap is closing. So I'll continue to write about this in a current Agencik AI framework report that should be out by the end of April. So be be on the watch for that. So one of the the key research findings that I have with this Agentic AI discovery research is that when Agentic AI fails, it delivers it fails to deliver that accurate, contextual answer in production, your your trust will erode instantly with your customers as well as with your internal support agents. My research shows that when customer trust with a is with AI is fragile, you'll you'll start to look for that, you know, off ramp really quickly. And so you have to give your customers that ripcord approach to reach the human agents whenever they they start to feel uneasy or start to to need that security that the human agent will always continue to provide. If you don't provide that record and that off ramp in a logical manner, and if you give it to your customers too prematurely, the customers will pull on it for sure, and your Agentic autonomous resolution rates will plummet down to single digits. So bridging the the trust gap is essential to maintain and preserve your ROI for your Agentic AI investment. And the fix then is to deploy that gated ripcord policy that requires a certain amount of AI interaction and attempts to resolution before the human access is granted. We also know that there is a fix to the trust gap. It comes down to what I call the integration tax, and paying that integration tax is critical. You cannot plug Agentic AI into a mess of legacy data that is unstructured in a broken way. It has to be trustworthy data. It has to sit on top of verified and improved efficient business processes. And if you don't do that, then the AI isn't grounded in that clean enterprise knowledge and it's going to hallucinate. And that hallucination will cause customers to lose trust in your AI capabilities immediately. So we know that Coveo is truly an expert in this space, and Danny's gonna walk us through that exactly, you know, why why robust search and retrieval is an absolute backbone of building your AI experience. And and that will ensure ongoing scalability and will continue to earn long lasting value for your customers. So, Danny, over to you. Thank you so much, Dave. Hey, everybody. This is I'm so it's a pleasure to be here. So I'm I'm the lead product marketing manager for Coveo for our service sign up business. And a little bit of Coveo for those who are unfamiliar, Coveo is an AI relevance company. What I mean by that is Coveo makes every digital touchpoint in the CX journey more relevant. That could be from commerce all the way to post sales support customer service. And we make every CX journey relevant using AI search, generative answering. And now as we move into the conversational and agentic era, our heritage and core differentiator in the market is always to help users find the most accurate and relevant content and products in the most easiest way possible, and by doing that, helping achieve business outcomes. And talking about business outcomes, especially through the lens of customer service use case, Coveo has done this for the world's leading brands and improving self self-service case deflection and even reducing their cost to serve. And what you're seeing on the screen is just a snapshot of some of the stories of how Coveo has helped these customers in the real world. And if you would love to learn more about how we went about partnering with these brands, please reach out, and we'd love to talk about it. But what you're seeing over here is a snapshot of customers that we worked with and the journey that they've been part of. They started with AI search, and then they have moved into generative answering. And now they are at a stage where they wanna keep up with AI advancements, particularly in the paradigm of conversational agentic. And what we're seeing right now is when it comes to conversational agentic, consumer AI is now the new standard largely because we have been using it in our day to day personal lives, and enterprise AI is now doing its best to catch up. But we're we're still seeing contrasting experiences. And this slide, the visual that you're seeing over here illustrates that contrast between consumer AI and enterprise AI. And research is saying that by twenty twenty eight, enterprise AI will use conversational and agentic to to become the intelligent front door to self-service outcomes. And self-service is often a prime use case when it comes to deploying conversational agentic. But the way to deliver this AI wow experience in enterprise is not easy because consumer AI is not the same as enterprise AI, and this is where the complexity begins. And some of the the search queries that you see on on the slide over here kind of is a mere sample set of of the complexity of queries that come in, and these are real queries that we work with customers in delivering that conversational agentic experiences. So questions are really long tail and complex. I think it's a good opportunity for you to think about the type of customer queries that come through your self-service channels or case submissions. Users describe often describe partial symptoms or multistep problems. They expect that same behavior from the consumer AI side of the world to the enterprise world, and then they now expect these systems to keep up. But what compounds the challenges that the knowledge required to answer these very complex questions is often buried and scattered across the enterprise. It's locked behind permissions as they should, and the content, it keeps changing. It needs to be updated. So at the same time, enterprise AI is expected to resolve all of this and keep up in real time at scale. And when enterprises start to ignore some of these nuances and challenges and deploy AI, we start to hear some very tragic stories in terms of breaking customer trust. We are seeing that AI ambition today is often outpacing operational readiness for a couple of reasons. Like, you might relate to this. In Agentic AI is now a mandate at the boardroom level, and this is what customers are telling telling me. It often just falls down at their desk, and now they have to figure out, okay. I need to figure out a use case, but there's also an urgency to prove value. So sometimes the nuances and complexities are put aside, and they straight away skip that step, some of the fundamental step to making AI successful. And they start to see that, a, answers and experiences, they start to break under enterprise complexity. And this creates this domino effect. Right? You have more tickets and more escalations, and what's worse is it breaks trust. So what was meant to be an advantage and a competitive differentiator now suddenly has become a liability when it's when you start to break customer trust. And this is a recurring theme and a narrative that I keep hearing from customers and in the industry, which really has led me to define this observation as the AI workforce gap. It's this gap between what AI promises and what delivers in production. And this gap keeps widening, especially when enterprises roll out AI in the spirit of urgency and disregarding enterprises the nuances in the enterprise world. And wow the way you would define wow is what AI is capable and and in in polished demos and controlling environments. We've all heard AI, Agentic AI, working really well, and there are isolated successes for sure. I I'm not gonna take away from that. But in production, we're also seeing that trust is often damaged, and it's it's it's it's a it's a real world problem that enterprises have. And the only way, especially in service, you would would agree with me that reliability is actually more important than being impressive. And so the only way to bridge this widening wild crust gap is to ground AI in your enterprise truth and invest in reliable and accurate retrieval systems that can deliver this knowledge. So at this point, I would love to throw it to you. I wanna hear if this story is resonating, this narrative is resonating, this water trust gap, and ask you what's the biggest gap between your AI expectations and reality in your service organization today? Dave, are are you hearing this from some of the conversations with the PSI members? Yeah. Absolutely. And it'll be interesting to see what the audience selects as far as the biggest gap. And I don't wanna skew the the results of the audience, but Yeah. You know, I I think that based on my opening salvo of overview for what we're seeing within the the industry for as it relates to AI challenges, you know, certainly data and knowledge not being ready and adequately prepared is truly a foundational impairment that so many organizations are fighting to get through. And, you know, your comment as far as, you know, the the number of knowledge repositories that are out there in our former prior knowledge management maturity model survey, we found that the average support organization has to rely on seventeen different individual knowledge repositories. So without having a a really robust unified search capability, which forms the basis for, you know, retrieval augmented generation, you are gonna be really, you know, world of hurt playing in the world of AI. And then, you know, Agentic AI just ramps it up to even the the next higher level of complexity. So Yeah. Yeah. I'd love to see the results. Let's see. AI doesn't deliver a meaningful deflection in our automation. Yeah. I I get that. I share that sentiment. We can't clearly measure or improve impact. That's also one of the side effects of, you know, all the buzz with Agentic AI. Customer service leaders are really looking to measure impact at this point. So we're gonna touch on some of these some of these aspects, but what's interesting to note is, you know, the some of of the responses that you see over here in the options in the poll aren't really related to AI models themselves. Because when things break, the one of the first things that I hear customers do is to, hey. Let's change the LLM model, or let's prompt engineer our way into getting a better answer. But the reality is AI doesn't have a model problem. It has a knowledge and retrieval problem. So they invest in prompt engineering just because you know, and change the LLMs because LLM LLMs these days have been increasingly commoditized, but that's not the root cause. What I mean by that is the issue isn't really the response that's generated by AI. It's often linked to the retrieval part. So I wanna take a second to explain what I mean by that. Prompts only when you control when you try to when you when you try to play with the prompts and experiment with it, it only controls how a model responds. They don't control what the model actually knows. It's retrieval that determines accuracy. LLMs, when you get an LLM, they don't know your enterprise by default. They only know what you retrieve and put in front of them at runtime. And the and the enterprise knowledge, that debt which even Dale spoke about is is messy by nature. It's all over. So you have to make sure that you unify it and integrate it. Right now, it's distributed across systems, and it's permissioned, and it's all constantly changing. So if the right information is in retreat or if it's incomplete or outdated, the model has really nothing reliable to reason over. So this is what I've been telling our customers. No amount of prompting fixes that. You can't prompt your way into accuracy, or you can't prompt your way into relevant answers, and you definitely can't prompt your way around permissions. You you need a system of search and retrieval that's grounded in your trusted enterprise knowledge. And this retrieval step really doesn't happen on its own. It's really linked very, very closely to search. And what I mean by that is once you start realizing that retrieval is the gap and that needs to be fixed, you need to bring search into the picture because search is what makes retrieval possible. Together, that's what forms the backbone of Agentic AI. So, historically, when when we talk about search, we often think about UI. Right? At the front end, there's a search box, there's a list of results. But the role of search has evolved into an intelligence layer. It's it's a system that connects structures and prepares your enterprise knowledge so that AI models can actually use an over it. So thinking of this analogy, when, like, an employee joins your organization during onboarding, instead of giving all the documents in your company for that for that employee to learn from, What if you were to give that employee an organized binder? And that's the role of search when it comes to retrieval. So a strong, solid search and retrieval system will connect and index all of your enterprise data. And then not just that, it'll transform it into something that your AI agents will understand, whether that's knowledge graph or structured format like markdowns. And then on top of it, it will apply relevant signals like behavior and how users are engaging with content and then boost certain con piece of content and boost certain results. And you can apply business rules and ranking as well. So the role of search really goes far beyond the UI. It it's really what makes sure the right context is available at the right time. And when you combine it with the table, that's what gives AI agents the context that they need to actually deliver in real world situations. So now that we understand, you know, the fundamentals of what search and retrieval are and how essential that's that that it is that is, what does this actually look like in practice? So it it's really yeah. I would love for you to watch this this this loop over here. So search and retrieval often is a part of an orchestrator. When a search query comes in, you need an orchestrator or a brain that will understand reason and retrieve the most relevant information from your enterprise knowledge and then evaluate. Hey. Is this answer or this piece of information relevant? If not, then reevaluate. And then finally, if the answer is good enough, generate that answer for the end user. And then also has to retain memory for retain memory across the interaction. And so if you're able to see my screen right now with terms of if you're able to see my screen, conversational agentic AI really requires a new foundation. So search and retrieval, as I've said, is is a part of an an agentic orchestration. You need an orchestrator that can understand complex and multistep intent. It, like, it maintains context across interactions and then decides when to retrieve, clarify, or act. And then because of the retrieval system, it needs to reason over fragmented and complex knowledge, then finally deliver grounded permission aware answers. So search and retrieval is really not just a UI upgrade. It really sets you up to deliver reliable self-service and case deflection. What you saw with that that agentic loop is something that we even recently introduced to our customers called the search agent. And we're excited about what we can do in in the for our customers in production. You you might be telling me, hey. Okay. This all sounds good, but, you know, what does it mean for me? Because the reality is every organization is at a different stage in their AI maturity journey. And some of you, you know, just starting it out with AI powered search, and others are experimenting with generative AI. And some of you have graduated onto the right hand side of the spectrum in terms of AI maturity with conversational agentic. But regardless of where you are in the the journey, I think everybody is trying to solve the same problem. And wow is often subjective. Right? Like but the question remains that how do I move from that initial wow to something I can actually trust and scale at my organization? And so the the answer is the only way to bridge that gap is by grounding every experience in a shared foundation, which will result in consistent consistent experiences and scalable self-service. When you invest in a search and retrieval system that serves as your foundation, you are not rebuilding for every new AI capability when you are ready to take that step. So you are building once when you have that foundation set up, and then you are set up to scale from AI search all the way through through Agentic. So we're not encouraging our customers to jump the hoop and and skip to the agentic AI part overnight. But we are strongly recommending that they put the right foundation in place so you can evolve step by step with confidence. Finally, I want to leave you with some guiding principles and how you can think about it to deploy trusted AI in enterprise service. It comes down to all comes down to starting with knowledge and not models. There's a lot of focus right now, as I mentioned, in picking the right model, but I would encourage you that models don't create trust. It's your knowledge. Because if your content and knowledge is incomplete or unstructured, AI will only amplify those problems, as Dave mentioned. So so the real investment really starts with your content. You wanna make sure it's quality, structured, and and taking care of permissions. And secondly, I would say making retrieval a first class system. They are super critical infrastructure, and this is what determines AI in terms of makes it accurate and relevant and prevents hallucinations. The last two points is closely tied together. It's it's how you build that trust and credibility. You want to ensure that every response that's generated is supported by citations. So users who interact with with those answers have are are can you you're able to build trust with those users. And in the back end, you need the knobs and control to often audit the responses. When things start to break, you wanna make sure you have the observability tools that that will allow you to monitor, measure, and refine your AI performance with with real usage signals. If you put these principles in place, and this is what often our customers do, they are now set up to to to gain success out of these conversational and agentic AI systems. So you are really building a system that you can trust and scale. So with that, I would love to take some questions or get thoughts from Dave. Yeah. So, Danny, relative to the the guiding principles, I think these are all excellent guiding principles, and we've seen these put in action for sure with GSIA members. There's one additional guiding principle that I would also suggest, and it's the audience that you first deploy your AI for. And it really gets back to trust. Right? Because whenever you deploy any AI solution, really any support solution, really any technology solution, that's gonna be customer facing. You wanna make sure that you've gone through the the proper internal due diligence. Right? So with so much of what is available from an AI solutioning perspective, so much of it truly is applicable and pertinent to both internal support organizations as well as to your customers. So our recommendation from a deployment perspective, a deployment strategy, is to really first figure out what you can deploy internally, Build the trust. Make sure that the use cases are complex enough that are gonna provide first internal value to your support agents, to your support organization, to your company. And you can try out and ensure that the underlying data structure knowledge is trustworthy. And if it's not, then you have the opportunity to improve it internally before you start to deploy anything externally. Because one thing's for sure, and I know we've all seen this, there's the three strike rule. Right? If you deploy something from a forward from a customer facing vantage point, customers are gonna give you some grace. They're gonna give you three strikes worth worth of grace. I'm gonna try it the first time. Hopefully, it's gonna meet my and exceed my expectations. Okay. It didn't quite. I'll come back to it a second time. And then possibly, if they're really in a good mood, they'll give you a third time at bat. After that, they will completely pull that ripcord and go directly to your support agents from a live human perspective. So I think that's a really critical additional guiding principle for all of our customers to really consider before they deploy any of these AI technologies. Yeah. That that's that's a really good point. And we often also see our customers do that as well, roll it out in phases, start with internal use cases, And then when they're comfortable enough, roll it out externally. I would also say I would I would kind of extend the liability aspect. Sometimes users are not ready to talk to an agent because the experience is so bad. They might even leave disengaged from your brand completely. So that takes liability to a whole new level. That's something that you don't wanna do because they're gonna move to your competitor. So you wanna make sure your fundamentals are in place before you roll it out in production. Yeah. Absolutely. And and that gets to the you know, every one of us is a consumer. Right? We're either consuming B2B technology or we're consuming, you know, in our everyday lives, B2C technology. And so, you know, when you're a consumer of multiple B2B technologies, you you see how other organizations are are building the mousetrap to provide a similar solution. And so we're constantly being compared and differentiated, and Thesaurus is no different. Right? We we have our own AI solutions that our members are able to utilize. And, you know, are the preeminent research firm globally, but we're not the only one. Right? So we know that our members also consume other research firms, just like your customers. So that's where, you know, ensuring that you're providing them with the absolute best digital end to end customer experience. You're measuring it with the right KPIs so that you can make the adjustments, have that closed loop feedback and and process to drive product development and enhancements in as real time and agile manner as possible. You know, that truly is table stakes. So, you know, great great great point, and I'm glad we had the opportunity to to have these additional discussions. Yeah. And one other thing that I would add is you wanna make sure you invest in a system that identifies content gap for you. When you roll out AI and you often start to see your answer rates aren't where it's supposed to be, you need the the knobs and the control in the back end for you to understand, okay. This is a knowledge gap opportunity for us to invest in this particular topic. You just wanna start by identifying, you know, what those topics are and start to fill those knowledge gap and then ground AI to act on it. Yeah. Absolutely. And fortunately, organizations in some recent survey data that I saw, support organizations interact with customers five to fifteen times more than any other organization within a technology company. So your support organization is is getting around. They hear all that your customers are experiencing. And so, you know, you need to deploy that intelligence and the analytics that they're getting just from the simple case submission process so they can understand, hey. We've got a first contact response rate of, you know, thirty percent. Okay. And that's great. Right? But that means that thirty percent of the time your customers didn't go to your self-service portal or consume your Agentic AI solution to get that answer. Right? So you have to focus where the the data tells you. And so having a really good stream of intelligence around how your support cases are coming in from a channel perspective will be the first good signal that you can utilize to, you know, ensure those knowledge gaps that Danny just just mentioned, it quickly solve and close. Absolutely. Vanessa, I would love to take any questions if we have. Absolutely. Yeah. So thanks so much, Danny and Dave, for the presentation. I wanna encourage our audience, if you do have a question for our speakers, if you could put it in the upper left hand corner of your screen. We do have, of course, a couple questions already in queue. It's a hot topic. So I'll jump right in. Our first one comes from Corey, and they say, where should we actually start if our knowledge isn't in great shape? That's a great question. We kind of touched on it in our discussion. You wanna really start by identifying those hot topics. Right? Analyze the queries that come in through your self-service channels and your case submission channels. What is what are those what are the queries with which are really costly to solve, high volume intent queries, which often don't have a self-service answer for it and or is currently failing. So I would prioritize improving knowledge there first in terms of structure and quality. And and like I said, you need to invest in a platform that can that can surface these content gap insights to help you identify this. At the same time, it goes without saying, our presentation has entirely based on investing in search and retrieval, that search and retrieval intelligence layer. So because even when you have, let's say, imperfect content, even that can perform significantly better if it's properly indexed and structured and covered. So I would start with, you know, looking at doing an audit of your the type of queries and improving knowledge there first. K. We have Emily, and they said, you talked about trusting grounding, but how should we think about measuring success or proving ROI to the business? Again, a super hot topic. It's all about measuring trust and proving value. So I I will actually defer to DSIA because they have a great framework for measuring outcomes and related to self-service success and case deflection. I think that's a great framework to start, and most of our customers follow that framework as well. The other piece in terms of making sure measuring successes is observability, being able to see what content is being retrieved, where which step is AI actually failing, where are the gaps that exist in your knowledge. So when you connect these insights to these business metrics, that's when you can clearly start to demonstrate impact and continually improve. Yeah. And and and, Danny, I would I would add to that. So thanks for for highlighting the the t five framework for self-service ecosystem of KPIs that in include, you know, measuring self-service success, implicit case deflection, explicit case deflection, and customer satisfaction for the end to end digital experience. But as I mentioned, the KPIs that we are seeing from a research perspective, working with our large enterprise members primarily as it relates to deploying Agentic AI, these KPIs are changing. Things like measuring autonomous resolution rate, separating out the difference between measuring human based transactional customer satisfaction with autonomous customer satisfaction and including customer effort level for autonomous as well as human interactions to separate and understand and be clear around the performance of autonomous robot as well as the human. And then, you know, we talk about conversational AI and Agentic AI. And, you know, there are a a number of really complex, very mature conversational AI capabilities and technologies that many of us are delighted with utilizing day in and day out. But in order to improve and ensure that you understand exactly where that conversational AI starts to get a degree of of error, because a degree in error early in that conversation will result in a fifty degree difference of where your customer wanted you to go to to solve the problem. So being able to monitor the accuracy of each individual return and having an automated way to do that is vitally important. So just is these are just a handful of the Agentic AI capabilities and KPIs that I've, you know, bringing in beginning to have conscious streams of understanding in working with our our members. So, again, the Agentic AI framework is going to identify a lot of these. And, you know, this clearly is a a two point o. You know, if if I refer to one point o versus two point o, the one point o model for support services maturity, you've got to perform at least at the fiftieth percentile from an explicit case deflection perspective. You're not doing that, then if you expect to then implement Agentic AI on top of that poor performance of explicit case deflection, your return on investment is not gonna be demonstratable. Right? So you gotta really shore up your one point o KPIs, your performance, which includes, you know, everything involving, you know, regenerative augmented generation, generative AI, co use of Copilots before you even start thinking about getting into the Agentic AI realm. And that's gonna be the two point o support service maturity. And I'm I'm working on, you know, introducing that research in the second half of of this year as well. So I think it it it works very tightly and dovetails with the the information that you're sharing today, Danny. So really vital for for our customers and members to understand all of this. Thanks, Dave. Yeah. I'm looking forward to that that report when it comes out. Okay. Let's go ahead and squeeze in one last question here. This is from Beatrice. And they say a lot of people already have search a knowledge base, and probably some form of rag implementations. Do we need to replace everything to move forward with this model? It's a great question. I will say it depends. Right? Because there's always a cost to ripping and replacing software. You need to ask yourself, is the cost of maintaining the current experience higher than the cost of actually evolving changing software? What you would need is at the foundation I spoke about this. Right? Like a broken record. It's unified search and retrieval is the foundation. The AI agents right now, companies are actually agent hopping. This is not working. Okay. I'm gonna switch to another agent. These AI agents and frameworks can actually be connected to this shared data foundation that we spoke about through MCPs and through APIs. So you it's not always that you need to rip and replace the whole thing. You need to ask yourself, is your existing experience that bad with with results in inaccurate answers and escalations and increasing your cost to serve? If the cost to actually maintain the current experience is higher than the cost of building on that foundation, I will say yes. It might be actually worth to set yourself up, and then you can scale from there. Thanks so much to Dave and Danny for delivering an outstanding session, and thank you to everyone for taking the time out of your busy schedules to join us for bridging the AI Wow trust gap in enterprise service Brought to you by TSIA and sponsored by Coveo. We look forward to seeing you at our next TSIA event. Take care, everyone.