Good morning. Good afternoon. Thank you for joining us wherever you are. My name is Juanita Oguin. I lead product marketing for our platform solutions at Coveo. And today, I am excited to be joined by Gera Raff, who is the senior manager for revenue and automation at Sysco. Welcome, Kira. Thank you. I'm so excited to be here. Yes. I'm excited to have you here. I'm excited for our conversation. And for those of you just joining us, we'll be talking about Cisco sales connect to application and how it improves seller productivity with AI search. Before I jump into the presentation, I did want to let you know we'll be recording today's session, and you'll get a copy of this within the next twenty four to forty eight hours. We do wanna hear from you throughout the presentation, so please feel free to ask us questions in the chat or the q and a. And we will have a section at the end of today's presentation. We will cover your questions. With that, let's jump right in. So today's agenda, we will be talking about the driving force for change and why does the school sales connect team decided to, consider AI search as part of their solution We'll talk about the importance of user research and why it's essential for adoption and personalization, and then we'll talk about some of the key capabilities implemented by Kira and her team as well as the impact on their business to date. But before we jump in, I do wanna say a massive congratulations to you and the team, Gara. You are all awarded the innovators award, which is an annual award Coveo gives out to some of our most progressive customers. In particular, it's customers that are using AI every day to innovate for their customers, partners, and employees, and you are a mom great company, Garrah. So congratulations to you and the team. Thank you so much. We are so excited about this honor. I mean, we just implemented I'd say less than a year ago, and it was it was a long journey, but it was a good one, in terms of what we learned and how we implemented the product and we are so excited to see the results. And, actually, we're really excited to see that your company recognizes that. Absolutely. And that's why we're excited to have you here today too to talk a little bit more about that. So congratulations again. And, To kick us off, would love to just have the audience get to know you a little bit more? You have such a rich history in your career? Do you mind sharing a little bit more about your background? Yeah. Absolutely. So I've been at Cisco for almost, I think, one month shy of twenty four years. So it's been a while. Normally a person like with me would be tired of staying that long in one place. But the beauty of Cisco is you can move multiple departments over your career and gain a lot of experience, which is what I did. I started out as a program manager in IT moved over to the service sales operation side, then ultimately, product and sales operations, I've owned a change management team, and then it really leads me to my career today where I'm at Sales Connect, which is under the product and, product sales operations area. And what we do is we support a lot of our internal sales people as well as our partners and distributors. That's amazing. And when you and I were prepping for this, I asked you what's your philosophy or your favorite saying, and you said it's to leave your ego at the door. So can you say more about this? Yes. I think that when you're thinking about user insight, user experience, etcetera, or even in your day to day work. It's really important to set aside your biases or your possessiveness or defensiveness around what you own and really lean in to be able to understand what are what other people experience around you because you never know the golden nuggets of insight that you would get and how it can take you to actually elevate your experience further. And I can can imagine this is also important in dealing with sellers directly as well. Absolutely. Our salespeople give us a lot of great feedback. Very constructive, and we we need to have that in order to build something that they will use. Amazing. Well, you have a great background and experience, and I'm sure that's really set you up to be successful. Wanted to just change years here and talk a little bit about sales connect. What is this? Can you give us an overview of, what this application and its objective are? Absolutely. So Sales Connect as an application, we have around three hundred fifty thousand Cisco internal, as well as partner distributors who support our channel business. They come into our, application every year. We have seventy plus unique distinct groups that we create based on those three hundred and fifty thousand to be able to target, capabilities, features as well as information to those user groups. We people go to sales or actually users go to sales connect for three things. One is to find content. Two is to take learning. And three is to be able to share that information with their customer or partner. The ultimate goal for end use on Sales Connect is to be able to successfully position the extensive Cisco product and and services portfolio and be able to land and deal as a result of that. Yeah. Kind of important, especially because I can imagine Cisco has a diverse set of complex complex complex or complicated products and your your sellers might need to have the latest information right at their fingertips? Absolutely. They have to be well prepared in customer conversations. So they need to be able to have this information at their fingertips. Absolutely. We also talked about the importance of business and IT work together. So so I can see here that you have what I would say is a relatively small team supporting a large user group. Are how are you and your team able to do that successfully? Yeah. As you've indicated, we're we're on the business side. We're a group of thirteen. So we're a small, but mighty team. And what we do is we encompass multiple areas of the application, which is around the development of the or ideation of features and the capabilities, the execution in terms of development and the testing, and then the ongoing support of our user base. This not only includes our end users, but it also includes our content cure curators who actually bring information onto the platform. Then in turn, what we do is we work with a wider team and of our IT team who we actually have them either do a little bit more configuration or as well as development on the platform. That's great. It sounds like you have a really great diverse set of team members that are each bringing their own, areas of expertise to support here. So I wanted to get into, you know, we were talking about sales, connect a little bit more, but what was the impetus to change? What was really the driving force to make you do something differently? Well, we wanted to leverage the vendor's deep industry knowledge and insight They're the ones that actually go to the customers, multiple customers. They're the ones that see all the use cases that are before them, and they're the ones that help us understand what are the best practices and potentially industry benchmarks that are part of an application or an experience. So we didn't feel at the time that we had that type of research, and we were constantly having to be creating new requirements and features, but we needed we felt like we needed that insight first. The other area is ultimately a couple of years ago as we started into this journey. The search experience wasn't ideal. Our users had I would say on the platform, it was one of the top issues that we had. So users were complaining about the actual UI, the look the field. They were not getting relevant results. And they wanted content recommendations. So they wanted more information. So we knew that we needed to do something to elevate that experience dramatically. And ultimately, one of the another big factor is the fact of cost. Previously to what we have now, we were a very custom application. Meaning, there were we didn't really work with one main vendor. To support a lot of our innovation. The great news about that is we could continually build, but it was very costly to support. And we couldn't scale with that as we were looking at future, future releases, future budgetary constraints. We were looking at all of that in context. So we wanted to ensure that whatever we we did was we we we took advantage of our vendors and what they had built and what they had spent a lot of money and time researching. Terms of promoting a user experience. And then finally, I think one of the big things just from our internal business team, we need you to be more in control. Of the user experience and our ability to impact it through activities such as performance tuning, which we couldn't do before. That's great. A lot of depth there and a lot of reasons to try and, incorporate a different platform. One thing I wanted to just get your thoughts on is, you know, obviously, we're here with Coveo. We're we're hosting this webinar with you. But one thing we we often hear from people is that search is not always prioritized at, let's say, the management level as it is under stood and valuable, at more of a practitioner level. So it's kind of at the core of what you're describing here in terms of somebody improvements you made on sales connect. Could you say more like why search, why is it important or valuable to even consider? I think if you really look at search, It's the calling card to an experience or to an application. That's one of the first things a user does. The other thing with search is search is universal. You see it everywhere, whether you're grocery shopping, whether you're at work, and you're in multiple applications, there is a lot of experiences that's free being brought to an end user that relates to search. So as a result, Our users are much more proficient and they're much more demanding of what they expect out of a search experience because they have many areas to compare from. Which is different from other aspects of an application where there where it has core rich information, but it's for particular use cases. Search is universal. Really great point. So I wanted to, change gears and talk a little bit more about your process and how you wanna making a change. And so, user research was one thing that you really wanted to emphasize. Could you tell us more about this and your process? Absolutely. So before we started this, re platforming, Sales Connect, we wanted to take a step back and really get to understand what the issues were around what around our user base. So we can make sure that we do address that over time as we work on the application. So I would suggest that when you're doing user research first, understand your audience. And when I mean understand your audience, understand their experience on the application. What are their rules? What are they doing on the application? And also more importantly, What do they do with the information that they have that they get from the application? I think that is an important perspective. If you do that, you can you can gain so much insight. You can do this by simply doing surveys, or you can actually have one on one interviews or team or group interviews in that process. The second thing that I would highly suggest is get examples of what's best in class. You you personally go out and you search on different, engines all the time. So you see what's out there. You see the features and capabilities that are being introduced. Try those in terms of looking at them and and going online to do that. But in addition to that, talk to your Coveo success team. Get examples of what Coveo has implemented at other companies. And if these are public sites, go in them and they'll look and see. So then you can understand what's feasible. And it's something you can take back. And then third, you should really think about as you look at your insights and as you look at best in class examples, How do you marry them into an ideal solution? So what we did is we used actually mock ups to be able to, show that, you know, a picture's worth a thousand words. So you wanna try to figure out what's a way that's meaningful to get that ideal experience down on paper. Love it. I love the steps and the recommendation, that you're suggesting care. Love to know because mock up is interesting idea. Could you share a little bit more about how that impacted the success of your program or your rollout? Sure. So what was really important about mock ups, number one, it allowed us to take something back physical back to the end users that we we spoke with initially and and say, hey, is this correct? Are we directionally correct and really validate your design? But it also allowed them to feel that they were heard. And that's a very powerful thing to do because once they you understand their pain points and they know that you understand their pain points, it it allows for future collaboration as well and also a better experience. The other thing is mock ups are also important for when you're meeting with IT or your professional services team at Coveo because it allows them to quickly grasp what you're designing or what you're ideating about in terms of of a future state experience. So not only are they able to look at it, but able to give you more ideas on what we could do to better implement or develop the in terms of the feature set. Awesome. Great ideas, great suggestions, and I'm gonna need to use some mock ups in my daily life now. I wanted to get into a little bit more of the capabilities and, the the things you actually were able to implement. Could you share more here? Sure. In terms of the top needs based on our user research, our users want a simplicity. Don't give me multiple tabs. Don't give me continual scrolling, make it very easy and make it very intuitive in terms of the experience. In terms of recommendations, These users really understand the future state. They wanna be told to certain extent what's out there. They wanna be able to be given recommendations on how they're searching. They even wanna be able to get better at searching themselves. So they wanna be given ideas on how to better search. And ultimately, they want better relevance. They wanna make sure that the search results are meaningful to them. And just not being brought forward without any context to what they do. Love this. I also love how it's very precise and not to use the same word on the screen, but simplify so that you have focused impact here. Absolutely. And I think the other thing is it allows them to it allows us to really realize especially with the feedback that we we got is we really needed to do something dramatic to elevate their experience, and that's why we we thought AI is what we need to do. Absolutely. So why don't we jump right in and speak to what you implemented CAI in action? And I'll follow your lead here, Kira. Sure. The we actually looked at a couple different AI models when we spoke with the Coveo team. The first model is the automated relevance tuning model, which really drives the search results experience that you see in front of you. And it really drives bringing, particular items up higher into this, results view. So it's easier for the, sir, searchers or in this case, our end users to find. So we utilize that first. Then what we did was we looked at content recommendations. Our sellers based on our discussions wanted the ability to be able to say, hey, when I do a search, I wanna be able to, be given content, but not just any content. What are other users using that are finding that they see that they think are successful? So it was really important for us to be able prominently display a set of content recommendations as they search. And then the third area, which was really the surprising, insight that we got was they wanted to be able to better search. So not just bringing items and recommendations, but How do I improve my search my search experience by the way I search and the terms that I use? So as thought about that. We looked at positioning the, query suggestion model, the same model in two different places on the screen. So you'll see one up top in the search, but you'll also see one on the right above the scroll lines so that we have it there so that if they don't think about it or see it in at the top, they're also being able to view it on the right hand side. This is great. I know we'll get into Metrc soon, but did you get great feedback from these improvements? Absolutely. We we got a lot of feedback in terms of wow. This experience is giving us a lot of the information that we didn't realize we needed. But more importantly, we were getting it through the metrics. We saw we saw users clicking on it. It's one thing to get feedback. This I mean, I'm saying feedback is very, very important, but you also have to marry it with the actual metrics to to make sure that you can get an objective assessment out of it. Absolutely. So why don't we go into that next before we show the numbers and the actual numerical impact? You know, we talked about the importance of why Metrics matter. And, you know, we have tons of customers who are also struggle of being able to, you know, show baseline or or even just track the success of their program. So we'd love for you to share, you know, your view of metrics and why they're important. Sure. So prior to the implementation of Coveo, we really struggled with metrics. And when I mean really struggled struggled with metrics. We didn't have a dashboard. We didn't have a concept or understanding of what a successful search experience looked like in terms of industry benchmarks. And we had no as I indicated before, we had really no control to be able to, performance tuning to help drive an increase in metrics. So we've really struggled with those three things. So when we implemented Coveo, I was really, really excited to see a dashboard with all these fabulous metrics, but I still struggle because even though I'm leading search, I also lead other areas of the application. So I'm not a deep expert in search. I don't have the deep ex experience around data modeling, search and relevancy, etcetera. So that was a that in itself was a struggle, but I was really lucky because through the Coveo success, team, they came in immediately, and Sabrina McCullough from the Coveo of Success team became my secret weapon with all things Coveo. She fundamentally helped me understand. It's not about the number of users or the number of searches that a user is doing to constitute what is a successful search experience? There are other industry benchmarks that can help you do that. So what she did was she introduced me into two metrics such as click rank, content gap, visit click through, query click through. Those four main metrics are the ones that we use to really drive an evaluate the performance of our experience. The other thing is she's also making me aware of the different types of metrics that evaluate the features specifically around AI models that we use and to see, are they working well? And not not just that they're working well or meeting the industry benchmarks, but are they resulting in better click rates, which ultimately is the most important goal? Absolutely. Thank you for sharing that. So why don't we look at the impact of AI? So you mentioned the three models that you are using today and you shared those on a prior slide. Can you talk through, each one of these and the impact you're seeing get bring? Sure. So the first one we talked about was the art model. So think of the art model as the results that come into the search result area. What we're seeing is that fifty five percent of all the clicks that are being done in that area are being done on on items that were actually boosted through the art. So what we know is that the best result are coming to the top and the users are able to quickly find them and click on them, which to us constitutes success. And it's also the industry benchmark is when you when a user clicks, that it constitutes success on the experience. The other area is the query suggestion. So I had put that in we had put that in two places What we've seen is when a user actually goes in and clicks on a search suggestion and it populates a new set of results, there's sixteen percent sixteen percent more likely to click on an actual result from that search or that query suggestion. And third, content recommendations, which we we had at the top right. So right now on the platform eight or over eight percent of our click that happen on content are happening through that personalization. So that's that people like you experience. So we know that it's resonating with our users and they're actually clicking on it. We would ultimately like to see these numbers continually go up over time, and that's what we do through performance tuning and other reviews, especially with your Coveo success team. Awesome. Yeah. Thank you for covering those. And a shout out to Sabrina McCaulo for helping make your team's successful as well. Just before we move on, I did wanna just give a little, context for those joining us today that might not be as familiar with these different AI models. But I feel like it's important just to showcase these are three of eleven different AI models we have today and the first two or what I like to think of as a search base. So it's, you know, they work, and help guide users as they are making a search Right? So a little dependency on that user searching. But the third one here, content recommendations does not require a user to search. It's that proactive recommendation. So it's a nice blend of different models, again, all in the aim of helping with discovery and findability. So Gera, you shared these three wonderful AI models you guys are doing very well at implementing and supporting. What do you see next with AI? We're currently underway in exploring some more capabilities, powered by AI to help us address a lot of the pain points that our sellers are and our partners and our distributors experience. Ultimately, what we would like to use is the power of Genayon to be able to generate presentations that are relevant and up to date with the Cisco approved templates with the twenty twenty four on them. We wanna do that because as I just spoke about before, it's not just important of understanding what your cell your users are doing. But what's their end game with the experience? And our users end game with the experiences to be able to have that conversation with the customer and that's through the use of presentations. So what we were trying to do is if we can figure out a way to bring the gen use that GenI to actually generate these presentations. We know that our sellers will ultimately still need to go in and modify it because they have to put their own stamp and personal flare on it. But if we can do as much as possible to omit the cutting and pasting that they have to do, we'd like to try that. The other a thing, our sellers value content freshness. So it's really important as because they're going to the customer they have real time information, but they have up to date information. So what we want to ensure is that we provide very strong visibility on the dates that the content items or the search results, the content items within the search results. So what we wanna do is use AI to be able drive getting the dates not from what the publisher does in terms of publishing the content and the dates that they assign there. But the actual dates on the content, we would like to see that, brought forward and a little bit higher in terms of visibility on the search experience. So that we don't waste sellers time on clicking on things that we know that they have no interest in. Our sellers have told us three to a content for content to be relevant for them. It needs to be three to six months old. Very aggressive, but that's what they want. So we wanna ensure that we provide them that visibility and in turn, we can also let the content creators know so that they can be focused on continually refining and updating their content as well. The third thing that I didn't wanna forget is we're looking at a chatbot. So we would like to drive a chatbot experience and It's one thing to go in and have a user go and search, but it's another thing to have them have a conversation with the application or the experience. And we feel like the chatbot would actually drive that. So what we wanna do is take the in in men's support folio content that we have and be able to have the sellers go in and ask questions and to be able to have very descriptive direct answers that it relates to the question that they have and also be able to attach the source of where that information was found so that again, we can help them get ready and be a very effective for those customer presentations. Sounds like you have some amazing things lined up. And then planning some great work there. We'd love to stay in touch to hear how that goes in the future. Absolutely. Thank you for having me here. Of course, you're getting near to the end here, but be before we go, I did want to share a couple of resources to those of you joining us today. The first is if you're curious about AI, an AI search platform, We have our Coveo relevance three sixty event. On March twenty seventh, it's a virtual event where we'll have some special guests from Forrester. Former open AI employees, and so we'll share our thoughts on innovation and generative answering. And secondly, I mentioned we have actually eleven AI models today including generative answering. And so if you want to just learn a little bit more about what those AI models are or how they work, we will post a link here of our ebook so you can get to know a little bit more there. With that, Gavin, I would like to thank you for time and attention, Garrett, thank you for taking time out to share your best practices with us and sharing just what you've been able to accomplish with the sales connect team. No. Thank you. It's it's a pleasure to be able to talk about a topic that I have so much passion about. I can see it. We can all see it. So thank you. With that, we will open it up for questions. And so, Kira, I will pose the question the first question to you, and it is around your user research topic. The question is, do you need a user research team? You know, having a user research team would be ideal. But in many cases, especially with your budgets, etcetera, you may not have or you may not have access to them. That shouldn't include you from actually doing some research. So even if you're a team of one, like I initially I was a team of one, you still can go out there. You can still have conversations. You still do surveys, go out there and try to find what what the experience is for your sellers. Remember, their perception is reality. So having a conversation and getting to understand what their pain points is and how they use the application is ultimately important and it doesn't need to be done by a user research team. It can be done by you on a smaller scale. I love that. I always say also going direct to source is the best practice. So I love that you really took the initiative to do that. Alright. We have another question here for you. What advice do you have for others looking elevate their sales enablement practice or other departmental needs? First things first. Understand your metrics. Understand your endgame with the metrics and what your outcomes are going to be. Understand the search and relevancy metrics so that you you can get a good idea of what's needed to be done to elevate the experience, know your audience, know their goals, know what they're gonna do with the information that they get from your application or process. Don't be distracted by naysayers. That's an another important one. Lean in. I love it when I get cases where they complain about the experience. Specifically if they complain about search. And I actually go proactively and set up time so I can actually understand what their experience is. Lean in. You're you'll wouldn't be you'll be of the golden nuggets of information that you're gonna find from that. But you take that information and also make sure you marry it with objective data. So then you can actually determine what really needs to be done to improve the experience or if there's training that's necessary. And then the last thing is use mock ups. Mockups are a great way, a great tool as I mentioned earlier. It doesn't have to be perfect, just directionally correct so you can actually have a conversation. Perfect. Thanks for sharing that. We have, one other here around the AI, specifically, did you face any with AI or implementing it? I don't think we face resistance at all. You know, we're in a time where everybody's talking about Everybody wants AI. They don't know exactly how they're gonna use it. But where we were looking at AI, I don't there was a resistance to it, but my suggestion as you're looking at it is make sure your leadership is prioritizing that work. And make sure that, you are following or adhering to your company's policy of responsible AI. I can't highlight that enough because that type of compliance is being done at every company level to ensure that the security privacy and just, you know, general well-being and the health of that experience exists for their for their user population. Great. Thank you. We'll do one more. And then for those of you that have a question that we're not able to get to, we will follow-up with you offline. The last question is you mentioned having a legacy platform in place. Could you share what the new platform is that you have? Sure. We're on Sales Connect is built on Salesforce experience cloud and we use Coveo. Our a Salesforce app exchange provider to be able to provide our search experience. Perfect. Thank you again. That concludes our time for today. Thanks for those of you joining us. And, Gary, another massive thing. Thank you to you for your time. No. Thank you very much. Take care. Bye.