Hi, everyone. We're just going to give a couple of minutes to allow more people to join before we kick off our session today. For those of you just joining, we'll give it one more minute to allow more people to answer the call. Alright. We'll go ahead and kick it off now. Hello, everyone. Thank you so much for joining us today. My name is Bonnie Chase, and I'll be your host for today's roundtable session on intelligence warming. Now before we get started, I have a couple of housekeeping items. First, everyone is currently in listen only mode. However, since this is a roundtable, we do invite you to join in the conversation. So if at any point you want to ask a question or add commentary to what's shared today, go ahead and click on the reactions and raise your hands, and we will go ahead and unmute you so you can join in. Today's roundtable is being recorded, and you'll receive the presentation within twenty four hours of the conclusion of the event. And finally, at the end of today's session, we'll be announcing the winner of the Intelligence Forming Certification Course offered by the Consortium for Service Innovation and sponsored by Coveo. Now to kick it off, I'd like to introduce you to our roundtable speakers today. First, I'd like to introduce Matt Seaman, executive director for the Consortium for Service Innovation. Matt has been an accomplished and passionate customer success and operations leader for over twenty years. He was an active member of the consortium before joining, and now he's focused on the intersection of knowledge sharing, customer experience, and emerging technologies, and how that intersection drives innovation and support services, customer success, and operations. Thanks for joining us, Matt. Thank you for having me. Our next speaker is Patrick Martin. He is our VP of customer support at Coveo, also with over twenty years of experience driving quality improvements, cross functional collaboration, and business process optimization, really to reduce operational costs and improve the overall customer experience. Now our discussion today will be focused on intelligence forming, which is a single tiered support model developed by members of the consortium for service innovation. So I wanted to just give take a moment to give you a brief overview of the consortium since we'll be talking about that a little bit today. Now the consortium is a nonprofit alliance of seventy companies with a focus on innovation and customer engagement, and this is based on the members' experiences testing new ideas and sharing, and ideas mature into concepts and models. These models continue to evolve, becoming more detailed as more people implement. And you may be familiar with KCS or knowledge centered service, which was developed by consortium members, probably what the consortium is most well known for. But they've also developed intelligence swarming, which consortium members have been playing with this concept since two thousand and five. So when we think about intelligence swarming, I'll just give you a quick overview. You know, the these slides are provided by the consortium. It's really three pillars. The first pillar is connect. So really making sure that that work is visible to the best resources you can solve on the first touch. We don't wanna pass our customers around like a hot potato from tier to tier. So it's really about making sure that you're able to solve their issues on that first touch. And to do so, it really requires collaboration, which is a really key concept of intelligence forming. So getting help from anyone with the knowledge, who can really help solve that issue. And then finally, that third piece is recognize. So really being able to recognize those contributions by the individuals and the team that lead to the overall success of the customer. So with that, I wanna go ahead and take a poll and just see from the attendees here, you know, where are you in your intelligence forming journey today? So the options are getting informed, getting started, or practicing. So go ahead and choose your option. Really curious to see how this goes. Any ideas, guesses, Matt and Patrick, on where the majority of people will be on this? I would guess getting informed. Maybe getting started. Yeah. I I I think that is pretty much where things are are gonna land. At least that's what I expect as well. Alright. Well, we will see. We've got eighty two percent participating. I'll give it another thirty seconds here. Anyone else wants to answer the question? Okay. So it looks like you are correct. We have forty eight percent getting informed, thirty three percent getting started, and eighteen percent practicing. So let me go ahead and share those results. So kind of kind of what little more balanced than I would have would have thought. Yeah. Yeah. Well, I'm I'm happy to see the eighteen percent practicing and and thirty two percent getting started. Yeah. Me too. All good. Yeah. Well, let's go ahead and kick off the conversation. So I'm gonna stop sharing now so we can go ahead and and have this roundtable discussion. And, really, my first question is for you, Matt. You know, I'm seeing a lot more about intelligence forming lately. So what do you think you know, why why is this becoming a popular, model for support? And what do you think is the reason behind this shift to the single tier tierless support model? Yeah. We definitely are starting to see more and more people talking about it, more and more people playing with even parts of the, the three practices, you know, connect, collaborate, and recognize. I think a big driver is the continuing maturity of maturity, the continuing complexity that people are seeing in the support and services world. You know, no longer I I have yet to talk to a company where their support team only has to care about what their specific product or technology or service does because everything is so interconnected today. And it's almost impossible for any single resource to know how to troubleshoot, solve this complexity that we're living in. So, you know, the tiered models are really starting to break down when, as you said at the start, Bonnie. Right? Passing work around, we're bouncing customers around. It really causes nothing but frustration for the people that are in our organizations as well as the customers that are trying to interact with us. I also think the technology is starting to really catch up to the concept. So the concepts have been around for a while, but I think the technology is finally starting to catch up, right, with all the different collaboration tools you can use today, all the integrations that are being done with, you know, companies like Caveo and Salesforce and ServiceNow and all the others. It's just making it so the tools are there to allow us to do swarming. And I think that is really starting to be something where people can see how to implement the vision instead of going, yeah, this makes sense. Well, now you can see there's a road map to get there and, you know, people are playing with it. There's more practices. There's more design techniques. So, you know, I think that's helping accelerate it. But the need for getting people up to speed faster and the complexity of our environments are really key drivers, right, for why we see people talking more and more about it. Yeah. And I can imagine, especially as a you know, if you're an a new team member, you know, you're not going to know all of the answers, and it's really important for you to collaborate with your peers and and get that information. It seems like swarming is an easy way to kind of level up your knowledge about the product and tap into some of that shared knowledge that's within the organization. What are what are some of those benefits that you're seeing with those organizations adopting this? Yeah. Definitely intelligence forming actually started the first discussions of intelligence forming started as a training tool and basically saying the best way to get our support agents trained is by having them work with support agents that are, you know, have been doing it for a while and have all that information knowledge. So it actually started as a, a way to try and break down the barriers of training and sharing of information. And then those early adopters started to see all these other things start happening with, you know, faster resolution times, higher customer sat, which we still see today. So the benefits we like to say that intelligence forming is wholly beneficial. You know, we're seeing customers love it because they get a much more consistent experience. They're not passed around to different people where they have to redo a bunch of work. And, you know, the customer satisfaction scores start to go up or the NPS scores or however you're measuring measuring your interactions with customers. The support agents love it. You know, support agents collaborate naturally. I mean, it's just it's part of a service organization. By making it easier and as a part of their role and getting recognized for the collaboration and the information sharing, they feel more in tune with what the company is trying to accomplish. They feel like they have more of a career path. They're getting trained and up to speed faster. We have member companies have reported, getting people from new hire to taking cases fifty percent faster with intelligence warming because of that information share in real time, which is just a great way to learn. So it really benefits the entire organization. And, you know, so we see all of those benefits just kinda start. And they start pretty early in the adoption. You don't have to be adopting for a year or two years before you start to see some of those results come in. Mhmm. Sometimes it's within weeks of some of the, processes being put in place. We start to see those benefits. That's great. And do you have, do you happen to have, like, benchmark metrics available that people can review for that? Yeah. In our website, there are case studies from Akamai, Cisco, Tricentis, Red Hat, and others that kinda talk to those those benefits, right, and the the actual numbers they saw when they started to implement, intelligence warming. Great. Great. So service innovation dot org? Service innovation dot org, intelligence warming, and there's case studies in there and lots of great information. Awesome. Well, I wanna shift to you, Patrick, because you are currently practicing intelligence forming at Coveo. And, you know, I'm I'm really curious. You know, one, what were the reasons that you decided to shift your team to swarming? And I guess the second question, you know, there that's related to that, Brian has asked in the chat, is how do you distinguish between swarming and traditional collaboration when selling the idea of swarming to executives. So you've obviously had to sell this idea to executives at Coveo, and you've implemented it. So talk us through what that was like. Yeah. Definitely. You know, I've been here in Coveo for four years, but this journey around fixing collaboration has gone on for many, many years in my, support career. But one of the things that when I joined Coveo, we were not a tiered model. We were a single level of support from the get go. But, we were relying a lot on, you know, our own tools for knowledge management, and and collaboration was done very disjointed. And some people doing it in generic Slack channels, which was extremely noisy and disturbing for everybody. I had some people who just were just getting up and going to ask people because we were might mainly in the same location. But as we grew and we spread into different locations, you know, growing in different areas, even having people now in Europe working on various shifts before we were kind of a six to six to six type of business. Now we're twenty four five. There's a lot more complexity in in managing such an organization as well as how we restructured the team being more use case focused rather than than technology focused. There was definitely a need for cross functional collaboration within the team. We had used the collaboration forum historically in the last few years, and that had been working well. But it really fell down to the same people always, you know, collaborating because we needed to have a fail safe program to make sure that these questions were answered. So no one was really collaborating except for the same four or five individuals, which were actually our safety nets. So, we decided to jump on the journey and say, hey. We we have the tech stack. We have Salesforce. We have Kaveo. We have Slack. I mean, this all seems to be a good formula to get this going. And we started playing with these concepts, and we were able to build a a swarming, flow that really leverages, you know, the people profiles based on historical case data, bringing experts into dedicated Slack channels, so we're reducing the noise, focusing on the collaboration piece of things, and then tying everything with knowledge management by indexing these channels with our technology, making them available for future use. So that's really the path that we were on. And, you know, out of the get go, we started to see much more synchronous collaboration, more people involved in the, in the swarms. And, as of this morning, when we looked at the data, our time to resolve, if we compare twenty twenty two to twenty twenty one, improved by twenty three percent across all of our cases. Just comparing comparing our previous collaboration method in twenty twenty one with swarming in twenty twenty two, it's a thirty percent improvement, in resolution times, which is quite significant. So when you're going to, you know, your executives, especially in in these times, and you're trying to be more efficient and and do more with less, these numbers pretty much speak for themselves. Yeah. Yeah. It seems like it really helped kind of streamline that collaboration process and make it more effective and efficient. Yeah. And and when we look at at the the collaboration and and who collaborates in the swarms, we we bring five people that is including the case owners. So we basically bring three SMEs and an expert in our swarms, and we have an average of two point eight people collaborating in the swarms all the time versus one person historically in the collaboration forums. Mhmm. So now the collaboration is more, it's more widespread. There's more people being brought into, the collaboration, and people are actually participating, which is great. Yeah. That's great. So so, Matt, you know, what would you say to somebody who is currently in a traditional kind of support model who wants to move to swarming? How would you advise them, you know, talking to executives? I think the kinda keys are attaching the shift that intelligence forming can bring to the organization to kind of the corporate level goals or what are the go overall goals of the support organization and painting the picture how intelligence forming can kinda get you those goals faster. We We like to say that intelligence forming doesn't change the overall business goals of a support organization or a company. It changes how you deliver against those goals. You know, we still want faster time to resolution and higher customer set, lower customer effort, all of those things. The key that executives seem to really start to buy into is the concept that if you can treat your organization as an unbound network of skilled people and getting the work to the right skilled people as fast as possible, and then just making it easy for them to solve your customer's issues when they need help. It's hard for anybody to say that is not more efficient or better than we have, you know, junior resources that takes some stuff, then they pass it over a wall, and then they pass it over a wall. So you talk to all those things and attach it to the top level metrics using a balanced scorecard or, you know, whatever type of strategic framework you have in place at your company. Usually, we don't really see much pushback from the executives on, you know, trying trying to implement and transform, especially when they start understanding the complexity and looking at how fast it takes in a swarming environment to solve a complex issue versus a non swarming environment. Right? Reducing escalations, all those things that, you know, cause lots of pain to a support organization or a company. Oh, that's great. Now let's dig in a little bit into how it works. So, you know, a lot of people are interested. They're kind of exploring this idea of forming. Matt, what would you say are some key things for people to consider when they're to determine, you know, if they're ready for forming? There's a couple things I think. Patrick hit on one of the points really well that intelligence swarming is not collaboration. Intelligence swarming is bigger than collaboration. And a lot of people think because they put a collaboration tool in place, they're kinda doing intelligence forming. Patrick's point of kind of free for all collaboration means you're losing all kinds of content. You're losing all kinds of knowledge. Right? All that stuff is kinda being lost where if you structure the collaboration processes a little bit, you can you can gain information from all that. So one of the keys is that collaboration does not equal intelligence forming. It it's what makes intelligence swarming intelligent is the ability to think about how you're gonna connect people to the right work. How are you going to collaborate, and how are you gonna recognize the contributions? And when you're doing all those three things, right, then you've added the intelligence to swarming. Some of the things to think about when getting started, what's the culture of the company and the organization? If your company is an incredibly hierarchy siloed type organization, some of those cultural factors need to be thought about in how do you transform and change those thoughts and the mentalities and the behaviors of people. Because no matter what kind of tool you put in place, if your people aren't bought into working together, if it's, you know, very political and hierarchical, it's really, really challenging. We're seeing less and less companies like that, though. That's it's rare that we go into a company. I would say ten years ago when I was in a different job, I saw more of it. Today, I don't don't see as much as I I used to in in companies, but it certainly still exists. And also understanding the profile of your, your cases and the work you have. Yeah. I think intelligence warming can transform, an entire company. It's not just a support thing. It it can be a company thing. But if you're a support organization and you're incredibly low complexity, incredibly high volume, you're probably not gonna see the benefits like Patrick was talking. You're probably not gonna see a twenty percent reduction in case resolution time if you're taking really high volume, really low complexity work. We've had member companies where, you know, they have, like, two different divisions where one division is incredibly high volume, low complexity, and one division is low volume, high complexity. And they didn't put intelligence forming across both because the benefits weren't there. So understanding the profiles and where you think intelligence forming will have, the greatest impact We'll definitely help with the adoption, what regions, how people are gonna work across time and space. We like to say collaboration is across time and space. So those types of things are definitely big considerations. But getting the people on board early and making sure that the support agents, the frontline managers, the middle managers are all involved in the design session, and designing the system is super critical. Right? The people that are doing the job we like to say the people doing the job should be the ones that design how the job gets done because they know all the things and all the tricks and all the things going on behind the scenes that sometimes management don't see. So if you get them involved, then they're very bought in right to the processes. They're bought into making it successful in work, and going on that that journey of transformation and change. Now, Patrick, for you, you know, you kicked off intelligence swarming at Coveo, and I'm curious, you know, to see, you know, if you could give us a high level overview of, you know, how how you made this happen at Coveo. You know, what were the things that you had to put in place? And then, you know, even going a little bit further to you know, how do you even know who to include in a swarm? Yes. Well, I'll I'll piggyback a little bit on what Nat was saying around, you know, the culture and and getting people to design it. And that's basically the approach. I I kind of, you know, back in back in December last year, I I did a quick workflow, that just said, hey. You know, what if we could create an automated way of identifying experts that can collaborate together, leveraging our technology, bring them in a dedicated Slack channel, collaborate, and then close the loop with knowledge management. Would this be feasible? And I threw that idea to the team. And I said, does this resonate with anybody? And everybody said, yeah. That would be fun. It would definitely be an improvement on what we have today. I said, okay. So who are my Coveo Salesforce Force and Slack experts in the team that I we could leverage? And I got nine people just raise their hands and say, we're interested in helping you build this proof of concept. So we sat down together, and we just started brainstorming on what that could look like and what technologies could we use. How could we approach it? And, we we had many iterations, but the first the first one that really stuck was, let's build a Google Sheet with everybody's profiles and proficiency across all our different technologies, and that's what we're gonna use as our people profile. Because a lot of the people or a lot of companies that I saw do swarming, they were bringing people in these dedicated, not the generic Slack channels. And they had to have, you know, either swarm leaders or managers monitor that the swarms were actually happening, which in the end, you know, kind of proved that if it's everybody's responsibility in the Slack channel, it's nobody's responsibility in the end if you end up being in the same loop that we were with our collaborative channels, and we wanted to to avoid that. So by being able to build these people profiles, we said, okay. Can could they index this? And with the case context, identify who the best people would be. And when we went with that, it worked, but it was always the same people bubbling up because it was very static in nature. So we sat down with our product team, and we showed them this. And we said, is there a machine learning model that we can use within Coveo that would work? And we said, yeah. The case classification model that we use for our case submission form could do that because you're indexing case data. So we jumped on that, and, we started testing it out. And finally, it worked, and that really brought the whole dynamic of people profile that is based on on case history. So in the end, what happens is we build a custom button in Salesforce in the case view, and all the agents need to do is click smart. That sends a call to Coveo with the case context and says, these are the top three SMEs and product expert you should bring into Swarm. Do you wanna do it? Yes or no? They confirm yes. That creates a dedicated Slack channel, and the swarming happens there. And once that's done with a Slack command, we can archive it, and we index it, and we bring it into knowledge into our knowledge management piece. So in a nutshell, that's how it works. And what we're doing right now is we're actually starting to bring r and d into the mix. So we have two of our r and d teams that are piloting this with us. So if we need r and d in a swarm, we just add that r and d team, and we bring them into the swarm. So now we, you know, we we're even going outside of support in terms of in term of, intelligence forming. So it's it's it's making its way outside of the organization, which is great because it just takes down, you know, the barriers of going through system through system. Like, if we have to go from Salesforce and say to submit a Jira because we need someone in r and d to look at something, and it's just a question, it just adds a lot of time and unnecessary, you know, delays versus bringing in to the conversation. We'd have them live with you. So we're starting to see and and actually measure and see if that's gonna have a difference in in the resolution times on things that we need in our engagement as well. So when you kick off the swarm, is that is that swarm initiated by the person who gets the case, or is that oh, so it's like, they get the case, they need help answering it, then they start the swarm. So there's no one kind of watching all the cases and directing who gets what. No. That's that's all automated. We use a tool that has, some intelligent case assignment, rules based on AI there. So once again, using case history and different factors around workload, fit with the customer, time frame complexity. So we have intelligent case routing going on that routes the to the most qualified agent for that particular case at that particular time. And if that agent requires extra assistance, then we can they can swarm. And we we have about between ten and twelve percent of our cases that actually have swarming activity on them. Oh, that's great. And is that pretty common, Matt? Is that a common percentage? Or I guess it would depend by organization. Yeah. It definitely absolutely depends based on the organization, the profile, and the cases, and all those factors. But we have never seen somebody who's using intelligence swarming where more than maybe, like, twenty to twenty five percent of the cases get swarmed. Sometimes early on, right, in the early adoption, you see a lot of it taking place, but then over time, it modulates. Because most most of your issues are still gonna be solved the way they are today. I find another knowledge article. I just know how to do the troubleshooting myself. I don't need help. And then it's that, you know, leftover ones where you need help. And, another thing which I think is important when people think about the collaboration, it's not we don't wanna set up a system where collaboration is only acceptable in, like, super hard, critical, escalated cases. But, also, if I just need some help, right, I just you know, Bonnie happens to have an answer that can just get me unstuck and move me forward. We wanna think about collaboration also. You know, Bonnie isn't gonna you're not gonna stay with me for the entire life of my case, potentially. You're just gonna help me get unstuck, and we wanna make that collaboration easy as well. And depending on then how you measure the collaboration and what you consider a collaboration versus somebody pinging somebody, that can change those numbers. But I think what Patrick is seeing is probably where a more mature especially if you have the connect people down pretty well, I think that probably is a percentage ish that most most companies would expect to see. Yeah. And I would I would add to that that it depends what you're doing upfront in terms of self-service. Because if we compare, you know, the the shared model versus the the collaboration model intelligence farming, most of your easy, repetitive issues that you would get that tier one would necessarily would usually handle, in theory, you'd be able to move that over to self-service. Which in theory, what's coming to your support organization now should be either, you know, things that are new or more complex that actually require more troubleshooting that cannot be served as self serve as self serve. So depending on on your ability to do that, your ratio of swarms might vary a little bit. Because if you're getting a lot of new stuff, yes, you might still route it to the right person that has some knowledge, but you might need to collaborate a little bit more if your ratio of new versus no is flipping the configuration. Mhmm. Now we have a couple of, questions from the audience. So, Patrick, the subject matter expert people profile finder, is that standard in Coveo offerings now and and available to use? No. We kind of we kind of knew something that was built for another purpose, which was, to actually, you know, do recommendations as customers are submitting cases, to make the the case creation flow much easier, which is our case classification machine learning model. So since that model actually indexes case information, what we were able to do is tell the model index whatever case field that we wanted to to get a good context, and what we asked the model to send back is a case owner. So that model allowed us to do that. So it's not out of the box, but it's definitely something that exists. And if you're using service cloud, it's doable. It's definitely doable, and and that's the machine learning model that we that we leverage for them. So you took the existing so the people profile piece is standard, but the expert finder kind of connecting the dots is where you kind of made that made the magic happen. Yeah. Well, the people profile is just case history. And so we basically index, you know, twelve months worth of case history, and we ask the model based on this case history, who would be the top three SMEs to bring to this to to to this form? So who has handled this type of issue in the past? Who has resolved this type of of case in the past, and those are the ones that are identified as as top SMEs. So we fed, you know, six months of data to the model to test it out, and then we just we just ran test cases with different things, and we just tested if this made sense. If if something came in our service line of business, will we bring the service people to this form? Or if it came into our commerce, then this we're bringing commerce people into this form. And once we tested that out and we felt comfortable with the model, we were able to turn the switch on and remove the static Google Sheet in this machine learning model. Okay. Well, to the person who asked that question, Brian, if you have any other questions to follow-up on that, feel free to add that. We do have another question around, you know, the involvement of r and d and the tools to use. So there are additional factors to take into account, like their culture, practices, and tool preference. In your case, was r and d already working with Slack before, or did they adopt it when support adopted it? And did you find a situation where r and d preferred a different goal for swarming? Good question. Slack is widely used at Kavail, so getting them involved in Slack was not a huge sell. What is, what is a little bit harder is depending if you have things that are dedicated to maintenance versus innovation in r and d. Wanna make sure that you pull on the right teams. In our case, it's all the same teams, and they rotate people who are in maintenance. So we asked them how they wanted us to contact them. You know, we didn't wanna have people dependent processes. So we were wondering, are we going to index, you know, the maintenance calendar and see who's on maintenance when so that we would be able to go work with Slack around it? And for now, since we're just piloting it with two teams, it just said, no. Just at the team. So we have, we have groups in Slack that we can just at. And they said whoever's on maintenance is gonna get notified, and they're the ones who are gonna join in. Everybody else is gonna leave this form. We have a very collaborative culture at, at Kaveo. So r and d and support collaborate tightly, you know, every day. They were actually involved in the forums that we had before. So it's not new to them to collaborate with support. We're just saying, hey. You know, maybe it might make more sense to do it with Slack and and see how, how it works by actually identifying the right people to help us versus just throwing a line in the water in a collaboration form and saying, is someone going to see this? Is someone going to help us? And having fail safe mechanisms as well. So it's just using the same concepts that we use for our self and support and and extending them to r and d. And so far, they they're more than willing to, to jump in this adventure with us. That was great. And, Matt, for for you and what you're seeing with member companies, do you see this challenge with tools in the different departments? Or, you know, what are the recommendations there? You you definitely do. But tooling is I have found tooling is usually an excuse for somebody not to do something as opposed for somebody to do something. And, I think, you know, what we see in, I'd say, less collaborative cultures or where r and d and support, development support, software development support have had that more, I create a bug and I throw it over the wall and hopefully you fix it and then you throw it back to me mentality. Over time in organizations like that, one of the things that the software development teams often see value in is one being able to opt in. So not feeling like they had like, if it's a true bug, there's maybe a process for that. But getting earlier visibility to what customers are experiencing, the problems customers are having, they start to see that as a rich source of information for them to get in front of problems. So it doesn't all of a sudden be, oh, we have thirty customers now calling in on the same thing. We released a patch and the patches has a problem in it, and now it's a nightmare. And then now there's major escalations and, you know, all that. Sometimes the software development teams like to just get closer to the customer. And talking to them about getting closer to the customer in a way that they can opt in to do that usually goes pretty well because at the end of the day, they wanna make sure that what they're developing is being used and is, you know, high quality and everything else. So that that's one of the ways that we see support organizations and kind of the other organizations starting to play nicer together with, Calabrio environment. And then just getting past the tool issue of, well, what tool should we use to do that? Right? How do we pass information back and forth? Mhmm. That makes sense. So there's a question for clarification in here for you, Patrick. You know, you mentioned that ten to twenty percent of your cases, you're leveraging swarming. What's happening with the other percentage of those cases? Is it a tiered model, or how is it just they just don't need support to solve those cases? How would you describe that for the other percentage that's not No. It's it we're we're we're not a tiered model. Everything is oh, you know, the case owner that is assigned the case owns the case from creation to resolution. So, basically, the other case the other cases that don't require swarming, they're just resolved by the case owner because they have the knowledge. They have the expertise. They found the right documentation, through our knowledge management program to resolve the case. And we just, you know, felt comfortable to view and to to, resolve the case without requiring this warning. Okay. Great. And we have a few more questions before we jump into metrics. How important is the quality of case context data to create the people profiles and maybe an expert finder? What initial steps should be taken by those who are informed or getting started? That's a very good question. Before getting into these, into swarming, we completely redesigned our case management process. And as we went through that, through that journey, we thought of our data model. We thought of our data model for, you know, potential, you know, collaboration, for case assignment, for case, submission, for training needs, you know, everything that we would, potentially need for, you know, just improving the overall experience, improving, you know, agent efficiency, and all of that. So as you're getting started, you really have to think about your data model. So I'm saying, okay. What what would we need in our cases to be able to accomplish this? You know, do we wanna go about it by, you know, technology, by use case, by different parts of the application or the product that you have? You're really trying to break down your your your complexity. And so we use all these fields. We use, you know, the subject, the description from the customer, but we also use very specific fields that are tied to our product taxonomy. It really helps get the full context of that particular case. The challenge, though, is you're gonna need to to have cases with this data for a certain set of time, maybe, you know, three to six months before you can actually think about, you know, training your machine learning model because you're not gonna have this data historically if you change it today. But unless you backtrack it, which, you know, could be a pretty significant, effort to do. So I would think that as you're getting started, you really have to think about that data model and what and what data points you actually really need to be able to provide those. Anything to add on that? Yeah. One of the things that we've we've seen, some as they start to implement, it is very manual, right, with a Google Sheet or Excel file, right, to try and capture the people profiles. And early in the pilot or early, right, when you have a smaller team doing intelligence forming, learning what you need for the data fields, all that stuff if you're not very mature. I mean, I sounds like you guys were very thoughtful and conveyed with very thoughtful about that process. But doing it a little bit manually, and then using that manual to then go create the right things in your tool. So if you don't have a good vision for that upfront, we've seen companies that when they're starting small, it's a pretty manual process, and then they automate that manual process. But I will agree a hundred percent, Patrick, right, that if it's super dirty data, you're gonna get super dirty matching, and it's gonna be frustrating versus, you know, really getting the machine trained up the right way. Makes sense. Now I wanna move on to metrics. There's some other questions here. We'll circle back to those in a little bit. But let's talk a little bit about how you measure success. So, Matt, you know, there's a question in here about what what metrics should be used when doing swarming. What do you what are you seeing? What are some common forming metrics? That's a good question. I think there's three ways to think about your metrics. There's the business metrics, the health of your intelligence forming system metrics, and then your people contribution metrics. And being clear on what those three things are and not mixing them and merging them all together into a big soup of measures is pretty critical in my mind. For intelligence swarming itself, some of the measures that are to see the health of the intelligence swarming system, right, is how accurate is your matching. Right? So when you're if you have an automatic matching engine that's, you know, saying, hey. Here's here's work. Here's people that should take that work. Is that accurate? Is it not accurate? Is it getting more accurate, less accurate? So tracking the ability to do the the connect is pretty important. Designing collaboration measures. So, you know, how are people collaborating? How are they opting in to take work? How are they opting in to help other people? What does that look like? So how are you gonna measure the collaboration space itself and being careful about making it so people can game the system. Right? Like, hey, Patrick. I'm gonna give you a thumbs up if you give me a thumbs up, right, so you can start to start to play games and things like that. But, so measuring the collaboration itself is pretty important. Kinda what's going going there. And then the contribution of people is really a triangulation of different measures to say, how are people contributing to overall success and getting away from just counting things like who takes the most cases, right, who closes the most cases, some of those kinda traditional support metrics that sometimes lead to pretty poor behavior, but using a triangulation of measures. And there's some interesting work going on at a few companies about how to do triangulation, right, weightings based on a bunch of different factors and things like that. But those are the things to kinda think through. Mhmm. And I I can talk all day about the power of measures, but also the pain of measures. Really be conscious about why you're measuring something and what you're gonna do with the measure, and don't just measure it because a tool makes it easy to measure. But think through what's the intent of the measure, what's the potential upsides and downsides, how are we gonna use it in goaling people because that, right, can completely make a a measure useful or useless. You know, attaching whether it's a lagging measure, a leading measure, all those types of things are super critical to think through. Yeah. Yeah. And and, Patrick, you you kinda shifted the way that you were measuring your team. Right? Can you share a little bit about your the metrics that you look up with swarming? Yeah. Definitely. You know, every time we speak about swarming, we always get the, the question of how do you get people to collaborate because everybody is so focused on their own little thing and because, you know, as Matt said, we measure them on, you know, cases assigned or cases closed or their own individual metrics from, you know, an efficiency perspective. And what we decided to do is focus more on the team. So we have maybe two, maybe three individual metrics that are in the hands of the individual, which is their case quality measure. We have a we have a measure that's called the control factor, which is basically a calculation of how well you're in control of your queue, and that is basically it. Everything else, we look at it from a team level. So we look at resolution time at a team level, first day resolution, CSAT. We bring everything at the team level, and we say we all work in the same direction here. If someone reaches out for help, you are contributing to to the team to reach these goals. And that has really helped get you know, maintain and even emphasize the collaboration culture we already had in the team because it it really sets the vision of bringing everybody working in the same direction. As for metrics on swarming itself, you know, we're still new. We're about nine months in here. And, you know, since this was kind of a, in house solution, we're we're playing along with what data points we have currently with Slack, with Salesforce, and we're trying to combine everything together. So right now, we're able to do the before and after. Like I touched on a little bit earlier, we're able to measure collaboration, in terms of people who actually collaborate, who are our top collaborators, and things like that. So we're just you know, we just scratched the surface on what we're able to do here, but we we're really cautious approach we're taking. Because as Matt said, you have to be very careful around which measures you decide to put forward because those are gonna drive the behaviors. You have to be very, very careful and think through and say, okay. If we measure this, or we hold them accountable to that, what's the outcome or what's the behavior we're gonna get? I would like to flip it the other way around and kind of say, k. What are the behaviors we're expecting, and what measures can you put in place to drive those behaviors? So think about the behaviors you want first, and then say, how can we measure it? And stay don't go over five metrics, honestly, or else you're just gonna get bossed about too many things. Everybody gets so confused. So focus on on what you're looking to drive in terms of behaviors, and we can stick to that. And as I mentioned, for us, the team thing worked very well. It it really helped us align everybody around these same objectives. Mhmm. We're not saying, well, you had bad CSAT. You had good CSAT. It's like, okay. We need to work on our on our CSAT. This is what our customers are telling us, and this is what we're gonna put in place as a team to to move this forward. So we really bring we really want the team concept, and we put that, you know, at the forefront. So if you're measuring as teams and you're you know, you've got this warming happening, how do you what do what does career progression look like? So this is a really interesting question from Tyler. You know, if there are no tiers and the case is being owned by the best person available, do you have a way to distinguish certain levels, or how how does that career progression look in a forming environment? Oh, that is a whole other discussion. But I'll I'll try to have a whole discussion about this when we were talking the other day. Yes. Yes. But, I'll I'll try to, I'll try to, yeah, to resume it as quick as best as I can. So we've actually built a job family within support with different roles that have different levels. So I'll just take our our product specialist. There's four levels, and these four levels are defined based on different competencies. So they're not just based on your work as the support agent, but more on your performance, how involved you are in escalations, how involved you are in collaborating with your peers, are you getting involved in improvement projects, and things like that. So we look at everything that we expect from a product specialist and and different ways that people can can develop, and we have development plans like that. We review them twice a year, and we have set actions on how we we encourage our product specialists to, develop, and we built a huge internal development program along with that with, you know, set competency objectives and say, okay. Well, if you wanna go from level one to level two, these are the competencies you need to develop. These are the trainings you need to take. These are the behaviors we need to see. Once you've reached a certain milestone, you will get that promotion from body specialist one to two and and so on and so forth. So, you know, the day to day stuff doesn't necessarily change, but the expectation around how you're gonna collaborate with the team, are you gonna get involved in different areas, is gonna help. And then there's a bunch of different roles you can support, like product experts, technical account managers, program managers, team leads that people can do and find promotions and really have a career in support. In two minutes, that's the best I can do. I can go on and on and do it in detail, but that's that's how we kinda do it. So instead of moving people from one tier to another, you really build this job progression in their roles and allowing them to be able to be promoted from one level to the other or even move from one role to another, whether it's a lateral move or promotion within the organization. Yep. Yep. Now next question, for Matt. I believe Matt had mentioned earlier the benefits of intelligence forming work better for those with lower volume and more complexity. I'm curious if there are any additional LinkedIn groups, podcasts, or forums with more support organizations with these types of challenges. I found that the best practice groups tend to lean more towards traditional contact centers and, although helpful, not very close to what we deal with in our support environment. Curious if Patrick and Matt have found some really good resources for support teams that are dealing with more complex issues. Well, I'll plug the consortium. I mean, consortium for service innovation is, you know, a a smaller organization made up of companies that come together to talk about these challenges and, you know, learn from each other and share ideas. Right? And think through, well, how do we solve these challenges as the world continues to get more complicated as, you know, all of those things. I mean, the consortium is a a great resource, and we're a not for profit as Bonnie said at the beginning. So almost all of our stuff is available through, you know, non compete, noncommercial licensing. So it's mostly free for everybody out there in the world. But, as an organization membership based organization, that's what our members talk about. But, I mean, there are a lot of great groups out there that talk about these things. TSIA has some really great presentations and benchmarks and discussions on the complexity and the ever growing complexity of support and services organization. So there's a lot of things out there that, you know, you can kinda tap into. Yep. Yep. We pretty much, hit on it hit on it hit it on the in the nail here. It's what I've seen is is the the support community. So going out to these these types of events, like, what the consortium has, like, with PSIA and and get out there and and talk to the community and and see what the challenges are. You know, it's it's pretty I was I was curious to see, the the results of the poll, earlier because in one of the sessions that I did at a past conference, I had asked basically the same question as where people were on this warming. And it was it was it was a split between not considering it and, you know, actually implementing it at fifty percent. There was no one at all. Actually, it was like it it was all or nothing. I thought that was really interesting. But, no, definitely, you know, it was it it's starting to get much more attraction and attention in the industry. You know, there's a lot of there's a lot of exploration around how can we move from a tier model to a single level of support leveraging intelligence forming, but there's a lot of reluctance as well because the tier model has worked so well for so well for a lot of companies that they kinda struggle and say, okay. How can we do the move? And I know I touched on it a little bit, but I I don't think you can do this move if you don't have very robust self-service to move your l one to self-service as much as possible. Or else, you know, if if you're if you're bringing your your support agents and say, okay. We we wanna have this level of complexity, this level of challenge. If they're if they're doing password resets all the time, they want that, you're gonna have high attrition. They're not gonna be engaged. You're gonna have a hard time getting them to collaborate because they're just dealing with the same stuff over and over again. So you have to not just look at it from, you know, a tiered model versus a single level of support. We have to think about your support offering as a whole and really think about it and say, okay. What can we do with self-service? What do we need to be able to drive that? Do we have the knowledge management program in place to be able to do that? Because it's not just tiers versus single level. Mhmm. It's it's a much broader discussion. Yep. And I think, Patrick one of the things Patrick hit on the head is it's you know, we, the support organizations in the world over the last thirty years, built the tiered models and built the things that we're now trying to, I'd say, remove and tear down. I think the tiered model has just outlived its usefulness in our current modern world, but, but it is a big change initiative. So also putting the right program management behind it, change management behind it, because it is it is a change to how you think about doing support and services. So, you know, having those things in place can also make it much smoother to kinda get those things done. Alright. We do have a couple more questions, so I'm gonna ask you to answer these quickly. Patrick, do you ever have to reopen a swarm? Do we ever have to reopen a swarm? Not that I know of. But, honestly, I don't have the data on that, so don't I I wouldn't I wouldn't bet my hand, but I don't think we've had to reopen a swarm. And do you see that, Matt? Do you see people having to do that? I haven't seen people reopen a swarm. They open a new swarm if they need new information or a new group of people, but so Okay. Maybe. Okay. Alright. Next question. Another rapid fire. What is the difference between intelligence swarming and a formal escalation process? Could they be one and the same? No. They can't be the same. I mean, it's depending on what your escalation process is. But, if by escalation, you mean moving work from one place to another place, then that's not intelligence warming. If you mean escalation, you know, I hit the panic button because things are really going bad, customers getting angry, you need management, you need all these people involved, then that's a different situation. Right? What we say is don't design for the one ops. Design for the majority, and then treat the one ops as those exceptions to the process. But, so you'd have to define escalation escalation. But moving work up a tier from one place to another, if you call that escalation, then, you know, you can't do that end swarming at the same time. Yep. Okay. Last question before we announce the winner of the certification. What mechanisms are available for knowledge workers to manage being connected with multiple swarms? For example, using prioritization or case age. So prioritizing multiple swarms. Okay. So when you say knowledge worker, are we talking about a case owner, or are we talking about, you know, a content creator? That's what I'm not let's just say it's a case owner. If It's a case owner. I I guess it really depends on how you develop your you're swarming. If you're swarming in in Slack in generic Slack channels, if if everybody's disciplined and they go and they reply by threads, they can have multiple threads and might get a little clunky to follow. But if you have dedicated Slack channels per case, it becomes very easy because it's just like an it's just like any other Slack channel that you use day in, day out, just like a a group conversation. So, basically, you're gonna prioritize them as, you know, the urgency of your case and and things like that and how this one is evolving as well. So, hopefully, that helped. Well, as I said, we are going to announce a, a winner for, the intelligence warming training and certification. So let me see here. We have Rala Begum Begum. I'm sorry. I don't know how to pronounce your name. Are you on the call right now? See here. So we will be following up with you after this event, to cover your intelligence forming training certification offered by the consortium. You can see here, this is just a screenshot from their website. You can find intelligence warming case studies. You can access the fundamentals training and sign up for trainer led workshops at innovation service innovation dot org slash intelligence warming. I really appreciate everyone joining the call today. As I said, you will be getting the recording following this. And if you'd like to chat with Patrick and have a follow-up session and really dig into how we're practicing intelligence worming at Coveo, you can connect with us on LinkedIn, and Matt Seaman is also available as well. Now I do wanna say, you know, we are having a urine Coveo webinar coming up on December eighth. So really to understand, you know, what's what we've done this year at Coveo. So here in a couple of days, if you're curious about the innovation that's coming out of Coveo, feel free to join that webinar as well. So I wanna take this last moment to thank our attendees. Matt, Patrick, thank you so much for joining this roundtable session with us. Really appreciated the great conversation. And, again, if anybody has any questions, wants to, dig in further, please reach out. Thank you very much. Alright. Thank you so much. Thanks, everyone. Have a good day.