December 14th, 2020 – Cyber Resiliency

All right, we're live. Welcome everybody. Got the crew with us, and plus one Mike Beard, special guest Cecil of Marco. Mike, how are you today? Uh, doing great. Thanks. Happy to be here. Yeah. Thanks for joining us. Gary Pika, Ryan Weeks, Wes, Spencer, Gary. Wanted to kick us off. Ha. Oh gosh. What's going on there? That was like a, that smoker's ha, That's, uh, I think there's some puberty going on, on, I Got a lot of band on, But, uh, smoking cigarettes is not one of them, luckily.

So, um, just a few things. Kicking it off. Um, put a link, um, data app, uh, actually pushed to this in the, um, cyber Nation just a little while ago. Um, very, you know, it's very detailed, but if you're really into, you know, getting in depth into what, um, the, what this compromise looks like, um, others are putting stuff in chat as well, stuff from Sands. Um, but, you know, Ryan, I gotta give you some props here. Last week, you know, our mind isn't even on this particular subject, per se.

You start talking about your presentation at MSP Tech Day at Datto on cyber resilience. I'm like, man, this is something that I think we really need to speak about. I get on the phone, I just happen to be talking to Wes and then Mike regard, and Mike's like, oh yeah, we're, we're, we're definitely adopting. That's critical, you know, this assumed breach mentality. But internally and externally, lo and behold, just a little bit of, uh, things happen on the weekend.

And, you know, Ryan, I wanted you to kick it off because, um, you were saying some things off the air, and I think it's really important your message to the MSPs on how to be looking at this. 'cause this is a supply chain type of event. It's not just a tactical. Do I have it or don't I have it? So let me flip it to you, Ryan. Yeah. And make sure we talk, let people know what we're talking about here. We're assuming you're Yeah. Fair, fair. Gary, you wanna No, go. Go ahead, Andrew.

You can recap. Recap It. Yeah. So, over the fair, Gary, thank you for, for pointing that out. So, over the weekend, um, going back, you probably probably heard that FireEye was compromised, but, um, this whole thing revolves around a, uh, a PT called a PT 29 out of Russia. Um, and then lo and behold, over the weekend, there's an alert coming out from CSA that several federal agencies have been compromised. And the mechanism for that compromise is Solar Wind's, Orion.

Now, for those of you that don't know, Orion is the legacy application going really where SolarWinds made its name. This is an NMS or network monitoring system. This is something that you will typically see competing in the space in the, in the mid-market and enterprise. Uh, it does have applicability in MSPs. I know MSPs have used it. I used to head up sales and for service provider at LogicMonitor, so we did compete against it. So, um, is that setting the stage enough? Gary to Yep. Yep.

Kicked off. Okay. So with that, Ryan, let me turn it to you. Yeah, sure. So, I think there's a couple ways to enter this conversation, but right is just level setting of what's, what we know versus what is suspected. Um, what we know is, last week FireEye had a breach. Um, the breach was linked to a very capable nation state threat actor, and has been, um, published as being a APT 29, which generally the publishing of that attribution doesn't happen without strong evidence. Right?

The FireEye was like, eh, we don't really know how this happened, but we found it while we were investigating an anomalous VPN connection to our, our VPN infrastructure. Um, and the result was the theft of, um, I think it was somewhere north of 200 tools, but really like 60 that are really of material importance.

Some in-House created some public, some hybrid, um, and much like the NSA breach, those tools could potentially be used to become weaponized to, uh, to, you know, to create worms, ransomware like worms, um, and, and do all sorts of other bad things. Uh, maybe they'll eventually find their way in Cobalt Strike. Who knows? So there was a list of CBEs that was published, um, that those tools targeted.

So number one, every MSP should be going through every single environment, including their own, making sure that none of those CVEs are present. Um, that will largely make a lot of those tools, uh, neutralized, right? Because they rely on the presence of that vulnerability to, to work. The other thing, uh, and this is a little bit more tricky, is FireEye released a set of signatures in the form of Yara rules, um, and snort signatures, which can be plugged into your, uh, existing control stacks.

Yara might be a little bit more exotic for you, but I'm sure most of you have something that can run snort. Um, you can just upload those signatures into those tools and boom, you have monitoring to determine if those tools are now being used against you. Right? And so then we're, but we're also scratching our heads, like, how did FireEye get breached in the first place?

And then over the weekend, what we found out is that through continued collaboration between Microsoft and FireEye and the FBI, that it was traced back to a product called SolarWinds Orion. Um, the SolarWinds Orion products, uh, between March and June of this year, uh, became weaponized by an attacker. Um, the attacker, uh, compromised a, a software build system, um, within the, the environment and used that build system to inject a back door into, uh, the Orion. Uh, it's a plugin.

It's a DLL, uh, in Orion. And so along with legitimate binaries, the build system was injecting this malicious, uh, DLL as well. And that was ultimately what was used to deliver what they're calling the sunburst malware, which is really just made up of, um, uh, uh, malicious piece of software called Teardrop. It's a dropper, but it's a, it's a, a never before seen piece of malware. Um, it's an in-memory malware.

And then the, uh, other component of it was something, uh, known as Beacon through, uh, cobalt Strike. And Beacon has very finite indicators. Um, and so, you know, similarly to the FireEye attack, um, they release signatures. And so you can go to FireEye's GitHub page and download the sunbursts, uh, set of signatures and indicators and leverage your tech stack to s your environments for those indicators.

Now, um, it is probably unlikely that you're gonna find anything because a PT 29 is really focused on long-term espionage at the kind of nation state, uh, level. Um, SolarWinds was targeted because of their supply chain. They have hooks into, you know, 300,000 companies in their eight k filing. They said they think it's somewhere around 18,000 companies might have had this malicious software, um, many of which are government, uh, agencies.

So, um, FireEye may have been one of those things where they, they poke the bear too many times, like FireEye made a business out of exposing the tactics of APT 29, uh, and other apps. And so when they realized they had a full hold in FireEye environment, they probably decided to try and give them a black eye. Um, whereas the real motive of the attack was to gain persistence, long-term persistence inside of government institutions.

Um, and so this threat actor is generally not one that we would consider in the MSP or IT channel threat profile. Um, they could be if they were seeking to leverage a supply chain of a target that was using an MSP or MSP related software. And so this adds leads me to the third thing that I think that every MSP should be doing.

You should be reaching out to all of your vendors and asking them whether or not they've analyzed their environment for the indicators that have been published, and whether or not they've used the vulnerable version of Orion, uh, software package at any point in time, uh, this year. And, um, if the answer to the first question is, no, we've not done an assessment of our environment, you should push them to do that.

And if they've said, yes, we did have that vulnerable package, then your question should be, how do you know that you have not suffered a breach as a result of that? Right? And so I think that really where I wanna wrap up is something of this scale and of this nature. And I, I actually took a, I actually captured a, a slack comment, I think it was from, uh, Dave, the companies with six to seven figure security budgets can't be safe.

How can any of us, these types of stories, tend to paralyze us into fear and not knowing what to do. There are clear signals inside of the information that is being published about what we can do to determine if we've been breached or how we can prevent future breaches. Seek out that signal inside of that information and make that the thing that is actionable, right? Don't let the size and scope and scale of an incident like this paralyze you.

Let that, uh, fear turn that into positive energy, um, in terms of, you know, protecting yourself as well as making sure that your supply chain has not been impacted by this vulnerability. So can I just jump in with one quick thing here? Follow up? Yeah. Yeah. So first off, that was awesome, Ryan. That was awesome. Um, super glad you're here, man. Um, but, you know, following up, uh, what you mentioned about, you know, Dave, I happen to know Dave Mason, uh, hey Dave.

Um, but really the comment was his clients are kind of, he's saying, what if your clients say, how are, if they can't be secure and they're spending six figure or seven figures on security, and I'd say the answer to that is, yet you're not gonna be secure, especially for what you're paying me. But we can make you reasonably secure, and we're gonna talk about what happens if that doesn't work, which it's great, it's great. Lead in to our topic of today. Yeah. Right around assume breach.

But, um, really great job on that, Ryan. Yep. Yeah. Yeah. So again, I mean, listen, this thing is gonna explode. There's, there's gonna be more breaches that come about as a result of this. The important thing is keep reading, but read with the intent to figure out if there's something actionable that you can do to assure that your environment or your environments have remained safe, right? Vendors' environments as a, as a supply chain, right?

Um, you know, you should be reaching out again to every technology provider and asking them if they use that technology, um, and, uh, and, and, and responding accordingly. Um, so, uh, I'll, I'll leave it there.

But yeah, this does kind of lead in a little bit to cyber resilience, which is this belief that, um, Hey, Ryan, Before you go on to your high level, uh, because you're gonna highlight what you talked about, if I could just interrupt question for you, question to we, question to you is what's reasonable from a vendor perspective?

Because I think, you know, you know, m MSP have been in situations where even for third party questionnaires, right, Mike, you're asking vendors for information, what's reasonable and what can they do? So that question to you, Wes, and I'm gonna come over to you after Ryan answers that, you know, again, not everybody out there understands IOCs and indicators are compromise and what tools they can plug those into. Could you just touch touch on that? Yeah. Yeah. So Ryan, you're, you're up first.

Yeah. So what I think the question is, what's reasonable in terms of talking to your vendor And what can they do if, like, you know, well, whatever, you know, you know, So I, I mean, I, I, again, I'm, the more we talk, the more you're gonna realize, I, I live by mental models, right? And I have this, this mental model of what do you know? How do you know, prove it? And so, when I interact with a vendor, um, you know, what do you know?

Do you know if you've leveraged, uh, a vulnerable version of this software in your environment? How do you know it? Can you prove it to me? Um, and so I think the, the prove it piece is probably or becomes a little bit unreasonable. Um, but certainly asking your vendors, what do you know and how do you know it? Um, you know, have you been compromised? No. How do you know? Well, we conducted an exercise to look for all the indicators in our environment. We don't use that software. We never have.

Um, we've even looked through our entire environment for that DLL and haven't found it anywhere. Like, right? That's what DA's done. That's what every vendor should be doing. Got it. Um, And so if a vendor can't give you that answer, then um, you either need to push them to, to, to do that work, to give you that competence, um, or question more deeply, uh, uh, you know, escalate that to, you know, seek to speak to someone, to the security team, right?

At the end of the day, this is the security of your business, and your vendors should, should realize that they're partners with you in keeping your business safe. And if they're not willing to answer perfectly reasonable questions, um, that should be a huge red flag for you. Yeah. But let's talk about it.

Like, in the real world, if you're, if you're Datto or ConnectWise or Kaseya or SolarWinds, one of the big four, you, this, you know, really needs to be addressed in mass, which is you have to address it. Probably put something in writing, because when you say, talk to your vendor, their vendor now is a 26-year-old kid who's their account manager, who's not gonna understand one word about that's who, that's who they are, right? Yeah, yeah, yeah, Yeah.

So the vendors have to address this proactively, or, Yeah. Yeah. So I mean, that's what we're doing at Datto, right? Um, so in the presentation I did on tech day, I talked about three concentric circles of protection, right? Protect, da, protect MSPs, protect SMBs in the middle, it starts with data, right? So when something like this happens, a vendor's first response is going to be analyze myself, understand myself, am I similarly vulnerable? Have I had this happen in my environment?

Once you have those answers, then you can create that messaging for your customers. And that is exactly what every vendor should be doing, so that when their customers approach them and say, Hey, you know what, you know, were you breached? Am I at risk as a result of this? Like, there's a standard message that is based off of, you know, uh, an attempt to analyze your environment, uh, in, in a reasonable, um, kind of in the moment understanding.

And the reason I put that in air quotes is because this thing is gonna continue to evolve for weeks, right? And there's also need to communicate an ongoing active monitoring capabilities surrounding this threat, um, in order to continue to provide confidence to their customers. So I, I completely agree, there does need to be something in writing. It does need to go out to the frontline sales staff, tech support staff of the vendors.

But the MSP should not be afraid to reach out to the vendors and ask these questions. It's a perfectly reasonable thing to do. Hey, Andrew, can I just mention the one thing you and I spoke about as a kind of a dovetail? Absolutely. Uh, I was watching the chatter, you know, of my peer group, you know, and one of the things that someone said this, wow, can you imagine if this had been automate or Kaseya VSA or End Central? And what I said to them back was, no, change the word. It's not.

Can you imagine if this was my RMM? Imagine when it is, right, right when it is, because you have the opportunity right now to do all those same things by watching this without having your, your environment breached. And really, that's why we're, I guess that's a good segue into this topic, Andrew. Yeah, no, that's a really good point. Um, Wes, just, again, this is important because Ryan is saying, Hey, look, let's not paralyze ourselves, et cetera.

What things, you know, talk IDS talk, IOCs talk, you know, bring this down, distill it down for people that may not know how to, what they can do with that information. So, sorry, I'm off eyes off camera right now. We have a lot of things going on at Perch. Um, uh, we have actually observed at least four or five customers currently that we're, we have high confidence have been involved in this, and some have been quite large. So we're, we're in the middle.

So in Florida, we have, uh, we have ant nests, like, uh, those fire ant nests, and if you poke 'em, the ants go businesses everywhere. You know what I'm talking about? Oh, yeah. Uh, this is my threat research team right now. They, they are in the middle of this. So, um, I'm, I'm just trying to multitask a little bit.

So, uh, one thing I wanna say before I answer your question, and no one's done this on the call today, and no one's done this on chat, so I'm just saying this now, not implicating anything. Uh, let's, you know, another thing about Florida, we have a lot of hurricanes, and I always feel like the weather people, when something like this spins up, like this is their Super Bowl, like, I'm like, are they just doing this for the ratings? Like, 'cause I, I kind of think they are.

Um, let's not do that as security practitioners, I'm very serious about this, right? Let's make sure that we give, um, as much credibility to our industry where it, where it sorely needs it most, which is times like this. And so what I mean by that is let's not get involved in conjecture. Let's not get involved. And I promise perch we'll never do one of those. You know, if you'd been using perch, these things wouldn't happen. We will never do that as long as I'm, uh, here running things.

Um, so let's just make sure that we speak in what we know and, and, and leave the conjecture to just small private circles, um, in, instead of saying things in public, and let's avoid the fud, all that kind of stuff. And again, no one's done that today. I just wanna make sure that we, we avoid that. So, um, Andrew, your question to me was around how do we look for and see these things? Well, this is what purchase is doing right now.

So FireEye has a great report, I think most of you're aware of it, so won't post it in chat, but there's a number of things you can look for based upon the intelligence and tools and visibility you have across your network.

So, for example, if you have really good network telemetry, there's a number of things you can look for from the, the C twos, the command and control call outs, either IP addresses, domains that you see coming out, if you have good insight into logging, there's a lot of things you can see inside logging. So, for example, you can do frequency analysis across SMB activity. I'll give you an example.

One of the things this particular piece of malware was doing is it would replace legitimate files and then it would, uh, run that legit, that illegitimate looking file, and then would delete it and replace it again, very sophisticated. Well, if you happen to know how that works, and you have good visibility into SMB logs, you could look for frequency analysis of delete, recreate, run, delete.

Again, uh, that is very interesting to find anomalies that may be in the traffic or, um, uh, there's another, there's a number of examples that are in the report that I won't get into, but these are all indicators of compromise. These are things that we can look for to say, Hey, this is a potential sign of something that may have happened that may be of serious, uh, uh, research and investigation for us. And so these are things purchase doing.

We recognize that most MSPs that are not named Marco Micro regard don't have incident response people inside their organization. And so, um, these are things that we're doing alongside our, our partners is, is looking into some of this and saying, Hey, we do have visibility into this. We're looking at X, Y, Z, we seem to think that you're implicated. Um, can we talk about this? Right? So that, that's Andrew kind of what's going on with with all of this. Yeah, absolutely.

Yeah, I, I completely agree with Wes. There's going to be, you know, I interviewed Krebs in the MSP tech day last week, and he said, you know, if it bleeds, it bleed and this one is gonna bleed, and it's gonna bleed big and it's gonna bleed for a while. Um, but don't let the hype, um, you know, paralyze you again, because again, AP T 29 is not in most MSP's threat profile.

If it is in your threat profile, it's because AP T 29 targeted this software package to go after another target, and potentially you are one of your vendors just happened to be using it. So really the actionable thing for you to do is just make sure that it's not inside your ecosystem. Right? And another great question is ask your vendors, have you asked your vendors if they use that package, right?

And like, and, and realize it's gonna take many of these vendors days to weeks to get an answer back on that one. But these are just reasonable things that people be doing to understand their own posture relative to what we currently know about this set of attacks. Um, man, there's so much to say, Ryan.

One very quick dovetail on that, you made me think of this is why when we do risk assessments of our vendors and we do what's called fourth party, uh, or really third party risk, uh, one of the questions you need to know for every single vendor is, does that vendor that I work with store process or transmit my information? Because if they do, that question that Ryan just asked is critically important, do I care if my lawn care guy, uh, is, is using, you know, Orion or anything else implicated?

No, I don't care. Um, so, so these are things you have to know. And if you don't ask those questions in the original risk assessment, sometimes you're gonna struggle to know, well, who do I reach out first to? And who are my, my tier one priority people to know these answers to? And so one clue to that should be asking, does this partner that we work with is this vendor store process or transmit information that we have, um, or have access into our networks?

Those are a, a key thing that you should be asking for. Very good. Really, I know we're, uh, 22 minutes in already, but man, I mean, we're doing in essence, somewhat of a virtual desk, uh, virtual tabletop, for lack of a better word. Again, thank good, like, Gary, your point, not if, but when it is you arm, and, but thank goodness this isn't right now. Um, Ryan, can you really, what today's topic was gonna be and is about is cyber resilience, but can you kick us off?

Um, I think it's such a relevant topic, and I'm excited to, you know, get this going around the horn because this is really what's going on. How do you stay operational in the event of an event? Yeah, absolutely. I'll, I'll make one more comment on the, the previous topic. Um, and it was spec specifically to SolarWinds. Um, I am in no way, uh, throwing stones at SolarWinds. Um, I have done webinars with Tim Brown, their CISO in the past. Uh, I do not envy the position he's in right now.

He is an incredibly smart and incredibly capable CISO who has happened to come up against a nation state level threat actor. Um, so, you know, we need to be supportive of SolarWinds not knocking them down right now, because, um, excellent nation states are no joke, right? Yeah. Um, so with that, that said, um, cyber resilience. Um, so cyber resilience is dovetails well with assumed breach, right?

What it says is your business must be able to continue to operate through an adverse security incident. And, uh, cyber resilience is really this kind of merging of these concepts of, um, business continuity, disaster planning and recovery and, uh, incident readiness, uh, and incident, um, recovery. Um, and so, you know, my talk was really about, you know, you've heard me say this before, people process and technology, like that's really what drives cyber resilience.

You, it starts with the people. The people build the process, the process, and people eventually need technology. Those things will drive maturity, and that maturity drives down the risk. It all starts with people. And you really can't have cyber resilience if you haven't invested in all three of those things. Um, and you know, when we think about cyber resilience at Datto, we think about it again in three concentric circles. What do you do to protect your company, right?

Job one for West and I as CISOs is protecting our company, right? And then leveraging that information and helping MSPs protect themselves, and then helping MSPs protect SMBs is that third circle, right? And, um, we, we talked a little bit about that, but most CISOs in the channel are gonna generally think about cyber resilience in those terms, uh, and think about their influence in those concentric circles, right?

My core impact zone is my company then supporting MSPs, then supporting SMPs out from there. Um, Marco's in an interesting position, uh, because they actually get to, um, do a lot of that protecting SMBs, and they've adopted this methodology of cyber resilience, um, and are helping to drive that methodology into SMBs.

So I think it's a, a very timely and just a very interesting, uh, topic and, and an important framing of building a security program because building a, you don't build a security program just to build a security program. You build a security program to achieve cyber resilience. Yeah, that's excellent. You know, um, Mike, since, uh, Brian, this is my, you know, I see where I am going every weekend, you know, my questions and what we're gonna talk about.

But since he, you know, me mentioned you, let, let's just go to, to you first, and I'll come back to, to Wes in a second. But, you know, like I said, you and I, before we knew this, we're talking Friday last week, assume breach mindset. Um, you're like, yeah, it's exactly what we're doing. And, uh, we're really focused not only internally, but also the delivering that message externally back out to our, to our customers.

Can you, you know, give, give us a little bit of insight into what's going on in your world? Yeah, no, it, it ties in real well to what Ryan just said. I mean, I wrote down a couple of things I wanted to talk about people, process, technology is absolutely one of them. Uh, there's a security element layer to all of those functions as well.

And like Ryan said, you have to make equal, maybe not equal, but you have to make substantial investments into each of those three categories in order to be successful. When, when I talk cyber resilience internally, really the things I'm after I pivot and I talk three Ps all the time. It's, it's predict, prioritize, and practice, right? Those are the elements that we're after. You know, it's predict, predict the systems that are gonna get hit.

For those of us that are MSPs, it's pretty straightforward. They're gonna go after our MSP systems, our RMM tools, backup tools, et cetera. We've seen that evolution over the last couple of years. I don't think there's any surprises there. Prioritization though becomes the next thing. I think people have heard, uh, lair and, and Wes and a few others talk about that, right? Do you, do you know how you'd prioritize your clients when something does happen? I think Gary hit it with it.

It's not an if it's a when. So when we go through that, how do we prioritize our clients and, and to be upfront as an MSP, that's difficult. And the larger you get, the more difficult that becomes. We still, yes, we have a lot of resources, but we still have finite resources and they stretch real thin real quick. Um, so the prioritization components really important. Then practice. And, and that's one that we've definitely evolved in the last year.

We went from really outsourcing all security functions like most of our peers, um, a year ago to, we actually do have some in-house talent now. Um, and that's helped us go through the practice exercises. And, and I'll tell you a couple of things with practice. First off, you have to practice all the time tabletop, right? We let a, a public, um, tabletop exercise with West last week was phenomenal. And that's the clients love it. And that's piece should be doing it.

The other thing with practice though, is you're gonna identify weak areas. Um, the number one area I think we as MSPs struggle is the time to gather information in an incident. So when that perch alert fires, for example, or whatever solution you're using, how long does it take for you to gather all the information that's required to figure out what the heck happened that time is typically way too long.

So that's a key focus area for us as part of cyber resiliency is bringing that time down, um, as we go forward. And that's, that's a good KPII guess for those out there wondering, how do you, how do you monitor security with a KPI? That's one of 'em, right? How long does it take us to, to gather and pull that information? How long does it take to detect? I'd say that's another one. Um, Ryan hit some of the other ones, I guess.

But my other three things that we always talk as part of cyber resiliency is that basic hygiene, um, you can't protect what you don't know. I've heard Andrew say that so many times. You don't know it's on your network. It's hard to protect it, right? So we have to have that basic hygiene done. Next one's the, the basics, right? It's, it's actually configure security settings, um, in systems. Too many systems go out configured out of the box, right?

We do have to go in and tune them beyond out of the box configuration. So best practices implement those best practices for security settings, permissions. The, the MSP industry, I'd say as a whole is far too permissive. We tend to give everybody administrative access even when they don't need it. Take a hard, hard look at that and evaluate, does it make sense for everybody to have it? Dual control is important, even in small organizations.

Um, it's a very effective strategy to reduce, again, your attack surface and patch scan and patch often, right? That's the other thing. And, and, uh, one of the, I think I saw it in the chat, I guess, but critical know what to patch. I guess I would say that too, right? We, we end up chasing a lot of cvs. That, to be frank, probably don't matter. Um, go after the ones that do matter. Something like this obviously matters. It's, it's got buzz and I think Ryan did a great job covering that.

But come up again with that prioritization of what we're gonna do from a patch perspective, knowing that in a lot of cases we also, again, have finite resources. Um, then the last thing is just participate in threat intelligence. That, that's a critical component to any cyber resiliency program. Um, be it something like a threat intelligence platform, again, such as perch or somebody in the market like that, but also participate in threat and tell peer groups, right?

So there's plenty of peer groups, there's us certain CSA notifications, all sorts of resources out there that provide threat intelligence know what that information is. Because again, that, that starts the whole process over again.

We can now be, again, begin to predict current higher level risks that come out through cisa, for example, I'm gonna take a shot and predict that, hey, that now is probably a target and I'm gonna run through the next exercise on what would we prioritize, how do we address it, and let's practice it. Gary, um, I'm gonna come to you.

I, I know I said, oh, I'm gonna go west first, but just because of what Mike said about, you know, knowing, you know, knowing what you have, understanding when something happens, what's most critical. And you know, like our good friends, uh, Ken at networks talks about, um, the crown jewels, the data. You know, I'm, I'm scratching my head here 'cause we always talk about, you know, packaging and pricing.

We talk about all in seed price are how is it that, you know, we're gonna have to get our clients and prospects to understand like, look, this is now, you know, we have to assume that a breach is going to happen at some point in time. What systems, what data is most critical? And and how do I get you to see that, Hey, I I need to be able to do data classification, data identification, I need, you know, we've gotta work together on the same side of the table. Am I, does that that make sense?

Yeah, yeah.

Absolutely, Andrew, and the concern that I have is when you look across the marketplace at the percentages of the number of MSPs that like, have less than a million dollars or, or a million and a half dollars of revenue, when you think about what that business model looks like today, what the pricing for many people, and that looks like, um, and you think about today, everything we talked about and we talk about every week, uh, you ask yourself how, how, like all the things that you mentioned, like how do they have the resources, uh, to be able to be able to do that?

And it's like leading, I think, and this might be a good topic on its own for a future cyber call, is what will be, how will this change our industry over these next few years? You know, because it, because it will. Mm-Hmm. So in answer to your question, yeah. If we're telling our MSPs, at least start with those things. You gotta, if you're not gonna do anything else other than the basics that everybody does, you gotta look at your own data.

You gotta look at your own systems that touch those clients. You have to look at those four or five, and it's probably four or five or six things, and you can't afford not to spend your time on those things yet. And before you do, you can't really help your customers. Like you have to reduce that risk for them, right? Does that make sense? Well, it's almost like what Ryan, yeah. I mean, it's, yeah, no, makes a ton of sense.

It's what's Ryan's saying and, and, and really, really good points, Gary. I mean, this is the, you know, we, again, we've talked about the sta but the, and you know, kudos to everybody on this call because these are the guys and gals that are practicing. I can guarantee it, you know, building an, an internal security program, et cetera.

Wes, um, to you a little bit outta line in the questioning, but I'm gonna bring up, you know, CSF cybersecurity framework just for a moment and posture it to you to say, Hey, you know, is it possible that MSPs are really only selling half the stack? And what I mean by that is, you know, for years, most of our stuff, if we, and we look at our, our stack, it's around prevention, right?

It was firewall, it was all the, you know, we, you look at your stack, it's preventative controls over the course, since you guys have come on the scene and you know, pretty much, you know, alien Vault was the first, but you guys really got the MSPs engaged in detective controls. But if we're gonna look at assumed breach, we're into the respond and recover controls, those are not part of packaging and pricing for most MSPs. What are your thoughts on that?

And, and, you know, what would you be saying? And, and Gary, you know, would love your thoughts on that. I'm gonna hand it to you guys to take from here, but I, but I see a huge opportunity here with that. I'd loved your thoughts and well Take it around the way. Yeah, I, I have a lot. Let's see if I can actually summarize all of this into one. Um, so we, yes, the reason perch kind of came to market for MSPs is we were doing this for big banks.

And what, we don't talk about this a lot, but, um, Aaron, my CEO was involved. He was one of the creators of Sticks and Taxi, which drives all threat intelligence sharing that anybody uses today. Government, uh, NGO private, whatever it is. We, we've been involved in this for a really long time. 'cause we saw it as a problem. And so perch was, we brought that to market in the MSP space because we see the value and the need for doing detection.

But, you know, if you guys go back and look at the cyber call where we had, um, Sunil you on and he shared a cyber defense matrix, we run into a problem that the further we shift right into detection response and recover, the less tool dependent it is. And the more, uh, people dependent, it is not to say that tools aren't involved, they are, uh, but it becomes very, very, very complex and very, very expensive from a TCO perspective. And so how do MSPs begin to walk down that?

Well, I think these are good experiences for all of us to understand. There should be some processes and procedures in place that have nothing to do with tools. Um, it, it involves risk. It involves, uh, things like what Ryan was just talking about. Like how do I actually go back and ask vendors, and which vendors do I ask and, uh, how does that apply and weight into my third party risk management? So those are a lot of things that exist inside of there as well.

But to be sure there is, uh, there is use cases in need for technology to drive some of this as well, especially in the response and recovery. If our response and recovery is literally just, you know, a hope and a prayer and cyber insurance and that's it. Uh, we're, we're definitely gonna run into problems when disaster strikes. And so we need to go beyond what we're currently doing.

But I do think the journey naturally begins with identification, then does shift into prevention and then goes into detection. And not to say we leave those other things behind, but we just build the maturity flow through in our security offering, and then we can get to response and recovery. And not to say that we don't start with some of that as well. We all have BDR and things like that in place. But, um, those are some things you really need to think about.

And so maybe we need to have some additional cyber calls. I I'm looking at the chat now. I'd be very curious to know, do you guys feel weak in those areas of detection? I'm sorry, of response and recovery, especially with, you know, the light of what's happening right now? Or should we, um, or should we not? I'm very curious, and Gary, what are your thoughts overall? Uh, uh, you're definitely gonna hear that overwhelmingly that people feel weak in those airs.

We learned it like when we did the, the tabletop, like people just was an eyeopener for people. 'cause I think what you're saying is, is the opposite of how MSPs have evolved. They've just, they've, they've started with defense and kind of worked the opposite way, right? As they've matured, and you're saying, if you're gonna start today, you would move from, you know, left to right instead of right to left.

And that's a different way of thinking in all aspects in how you build your security practice, but also in how you go to market and what your conversation is with customers and prospects to get them to think in a different kind of way. Wes, does that make sense? Of course it makes sense. Yes. So Wes, I'm gonna hand it to you. Can you talk to Mike A. Little about, you know, capability maturity? Because I want everybody out there to hear, you know, from an MSP, um, like Mike on this subject.

Yeah. So Mike, one of the reasons we wanted to bring you on the call today. Um, a we we wanted to see your smiling shining face, uh, but B Mike, you know, I have a ton of respect for you and for Marco and what you guys are doing. Can you just talk to us a little bit more about this approach specifically, you know, if you, let's, let me just play a scenario, maybe this will help.

Let's say, Mike, you went to a much smaller MSP for those of you don't know Marco's like, what, 14, 1500 employees, quite large. Um, if you were walking into a much smaller MSP or starting your own, how would you guys go through this journey that we're talking about today? Like specifically, you know, just around, um, detection response. How would you just, how would you, how would you start approaching this? What advice would you give us? Yeah, great question.

Um, you know, it's, uh, I'm a little, I'm gonna be a little jaded by reading some of the comments on the side. I, I think a few people are hitting it, right? It's understanding your customer segment and what they're after. And I think Chris said it, you have, you have to step into it, right? So it's coming to market with a package, understanding where you're weak, and then you have to sell to the needs of it. I think there's a lot of ways to do that.

So if I started my own organization, I, I'd like to say I do it all right to begin with. 'cause I've been in the security space a long time. But that doesn't mean anybody's gonna buy it, right? So I, I think there's that balance. Um, identification is, is honestly that, that's a key part of it. And I, I constantly see that as a weak area in our industry as well. We're really good at identifying and managing workstations and servers. We're not always great at identifying other components.

Um, IOC type devices, um, iot, excuse me, that, that kind of thing. So that would be one of the areas that I would, I would focus on first. We have to make sure that identification is covered protection. Most organizations understand protection. They may not be doing everything right. They may not have everything multifactor. I think we talk about that a lot, um, is typically still a common miss, but protection is there.

It, it's, again, outlining those best practices and, and building a platform to adjust, um, and really provide those services moving into detection. In response, though, I, I would bake it in, to be frank, if I started my own MSP, I would not sell a solution that does not have some detection capabilities in it, predominantly log capturing, um, the ability to gather SIM type information and pull those logs out.

And then from a response perspective, if I started a small MSP, I'd go the way that, uh, a lot of us already do today. We'd out, I'd outsourced it, I'd hire, I'd have a good platform, and I'd leverage their services as part of the response such as a perch or somebody like that, right? Where it's, I don't have to build the bench out in internally.

I can use it when I need it, but I think you do need that log, like that log type information's critical one, it checks off a lot of different customer segments environments. But two, again, it goes back to the me as an MSP, when something happens, I need to be able to identify it quickly to be able to contain it. And it goes back to cyber resiliences containing, but allowing the business to still continue to operate. I can't do that if I don't know what's going on.

So it's that identification and then gathering that detective information that really is important. Protection, you gotta balance, that's always gonna be a business decision. And I think we as MSPs need to understand that, that it is a risk-based decision. Not every organization needs the same protective controls. When something happens though, there is a very, very common playbook that every organization follows. Right. That's awesome. I hope everybody heard that, what you just said.

And maybe a, maybe a quick follow up to that too. Do you guys, and it's okay, you, you can be as honest as, as you, as you need and want to be here, uh, Mike, but you know, what's, what's your feel as the CISO at Marco of your guys' maturity around and assume breach mentality? Do you guys do a good job of that? Could it be better? What advice would you give to others? Yeah. Um, it, I guess I would say it starts, it, it starts somewhere, right?

So in, in our case, it started in our security group. We established a security group and have grown it. Um, I think an area that we can improve, I think it could be improved. I guess I'll start with that. I think it's one that it, it's constantly evolving. We're evolving to different threats and everything else, but we also have to integrate that into the business, right? We, you know, Ryan hit that. It's people, processes and technology.

We've done a really good job of establishing those elements in our group. We're still going through the process of integrating our, the things that we do into other business units. We have a group of a couple hundred people that manage all of our customers. I guarantee you, not every one of them understands what we do. That's an area that we're continually looking to improve. And that, that's a goal. To be frank, it's a goal for the next year. Um, but yeah, it, it, it's difficult.

I mean, the stuff changes so fast. We're constantly busy doing things, and then to have to grow it throughout the organization, it's a, it's a challenge. So that'd be my piece of advice, I guess to MSPs on that. Don't you? You gotta start somewhere. Somebody's gonna get good at it. It could be one person. That's okay. That's a natural progression. Got it. And maybe Ryan, a question for you.

Um, you know, we talk, we do, we do, um, talk a lot about like, you know, or maybe we haven't talked about this enough on the call, but let's think about this partner versus build, you know, tho those kind of like, how do we approach this from that per perspective. Um, do you think though that MSPs need to have command around of this, or at least some degree of command? What do you think? That's like the million dollar question, right?

Um, what is the role and responsibility of an MSP when it comes to building cyber resilience for themselves and for their customers? I don't know that it rests solely with the MSP in terms of achieving it or, um, I guess maybe stated differently. I think you would, an MSP would not be exercising due care and due diligence in managing their customers if they weren't thinking about this. Now that doesn't mean that they're responsible for implementing it, right?

That's why we see 50% of MSPs now partnering with Ms. SSPs in order to deliver that capability for themselves and for their customers. So I think you're negligent if you're not thinking about it. But it's not necessarily all on you to go from zero to five on a capability maturity scale when your head's still spinning, trying to like figure out like, how do I enter this cyber cybersecurity space?

Like, sometimes the best thing to do is just go buy maturity and partner up with someone that can give you there more quickly and help you learn along the way. Yeah, that, that's really good. Um, I have a lot of thoughts about this, but first, Gary, what do you think? Let me turn over to you, Gary. Yeah, I, I mean, listen, where we are right now, there's a big por a big portion of the MSP market that's gonna have to do one of a couple things.

They're gonna have to mature and they're gonna have to mature quickly, may or may not be possible on a case by case basis. Uh, they need to, you know, partner up to go get some of that maturity, uh, uh, what short or long or long term or the, you know, the, the next alternative is they maybe get to the point where they need to merge or be acquired by someone you know, that already has it or do nothing, which is not right now a good alternative, right?

Because the SMBs every day are being more educated. And if you haven't been asked the questions that you don't want to be asked, okay, like you talked about now, Ryan said, ask your vendor questions. The MSPs are gonna start getting asked. I can tell you 'cause I get a whole peer group that we're training them on how to go out and have prospects ask the right questions, you know, on it. Are you with me on this? Yes. Yes.

Hey, just quick, um, highlight on that, Ryan, he, uh, gosh, I'm gonna butcher his name, Ryan, it's still root. Um, I'll try and get the, uh, article out there, but we're gonna have Ryan on. He wrote, uh, Huron, uh, he, he wrote a fan fantastic article. Gary, you pointed it out. We had it in the cyber nation on the 21 things that every SMB should be asking their MSP. And uh, so we're gonna have that on, but it's a great guideline for that.

Gary, just, uh, as we turn it over to you, um, maybe think about, you know, partnering with an M-S-P-M-S-S-P as Ryan said, but what a great kind of reframing type of thing. Even if you were in an MSP and in a, an MSPs in a customer or a prospect, they're saying, no, no, no, we're not gonna spend this money. Say, Hey look, let me take myself outta the equation.

Typical scenario is you look at it, what an MSP partnered with an MSSP to deliver this capability, hypothetically all in seat price is $300 with that combination. You know what I mean? So maybe that's another approach as well. But with that Gary Floor zero. Yeah. Yeah.

And, and so I want to ask a couple questions here, people, but, um, look, if that's where you are, if you're still at that point that we've been talking about, like for, since we started this about struggling with price and value, we, we gotta get moving past that pretty quickly. Like we, we gotta get onto the next thing. 'cause there's a lot of actual work to do as, as we're on that, that you have to get to. So you can't stay there.

You've gotta somehow get your mind past this enough has happened just in the last year, that should give you enough wherewithal to have different conversations around pricing. I mean, if you think about what we talked about every week, it's ridiculous to think that, that an SMB would care whether they spent 4,000 or 5,000 or 5,500 Andrew. Yeah. And every MSP, you know, needs to get there and they need to get there, uh, fixed. Mike, I had a question for you.

I want to talk about out, we get to talk to you, right? The ciso you have, you know, you have leadership, right? In your company. What's the mindset of the business, you know, leadership in terms of getting to like an assumed breach mentality and all that, that means, like with the investment that your company has to make? Like how does that start at the leadership level? Yeah, it, uh, that's a, that's a great segue. So, um, we look at things typically one of three ways. It's an investment.

Um, it's a growth area or it's a, uh, efficiency gain. And security is absolutely an investment, right? So we take that approach going into it. There is strong commitment right now to this, I think, and that, that comes through education. I mean, it took a couple years of education to really help understand where that's at. And, and we went through an evolution, just like everybody. We had assessments and audits and all sorts of things, and, and we built a plan, um, through that process.

Somewhere along the line, everybody got on board with it. And I think that came through, um, just establishing trust, right? It's no different than any other relationship. Yeah. To the point that my, my board, and really my executive group does have that security first mindset. Now that they're asking the questions, are we secure if we do this? Do we know if we're secure? How do we evaluate the risk? I'll be upfront, risk is, risk is where you wanna steer the conversation.

I think for it to be effective, what is the risk in doing that? Um, again, we may take some risks. We may take, we may not in some, in some cases as well. So great statement there. But what I mean by that is we, everybody's got a finite budget. We can only do so much at a time. What risks are we gonna address this year? What do we do next year? What does that three year plan look like?

As we've built out that plan people have gotten on board security is one that, I guess the word of caution I would have is you gotta look backwards a little bit in what you've done as much as you look forward. Because if you don't, what ends up happening is that leadership group feels that, geez, this is just nothing but a money hole. We're never gonna make any progress. We're never going forward. So take a second look back at, look at the things we've done to get where we're at.

So do you start with, do you try to start with making those investments in, you know, your ability to respond and restore, like in prioritize those, because ultimately you think that's where your biggest and then your customer's biggest risk lie. We, that that is the approach that we took. Yes. We, um, our, in our third party assessing process, um, with the, the group that we use, that's where we have the most significant impacts, um, year over year and is incident response.

Um, so detection and respond. I, I think it's really important that all that everybody here, all the MSPs listening, you know, kind of hear that. Um, right. It's little, not exactly the approach that's been taken by most. And so Andrew, I think that's really, really important. Well, I think it's important.

I, I think Ryan wants to say something, but I was, as I turn it to him, Gary, I think what's really interesting is having these conversations with prospects and customers, you know, and assume breach mentality, alea, you know, what is it gonna take to get leadership on board? How are we gonna manage risk?

You know, and instead of, you know, hey, we need to add this, we need, like, yes, there's time and place for that, but I think it's also how are we going to change the relationship as the MSP with our customers and prospects? Does that, does that make sense to you? Yeah. I'm saying you gotta go in with that new kind of pricing and tell 'em it's still not enough. It's what's reasonable. And, and have that conversation just the way we are today.

Like, so that they understand kind of the framework and that in that assumed breach, here's how we're, we're prepared and here's how we're gonna prepare you. Yes. Uh, that's not a conversation most MSPs are having with customers. So huge competitive advantage. I agree. I think it takes away a lot of the FUD factor and goes right at the heart of it. Ryan, you had some thoughts? I think I, I mean, I, When Mike was talking about advertising wins, like I just, I couldn't agree more.

Like, one of the things that we're horrible at is communicating our wins because we're cons, we're downside risk people. We're constantly looking at where the next loss scenario is going to come from. That we do need to look back and even in real time, like you're building out a new part of your program, communicate those wins to your MSP, to your leadership, to your MSP's board, to your customers.

Show them the return on that investment so that they understand that it's not a money pit, that it is, you know, driving good outcomes. Right. And you'll, those conversations about investments for future risk mitigation become a lot easier when they have confidence that you're putting that money to good use and accumulating those wins. Yeah. That's excellent. You know, and, and again, Gary, you talk about this too. This is what, you know, our, our clients look like.

You know, this is why, right? This is the outcomes you've pre you've preached this for years and years and years. Um, so, um, And Andrew, before we get to the end, I just gotta say, you know, for you and I, we come at this from the business and MSP standpoint, right? My life is MSPs and their businesses. Uh, to be able to have people that had the deep knowledge of these issues, like Wes and Ryan and Mike, is just awesome.

I, I hope everybody listening, you know, feels the same way I do, that we're just really fortunate to have resources along with us along this journey like this. Oh yeah. It's, it's phenomenal. Like, I get, like I said, you know, I'm come back to Ryan kicking this off. That was awesome. Um, so I, I had this question out there. I think it was for you, Gary, but, but, but, or actually for Ryan and, and, and Mike.

Um, but I'll just kind of tee it up and maybe we can hit just a question or two again, Mike, you and Wes just did a tabletop virtual, right? 'cause of Covid. If it wasn't Covid, I know you guys, it would've been a big event in person. Walk us through that and tell us about how it worked with customers and prospects. 'cause we're here we are. Ryan brought it up last week, you know, how do we get into a respond and or, and a recover type mental, you know, mentality and, and what pieces?

The Well tabletops a big one. Gary, we're definitely committed to doing that Q1. We're already working on those dates. We'll let you guys know. But Mike, talk a little about that. 'cause I think you guys said it was, it was pretty awesome, wasn't it, Wes? Yeah, it was. Um, it was great. We had I think 150 people or so on. And, um, we walked through it. It was, uh, an what I'd call a narrated tabletop. So we had, it was very predictive in what we did.

And we of course took a not ideal solution, um, path through it. Uh, but the education factor on it was phenomenal. And, and we had a interesting group of folks on the call. We had everything from large organization security analysts that very much knew what they were doing and probably could hold, hold me accountable to CFOs, CEOs. And, and I would say that finding that right balance was interesting. But we found it because the follow up that we had from both groups on, Hey, can we see this?

Which by the way, it's out on YouTube. Any, anybody on here can go watch it. Let's look for Marco Tabletop and you'll find it. Um, it was fantastic pulling everybody together. That's something that too many organ, nobody knows how to do it. In the SMB space, to be frank, all of us recovering bankers get it. Um, 'cause we've done it before, but to everybody else, it's really a newer foreign concept.

So seeing one done, I guarantee you out of that 150 of the, the folks that attended, probably half of them will take that back to their organization and, and take action on it. And I say that with confidence 'cause probably 50 of 'em have already reached out. So, so real quick, Wes, I wanna hear from you, but Mike, will that turn translate into money for, for Marco? Oh yeah, absolutely. It already has.

It's the, it's everything from, Hey, can you help us write a plan to, can you identify where we're weak in our plan to? Can you walk us through this same exercise with our leadership staff, right? So a lot of avenues that can go and, and again, it opens the door to really everything. That's why I love, love security events. They start with identify and end with recover and you get everything in the middle. Yeah. And what's great about it is you can make it a hundred percent content.

All you're trying to do is help those people and change their perspective. And the business you get is just a byproduct of it, right? And that is the best, always the best approach to get good qualified people who want your help. And that's really what we want from a sales standpoint, Andrew. Like if we can spend less time convincing people and more time educating and helping them, you know, get where they want to be, that's the best relationships to most profitable relationships.

Well, Les I'm gonna ask you to kind of close out last point here, but as I turn it over to you, Gary, you're a hundred percent correct 'cause what we talked about earlier on, just what Mike was bringing up, is getting leadership involved. This was an educational type of event, fair West and leadership's involved saying, Hey, can you help us with our plan now, which happens to be governance and policy. Is that right? We, Yeah, I mean absolutely it is.

And uh, just freebie for anyone that wants it. If you work with financial check in chat, I just posted a link to FDA's tabletops. Um, some of 'em are quite simple, but yeah, it's like what Mike said. Um, in fact, let me just use, I'm gonna steal David Powell's examples. So David talks about this a lot. If you guys know David, um, you know, he says, you know, I can walk into a house and I can say, Hey, you need to replace like all your walls that's covered in mold.

And they'd be like, okay, cool, I'll budget for that. Uh, let me know. It's gonna be maybe next year. But if I actually go cut into that wall and I peel back that drywall and I show them all the mold that's in there, what are they gonna do? They're gonna say, oh my, we've, we we're outta the House. I'll find the money. We We're, I'll find the money we're gonna hit in a hotel until it's done. And that's the purpose of a tabletop is not to cause scare. You don't have to cause fears.

No need for that. All you need to do is say, Hey, how can we use this to identify where some of our gaps are going to be? And I did that at the bank that I came from. It's extremely valuable to say, how would we stack up to this? What are some of the deficiencies? And it's not always about technology and it's not always about getting a new sale. It's about being someone that's front of mind as a true VCIO for those partners of yours so that you can show the gaps.

And what will happen is people will come back. So Mike and I know this, looking at the chat that was falling through in the WebEx as we were going through all of this, you know, we asked how many of you guys are confident around multifactor and have it deployed everywhere because that was a piece of the tabletop and many people said, um, not at all. Or I only have it for like IT people or VPNs or whatever it may be. But cloud, you know, access. No, and that's a huge deal.

And I promise you Marco probably got an onslaught from their sales guys of, Hey, how do we do this MFA thing, Marco, make this happen for us, right? I mean, that's great. That's, but we're not doing it to drive a sale. We're doing it to test maturity and where we're at. And the tabletop is great to show those things. Um, and, and that's the value. That's truly the value.

Well, I think that, that you kind of sum summed it up right at the top of the hour here, Wes, because really that's what cyber resilience is, right? You're showing them, Hey, we are living, breathing in the simulation and event. Can we still be operational? And I bet a lot of people were sitting there going, man, we're in trouble. Or, you know, maybe we're not in deep trouble, but we need, you know, help here, here, and here. So, awesome job.

Um, Ryan, Wes, Gary, Mike, man, what a fantastic, um, job you guys all did today. Folks, thank you so much for joining us. I think what we'll do, Gary, and and team is one more for the year. We'll do next Monday, we'll take a few weeks off with the holidays and we'll come back on it strong, uh, in 2021. So, uh, closing comments from anybody or should we just wrap it up and let people have their day? I have a quick said it all. Go ahead. We, I have a very quick one.

So one thing that FireEye said in the report that I thought was good, another way of looking at, uh, this whole topic is what they call persistent defense. I like that term a lot. I haven't ever seen that term before. We talk about persistent threats, what about persistent defense? Uh, so just thought I'd throw that out there. Awesome. Well, you guys all have a fantastic week, both the folks in the, uh, uh, audience. Thank you again for all your support in, in 2020.

Looking forward to a awesome 2021 we're gonna do next week. As I said, we'll be planning tabletops coming up in Q1, Gary, we'll get those out. We're gonna be doing them with uh, Chris Lair who is out there who won't respond to any of my emails. Come on, Chris. Very Busy, passive aggressive, Very passive aggressive. I don't know man. He pretty much texts me every day. I don't know. It sounds about right. Alright, with that guys, thank gals, have a fantastic, uh, week. Take care everyone.