April 12th, 2021

All right. Welcome everybody. Week 45 here on the cyber call are joined with Wes Spencer, Eric Till, and Chris Lair. Hopefully Ryan Weeks shortly. Um, but hope, uh, everybody had a fantastic week. Took announcements. Um, next week Brian, let me get him and bring him on in here. Can you believe we're seven weeks away from a year, Wes? Seven weeks. Hey man, that's awesome. I remember we had this idea and uh, is your idea, Andrew? And I was like, man, this is ambitious. Let's see what happens.

Yeah. And here we are with almost 2,800 folks on the call. Isn't that awesome? I Know, I know. Can you guys see me on video? No, man. We just see the Cyberon logo. You're our invisible host, Which is not, which is not terrible. No, it's probably good, Chris. Yeah, probably good for you. Never look better. Well, I have to hear too much from off, so, uh, quick announcement next week. Just talking to Ryan and Wes offline. Uh, I'm gonna put something. Oh, good Tim. I'm glad you guys see me.

Um, anyway, I'm gonna put something in chat. It's from Mitre and this is kind of interesting. This just came out and it's um, this, uh, this uh, group called Menu Pack that did a lot of the attacks, not just on IT service workers, it talks about others, but it just was, was released. It goes through, uh, the Mitre attack Ryan kind of stuff as well as Wes on TTPs. And ideally we get Force Carver. We can't promise it, but we're gonna try and get Force.

And Wes, this coincides nicely with your threat report that just got released. How is that coming along? 'cause I know it just, you guys are just Yeah. Starting out there. So, a couple things on that. I'll, I'll post a link here in a minute, but we're doing a webinar, we're doing two of them on Wednesday. We're gonna actually talk about the threat report that we just finished. Um, I did some interviews with Channel Futures, Joe Penn and Terry, and a few other news outlets as well.

Um, so you'll see that kinda hit the news cycles as well. Um, but we are releasing it on Wednesday and we're gonna do a really cool webinar on it. So I'd love to see you guys join. Fantastic. Ryan, any comments on that? I know I'm putting you on the spot here. I sent you the, the URL, but in combination, did you get a chance to look at any of the, the, the info in that Mitre, um, write up?

Yeah, I mean there, I think there's kind of two threat actor groups that they have written up that have focused on IT service providers. And, um, you know, while it's not comprehensive by any means, um, you know, it's a, it's a, it is a sample set. And so it's, it's interesting to look at both of them in terms of what the common TTPs are that they're using.

Um, and that's another kind of way that you can look at your security program to say, now do I have the capability, the people process technology in order to detect these things? So it's another great thing. I'm I'm, I'll be excited when we can get more threat actors mapped into it, um, for sure. But, you know, we use, um, we use the AB T 28 1 all the time internally. Got it. So it's, it's definitely useful. Like it's not, you know, it's not just a cool thing to look at.

Like if you figure out how to instrument it, it can actually be pretty powerful. Fantastic. Okay. So I think that is that, and then I'll put up our first poll question, and I'm gonna be handing this off very quickly to actually, um, Eric and Chris who are playing Gary, um, Gary, uh, was supposed to be here, had a flight change, and cannot unfortunately, uh, Eric had a great response to me this morning when I said to him, I'm like, yeah, Gary can't make these.

I'm like, I'm starting to take this personally. 'cause Eric's been with us for two weeks in a row. Um, so, um, you know, let me just quickly set the stage and then I'll ask Eric and Chris to briefly intro themselves. Uh, for those of you that may not know most, most of you know Chris, um, Eric came on the first onto the show last, last week for the first time. But, um, just let me kind of tell you why I was thinking and, and why I thought this was a, a, a good week for this.

So Eric came on last week, we looked at the legal around, you know, MSAs and, and we had a lot of great conversation, a lot of great feedback, awesome chat on that. And I get to fortunately get to talk to Chris quite, and you know, I, I talked to him about, you know, what's going on with these cases with MSPs and then thought back to what Justin Remu, uh, was on recently, uh, talking about cyber insurance and how attorneys, um, how they cross examine the MSPs and what they're looking for.

And it concern was, I don't want our cha, you know, the, our people here that are with us to be easy Pickens. That was really my, my, my concern. And I thought, what better, two better people that we could bring on with Eric and, and, and Chris and kind of walk through these legal implications are pulling ir, handling maybe your own ir but also maybe if you think you're into that business like some MSPs are.

So that, Wes, that was where I was coming from, and let me know if that makes some sense to you. You mean to me personally? Yeah, that makes total sense. Yes. Okay. So, um, so with that, Eric, um, thanks for joining us again. Give us a quick intro, Chris, I'd love it from you even then if us for the first time, don't know who you are. And then then, um, ask that, uh, Eric, you, you kick us off with the first question, but Eric, please, floor is yours. Fantastic. Thanks Andrew.

And thanks guys for having me again. Um, so I have been a, an attorney for about 23 years, 22 of them acting in the capacity of in-house counsel to technology services companies. Um, I did it for on behalf of my own company for, for a long time, had a successful exit from that company, did it on behalf of Logic, a, a huge global player in the industry. Um, I was their general counsel, chief risk officer, corporate secretary, et cetera.

Um, and now I am, uh, providing legal services to, um, MSPs and other, uh, technology services companies on a fractional basis. People who might not need a full-time attorney, um, but they do need regular assistance with, with any part of their business. Um, that's what I'm working on now. So thanks again for having me. Yeah, it's great to have you Eric. Um, and as, and as a personal note, um, uh, Eric, uh, and I go back 17 years, he's phenomenal.

If you need any of that kind of stuff, I'm already hearing great feedback from people that have reached out to him. So, Chris, the floor's, yours, a little bit about you, my friend, uh, Chris Lair with, uh, solid security. Uh, we do, uh, we're based in Austin, Texas. We were acquired by an insurance carrier by the name of CFC in October of 2019, but we're still known as solid security for the most part.

Uh, we've been around for 18 years, started doing cybersecurity for community banks around the country. Uh, and then a few years ago, uh, we have to get involved in an incident response case, and it grew very quickly and we, and we grew, uh, incident response practice out out of that as a result. And I spent a lot of my time, um, and efforts and blood and sweat, uh, dealing with incidents and primarily in the, in the ransomware and, and live attack scenarios.

Uh, we do have a team that focuses on business email compromise and they do a great job. And I, I stay on top of that, but I do not have to deal with that as much in person as I do with these, uh, pain in the butt ransomware attacks that we see around the clock all the time. Thanks Chris. So, So Chris, let's talk about those ransomware attacks and, uh, and let's kind of start at the end of the story and, and, and work our way back to the beginning.

Let's say that, that I'm an MSP and my client gets a ransomware attack and I'm doing my job. I have great backups. I wipe the machines, I restore everything, the client's back up and going, uh, and then a a short period of time later they go down again. Um, tell me what went wrong. Tell me why what I did probably wasn't the the best approach. Yeah, this is one of those that I like to say it was the best of intentions, but, uh, the worst of results.

And we, we see this quite often and what happens is, in today's world, knowing what happened and who the threat actor is specifically helps with the situation a lot. And so, first of all, when you wipe things and, uh, even just power things off or shut things down or reboot things, uh, you've lost a lot of the data that's extremely valuable in these situations to know who did the attack.

A lot of times we know who did the attack, we know the probability of them actually, uh, getting in there and stealing or exfiltrating data. We also know their capabilities as far as installing, uh, additional malware back doors and those types of things. And so, um, and we've heard this quite often.

I was on a call just earlier today with a, with a very large insurance broker up in Canada, and they were citing this exact example of where they had a a a a, a policy holder, call them their MSP just did that. And a week later they got re-encrypt. And it was actually more frustrating than the first time around because here they thought everything, they were in the clear and, uh, unfortunate that they were not.

And so, and then you can speak to as well as that, um, you know, you can speak to what happens when that evidence gets destroyed and, and what kind of woes that can, uh, cause the, the legal team, uh, with regards to a lack of evidence to support forensic investigation. Yeah, absolutely.

And, and, and that's something that, that you learn early on is, and, and, and we, we used to call it leading with forensics and, you know, we can't lead with sales, we can't lead with the technical team for the recovery. We have to lead with forensics because we have to preserve everything that's there and, and why do we have to preserve it? Because we might need it later on. Regulatory agencies might need it later on, the customer, the MSP, someone might need it later on.

And, and as you guys know, once it's gone, it's gone and, and it's not coming back. Um, so, so leading with a forensics first approach in documenting every single thing, every step of the way, every conversation, every phone call, almost every keystroke, um, is of utmost importance. Yeah.

And, and, and the other thing to think about, and, and I'd like to hear your take on this as well, Eric, is, you know, most MSPs do not have the luxury of having in-house counsel or some, or an attorney within arm's reach for everything that they do. Uh, maybe they find it just cost prohibitive or, or, or yeah, that's probably the main driver. They're just not big enough to have that yet.

What are kind of the legal and regulatory exposures for these MSPs if they do go and, and start doing things on their own, uh, without taking those things into account? Uh, so there's a lot of exposure there. And you know, one of the big things is if you've been subject to a breach and, and every state defines a breach differently, um, but in at least 48 states that I'm aware of, you have the affirmative duty to notify the state's attorney general.

And why do you have to notify the state's attorney general? It's not so they can come in and help you with the remediation, right? We've all seen the, the, the joke with the knock at the door and they say, hi, we're here with the government, we're here to help. Um, that's not why they're there. Um, they're there to make sure that, uh, you file the law laws, they make sure that they're, that, that they're there to make sure that you are fined appropriately.

And, uh, but, but it's not just them, it's the regulatory agencies as well, depending on who you are, who your customer is, the type of data that's involved here. Um, it could be the SEC, the FDIC that the alphabet soup of, of governmental agencies. They need the data and they're not gonna pat you on the back for getting the customer up and going quickly.

They're gonna pat you on the back for having the forensics so that you can hopefully share it with them and such that it doesn't have to happen again. Yeah, and I'd just like to add there that that's another point is, you know, you have to report in most cases to the state's attorney general, and that is, that comes back to as very important what is said and what is not said during that.

And, and a lot of times we're talking about the technical acts of recovery or restoration, uh, WIP and deletion, rebooting, all that kind of good stuff. Uh, but also there's the exposure from what an MSP might say. I was on a call the other day and the MSP was just speaking freely. They had been involved for about six days before we were called in and they were using words like findings and breaches and all these things that they absolutely should not have been using.

And so that's another thing is, unless you're kind of very practiced and rehearsed on these types of things on what to say and what not to say, you don't wanna be saying those things because they can be easily taken out of context, especially from an Attorney General's perspective. And there's no reversal, there's no appeal process, there's nothing you can do, um, about that. It, what's said is said and it's on the record. And, um, that can make life a lot, a lot more miserable.

Hey, uh, Chris, do you mind if I and Eric just jump in here? I'm not sure how bad Or good Yes, I do mind. Just be quiet and I'll go ahead, Andrew. It's your show. But, um, Felicia, who's fantastic, she runs a great M-S-P-M-S-S-P, she brought something up, Chris, that you allude to a lot, which is around logging, you know, interchangeable, if you will, with sim per se.

I know it's not identical 'cause you can do, you don't need a SIM to log, but she says, to what, what degree does proper sim logging of events from endpoint network, um, alleviate the need, uh, to keep infected endpoints up and, you know, with the forensic analysis. So I think her point being is like, Hey, look, if I got the, the logging, can I just restore? And if I'm reading it correctly, Felicia, and then Wes, any comments there too?

Because that's your business, Chris, you're, you're, you take You I think it's a good point. I mean, I mean logging, so I I I think it doesn't eliminate the need to still have access to the forensic stuff, right? So you can't assume you've been attacked. We've got great logging, let's go ahead and reboot and restore. But what it can do is it can shorten the cycle on a lot of things. It can number one help with containment and eradication very quickly.

It we can understand, um, you know, uh, like what Ryan mentioned earlier, the TTPs who the threat actor is, a lot of those things very early on with the logging because the information's right there. We don't have to go dig around for it. And forensic analysts don't have to crawl through logs and all the other forensic artifacts they have to do to get to that answer.

But really, really in today's world, uh, especially with file-based, is to, we can understand right away if there was an attempt to exfiltrate or there actually was exfiltration and the most important thing is what the hell did they feel? That's the number one question comes up. 'cause we have a lot of cases we have now the people are able to restore. Uh, but they're kind of in a situation where they're like, wow.

They say they have this much data, they've given us some samples, we don't know how much of that data is. So our forensics team has to go in there and kind of figure that out. And, and if, you know, forensics people that are not necessarily wired and geared to kind of, you know, work with a proverbial gun to their head to hurry up, they like to take their time and make sure things are done right.

So when you try to accelerate that process and have them come up with answers almost prematurely or guesses, that's kind of against common nature forensic people. So when you have that logging and we can get those answers or maybe even 80% of that answer, it can make a ransomware extortion situation much easier to deal with and face and then make some decisions more quickly.

Hey Wes, um, I I'd like to, before we we turn questions over to you, uh, a good segue, 'cause I'd like your comments on this, and it ties in with something that Patrick asked, which is, I'm onboarding a cu a new, a net new customer, right? And you can talk about this because a, I know it's happened with a number of incidences where you guys have been involved with, but b at both enterprise and mid larger mid market with m and a.

You can also talk about why this is also becoming, even though it's not, Hey, I'm onboarding as a new service provider, I'm potentially onboarding a new company. Why is this becoming more commonplace? And again, the the good and the bad by not doing it? Yeah. So first of all, Andrew, do you like my, uh, nano leaf? I use cyber call colors today for it. You might like that it's just for you. Uh, so yeah, a few things I wanna talk about there.

Um, first thing I wanna say is, is Chris is right in the, you know, when you're handling, let me answer this part first. When you're handling, um, you're trying to look at data exfiltration, this is super tricky stuff and you talk to most enterprise data exfiltration is something that oftentimes is done post boom. You guys know on the cyber call we talk about pre-boom post boom.

Um, and the reason for that is things like exfiltration, things like, um, you know, data loss prevention, very, very, very difficult. I mean, it's sometimes all it takes is a cell phone to take a screen capture and how do you really, you know, stop that unless you disallow cell phones coming into an org. It's just, it, it's a challenge overall and it's very much like science.

Like you talk to a scientist and they say, you know, we did this study and a whole bunch of data came back and we see 82% is whatever. They didn't study all of that. They used, um, they used segmentation analysis to say, Hey, this sample size shows me these things and we can interpret, we can interpret these other things. It's very, very common in science. And, and oftentimes in instant response, there can be similarities in some of that.

Like, what I'm trying to say is sometimes we go back and, you know, to Chris's point, having file level analysis and like file integrity monitoring, we don't always use that on the front end, but on the back end, we, if you have it available, it's so critical for us to go back and look and say, okay, well we know these things happened. We, we can't say for sure if that thing happened, but if this happened leading to this thing, we can interpret, you know, what happened in the middle oftentimes.

And so that, that's goes back to why having logs and having data is so important. And not just from the sim but from other things that produce as well. For example, the EDR. Like, that's really, really important for us. Um, so I thought I would just start with, with that piece.

And then the second piece I'll say really quickly just so we can get back to it, is, yeah, going back to Patrick's question of how we handle, um, you know, acquiring a client that has a, a compromise already those things do happen. I've been in the middle of those situations more than once. Um, one thing that I think is important is to have a very clear communication with your client to say, Hey, look, upon onboarding, we noticed these things. We wanted to share this with you.

This is what we think is happening. Um, you, you've gotta be very clear of this didn't happen like a day after we acquired you this, this happened immediately. And so you've gotta have clean onboarding processes that are very snappy and that also teach your analysts to see and to look for things that happen, um, right at that acquisition period.

And so, like I I, we have some very large enterprise think like 200,000 employees that use perch, and that's specifically what they use it for is during their acquisition processes. This is less, you know, MSP onboarding, but more acquisitions, which is very similar when they acquire a new hospital, they, dr they have a bunch of things that happen right away day one, so that they get that look to say, whoa, patient, uh, you know, patient privacy is at risk here, but it's not our fault.

We saw this during onboarding. So it's really important to have those things in in process. Wes just, sorry, le just, I I thought about the video clip that we posted on LinkedIn from Eric last week. Eric, this is what we had the MFA example, and it's in the, it's the client's responsibility. It's the client's responsibility.

Does it go to this as well in onboarding that maybe MSPs have gotta put some language in there like, Hey, if we, you know, come across blah, blah, blah, this is, this is how we, this is how this will be handled. Absolutely. It's, it's, it's utmost importance because if it's not written down anywhere, um, then, then there's question as to whose responsibility. It's, um, if it is written down, it's, it's nice and clear.

Um, you know, I, I know we all like to have really simple agreements with our, our customers and written in plain English, but this is one thing to not skim on. Yeah. Very, very, very important. That's, uh, over to you. Oh, back to me already. Cool. Yeah, keep, so we, we took care of everything with Eric. We did, and Chris, Yeah. So, so la I wanna start with you, my friend, um, because I like to ask you good questions and you typically sometimes gimme some good answers back.

So, uh, let's talk about one thing la I want to chat with you about is, uh, help desk and intake. Like how do we train our, our our tier one guys and gals to recognize the presence or potential presence of an incident? I don't wanna say breach, I wanna say incident like, to make sure they don't just see something, open a ticket, close a ticket, and have no concept of, whoa, that was weird. What happened there? What, what would you say? Yeah, I mean, it's a great question.

I've been given a lot of thought, right? Because we know, I mean, we know when we're dealing with whether it's level one people or, or dispatchers or, or whatever the case may be, uh, you know, we have to keep things pretty fundamental, right? And so, you know, I think there's some things that we can just kind of think about right off the bat, right? Like time of day, day of the week type things, right?

I know we're gonna get pummeled on Monday morning, but a lot of times these attacks are gonna be discovered e either late Saturday evening or early Monday morning. And so those, that's, that's kind of one of the first things. The second thing is, is, is is if you have calls maybe that are from someone that you necessarily don't hear from a lot, and so, um, you know, it's, it's weird, but we could probably draw some of these conclusions up across the board.

You know, we have, you know, we have one of these people that, that get a popped. Usually it's that person that comes in early in the morning, uh, it's always first in the office at 5:00 AM or 6:00 AM and they're the one that that, that finds it right? And you're like, well, why are we getting a call from them at that particular time? And, and, and our it instincts always lead to, it's a technical issue, not necessarily a security issue.

And so we kind of really have to change that mindset and kind of listen to some of these kind of common terms. Hey, my files won't open, my files look weird. My icons are all different. Um, it's just a, you know, blank screen or, or whatever the case may be. Um, unfortunately those things, even though they don't, aren't as descriptive as we would like, uh, we're gonna have to take those into AC to account and then say, well, yeah, what should we handle this a little bit differently?

I, I wish there was some more specific answers. I wish there was something just right, you know, in front of your face from a symptom perspective that could say, Hey, that's an incident and that's not, but you know, and the way the world works, I mean, we've seen some of these variants, uh, these ransomware variants just all of a sudden change one thing and then just affect the machines a completely different way.

For example, you know, for the longest time, one of the customer, you know, say, Hey, we're we're virtualized. Uh, you're running, uh, hyper V or VMware. Well, VMware great, fantastic. We're in good business. Well, because, you know, hyper V because it's Windows based, we knew it was probably attacked as well. And, and VMware at the hypervisor level was not, well, that's not the case anymore.

We're seeing these guys both small and well known, uh, attacking the hypervisor and doing stuff like that. So before you might have heard a VMware problem and say, well, I can't be security related. It's VMware, but not anymore, because you could see, hey, it's a VMware issue. It's quite possible that the, uh, threat actors did something with that. So people just have to be on their toes and, and you have to preach what we tell people, uh, in cybersecurity awareness training.

You have to say, don't punish, uh, you know, and I say that, you know, punish is maybe a, a, a a hard word, but, but don't, um, penalize people may be the better word of, of slowing things down and asking additional questions if they're, if they're on that help desk, say, look, I know we need to help the client. We know we have these quick turnaround times, but there were just a, a couple of things on this call that made me think, Hey, we maybe need to think about this a little bit differently.

And so kind of giving them a little praise for doing that will go a long way. Yeah, that's good. And a couple things I wanna just pull outta the comments here from our attendees I thought was really good. Um, I love what Felicia says. She's like, you know, this is why I don't want to get rid of you, not get rid of outsource, I should say our tier one, because they, they know so much more. They know what's going on with our clients.

Like that's just, they, they understand some of those things and they can detect those oddities. Um, I think that's a good point, right? Like we say this at Perch all the time with our SOC is we're like, Hey, look, when we escalate something to you, sometimes it's just because we need more information for you. It's not necessarily worth saying red alarm, we're just saying, we don't know this. Like, you know, it. Um, that's why you're so important here.

So that awareness and knowledge is, is really important. And then Ryan, I I liked what you said too, um, just about adversary emulation, like this is something I don't think MSPs do because it's not very easy for them. And maybe it some point, Ryan, we ought to do a whole cyber call specifically on adversary emulation and maybe bring in someone from Mitre on Caldera to talk about it because it's really cool.

And if I were to go start a new company, that'd be a really fun one to go start is, you know, how can we bring adversary emulation down market for MSPs to use to test, because there's an end user component to all of this as well. Sometimes the end user is the most important of everybody because we need them to be the ones to raise the flag and say, that was weird. I'm not used to that.

And we need our tier one and tier two folks to say, that was weird on that ticket that came in, or that alert that I'm looking at. Can someone else take a look at this and dive in more? Like, we've gotta set a culture for it. We've gotta set awareness to at least say something happened that seemed to be pretty odd. Um, so, so really liked all that. Um, uh, let's see. So Eric, a question for you. Let's talk about tabletops a little bit because this kind of segues into that, right?

So you've probably done and led a lot of tabletops. You, you know, probably that LA and I like to do them a lot and I can't wait until we can do one in person again. There's so much better than our, um, than our remote ones for sure. But just talk to us a little bit more about from your perspective, um, as an attorney, like how are tabletops useful? How have you been involved in them? Um, and you know, just, just kind of give us your feedback as an attorney on tabletops.

Yeah, so thanks Wes, and, and as an attorney, um, and, and maybe it's selfishly, but, but we like to quarterback the tabletops. Um, I know the tech folks do, and I know the sales folks do and everyone else likes to quarterback, but, but we as attorneys like to do it. And one of the reasons is because of that forensics model, right? Where we, we wanna get, we wanna keep our customers satisfied too, right? It's not just the sales guys that do, um, we want to as well. And, and, and yeah.

So we've, we've been through a bunch and not, not necessarily on behalf of our customers, but just internally, and I'd like to liken it to a, a pit crew, right? A a NASCAR pit crew and trying to write down every step and every motion that a pit crew goes through during a pit stop is great and you have to do it and you have to start there. But if the very first time you're exercising that process and procedure is, is during an incident, during a real incident, then you're in trouble, right?

You're not gonna be able to change the tires and fill the gas and clean the windshield in 12.3 seconds. Um, it's gonna take you days. And that's why we do the tabletops. It's to exercise what we've written down to make sure, um, that what we write down is accurate to make sure that we have printed copies of our incident response plan instead of just copies that are stored somewhere in our network that all of a sudden we can't access anymore.

Um, so that there are a thousand different reasons to do the tabletop and I see in some of the comments. Um, and they're good comments about using the tabletop as a sales tool, um, with our customers to make sure that great they have an incident response plan, response plan, or maybe they don't, but maybe go through a tabletop exercise without an incident response plan and see how that goes.

Um, so there, there, there, there of great importance and, uh, and to answer your original question as the lawyer I'd like wrong. Yep. I i I love it. Um, and I love the analogy I didn't really, given you're, you're really making my my brain go kind of boom a little bit on the pit crew analogy, right? Like imagine even being able to use that is like a, like an analogy with your clients.

Like, it, like pretend me and Matt Hoper for example, uh, went over to a NASCAR race, I don't even know what the races are called, right? And uh, you know, we were the pit crew, like it would take me probably a couple hours to even get the tire changed, let alone, you know, the gas that shoots out like ultra fast, like to even find where I insert the gas into the fricking car, right? Um, and I think that's a great analogy of like the reason the pit crew can do it.

And I'm guessing they do it in seconds, I'm guessing. 'cause I don't really watch nascar, but the reason they're so quick is it's muscle memory. Like, it's not like this, you're not born into it, you know, it's not like, wow, you're such a talented pit crew. Like yeah, I just was born into it and I'm really good at that kind of stuff. No, they do it over and over and over and over and over and then they train, train, train, train.

And I think there's an element in cybersecurity that way that we as MSPs should learn those things as well. Um, there you go. F1 pit stops for two to three seconds. Thank you Kayla. That's, that's unbelievable to me, right? There's an an analogy there, Eric that isn't there if muscle memory and training gets us to that kind of professional, um, a ability, right? Absolutely. I think it's somewhat to be a cardinal sin though, to be from and live in the south and not know anything about nascar.

That's disappointing. I'm a fake southerner. Yeah, you are definitely a big southern. Hey, uh, Eric, I'm really glad you brought up the sales side of things because what a great wedge like in a prospecting situation, Hey, when your current MSP walks through an incident with you, like you're, you know, you actually are locked down, how does that go? How is, how's that planning been going for you? Blank stare. No words.

Ab absolutely, it's, it, it usually is because they don't do it and it's foreign to them. And, and, and frankly when, you know, I joined Logic, it was a little bit foreign to us. Um, you know, we, we really didn't do that. And, and so what did we do? We, we brought in the professionals, um, and, and to, to exercise our, well to create and exercise our incident response plan. Um, and, and you know, we eat our own dog food and, and, and that's how we learn. That's how we sell.

Right, right, right. That's, that's, that's an excellent point. I think, you know, again, we've done a lot of these incident response with Wes and Chris, for those of you out there, um, I forget who had asked that. I put something in there, uh, to Michael and Ryan asked a better question than me just answering it. But, um, if, if you don't have an incident response plan, if you haven't heard Wes and Chris walk through a tabletop.

If you haven't heard Mike Beard and Chris, um, walk through building your incident response plan. These are webinars that are available within the cyber nation, the cyber nation's free, uh, et cetera. So Wes sorry about that. I just thought I was No, Yeah, it's good. And uh, Michael, I see your comment or your question, you know, when is this more than the MSP can handle? Um, oftentimes you need to think about it this way. The second incident happens.

There's, there's a element for almost all of us. It's more than we can handle. Chris, wouldn't you agree with that? Yeah, I definitely agree. I mean, the first thing we find is just the, you know, the, the capability to work around the clock. Um, you know, you're, you just aren't set up to do that. And there's a couple things. Number one is people get tired.

And so when you take it on yourself and you, you have some superstars, and Wes and I have incorporated some of those into our tabletop exercises, they'll fizzle out, their emotions will go sideways and you'll have to deal with that. Problem number two, just like we talked about that muscle memory stuff, they're just not used to doing it. So that's kind of a, that's kind of an issue.

Uh, the other side of it is, is you don't wanna get caught in a situation where you're looking like you are trying to cover your own butt on this type of deal. Yeah. So you wanna make sure things are transparent. The Biggest thing for me is you should have a sense of how long it should take you to move through the various phases of an ir. And if you're missing your milestones, that's a sense that you're in over your head and you need help.

Plus we've had many times where MSPs have been involved and then we find out, you know, just through casual conversation during the event or afterwards that they, they neglected some of their other clients because they were all focused on this one client and this incident. And so it has some, you know, it has some adverse kind of issues there where they didn't really foresee that to be, to begin with. Now they gotta deal with more customers being upset because they didn't get service.

'cause you're too much focused on somebody else and you don't want to necessarily tell them that they were less important than, than this other client was. That's good. Um, maybe my final question is, um, around, so we've talked a whole lot about the right and wa wrong ways to do incident response and you know, um, where the gaps and challenges and problems are for MSPs.

And, you know, if, if I see some of the comments, some of you may be new to cyber call and have not heard some of the incident ones that we've run. When Chris, you've joined us, um, Andrew, I don't know how easy it would be in the future to maybe post into Cyber Nation, like a list of some of the cyber calls we've done with Chris Laer on incident response. But those would be really good reading material just to go back.

'cause you're gonna get a lot of the, the things we're just assuming in today's talk, we're just assuming you know it, you, you just gotta go back and watch those. Same with the tabletops. We've done a bunch of those.

Um, but maybe my final question, we haven't really talked about this as much and I'll, Chris, I wanna start with you, but then Eric, I want your opinion on this too, is the attorney being present in the middle of incident response, how important is that and when do you need your attorney involved, obviously at the very beginning to establish, uh, privilege, but like they gotta be on every single call and, and maybe Chris, can you talk about like, when, when you notice they're present not is like there a difference in it?

Just talk to us about that angle. Yeah, so I'll go first on that one. So typically, initially you want them involved so they can, so you don't have to repeat all the same stuff or for the, or the victim isn't having to repeat all the same things that they had to repeat to you of, of what happened, right? And so just having them involved upfront establishes the team early on.

And so everybody understands the players and who's there and everybody hears the same, the same words coming out the same description about the event. That's the first thing. Most of the time, uh, the attorneys are not going to be wanna be involved on every single thing, especially on the technical side of things when people are talking about workstations and operating systems and all that kind of good stuff. Now there are some exceptions.

There are some attorneys out there with some technical backgrounds and and they like to weigh in and, and, and say their stuff and whatsoever, but it's not necessarily a requirement, but it's what they do. Uh, a lot of these guys and girls and women or men, whatever we wanna say is, you know, they're, they're, they're balancing many cases as well. So they can't stay on the phone eight hours a day or 12 hours a day with these particular things to balance it out.

But there are times where, you know, so, um, depending on how, you know, the situation is when there are daily status calls, you want 'em to be on there, uh, when there's some really relevant, uh, milestone type things being discussed. You, you want them on there and those types of things. And, and really the, the more things you can do verbally in these situations, the better you are.

Uh, a lot of stuff gets really messed up with emails, things are said incorrectly, people mistranslate, they don't have the ability to ask the question right away. So there's a lot of back and forth. And so I'm not a big fan of, of managing things through email. And so getting those things through phone calls makes the most sense. And so you just need to be able to, to know when those attorneys and kind of feel 'em out.

I mean, we work with multiple attorneys and multiple firms and the, the, even the attorneys that are in the same firm have a better way. They have their own way of handling things. So you gotta kind of know each one of them and, and know how they're gonna want to act and react and that type of thing. And Eric, I don't know if you had something you wanted to add to that. Yeah, no tho those are all great points and um, and, and I agree with with every single one of 'em.

Um, I think it's also important to, to have an attorney as part of your incident response plan and not just write down, you know, attorney on line five, but, but know who it's sure. Um, and, and it can't be your brother-in-Law, who's the divorce attorney who's gonna now sit in on these, uh, on these tabletops with you. It's gotta be someone who knows what they're doing. Um, and as, as general counsel, I've been through this, uh, a bunch of times with our customers.

Um, and I know when it gets over my head and I've gotta call in, um, other attorney experts to, uh, to help solve the problems. Yeah, I mean, I mean there, there, there are attorneys that are, you know, pretty easy to work with and there are ones that are micromanagers wanna call everything to the, every word by word, play by play. And so you just gotta handle that and it's frustrating.

But, um, but Eric's right, I mean we are starting to push more people from instant response planning and tabletop exercise to include the proper breach coach in there. Uh, 'cause it just makes perfect sense. Yeah, it's a little bit more expensive, but most of the time, uh, these breach coaches are not charging full rates for that type of stuff because they understand the importance of the relationship and all those types of things.

And so it just makes the most sense to have the right players when, you know, Wes and I come from the banking world and when we did our banking exercises, it wasn't just the IT people in those rooms. We were surrounded with compliance people, b, SA people, auditors, all sorts of our p and usually our GC was in there as well. So, uh, same thing goes in today's world.

I mean we were doing that stuff 15 years ago and it's just now taking hold in, in, in, in regular businesses outside the financial services world. Yep. That's good. So I'm gonna save my time. I'm gonna end here and let, uh, switch over to Ryan. Um, 'cause I know we have a bunch of questions that have already come in and probably many more.

So Andrew, I'm just gonna uh, pause here so we can have more time for, for questions from the audience And Wes and I'm also, I just put in the URL, um, with one of the RM breaches that you and Chris did. I'll, if you guys email me too and I'll put my email in so that's in there right now for those of that. Awesome. And then I'll also put in an invite to the cyber nation and then, um, I'll put in my email. So if you have any questions. Alright, sorry about that, Ryan, go ahead, please.

No, it's fine. So the first question is, I guess more of like a, like trending. Like what, when we, when, I mean you deal with a lot of MSPs that have had compromises or or customers of MSPs that they've had compromises. Are you seeing a trend of increased, um, MSPs increasing like statements of work or, or within their existing agreements to include IR response and recovery? Or are you seeing them like, like do you see more of them trying to take it on or more of them trying to outsource?

Like what direct are you seeing both, like what's kind of going on out there in terms of what you see large swaths of MSPs doing? So, so I am, so it's good news that the MSPs seem to be better prepared to deal with these situations from a recovery perspective. So it does appear that the trend is that MSPs are taking backups more seriously and kind of maintaining those and doing the right things.

The the flip side of that though is, is they're very well, there has been a trend that they're quick to pull the trigger on that restoration and recovery and then they are destroying the evidence or doing things. And so there have been a number of calls where MSP has said, Hey look, we recovered our, our client's environment a few weeks ago and now they're getting notified that they have some information out there on the dark web. Their client information is out there on the dark web.

So they just didn't realize that that's what went on. They just thought that they got, you know, just some everyday ransomware attack and they did that. So that's a, I am seeing more companies and a lot of them are MSPs advertising the fact that they do IR work. Um, I think that's, I that's perfectly fine if, if you can do it.

Um, I mean there's as, uh, example of the recent, uh, proxy log on, exchange half, there's so much work out there that the IR firms we're pretty much at capacity for, you know, for a couple weeks there. Um, but I can tell you from personal experience, when we took it on the responsibilities of being an IR firm, we took it very seriously and we hired, we hired what we believe were some of the best in the business out there at the time in order to do that.

And so we just did not go and, and just take existing staff or pick our most, you know, security savvy person and kind of do it thinking we can get it done. 'cause we'd been through similar situations and saw how it's done by whether it's us, Kroll, Charles River or anybody else, and think we can kind of repeat it. It it is not that way. And I would especially say on the forensic side, it is not that way.

I learned very early on, and I, and to this day, I'm still the same way that forensic side of things is its own beast. And you gotta have people that have had that experience, that have had to go to court, that have had to fly across the world to collect forensic data that have been in those adverse situations to really realize what the worst case situation is. So you can not fall into that pit.

I mean, I find just, there's a lot of firms seeming they wanna do everything but forensics and then punt the forensics off and that that's not a very good, very good experience either. So my point is, is I'm not trying to talk you out of doing it. I want you guys, if you, you, you're capable of doing it and you're capable of generating revenue and it's, especially if it, it helps you generate other relationships, fantastic. Just do it the right way.

Make sure you plan it out just like you would anything else that you're getting involved into. But at the same time, think about what you wanna do with your existing customers. Yeah. And whether or not you think that is the right idea if a customer has a, an event whether you should be handling that same event and whether or not that there's a, that you can explain yourself out of a conflict of interest, uh, argument.

Yeah, I think you, you're, you're really advocating like knowing yourself, right? And so a simple way to do that is inventory the type of incidents you might need, need to respond to, right? Is it a phishing email against you and all your customers? Is it a malware infection? Is it a ransomware attack? And start small, right? And make sure that you know, how you're gonna respond to some of those easier things.

And you're gonna realize is going through that exercise of inventorying and checking your capabilities and your process, you're gonna figure out really quickly where you're not equipped to do some of those larger ones. And that's really where you should snap the line and say, okay, we know what types of things we're gonna, you know, call out up Chris for to come help.

And like, but in the meantime, we're gonna keep trying to mature and move that line further and further and add the right capabilities to do that. But yeah, I completely agree. Never outsource just forensics because you're basically outsourcing the intelligence engine that drives the entire response. And so it doesn't make sense to try to do everything else but not have the core intelligence driving that process. So Yeah. I'll give you another quick example.

We ran into a situation the other day where they had a, they had reached out before they had reached out to, uh, to the insurance carrier side. And the, the, it was an MSP and the MSP had put together a statement work for $180,000. Um, and it was a, it was a smaller company and that I was like, holy, my $180,000 and I don't even think that was forensics. And, uh, we're like, I asked the question, well, why is it so high?

And they said, well, we put it really high so we don't have to do any change orders. Well, that doesn't work because if you know the carrier side, they have to put together a budget to begin with. And if you go out there and set something at $180,000, you end up billing out only 25 or $30,000. That kind of defeats the purpose on this thing. And so, I mean, I, I I, I, that that's the stuff that we're seeing.

We're seeing a lot of people just guessing at what should be done and learning things the hard way. And that's just not the way to do it. Yeah. Hey, Brian? Yeah. Can I, can I just, 'cause Eric has mentioned a few things on something you and Chris just touched on, and that is Chris, you kind of ended that segment by, hey, you know, you're actually auditing your own stuff and, and Eric's mentioned forensics first.

Eric, just, can you just give us a little bit of the legal side to leading people from doing it? If you, if you have the chops, awesome. We're not saying that. Talk to us on the legal side of those pieces, if you wouldn't mind. Uh, so you broke up a little bit, but I'll answer the question that I think you are answering. Um, I, I think it's, it's when do you think you're getting in over your head?

Um, was was that it It was really, you know, audit, you're almost, you know, cat talks in the hen house. Ah, I got it. And then you talk about things like, hey, if you can't do the forensics, and you always talk about forensics first. Yeah, definitely. Those two things I just stuck in my head as Ryan was Mentioning. Yes. And, and they go hand in hand because as an MSP, there is a chance that something you did or did not do has led to this incident.

Um, hopefully that's not the case, but there's a chance. Um, and at the end of the day, um, this, this, this forensic information that you're collecting throughout the remediation process, um, it can be used as both a shield and a sword. Um, it depends on who's accusing you of doing what. And you might want all of this data to say, look, we did everything like we were supposed to do. We, we, we did everything like any other reasonable organization would do it.

Um, and, you know, sometimes it, it comes up where it shows that that's not the case. Um, or maybe the, the fingers pointing at one of their other providers or at the customer itself. Um, so, so there's lots of other reasons to maintain the forensic evidence other than just for the regulatory piece. And, and if you eventually get to court, um, you wanna know the answers and, and if you don't have it, there's a chance that you're never going to know the answer.

And then you have to answer the question of, well, if it's exculpatory, if it's gonna help you, why isn't it there? And, and then the flip side of that, well, if it's not there, maybe it's not there because it was gonna hurt you as the MSP. So it's kind of a, a long-winded way to answer your question, but forensics is important. Yeah. Thank you Ryan. Sorry about that. I just No, it's, it's fine.

Um, so I guess to, well, I don't have a lot of time, so I'm gonna skip over a question because I would like to lose time for q and a. Um, Eric, once, once an event becomes an incident, right? We've, we've, we've said, okay, this is a thing. We've done enough investigation and that's, it's actually a problem and let's assume we have a decent IR plan in place. So legal's involved, we have a communications plan, insurance is notified, et cetera. Everyone's part of the process.

Um, what are the benefits to an MSP of getting this process of incident response out of their PSA? Right? So again, most processes for MSGs are run inside of A PSA. Why might you not want to run an IR in a PSA? Yeah, and, and that's a great question. And, you know, privilege issues aside, um, the, the data here is sensitive and you don't know exactly what the data is going to say. Um, but with this sensitivity, you wanna limit your audience.

And the issue with the PSA is that your audience tends to be pretty broad. Um, and it's maybe not just people inside your organization who can view the PSA, maybe it's, uh, maybe it's your outside accountants or outside attorneys who have nothing to do with this or, or other outsiders to your organization who you just don't want to expose this information to.

So it's always a best practice to, to keep all this and to keep all this forensic data outside of the PSA in a much more controlled environments. Okay. Chris, do you agree? Um, yeah, I agree. And I also would say from a change management perspective, um, you know, the fact that you have a PSA and there's more rapid change in more things that happen in, in that environment that could cause some issues could cause some data leakage to somebody else and that type of thing.

I think, you know, the PSA is a very, very powerful tool and does a thousand different things, which is fantastic. But in this particular case, you just really want that instant response kind of process and, and that, and that environment to be as kind of siloed as possible. So you know, it's there, you know, it's available, uh, you know, that, you know, other entities within the organization that are messing with the PSA aren't actually accidentally impacting that.

Uh, so that, that's, that's another thing that I'd like to add on to that kind of, that, uh, that, you know, that argument about PSA and I just think, again, it kind of comes back to this mentality purpose. I mean, this mentality concept. If you know that everything you're doing is in this particular environment, then people have their minds that helps that mindset get started, right?

When everything's in the PSA and kind of mixed in with everything else, you're, you're, it, it's, it's not as much of a psychological effect as it is when it's outside of that. So I think that's, um, as powerful as anything else we've said here with regards to, uh, doing, um, something outside of the PSA when it comes to handling events. Brian, you got time? Uh, we got five questions. Have you got anything else? Please, please Do.

No, I think we, we, you know, we never really have enough time to get the questions, so let's, let's try to do that this week. Okay. So Wes, I know I've been jittery. Would you mind quarterbacking some of those? Is that okay to put it on you bud? Yeah, yeah, yeah. Hap happily, that's The boy gets quarterback everything. That's true. Sorry. Sorry. You got number today, man. Let's see here. Um, We'll start with the, I like the first one.

So how should MSP, and by the way, you guys on the, in the audience, you can vote on questions. So if you want, like, if there's one that's of particular interest, vote it up. I'll, I'll go top to bottom I guess. Um, okay, so how should MSPs package the right of boom IR services, right? For those of you that don't really mean by that is after the incident happens, how, how would that look? It, uh, it would be easier to estimate and include MRR for left of boom and prepare for an incident.

Yeah, no doubt. Uh, will cyber insurance pay for IR services if the MS P is performing the security response? Okay, some good ones in there. Um, Chris, you wanna take that? Yeah, I'll take that. So, um, I think we're gonna start to see more MSPs be advised to put some language in their MSAs as far as incident response and being, having some level of pre-authorization for an MS P to do something with regards to, to an incident response, to kind of speed up that process and to make things going.

So, um, um, I think that's, that's very important. I think it puts it right in front of the customer's face about IR and how, how serious it is. Um, I know a lot of MSPs do stuff like after hours and emergency rates and that type of stuff. I just think with this in response, it needs to be a little bit more articulated and things needs to, need to make sense.

Um, from the, the cyber carrier insurance perspective, um, I can't speak for all carriers, but most of the carriers do have si they do have panel and meaning that you have to be on panel to do IR services. Now does that mean that they're not gonna pay any invoices that you generate as a result? No, especially if you're just doing something, uh, immediate, they're usually pretty good about that.

But if you sit there and you convince your client or, or otherwise to say, Hey, just let us do the work and, um, and then, you know, the insurance will probably take care of it. Uh, you're doing your client an injustice there. Uh, they really need the, you know, you know, especially with us, especially a client can call us and have that first call and it does not count against them. That's a, it's a, it's, it's, it's a, it's basically a, a nil effect.

I mean, it doesn't count against their deductible. They're not charged for it. They can make a decision at that point. We're not even gonna file a claim. And most carriers are like that. I don't know any of that are not, so the point is, you should always have your client call the carrier first and get that stuff out the way. Now, a lot of times we we're not gonna put people on planes or in cars to go do things, and we're gonna leverage the MSP to do a lot of the hands-on work anyway.

So it's in your best bet to do that. And when you, and when you, and you have somebody like me asking you to do that, it's, it's approved. If you go on and do it yourself without an adjuster approving it or someone like myself involved, you're somewhat rolling the dice and your client could be on the hook for all that type of stuff. And then you have a whole nother situation on your hands. So be very careful as far as what you can do.

Now, again, initial stuff fine, usually it's not gonna be an issue, uh, especially if your rates are reasonable. Now, there have been situations where an MSP's done a bunch of work and they've generated a huge invoice, and then I look at that invoice and it doesn't make any sense, and I go back and I just start saying, rate's too high. This was not part of what they should have been doing. This is not where we, we cut it up.

So that's another issue you can get in, is you could be doing some stuff that's just not even covered under the policy, regardless of who did it. You could be stuff that's considered betterment and like, uh, you know, we had somebody the other day that was already in the process of buying new computers for their client because the computers were so old and they'd already convinced their client that that would be covered by insurance, then nah, that's betterment, so it's not gonna be covered.

So there, so that was a little bit of a contentious, uh, issue right there between them and their clients. So it's best to get the carrier involved. I know somebody had mentioned earlier that sometimes there's a delay, uh, but I would say in most, most time they're 24 by seven now. So even if you have to wait a few hours, it's worth waiting a few hours to get the, the right direction from from from the carrier. Got it. Eric, anything else you'd add?

No, I, I, I, I think, uh, I think he hit it on the head, especially the part about getting the pre-approval upfront. Um, the last thing you wanna do is say, Hey, Mr. Customer, I'd love to help you, but we don't have to sign. So, or we don't have this, or we don't have that. From a a legal perspective, that's never a situation you wanna get into when you have a, a client down situation. Got it. Okay, cool. So the next one, this one got a ton of votes, um, from our friend, uh, Felicia.

So, uh, Chris, I'll let you handle this one too. What methods, and we talked about this a little bit at the beginning of the call, but apparently we, people want you to get a little bit more technical here. Um, what methods do you use in order to find out what kind of data has been exfiltrated so that a risk assessment can be made? Yeah, so, um, I, I am, uh, not a forensics analyst, nor do I play one on tv, nor did I stay at a Holiday Inn Express.

But I will say that they're, uh, typically what we're, uh, what the forensics people are doing is, uh, uh, the logs do help, but they're looking at a lot of, uh, master file table artifacts and some other artifacts that are in there. Uh, for example, recent items is a good one. Uh, they can look at the recent items that were, um, basically created in the Windows operating system and understand what the attacker did just through, through that process.

Um, but even more fundamentally, we can see things like, uh, mega sync. Uh, if everybody's familiar with mega chem.com and all that kind of good stuff, uh, mega and mega sync is used a lot by these guys to quickly exfiltrate data out and uh, sync it to mega, uh, to get that stuff.

So a lot of times you can just find the installation of that mega sync application in there, uh, zips, uh, you know, large zip files or, uh, tar files or um, you know, dot seven z, whatever you wanna call it, whatever flavor compression that they used. Uh, that's another indicator whatsoever. I mean, some people like to say, Hey, we wanna look at bandwidth logs and that type of thing.

That can just tell you that there's a possibility that a lot of data happened, but it doesn't really tell you specifically what it is. So you have to get down to kind of server level artifacts, operating system level artifacts, uh, uh, to crawl through those and to kind of understand, first of all, we need to know what they accessed manually and then what they tried to exfiltrate manually. And then, then we gotta figure out from an automation perspective that there was done.

And problem is, is some of these guys, these ransomware, their tool sets that they're provided actually have some built in exfil tracing capabilities. So, and a lot of that can happen in memory. So that's back to the point where we don't want people to reboot, restart, shut down because a lot of that stuff we can, we can, we can scrape out of memory and figure out what it's, what it's done. But, um, I wish there was some simple 10 step process.

I could tell you this every way, but it kind of varies for, for per scenario. Yep, it does.

And and this is why having skilled, um, forensic analysts that truly understand not just the malware at hand, but even to a large degree, you know, if we can identify even what the threat, who the threat actor was, we understand some of their TTPs, I was looking at some today, um, of some of the new, um, I think it's the soden guys that are now starting to use like contact forms and like fill in, you know, hey, you know, I'm a photographer and used a bunch of my, um, my photos that I own.

Um, here's a, you know, a link to my, um, uh, you know, whatever it is, a lawsuit, whatever, just scare someone into clicking on it. Especially 'cause those that receive those typically marketing people that aren't always, um, as technical. Um, and then, you know, you always looking at some of the analysis and, and some of 'em, what they would do is they would connect out to very commonly like a.top domain.

So even my point is understanding even, um, who the adversary is and understanding the TTPs involved, which are like how they operate and the things they like to do can clue you in to look for certain things as well. So yeah, you've, you've really gotta have somebody that truly has some skill and understanding of, you know, what kind of malware, how do they, how do they operate, um, typically to, to know some of the things to look for for sure.

Sometimes we leverage threat intel too, so we have some sources of threat intel that kind of tell us what, what's going on and the, the attackers actually talk amongst each other and chat and a lot of times we can grab data of that and that's not just easily accessible either. So there's, it's um, you know, there's a lot to it. I mean, it's crazy. Yes.

So, um, Andrew, that, uh, I think we're outta time here, so, uh, but we did answer the other one that had three votes from Brian as well, and, uh, the rest of them maybe we can get to on a future cyber call. It's always good to have questions that, uh, are still lingering, I'm sure. Well, um, that fantastic. Um, Eric, Chris, thank you, um, for coming back. Eric, Chris, always great to have you on, everybody thank you for your support and being with us week after week.

Um, Wes, Ryan, awesome as always, having you guys with us and we'll look forward to next week where we're gonna dig into some of these TTPs specifically, uh, around attacks of the MSPs. So with that, wishing everybody a fantastic week and we'll look forward to seeing you all, uh, very soon. Take care. Thanks guys, everyone. Appreciate it. Thanks.