Special Guest: Chris Loehr, EVP Solis

Holidays. Can't believe we are in December. And in the final few weeks here of, uh, 2023, it's hard to believe how quick this year went. Um, and joining us today, we'll go into intro shortly, is is Chris. And our thematic today is, uh, talking about ransomware and account takeovers and things that are going on for the holidays. He's got a few quick announcements. Chris was kind enough to do a top five things you can do. I put it out in LinkedIn. I'll put the link, uh, in chat shortly.

You can go out there and grab it. It's a great document. Uh, something that you could actually rebrand and then, you know, get it in front of clients and prospects and things like that. But, um, really, uh, important time of year as there is a lot of, um, people going on vacation. Um, and, uh, with, again, I won't go right into the abstract, but a few quick announcements. Wes, um, congrats on what's going on with you.

Give you the floor for, uh, a moment here and tell everybody what you did and yeah. Uh, maybe what the group has done, but I just wanted to brag on everyone. So rejection con starts tomorrow, which is really cool if you guys are unaware of what that whole thing is and what it's about. I'll pop a link in chat.

But basically what happened was had a whole bunch of friends say, Hey man, you know, we've been trying to get into these, some of these conference talks for quite some time and I just keep getting rejected. And I took a look at some of 'em and they're stellar, uh, really, really amazing talks. And so we decided, Hey, why don't we, this is months ago. We decided why don't we start a conference? It's all gonna be charity driven.

Um, a hundred percent of conference revenue goes to charity and, um, let's ban together. And so fast forward, this thing starts tomorrow. We have hundreds of people that have already registered for it. Um, we have I think 30 or so plus vendors, maybe 35. They're sponsoring it. And we blew past our 10,000 goal. We're now sitting at 15,000 and growing. Um, I think we might hit 20,000 by the time this thing is over. It all goes to Rural Tech Fund.

If you guys know Rural Tech Fund, we've had Chris join us before on cyber call. Um, they basically reinvest in inner city and rural, um, school communities and help them get STEM and awareness going, whether it's AI or robotics or cybersecurity or whatever. So every penny's going to it. And then Chris is going to, he's committed to coming back, uh, next year and telling us where every penny went that we've given. So it's really, really cool.

So if you guys have not registered already, jump into it's rejection con.com. I took a page from Black Hills. Um, in fact, there's sponsors too. Um, but to pay what you can model. And so if you wanna give 50 bucks to Rural Tech Con, great. You wanna give zero, um, no problem. Whatever it is that you wanna give. And then last thing I'll say, Andrew, we've had a whole bunch of MSPs step up. Deley Mendez was the first and he gave 2000, uh, which is really, really amazing.

And so if you go to rejection con.com/challenge, we're doing this fun little MSP challenge of anyone that gives $500 to a rural tech fund, um, gets, uh, we just post their image up there and they get, um, access to their entire MSP and, um, they just know they're supporting a great cause. So just wanna share that and, and make sure folks know. Awesome. Very, very good. Cause I would like to have Chris back here. I'd like to know where every that we raised went.

So Gary Have a team accountants, so Chris is awesome. Maybe we can get him here at the final. 'cause he does this thing every year. Wes, maybe we should get, see if Chris will jump on again. Oh, its Golden ticket. Yeah, Yeah, yeah, yeah. He doing it again this year. He is doing it, yeah. Yeah. Alright. Chris, are you in a hallway right now? What's going on? No, I'm in a, my room at a lodge in Telluride. So that Telluride, I was gonna think, I thought about doing it outside, but it's too cold.

I feel like we all work and he lives like the life of the rich and famous, Oh my God, on the road all the time. Staying in five star hotels. It's un unbelievable. I I'm on the road all the time, but I wouldn't call those five star hotels. But this, this isn't, Hey, I'm not complaining about these digs here. Yeah. So, uh, but yeah, I, I try to find something that's a little bit more, uh, Colorado list, but this is what you get me looking like. I'm in the hallway All your ride. It's pretty nice.

Chris, do you think you could do maybe a, um, uh, what, what's the, the, the, I'm gonna drawing a blank. GoPro, you know, theme for us, you know, from a black, if you, if you've never done a double black, we'll, we'll put $500 on rural tech fund. Uh, just go up two levels on a mountain and we'll, we'll give money to the Royal Tech Fund if you go pro. And how's that sound? Uh, yeah, so I would do that, but I, a I don't ski and b I'm not skiing. So I'm here actually for an executive retreat.

So it took a little bit of break from in a conference room with no mountains to be seen Okay. To come do this. But hey, if you want me to do anything else here more than, so yeah, just name it. Okay. I said it's good to have you. We've missed you. Yeah, it's Been, I appreciate that. Now it has been. Yes. Let's get into it. This is, um, first of all, Chris, thanks for joining us. Uh, I'll let most everybody knows you out there, but I'll let you introduce one second.

But kind of here's the quick abstract is, you know, um, we've had a number of, uh, comp, you know, famous infamous compromises over holidays, um, uh, over the past few years. And, and, um, you know, in looking at the research, obviously this is, um, a time where threat actors are, um, more at work than ever. You know, they're not foolish. They realize people are taking vacations.

They realize that this time of the year with delivery, uh, you know, FedEx and all the delivery items, uh, things are fast moving. People are not expecting, uh, maybe a certain type of phishing campaign based on, although we should, but unsuspecting users may not be. So, um, they, they take full advantage of it. And we wanted to make sure that you and your customers, um, and again, I'll put the, the link and today is all about things that we need to be aware of.

I obviously can't be a full extensive list, but, you know, one of those top, you know, five to 10 things that, uh, a threat actor is definitely gonna be looking for from a, a, a weakness and and exploit. So with that, Chris, welcome back to the cyber call. Always awesome to have you. And for those that may not know, if you could give a quick overview and, uh, we'll get right on into it with Senator Pika, uh, from the great state of Philadelphia. There you go.

Or Pennsylvania, I don't think it's a state. Go ahead, Chris. Uh, I do have to remind that the Dallas Cowboys were victorious last night on, anyway. Um, yeah, Chris Lair Solace, uh, also known as Solace Security, headquartered in Austin, Texas. I, um, I personally reside in San Antonio, uh, born and raised in Dallas, so that kind of clears that up. The, uh, we are a cybersecurity company like an MSSP, but we also do a, a significant amount of incident response work.

Uh, I spend a lot of my time on both sides of the business, but spend quite a bit of time on, on incidents primarily as the ransomware side of things. But I get involved in all sorts of aspects. I mean, we work over 500, uh, cases a year and we are extremely busy right now. And, um, are, we are owned by a company that also owns CFC Underwriting, which is one of the top cybersecurity policy writers in the world.

And, uh, does a significant amount of policy writing here in the US and North America and other places. Uh, I know we have other people from around the world join us today, but, uh, that is my, I have a lot of background also in banking and financial services like Wes. So that's what you get from me from a quick bio. This is the time of year when Dallas wins a few games. That's right. Think about the last time that they were really good. I think. Uh, I think Madonna was still hot. That's Right.

You hot. Maybe she was popular. Your defense, your defense is only allowed 75 points over the last two games. Games. Yeah. You got lots of room for improvement. Hey, there, before you start off in chat, um, I put the link to LinkedIn. There's a Chris at the top five A PDF. Like I said, you can go out there and grab it, use it. Um, so Chris, thanks for doing that. Um, very short order. Um, this is part of the 12 days. So with that, Mr. Pika, the floor is yours. Yeah.

We're gonna talk about some of the phishing attacks, but first, uh, let's just zoom out, right? You work on a lot of cases. So without, uh, breaking any client confidentiality, can you tell us about 2023 in terms of, has it just been more of the same or have you seen some different things compared to, you know, more recently than, than before? Yeah, that's a great question, Gary. So, I, I'll, I'll start from kind of more recently.

We, um, recently we've had a couple of cases of a, a type of, of ransomware. And I won't, I'm not gonna name them because I don't wanna give these guys any notoriety because they are, have been traditionally known as hacktivists. Meaning that when they do hack somebody, and if they do deploy their ransomware, there's some political or ideology messaging behind that. Um, in the two cases we have, we have no idea why the heck they attack these companies.

'cause they don't have anything to do with politics or ideology or anything of that nature. They're smaller organizations. And the only thing I can think of is maybe it's just to generate revenue. So it's, it's very interesting and, and the reason why I wanna call that one out is what we're seeing, especially the last few months in my opinion, is we're seeing a, a real mix of attackers.

You know, for, I would say, you know, traditionally we see, you know, bigger groups like Lock Bit and, and those types of groups. We can even throw Black Cat in there, even though they're, they got shut down that, uh, you know, they, they are the kind of responsible for the majority of the work we see. And then there's a bunch of smaller guys kind of scattered in there. But I would say it's, it's a little flipped on its head.

We just see a lot of scatterings of, of different types of groups or in even some of them may indicate that it's an individual just subscribing to a ransomware as a service model. And the reason I can say this is, is because there's no consistency to the way they operate or like the, the stuff they demand. I mean, we've seen ones where they don't deploy, they don't actually encrypt files, but they just go take one, one file like a database as an example.

Uh, we've seen the other ones where they, uh, they go in and they, they do encrypt, but they don't take any files at, at all. They have like no intention of doing that. Um, we have one case right now where the threat actor is very engaged with one of our, with one of our employees as far as very talkative and, you know, almost sharing personal type stuff of what they do as hobbies and stuff. So it's just kind of, um, it's a weird deal.

I mean, I mean, kind of, we remembers when we first got started doing this stuff and uh, you know, a lot of it was like that, you know, a few years ago and it got kind of normalized. Uh, but now it's just all over the place as far as the groups that we see attacking now. Yeah. The what, why, why they seem, you know, or you know, how they attack seems pretty consistent, but as far as you know, who's doing it, it's kind of all over the place. Yeah.

You know, if you think about that, it kind of makes sense. Like think about the MSP market, right? They're serving SMBs and SMB, you know, the addressable market for SMB Tech is growing, you know, so rapidly, right? And that's why we see, you know, three to four MSPs pop up for everyone that gets acquired. So these guys have these organizations, they have the same tam, right? S you know, a lot of s focused at SMB, um, in mid-market enterprise. So I guess their TAM is expanding too.

So we're seeing the same kind of per proliferation, I guess, in in, in their space. They're serving the same customer base, right? Right. And, and to be honest with you, if you look at, you know, not to delve too much into this, you know, particular piece of the news, but, you know, black Cat Alpha B was shut down by the authorities. So their, their infrastructure itself was, was shut down. And, um, you know, it was probably a result of them attacking larger organizations.

And so if I was, you know, in the, in the threat actor's shoes, it seems to me that there is much, far less risk of me going after small to medium sized businesses because I'm not going to attract the same attention as if I attack like a casino. Yeah. The state department's not getting involved. Yep. Right. It's so funny you say that, Chris, because I, it was like, I don't wanna say I predicted this, but when they attacked MGM, I'm like, that's not gonna go well.

Uh, at around the tune of 80, you know, million dollars, uh, of losses. Um, and and Gary, to your point just real quick, you know, you know, a year ago at, at Schnoz Fest, right? We, we were, you know, went through the, the supply chain of threat actors and we saw the growth of initial access brokers and now, you know, we know for certain right, that that industry has exploded. We also know that, we talked about this with Phyllis at LA many times about the growth of SaaS applications.

Last week we talked about the growth of SaaS. And so, you know, the, the combination of very accessible phishing kits and exploit kits at ridiculously low costs, we just opened up this market like tremendously. If that makes sense to you. Yeah. So let's talk a little bit about these phishing attacks. 'cause we see them, you know, on the rise. And what would you recommend MSPs do to get their clients more vigilant, like right now?

Because, you know, in addition to vacationing, everyone's dealing with vendors, like this is a time of year where you're really dealing with people getting more emails, you're expecting more things. I'm like, this is the time when you're more likely to open or click or, or, or call like any, any recommendations on how to express to clients that the level of how vi vigilant their people have to be has to get raised right now. Yeah.

Gary, I think the, one of the things is to explain things just like you did, right? I mean, if you think about this, this is when people are trying to get all the bills paid, right? They, they wanna, they want to get all the money out the door so they can close their books. They wanna get all the money in the door, they can close their books, you know, every, no one wants outstanding receivables or outstanding payables out there And see if, uh, that helps. Let us know.

Let us know if turning off, I'm not so sure being a decentralized system, if, If this will make a difference. Yeah. Help, I'm saying Bob says it's help people said it's helping. It does help. Okay, good. So we'll, um, so we'll go this route. Yeah, let's stay with this. Chris, next question I have is, so you have a unique view, right? Because you're handling IR cases for both Solace and your parent companies, right? cfc mm-hmm. Right.

Has the increase in scrutiny, like writing policies, has that had a positive impact on CFCs clients? I know there's a big push right on that. Yeah, there, there's a big push. I think what I, what I tell people is it, it varies from carrier to carrier. Some carriers are, um, looking for a longer list of requirements and other carriers are looking for a shorter list. And, but what they, what they all are looking for something to be in place, right?

And I, and I think what comes down to is, um, I think getting, uh, to today is if we're not gonna get a conversation, but if you wanna look up the industry and look up hard market and soft market, it'll give you kind of an understanding of why things happen and, and why pricing gets done in, in the insurance world. And it applies across, across the, across the insurance space.

It's not just focused on cyber, but when you're in a software and when, when you're kind of a, a soft area of the market, you actually, it's easier to write policies, it's easier to renew people. Usually price increases aren't as drastic as they were before on premiums and stuff. Um, so that makes life a little bit easier. Also, I think carriers are not as, um, they're not gonna have the same level of scrutiny. They're gonna have a lower level of scrutiny.

However, what I'm seeing where it's kitten people, and it, this isn't specific to CFC, this is specific to across the board is where when someone does have a claim and the circumstances don't match what they put on their application, right? So it could be something technical like backups, right? So they call in, they have a ransomware attack, and they said they had immutable backups, and guess what? Their backups get destroyed. Or even worse, they'd ever even had backups, right?

And so then their claim is in danger of not being covered at all, or, or in some cases being covered partially, uh, or you know, where they've stated, Hey, we have X amount of, if you're healthcare and we say, Hey, we have 50,000 medical records, and then we get into the situation, we find there's a, there's way more than that, right? That's, that's gonna cause some claims issues.

And, and when you have claims issues, it's kind of, it's kind of interesting because, hey, look, if you have insurance and you have a, if you have a valid claim, things are very smooth. If you don't have insurance at all and you're paying out of pocket, uh, yeah, that it, it's more painful. But they still go as smooth because people need the help, they need to do the things. There's no one standing in the way besides the client themselves.

The worst case in my opinion, is in the middle where you have insurance, but your coverage is questionable because something doesn't match what you put on your, your application. And so that's where things get kind of dicey. So that's where if, if I, you know, an m MSP talking to my clients, or if your clients are asking you with assistance with their applications, you wanna make sure that they're very clear that get what gets put on there better be what is in fact the truth. Yeah.

And Chris, that's, that's why the carriers are now moving heavily towards what they call this, you know, automated, um, type of underwriting and continuous monitoring, right? Right. Because they want to, they want to get to that point where they don't have to rely on what you say, which is a challenge for them, because since the beginning carriers have always looked at, insurance is good faith, you tell me what, what's true, and I'll, I'll believe you.

And we've seen in cyber, it just doesn't work that way. Yeah, no, you're exact, you're exactly right. And, and the challenge is trying to monitor these things that are technical versus these things that are not technical. Like, so how do you monitor how many records somebody has, right? Um, how many, you know, how do you, how do you deal with the situation?

If I'm a healthcare entity and I went out in the middle of my term of my insurance, I went out and acquired somebody and that doubled my health records, how do I deal with that? Is somebody paying attention? I mean, we still get a number of people that call us in that don't even know if they have insurance, right?

They call us, we get referred, we ask 'em the question, and, and, and still to this day, and I mean, it's, it's at least a couple, two to three calls a month where people don't even know they have insurance and they have to go find it. And once they do, they don't even know what to do, right? They don't know what the carrier's expecting. They don't know anything about breach. So a lot of things that I thought were improving, we still see indications that people are still struggling with it.

But you're right, uh, wes, you know, some of these more automated ways are, are making, are making headway. Uh, you know, the challenge is gonna be with is how do you scale those, uh, as you, you know, on the carrier side. Yeah, look, MSPs or technology companies, and we have an issue doing that for our customers. So, you know, um, yeah, it's a challenge.

Uh, Chris, is there, do you see looking at your customers, like any improvement in terms of implementing, you know, incident response plans or doing tabletops? Is there, is there progress? Is there any hope out there? Are people moving in the right direction? Yeah, I think people are definitely doing more than more. And I think a lot of it, it's due to their MSPs, uh, being persistent about them doing it, right.

Uh, we do find that there are people that are calling kind of proactively into, into us saying, Hey, look, uh, you're our insurance carrier. We're working on incident response plan. We want to get your feedback on making sure that we have the right contact information and the right processes in our incident response plan. So that's a good sign.

Uh, it's pretty rare when we have one of those people that's very prepared like that, that they, they usually don't have claims because it's an indication of their maturity, right? Uh, but I would say overall people are doing a better job. They're understanding the importance of their incident response plan and how it needs to be reflective of what they can do.

Uh, you know, a lot of times people just go find something, a template online and they do a find and replace, and it doesn't apply to them. I mean, it, it doesn't fit their size, it doesn't fit their industry, it doesn't fit their situation. So, but I would say overall things are improving, but are they at a place where I'm like, have a big smile on my face? Nope. Yeah. You know, Chris, last thing I I have is like, uh, and I'll tie it into what we talked about.

People didn't hear the answer to the question about, you know, with the, the kind of getting to your customers and having them understand why they could be more vigilant about, um, you know, phishing attacks at, at this time of year. But in general, isn't it to me, isn't this a time of year that every MSP should be having a conversation, not just about that, but a write a boom conversation about what needs to change, implement. Like this is our chance, right?

To tell customers about this year and plan for next year and start to have them, you know, think about the budget they need to invest, you know, um, with you or somewhere else, right? But this is the time of year to do it, isn't it, Chris? It is definitely the time of year to talk about, you know, I think it's important to explain to people that security is not just about tools, right? And process is very important.

And incident response planning is that, you know, what I tell people is whether or not you use somebody like us to do a tabletop exercise, which isn't, you know, ridiculously expensive. I mean, just depends on their size and how many people are involved in that type of thing. But, uh, what I tell people is like, look, you're more likely to have an incident than you are a disaster, in my opinion.

I mean, somebody hacking into you and causing disruption is probably more likely to happen than a tornado or a flood or a fire, or whatever the case may be. Uh, so yeah, it is definitely the time to get that scheduled, to get that commitment done, uh, to make sure that they're thinking about it and who's gonna be doing what in those situations. Uh, so you're exactly right.

Putting, you know, a lot of times we just speak tools, but there are times where we need to go through some process related efforts, projects that don't involve tools and, um, that it's the best time to talk about it. Man, that's a good little nugget you just dropped there. I mean, you just like slipped it in. I hope everybody picked that up. You're more likely to have an incident than a disaster. Like that one right there is gold.

And think of people don't think twice about like all the things they need to do for potential of a disaster. They're buying generators, they're doing 10 different things, right? But they're not making the same investment into something that's more likely. So, um, everybody write that one down and, and, and add that to your, uh, VCIO process.

And, and, and also what people do is, I'll just add onto that, is they treat incidents like disasters and they, they don't take the, they don't take the correct incidents response steps. They take disaster steps and they're recovering from backups and trying to get up immediately and destroying evidence and all those types of things that we've spoken about before.

So yeah, it's not only to get them understanding the differences between an incident response and disaster recovery and understand, hey, it's a different type of preparation. And so you need a different type of testing and, um, you need to be really familiar with what goes on if you ever have to go through that effort. Awesome. Okay. Uh, Phyllis, you are up my friend. Excellent, thank you. So, um, let's talk about prepping for the holidays.

Everyone's bought their gifts, we're all ready to go, but if you're running an MSP and um, you're supporting all these folks, what is it that, um, organizations should be doing right now? And should we also be discussing remote access too? Definitely. I mean, remote access is one, and I think Covid was the best example that I like to use is when you know that pandemic and everybody had to work from home. People were just saying, Hey, I don't care how secure a remote access is, just do it.

And I mean, there were people that did it that way and, and when our clients came to us, we're like, well, we're gonna do it a secure way. We can get that done quickly for you, and we're gonna make sure that the people that do have remote access are the ones that need it and the things that they need access to. Um, that's what they'll have access to. And so, uh, COVID was a great lesson, uh, or a great way to demonstrate that, that lesson, in my opinion, the holidays are no different.

So I look at it from two, two ways. Number one is I know especially here in the states, uh, people are more prone to work when they're off. And I think there's a little bit of an issue with that. If you, we want to talk about burnout and the reasons why people are supposed to take vacation and stuff. It's supposed to get away from the office.

I mean, west knows in the banking world, if you were what's called a key employee, uh, when you, you were required to take time off and that time off, you were not allowed to work during that time. And the, the basis behind that was to identify potential fraud that was occurring. Uh, but really it has other benefits, right?

And so I think over here, especially with small and medium sized businesses where you don't have a lot of people there to back up others that you find people working on vacation and they need maybe remote access during that time that they don't have during their regular workday. The challenge with that is, is that's, um, you know, that that increases risk a lot.

So you really have to think of if you're in charge of a business or you're the business owner or whatever the case may be, or you're the MSP talking to your clients or even MSP as yourself worrying about your people going on vacation, is what do I want my people to do? Do I wanna have that remote access just enabled full-time all the time so they can get into access whenever? Or do I wanna make it more conditional?

Meaning, yeah, something's, something's, I don't want to call this guy 'cause he is on vacation, but I'm gonna have to, therefore I'm going to en enable remote access because I'm calling him outside of that, I'm gonna turn it off. So I think, um, the downside of the covid was remote access became more of a 24 by seven thing, and people still are now, they have way too much access.

They need way too much access around the clock, and we really have to take into account, hey, do we need to leave those things in place? It creates a security risk, but al also, is it damaging the employee by, by allowing them to work on their time off and when they should be spending time with their families and such. Yeah, that's so true. I Have a thought, and I think you could phrase it, sorry, I think you could phrase it that way, right?

I think when you're talking to your clients, you can talk to them more in kind of a business sense or a personal sense, so you don't sound like you're just talking at talking to them from a techie perspective. And I think that will re that. A lot of times that type of messaging resonates more so with them. Sorry, Phyllis, but I just wanted to add that Point. No, I mean, I think that's, um, a great follow up.

Um, I'm curious, when you, when you talk about secure remote access and how MSPs are, excuse me, supply map for their clients, um, what do you do? And this is just off script, I'm just curious, what do you do when an organization is like, oh, that's too difficult, or No, this is how we wanna supply remote access? Yeah, so it's interesting on the, especially with multi-factor authentication, it's amazing, uh, how much pushback you still get on MFA, right? Especially with remote access.

I mean, we still see it today. I see we still have cases where people have old Cisco ASAs where they're using the AnyConnect client, and this is stuff that I used 20 years ago and, uh, to, to do their remote access.

Then they might actually have a more sophisticated firewall in place and leave this ass a in this kind of old school because, you know, they don't wanna deal with people moaning and groaning about change or moaning and groaning about having to do an extra step of authenticating, right? So a lot of people just throw in the, the the white towel, right? The towel and say, yeah, I'll just, you know, deal with the pain if, if something happens.

It's so easy for these threat actors to take advantage of like, like a AnyConnect situation with no user, I mean with no multifactor authentication. Uh, so that's, that's, that's one thing is is that pushback. But, you know, I explain to people multifactor authentication. I mean, my first job outta school, we were using multifactor authentication in 1999, and so it's not like this new concept.

So people pushing back and saying it's too hard, you know, it's not a novel concept, it's something that's been around a while now. I do think there's, we need to think about things from the end user's perspective and figure out ways to better ways to explain it to 'em. Uh, usually like when I, when people have pushed back on multifactor, I go, well, do you realize like you, your own social accounts and stuff that you use personally also have multifactor capabilities?

So the stuff that we're gonna be able to do here in the office, you can apply those same principles in your own personal life and, and do put those protections in place and people get it. And I also think that things like, and not to get enwrapped in those technologies, but I think with pass keys and some other things like that with, with Apple devices and, and, and Android devices and stuff, I think it's gonna be a lot easier for people to accept that going, going down the road.

But this remote access thing is crazy. I mean, we just see people just, just again, just kind of quitting on trying to push back on doing the right thing. Meaning they just allow their clients to take the easy route and guess what they get bit most of the time when they take that easy route. It may not be today, but it's gonna be sometime in the next 12 months, somebody's gonna take advantage of that weak remote access security and find their way in. Yeah, that's so true. Thank you.

Um, and so we know, you know, my favorite controls, controls one and two, that would be, um, enterprise asset management and, um, software asset management or inventory. Um, those are arguably, excuse me, the hardest things to do. And of course, you need to know what's on your environment in order to patch effectively, which is of course is, is, you know, something that we need to do in order to shore up our defenses.

So having, having known that, I'm sure you've experienced it with your, with your clients, um, can you share with us what are some patching best practices and what is it that threat actors are actually looking for right now? Yeah, so what's interesting is, is that, and, and I was think thinking about this a lot this morning for a number of reasons, but, so let's just take NetScaler as an example.

There was a critical, a couple of critical, uh, vulnerabilities announced recently with NetScaler devices, uh, and NetScaler devices are kind of remote access device, right? And so, um, and if you read those advisories, I, I believe at least one of them said, Hey, we don't know of any, currently, we don't know of any public exploits of this vulnerability at this time, right? Well, you need to not take that as like, there are no, but there's not anybody out there exploiting it.

You just gotta read that as it is that the vendor isn't aware of anybody exploiting that vulnerability at the time. And so if you look at that, one of the key things in, in updating those NetScaler devices is not only did you have to update 'em, but you needed to make sure that you killed all the sessions that were the existing sessions that were connecting through that.

And if you didn't kill those sessions, then those sessions are still vulnerable and taking advantage of that vulnerability, right? So that's one thing. The other thing is, is, is you may not have realized that you have been exploited, right? So I I, you know, in that document that I put together that Andrew referenced earlier, we go back to proxy shell and proxy log on with exchange.

A lot of people did jump on there and update it or whatever, but the web shells and those other things were there already and, and they'd already been attacked. And so they didn't take the time to recognize, oh man, I, I actually have been attacked. Now we need to bring somebody in and investigate and see how bad the attack was. Was it just something that they got, uh, just a couple of steps in and that's bad, but not as bad as it could be?

Or did they fully get into our environment and have they been sitting in there for a certain period of time? And, and so I think when you talk about patching and updating, and this is where automating is good and bad, automating helps you do things quickly, but at the same time it may just be patching and not doing that full remediation. So you really have to look at things, especially those critical vulnerabilities out there and understand it.

And the bad guys are looking for those, especially those externally facing assets that have those vulnerabilities. They are looking for them incredibly quickly and they're taking advantage of incredibly quickly. I mean, Microsoft Exchange, I mean, you still see those boxes just get hammered and, um, and they're just hammering 'em nonstop, waiting for one of those vulnerabilities to pop up or something like this to get in. So, uh, it, it is more than just patching and updating.

Uh, it is kind of understanding the vulnerabilities, understanding what you need to do, and going and having some type of process to understand, do I need, is there a chance, I need some further analysis by an expert in my environment? Because I see some indications that this may have been a target of an, of a current attack or recent attack. Right?

And I like your point of just because the vendor says we have not seen this exploited, doesn't mean that you should prioritize patching that blow, right? You're exactly right. And most people just don't know, right? I mean, when these guys get in, uh, this is, I had a conversation with somebody about this last week and they just didn't fully comprehend. They're like, well, I have E-D-R-X-Y-Z and, and that it's a really good EDR, why didn't it catch this?

And I'm like, when these threat actors come in, they're not de deploying malware, they're just going in, they're running act, they're running PowerShell commands. They're, they're like, in this particular situation, they ran power share commands to look for where the domain controllers were they ran, mean, these are just builtin commands.

There's nothing special that they are doing in these environments when they first get in that through the normal set of security tools is gonna pick up on, I mean, that's the reality of things. So people are like, oh, well, you know, EDRX, YZ should have picked up on that. I'm like, no, 'cause these guys are just, they're just popping on your network running commands as a normal user on your network, and that's what they're doing.

And, and it doesn't look nefarious to any tools out there really. So you're right. I mean, that you have to be on top of these vulnerabilities. And then if you, you say, Hey, look, I have this vulnerability, uh, what could be the potential outcome of somebody exploiting that? Now I need to go dig and see if there's any kind of weird activity associated with different users at different times or whatever.

And a lot of times you just, what you gotta do, I mean, you just, uh, remote access, for example, you can pop in into VPN logs and go and go look. I was like, why the heck is Susie logging in Saturday at 2:00 AM in the morning? And, and, and you saw that. So something strange there. And so you can dig in a little bit more, uh, when you, when, when you, but you have to do that work, right? You can't just assume that patching and updating is gonna, is gonna solve, solve, solve the problem. Right?

Chris? And just Real quick, Phil, Chris goes right into like, again, the threat actors aren't stupid, right? I mean, there's a reason they're using valid credentials and using living off the land tools, they're going, they understand how EDRs function looking for behavioral behavioral anomalies. Fair. No, that's exactly fair. And you're right. You know, the, the, you know, the operating systems give you great tools. I mean, they're just there.

And, and you're right, these guys, not only are they, they're educated, they're smart people and they're also well trained a lot of times, uh, they have, uh, documentation and others, other resources at their fingertips that make them very, uh, very talented experts on getting in. And they're laser focused on one thing, right? I mean, we have to remember that they are laser focused on doing one thing. They're not being pulled into 12 different directions.

They have one sole purpose in life and they can devote all their time and energy, mental and intelligence and all that stuff to getting that done Stuff. Yeah. This is a good segue into, um, the incident response and data recovery plans. You, you talked about, hey, it's not just the tools, it's also people in process. So talk to us about, um, reviewing those call and escalation trees, um, on the technical and business sides of your clients, and actually for yourself as well as as an MSP.

Yeah, it's so important. I think that, um, like turnover is, is a kind of a big deal, right? So I think this time of year you start to see people, uh, probably they're gonna get their Christmas bonuses and then put in for their new jobs, right? Uh, so you're gonna see probably a higher level of turnover the next couple of months than you do the rest of the year.

And so when you, when you have these changes in your environment and these turnover and this type of stuff going on in your environment, uh, you're not necessarily going back and updating any type of plans or, or anything of that nature right away. Uh, you know, for example, if you have an instant response plan and you have some schedule of review once a year, like you do other policies and stuff, you're probably gonna miss things, right? Mm-Hmm.

And so you may not need to review that entire plan, but you need to review that you have the right people available at the right times to fill those roles that are part of that incident response plan. Same thing goes with a BCP or a disaster recovery plan. And then, and again, just like you mentioned vacations, right? Uh, people go on vacation, you need to make sure that there's somebody stepping into their role. You need to know if somebody's going on vacation, where the heck they're going.

I mean, if they're going to grandma's house, two counties over, that's one thing. If they're going over to Singapore and they're gonna be in the totally opposite time zone, that's another deal. Right? So kind of understanding the, you know, especially people in critical roles, uh, what they'll be doing, how long they'll be gone, uh, those types of things.

And making sure from a incident response perspective, 'cause that's what we're talking about here, that you have adequate coverage and that you have alternate plans if you don't have that coverage is, is important. Now, you, you probably should have been doing that before Thanksgiving, but you better do it now. No, that's, that, those are excellent points.

'cause I was just thinking, you know, in controls we always say review yearly or when you have like, you know, personnel changes, but you really just have to make it a part of your process somehow. Like, you know, and you make a good point 'cause people are on leave. Um, Yeah.

And I, and I do, again, I think if you kind of shift your mindset to say, Hey, look, if somebody's on leave for whatever purpose, I, as a business owner or a person running the business, I need to try to do everything I can within my power, which, when what's financially feasible to make sure that they enjoy that time. Mm-Hmm. And I need to structure things in a way where I can depend on the people that are not on vacation and not on leave. Right? Right.

So let's pivot a little bit and talk about, um, account inventory and, um, can you tell us a little bit about reviewing admin and service accounts? We often see that, you know, service accounts are being compromised. 'cause honestly how many people pay attention to service Accounts? Yeah. They don't pay attention or the bad guys create accounts that look like service accounts and so they kind of fly under the radar. So we've seen that too.

We've seen some account that looks like a legit service account and we ask, what is that? And they're like, well, we don't even know what that is. We've never seen that before. But it kind of just blends in with the naming scheme of their other service accounts, right? So, uh, we, we see that, uh, you, so a review of that review, who has admin rights, a review of those people that have admin rights need to have those admin rights.

And maybe it's like when somebody goes on vacation, you know, 'cause we really should have a user account and an admin account, right? Your user account shouldn't have admin rights as well. So let's say you have two accounts, uh, and somebody's going on vacation, maybe you disable that, that admin level account while they're on vacation, but you leave their user account in place. Uh, so those are some things to think about. But you, you do need to review those accounts.

Uh, I've seen where there are system accounts, they didn't put, they didn't have the, the same attention to detail when it came down to passwords on those accounts. Like they weren't doing the same kinda password hygiene. Uh, they have service accounts whose passwords haven't been reset in 24 months or greater. Uh, we see that, uh, we see, um, we actually saw a, somebody running an instance of Windows server on a Mac mini. Um, you gotta kind of raise your eyebrow to that one.

And, um, and so yeah, that's, yeah, that's, I never thought I'd see that either, but hey, check the box on that one. So yeah, so it it's a great time. Um, you know, and it's a, and vendor accounts super, super important. I mean, you talk about asset inventory, and I think one of the things that people don't kind of include in that scope is, you know, what, what their vendors have, what they're accessing.

It's interesting because we see people with, uh, assets where they require, uh, some vendor to, to, like, let's take healthcare for example. A lot of imaging systems and stuff that healthcare use, they, they do depend upon a third party vendor that specializes in that to access. Well, one of the things a lot of people don't know is the tool that that person is using to remote in, what kind of controls are involved with that tool?

What kind of controls that the, that the client, not the vendor, the client has to either disable that or, or enable that. Can I do that? Or is it on full-time? And is there multiple tools we've seen where, uh, this comes up a lot where we go, yeah, this tool's in the environment. Because what we try to find out is, did a threat actor install that tool? And they're like, no, that's a tool that a vendor used two years ago. We just never removed it. And you're like, what? You know?

So those types of things, those types of things still occur. And I, I I say they're very prevalent in the small to medium sized space. So when you're reviewing those access and reviewing that inventory, you need to really have an understanding of who has access to what and when and how. And this is a great time to review that and put together plans on how to remediate, uh, or reduce those risks that, that, that you've discovered.

You're Bringing, you're bringing me back to my Nightmare MSP days, stuff Like this. That's what I try to do. Oh my God. All right. Over to you. What? Yeah, Thanks. Um, so Chris, let's talk a little bit about if you're a business, and it doesn't even have to be the holiday season. It could be a CPA during, you know, March, april. It could be, you know, heavy retail during, for sure Christmas.

But let's talk about holiday seasonality and just like busy time of year, what should, what should MSPs be saying to clients during their busy time or times of year? Um, I can even think for SaaS companies, right?

You got end of month, end of month, end of quarter, no doubt, you're more prone as a SaaS company to go pull up some kind of like, hey, trying to get this invoice, you know, paid to you guys, the chances of that being opened and clicked on, researched and, you know, are much higher, right? So how should you be thinking about that with your clients and what should you be saying? Yeah, I one to sound like a broken record, unfortunately, right? I mean, you know, you bring up CPAs, right?

There are times of year where they're incredibly busy and they don't have any time. They, they wanna talk to you, they want to hear from you, yet if they're disrupted, it's a nightmare for them, right? And usually they're very small, two to three people and that's it. They don't have any kind of resiliency whatsoever built into their operations or anything.

So this is where even if you're getting pushback or, or you're, you're having some obstacles and communicating your messaging out, you need to be persistent and find different ways of doing. So. Uh, it's incredibly important for people just to be aware that this is when the bad stuff happens. Um, you know, and I, I'll tell you from an incident response situation, when you have a situ, you have an incident pops up during this time, it's very hard to track down people to help you out.

Meaning like, not your typical incident response. But let's go back to that healthcare example, and this is what you can convey to your clients. Let's say, uh, you a healthcare situation, you need recovery and you have some specialized equipment that your third party vendor, excuse me, needs to, you know, remote in or come on site to get back up for you. Well, guess what? They don't have a big staff sitting around for these types of issues to pop up because they're just general support people.

They're not looking, and a lot of times this is their least busy time when they're talking about what they're doing. But an instant response situation, you need somebody right away, and these vendors won't have anybody for you right away. They can't pull somebody out of vacation, they can't do those types of things.

So it really, and when the bad guys take advantage of this situation and they, and you get popped a day before Thanksgiving or a day before Christmas or whatever, and you're like, man, we need to get back up and running in two days at the most, and, and we can't find these specialized resources out there to help you. You have no choice but to wait. And, um, so you don't wanna do that. And you know, the same thing goes with, uh, banking, right?

If, if you've lost money as a result of financial transactions or even in, in the case of a ransomware attack and your financial institution finds out about it, they might lock you out of internet banking. And that means that you're gonna have to revert to going into the branch to do transactions. Are you prepared to do that? Do you even know how to do that? What, how disruptive is that gonna be to your operation?

So these are pretty simple things that you can think of in ways to get, convey the message across to your clients that says, Hey, this is the time of year that you're most susceptible to being attacked. And it is the one time of year you don't wanna be attacked.

So you need to, you know, you need to be all hands on deck and, and just be incredibly vigilant about what you do and make sure that's conveyed to their staff that they're not to take any shortcuts or if they feel pressured into taking shortcuts or doing things, uh, in a way that could be haphazard or high risk, they need to raise their hand and and, and stop the production line. Hey, we can I just Ask Chris something real quick, please.

Chris, you mentioned, you know, lack of resources, but what about, you know, we still have supply chain issues. Like let's just say you're dealing with utility companies or comp manufacturing companies running on legacy systems that are, you know, they get, you know, let's just say they get really locked up and you're talking about, you know, they don't have a redundancy plan. Well, our fastest way is to maybe rebuild something.

Is that something that we need to talk about as well, you know, that maybe we can't even get the infrastructure you need? Yeah, that's exactly right. I mean, we've seen a ton of situations with legacy stuff where the vendor just doesn't have anybody that even knows that legacy platform anymore. You know, they just put off upgrading.

They haven't been, you know, there wasn't even maintenance available for them to purchase because of the, the product, whatever legacy system they were using was end of life years ago. And then they get in these situations and you just can't find anybody to work on the system. So it's really important to understand if your clients do have legacy systems. 'cause sometimes there's business reasons why they have it.

Sometimes there's, uh, financial reasons why that it would just be prohibitively expensive to, for them to upgrade. But to understand what those legacy systems are and to make sure that, um, you know, how would I put this, the stranger, the legacy system, the stranger and the older, there's probably an equation here for somebody, but the stranger and or the more unique and the older the system's gonna be, the least likely it's gonna be able to be recovered.

And, uh, I mean we've just seen some situations where people are done. I mean, they, they that legacy system's, their core system and there's no way to recover it and they have no choice but to close up shop. Hmm, interesting. Well, hmm. Alright, Wesley, back to you. Scary stuff. What a wonderful, uh, note to start closing out the year on, right? So I, let's talk about this a little bit, Chris. I, you know how this goes.

We've been so focused on overseas, especially Russia, Ukrainian, Eastern European threat actors because of ransomware for so long now and rightly so. But one thing that was scary about the MGM attack, and you look at what Mandiant said, uh, in some of their post-incident analysis is it seems that, you know, we know how they got in. They literally just, they called up.

Um, and it seems like we may have a bit of a blind spot in the sense that we've been so focused on, you know, there's no way an Eastern European is gonna call me up on the phone and I'm not going to detect that accent and have my alarm bells going off skyrocket. What, what, what happens in the case of MGM with scattered spider when it actually appears to be a native English speaker out of perhaps the us right?

That changes the mechanics completely to where I think a lot of us would fall for a phone call that comes in from someone that purports to be from our MSP. And that has a great, you know, Midwestern accent. Like what do we do about a situation like that? And do you think MSPs have done enough game planning around that? Yeah, I don't, I definitely don't think there's enough game plan planning. I definitely think it is a blind spot and it kind of makes sense.

I, I do believe, uh, 'cause I've seen the attempts where these organizations that are based overseas are recruiting people here and they're offering substantial amounts of money for these people to work for them. Right. And, um, I, I, I also think that people over here look at the, the, uh, you know, they look at it as probably a very low possibility they'll ever get caught. And if they do get caught, it's gonna be kind of a slap on the wrist.

So that's very, um, how would I say, um, for someone over here that's, uh, looking for a big payday, I think it's a very attractive proposal that these groups are putting over here and there, and there's people willing to do it. So you're right. I mean, what do you do when you have some that you've, you've been kind of trained to be detecting accents and now you can't do that. It's just coming in. It's kinda like the, the phishing stuff, right?

Back in the day, you look for misspellings, grammar errors, weird logos and stuff in the emails. Right. But those kind of went away, right? The people got better. And I think, um, not only with local assets doing the work for these guys, but also, um, you know, you know, we haven't talked about it and don't wanna get too much into it, but ai uh, being able to leverage AI to replicate, uh, people's voices and dialects and accents and that type of stuff is also gonna be a, a big risk.

So that's where I think where we're gonna start having to way have to have better ways of challenging a caller and making sure that they are who they are and not with simple questions and figuring out ways to do that. I mean, when I was, uh, it was interesting. I was, when I was coming, I was in Denver, flying into Denver to get over here, and I went into the American Express Interior Lounge. I mean, that lady, I had to provide my card, my driver's license.

She asked me when my flight came in, when my flight leads. I mean, these were four or five challenge questions I had just to get into there. I think, I mean, that's just to go in there and eat some free food and sit in a comfortable chair. We're not doing that same level of challenging, uh, on people calling in and for it support. That makes sense. Yeah. An interesting way of looking at this is because humans were driving so much of the fraud.

Um, it makes sense that we train humans to stop humans, but as AI is now driving automations, looking in chat here around things like whether it's, you know, natural language, um, that's turned into something verbal, whether it's, you know, large language models that are having us write quicker, it's almost like, so if the shift is now machines are doing more of the initial attack, uh, and getting to the attack surface, maybe we need more machines to stop that, right?

So if, if humans are against humans is no longer a thing, what are we doing from the automated perspective? It's just, it's just, I don't know. It's an interesting philosophical concept to think about a little bit. You're kind of getting me, getting me going on this. Let's, let's shift, um, just a couple minutes left here. Let's shift to write a boom for a minute. You know, sims are expensive coming from someone who, among other smart people built a sim.

Um, I understand why a lot of small businesses simply don't go and, and build out a sim. They're ridiculously challenging to, to get the cost and especially because they serve so much as a write a boom function. What, what do you do in a situation where you walk in and it's a non-regulated client, they have an incident, um, and now they have no logs whatsoever from an incident response perspective.

How does that change the game and how you act, how you react and even the outcomes of the incident itself when, when logs are just not present? Yeah. From, from what I tell you with logs is logs helps us understand something more quickly. Uh, for example, we can, you know, we don't necessarily depend upon logs from a forensic analysis. Uh, we deploy upon a number of other artifacts from the servers. Um, but having logs definitely gets us answers more quickly.

Especially when it comes into, you know, how long the threat actor, the dwell time, how long the threat actor was in the system, uh, that type of thing. So we, we can look at logs and go, wow, that threat actor was in here for, you know, something like 45 hours over the last two weeks. The chances of them exfiltrating data are extremely high. If we see 'em from a log perspective only being in there for a few minutes, then the chances of exfiltration are very low.

So that's where logs really kind of help you kind of get answers more quickly. Uh, they help you direct your forensic analysis more quickly. Meaning like, we don't have, such as, we don't, we now have to cast such a wide net from collecting analysis. If we have logs that says, looks like all the activity was on these handful of assets, we can focus on those assets first to get answers more quickly.

You know, in today's world and it's getting worse now, uh, you, you're still gonna have to perform a full forensic analysis, which could take, you know, a month to two months depending on, that's kind of what I would say a typical small to medium size effort is. But you have to get some forensic answers more quickly now. 'cause you have laws that are requiring you to provide some type of notification to some type of entity very quickly.

You might have contractual, uh, contractual obligations to do that as well. So that's where I think having some form of level of logs get in. I think when people get caught up with sims is, you know, they look at the expense of collecting absolutely everything in the environment. If you can kind of focus on those critical assets and only collect those, I think your dollars of investment in the sim go way down. And it probably makes a little bit more financial sense to do.

But lemme tell you, most people don't have logs and we find a way through it, but the ones that do have logs just allow us to get those answers more quickly, which makes everyone's lives easier. Hmm. Yeah. So that really does come back to an intelligent discussion around what we actually are collecting and what the minimum standard of, of logs and log types and log retention should be.

Even to your point, if a client does not have the budget for a full scale, collect everything, sim what are the bare minimums that we need to be retaining? It's really good. Exactly. Andrew, I know we're at bingo. I wanna turn it back to you, my friend. We're at bingo. Oh, two o'clock. I see what you're saying. Yeah, sorry. Um, no, no, it's all good. Um, so first off, Chris, thanks for making it an awesome, uh, conversation As always.

Um, to everybody out there, thank you for, uh, as always coming and again, apology for the platform issues. Um, I will begin to start to look for something alternative, uh, outside of Crowdcast. Um, as I can tell, And Andrew, we gotta have Chris back more often. Uh, this is, this is a session people are gonna want to go back and listen to again. Mm-Hmm.

Uh, you packed this with so many little nuggets I couldn't even interrupt every time, uh, without messing up the flow, but this was really good. No, I appreciate that. Yeah. You know, Andrew know Andrew definitely knows how to track me down, so I'm more than to get on here. We'll do the part two, Gary, with full, full peak of interruption. How's that sound? That sounds great. Alright. Oh, Chris, again, thanks a million. Phyllis West Gary. Uh, awesome job as always, everybody out there.

Um, appreciate all the support and wishing you all a fantastic week ahead. Take care everyone. Bye-Bye bye now.