The CyberCall – Security Legend John Strand on REvil and MSP attacks

Welcome everybody. Week 57 on the cyber call. And we have a awesome special event for you today. Um, I hope since the last time we saw you, life is a little less eventful for you all. Um, joined as always with Gary Pico, Wes, Spencer, Ryan Weeks. Good to see you all. Um, shortly I am going to introduce our special guest, John Strand. Um, so just a few quick announcements if I could. Number one, um, I am going to put in the link to the cyber cast.

We just pushed out, um, control three on CIS controls around data protection part, uh, A and B. There will be a part C, you can get it there, and it's on any, uh, of your major podcast, uh, publications. Momentarily. I'll put in the, uh, true methods Cisco content that's still, uh, going live. Gary, we, we've onboarded quite, Yeah, a bunch of people been in.

But listen, this is the time go in there ir planning, threat modeling, uh, cyber resilience, tabletops, like all the things that right now we need to be focused on, Right? That, that, that's absolutely true.

I'm putting that link in there and I think we are good because the other call to action we will get to, or the, uh, announcement if you will, John's gonna be talking about that at some point because as a really interesting and, and, and I think kind of groundbreaking approach to educating people on cybersecurity. So kind of set the stage here. Number one, following last week's, um, events.

Um, you know, Ryan West and I were talking offline and, you know, Ryan's like, like we've really gotta bring somebody on that understands good application security. Um, MSPs really need to start to understand how to evaluate tools, um, and, you know, what the impacts are of, you know, third party risk, supply, chain risk, et cetera. We could think of no other, uh, than, uh, John Strand. John, welcome and thanks a million for coming live from Costa Rica, uh, to us with your family. So thank you.

You bet. And thank you very much for having me. I really appreciate it. Well, yeah, it's an honor for us. Um, so John, um, I'd love it if you could just kind of give folks a little bit about your background. You're involved with just a few things, um, few businesses, little education. I'm making light of what you do. Uh, and then, um, I'll set the stage in terms of how we're gonna approach this. 'cause it kind of got themes from Ryan to West to Gary.

But Floor's yours and please share a little bit about who you are, my friend. You bet. So I actually got started in computer security back in 2000, 2001, um, in the world's largest class action lawsuit. Like I, I think it still stands, um, it's Cobell versus, uh, the United States Department of Interior Misappropriation of Native American Funds. So basically, this was such a big case. They actually shut down the entire Department of Interior for a long period of time for a matter of months.

And I was right in the middle of that. So that's where I got bit by the computer security bug. And then I moved from there to Northrop Grumman, where I did classified projects for about five years, which sounds way cooler than it actually is. And then, um, I started my own company, black Hills Information Security, was an instructor with the Sands Institute for about 15 years. I recently retired with them.

And now I'm the owner of Black Hills Information Security Active Countermeasures, which does network beaconing detection. Uh, that's our specialty, uh, for that particular product. But the big thing that we're working on right now is education, uh, with Wild West Hackfest, our conference and all the training that we do. Any of the training that I do is pay what you can training, and we can talk more about that a little bit later.

But education's always been my thing, and that's what I think we're, that's how I think we're gonna get dug out of this hole in computer security. That's my goal anyway. John, one thing I'd love for you to share that you shared offline, and this is something, you know, I I feel like I'm Gary right now. MSPs, please listen up to this, the theme that John's gonna share around basic cyber hygiene and what you are seeing.

I mean, just John, can you, can you tell us about the themes you're seeing and Yeah. You, you A absolutely. So what, what I'm seeing in a lot of the IR work that we've been doing over the past, let's just say 18 to 24 months, okay? Almost every single organization that we're working with ha, whenever they get compromised, it's by basics and fundamentals. Uh, weak password policy. If you look at some organizations we just saw case, they got hit by an application.

They knew it was vulnerable to an attack, but they didn't fix it behind on patches, things of that nature, colonial Pipeline. They came out and one of the things I thought was really cool about Colonial is they were honest about what got them. They were like, yeah, it didn't have two factor authentication on this VPN over here, that bit us. This is basics and fundamentals.

So one of the things I'm constantly trying to push for people that are getting involved in computer security is don't get caught up in the attack du jour. Don't get caught up in this morass of all these new technical things. Stick to your core fundamentals, patch and update your systems. Use long strong pass phrases, use two factor authentication, reduce the attack surface available to the attackers and train your users. Those are some of the things that are prevention.

And then the core detection, we can talk a little bit more about that as well. But this isn't rocket science. It's not hard from the perspective that I'm gonna have you solve eagan vector values. But it is hard because it requires grinding, it requires going to these basics and fundamentals and not letting up. Yeah. Reminds me of, uh, really like the bill, bill Belichick kind of philosophy. Mm-Hmm. Of the Patriots, right? Boring fundamentals, basics over and over and over.

But guess what the results are. So thanks for sharing those themes. Okay. So setting the stage before I hand this over to Ryan. So Ryan's gonna kick things off. He's gonna be talking to John about application security, what good, what good looks like, how, you know, MSP should be approaching it, some other security topics from there. Gonna move over to Wes.

Wes, I know you did some audibles, but I think some of the, are you still on with some threat modeling stuff and and, and things we've done in that area? If I understood correctly? Yeah. As far, yeah. Yeah. Yes. Perfect. Yeah. Perfect. And then from there, uh, Gary's gonna take a little bit from really the education and culture and progressing, uh, our industry forward with John and what he's doing uniquely in education.

Uh, for so many thou, like we, John was sharing this offline, like 20 505,000 people at a time are taking these courses. And some of these are core. I write now, for example, below John, I've got soc core skills. These are things your team can take. Now he's got a philosophy of pay what you can, he's got core cybersecurity, uh, fundamentals. And, and again, I'll let him tell more about that. So with that, Mr. Weeks, the floor is yours. Thanks. Yeah.

Thanks for joining us, John, especially while you're on vacation with your family. Um, so last week's, uh, event really contributed to what I've been calling the crisis of competence, uh, in, in the IT channels, specifically with MSPs and their vendors. Um, of which I'm one and I, you know, I, I got up in front of all of our customers and I said, Hey, here's what we do from an application security perspective, um, to, to prevent the similar types of things that that happened in this attack.

But it really got me thinking that there's very few MSPs that can really understand what good application security looks like, right? I could tell them what I'm doing, but they don't necessarily know if that's good or bad. So could you help maybe contextualize for our audience, how do you spot good application security when, when you're, you're thinking about your, your tool stack and your technology stack when you're talking to your vendors? You bet. First I wanna apologize in advance.

I, I have a rooster that has decided to come over here and hang out with me outside. So if you hear the rooster, um, you know, there, there's a rooster just off camera and he's just chilling with me. But it may turn into an Allison chains song here at some point. Okay? So, alright, so let, let, let's talk about application security if you're gonna get popped. Um, with application security, we go back to the basics and fundamentals.

You're gonna get hit with SQL injection, cross site scripting will be used against your users or cross site request forgery. It, it, it, it's, it's not really rocket science for the core vulnerabilities that hit the vast more majority of organizations. So we go back to the OAS top 10, go back and look at those are the vulnerabilities that are being exploited the most. Start there. That's, that's easy, right? If people are like, where do I start? And you can do that with Zed.

Attack proxy is a free utility that'll help you identify those vulnerabilities in your web infrastructure. Doesn't cost anything. Okay? And that's critical. 'cause many times when people are looking at application security, they think that they've gotta go spend $25,000 on a tool. I'm gonna be completely honest with you. When we're pen testing, we use Burp Suite Pro, which is just a few hundred dollars. We use Z attack proxy.

And yes, we do have some commercial tools that we use for due diligence, but honestly, you can do it cheap and you can do it effective. That's number one. Number two, and this one's key. Application security is something that is done continuously. It is not bolted on at the end. 'cause if we look at Kaseya, honestly, the way that this whole entire thing went down, I know exactly how it went down. Um, just guessing I should say.

Um, they're alerted that there is a vulnerability in their web infrastructure. But because it's not something that's part of A CDE, because it's not part of something that's an existing patch release, not something that they can easily update, they don't know what to do with it. So they sat on it because they are worried about regression testing. What happens if we try to fix this at the end? Will it crash our entire infrastructure?

These are difficult choices that an IT manager has to face, and you have to face those choices if you try to bolt security on at the end. So application security is something that can be done at the beginning of an application development lifecycle continuously and throughout it should be part of our vulnerability management process, and it doesn't necessarily have to be expensive. Perfect. So Ryan, I'm sorry.

I mean that, that lends to something like, one of the things I think our industry's dealing with is, you know, all of the major vendors, you know, have code that dates back quite a few years, right? It's true. Yeah. There, there is legacy code. And I think as an MSP, one of your questions to your vendors should be, what is your stance with regard to legacy code? How are you managing it? How are you making sure it's free of vulnerabilities? And what is your process to retire it?

Um, Oh, oh, that last question, that last question's the money question because, okay, so whenever you're working with different industries, um, I absolutely love that, Ryan, that is so, so on point. So if you're working in finance, okay, if you're working with pen testing a bank, they're like, look, we've got AS 400 systems here. They're like 15 years old. It's legacy code. We're a special unique snowflake in finance. We can't get rid of these systems.

Then you go to medical and they say, oh, well we have dialysis machines and radiology machines that are 15 years old. If these things mess up, people could die. We're a precious, unique snowflake. You go to the, uh, OT or the SCADA ICS environment and they're like, oh, we have legacy technologies and, uh, we can't get rid of it because, uh, you know, we can't replace these things.

I, I think that key question that you need to counter that is that retirement of infrastructure has to be part of the life cycle. And almost always you get a pushback that says, oh, we cannot get rid of legacy technology. The pushback should always be great. Then when, when is it going to because it's gonna be retired one way or the other. Whether you like it or not, it's going to be retired. The question is, are you gonna be in control of that retirement? Right? Yeah.

So this, this one's perfectly into the next question, which is MSPs are, right? There's the SolarWinds incident that happened earlier this year, and it wasn't that MSP side of their business, but it created some competence issues in that technology. Now the ca say incident and there's some believe that RMM is dead.

Uh, there are some that are saying everybody sucks when it comes to application security, which frankly annoys me because I, I work really hard on application security, um, to, to make our RMM, you know, secure. Not that we're perfect, but you know, we're, we're, we're trying much harder than I think what we've seen in some of these, some of these recent incidents. So how would an MSP go about like spotting good application security when they're making a selection choice?

Like what are the types of questions they should be asking? What are the red flags they should be looking for? Like how would you guide them? Like what's the run for the hills type of stuff and what's for the, you know, what's the probably safe if you move forward with this? Like how do you, how do you paint those lines? Easy peasy. Two questions. One, ask 'em what their current application security program looks like.

Organizations that have an application security program are gonna be proud of their application security program. They're gonna say that they're doing this testing, they're doing it regularly, this is what they're looking into. That's key. If a company comes to you and they says, well, that's proprietary. We don't like to talk about it, but we have our top men working on computer security run details are key, right? Details are key.

And if you talk to a security geek or you talk to somebody that's involved in security, we love to share because that's how we learn. And if they're willing to share, that's outstanding. Um, that is, that is a fantastic really gold standard sign that you're doing. Okay. The second thing, whenever you're talking to an organization is ask for something called their letter of attestation.

Anytime BHIS does a pen test for an organization, if that customer asks, we can write a letter of attestation that very clearly states Black Hills Information Security did a test for this organization, vulnerabilities were found, the organization is working on those vulnerabilities, and that we believe this organization is operating within an acceptable level of risk.

If you cannot get a letter of attestation from your vendors for the last time they were tested by a third party run for the hills. Cool. Um, I do want to dig into that a little bit more, but maybe if we'll leave that for if we have time. No, go for it. Talk to us about You Have time, Ryan? Good? Yeah, you're Good. So one of the concerns I have is, right pen tests are scoped and defined, right? Yes.

So there's a, there's things that are in scope and there are things that are outta scope and generally they're time bound. So if I'm a vendor, I could go to a third party and say, please test these two systems when really there's 200 that are part of my product and take a week to do it, when really you would need four weeks to do it well. Mm-Hmm. Uh, to what extent should I really trust the attestation or the pen test? So understand that everything is time box. You're absolutely correct.

Uh, four week test is not as good as an eight week test. An eight week test is not as good as a 16 week test. But if we go back to one of the earlier things I said at the beginning, the vast majority of the vulnerabilities that are gonna be hit by evil attackers are the low hanging fruit, the SQL injection, cross site scripting, command injection, remote file, include, um, these are the vulnerabilities the attackers are going to go after.

Lucky for us, a lot of those are really easy to identify. So you're definitely playing with that 80 20 rule. So yes, I may only have two weeks, but I guarantee you we're gonna knock out the 80% of the vulnerabilities within that two week period.

So that's something, you know, you've always, you know, we always gotta go back to, but that's why we have continuous testing as well, because BHIS can test and then you can hand it over to Trusted SEC with Dave Kennedy and he's gonna find more vulnerabilities. You can head it to in guardians and j Bal and Mike Poor. They're gonna find more vulnerabilities. That's why we do this continuously. So that's number one. Always come back to the 80 20 rule.

The second thing is the first question, ask the company, what do they do to secure their apps and test their apps If they're excited, if they're like, look, these are the tools we use. This is how we bake it into our SDLC, these are the IDE plugins that we're using in our environment to test our code before it actually hits production. If they're excited about that, that shows a culture of security. Um, and that's what you're looking for. Do they have a culture of security?

Are they regularly testing? And you're gonna be okay? Perfect. Thanks. So talk to us about active countermeasures and like dis you know, deception and decedent and, and, and this whole world of, because we haven't really covered this in the, you know, in the, in the cyber call or with our, our audience yet. And I think it is an interesting world. It's, it's, it's another kind of thing to attract them to, to continue to evolve in their maturity journey.

'cause it, it gets to a really interesting piece. So talk a little bit about, um, about what that means and, and maybe specifically how it could be used against a threat actor like Gold Southfield and, uh, Sono tv. Oh yeah, absolutely. So active countermeasures, we have two things, right? So at Active Countermeasures, one of the first things we started on was beaconing detection.

Uh, when we were doing, uh, pen test for organizations at BHIS, they were never detecting our command and control channels. Uh, we could bounce off CDNs, we could use Google. Uh, we actually created the first proof of concept code in a tool called gcat that the Russians used when they attacked Ukraine, which is a weird thing to have happen. Um, when we were having conversations with the FBI about that.

Um, so we knew that there was a lack of tools that could detect advanced back doors that weren't just detectable by signature based detection. So that's AC Hunter and it does that. And we have a free tool, by the way, called Rita, real Intelligence Threat analytics. That's the heart of what AC Hunter does. So go check it out. It doesn't cost anything. You can set it up in like five minutes and it works really, really well. Now the deception part that's interesting too.

And kind of getting at the core part of your question. So there's a lot of really interesting vendors out there, right? Like you got elusive networks, you got a TiVo, uh, symmetry I think got purchased recently. Uh, trap X. So there's a lot of vendors out there that are in the realm of creating cyber deception. Uh, canary is another one completely spaced.

Um, so the reason why this works, right, is it's detection time plus reaction time must be less than the time it takes for an adversary to break into your network. So DT plus RT must be less than at, and we have very little control in modern defensive architectures over a lot of levers as it relates to the attacker, right?

So most of your defense is like you're wearing a brown paper bag on your head and attacker comes and punches you in the head and you're like, congratulations, I just found out that we got hit. Our IDS system detected the attack. Um, you notice the attack when you get hit. So if you use deception properly, you can set up deception in such a way that you can actually detect the adversary and waste the adversary's time, um, which increases your detection capability, right? So some examples, right?

You can use tools like port spoof. Port spoof is a utility that'll take the 65,536 ports that are available to a computer system and the ones that aren't used, it'll reflect back an end map finger, uh, fingerprint to identify those ports as open with live signatures of services. So you completely confuse the vulnerability scanner or the port scanner, what's actually running there. It can take a port scan from a few minutes to days to complete. That's easy.

And that greatly increases the amount of time it takes for an attacker to go after your environment. You can set up fake group policy preference files in active directory. Um, you can set up fake honey user accounts in active directory. As soon as somebody accesses that account, it triggers an alert postex exploitation. You'll be able to detect that there's a ton of these technologies that exist, right?

And traditionally everyone would say things like, well do cyber deception after you get everything else right? And that is complete and utter garbage. Don't look at that. Cyber deception is now key to absolutely everything that you do in computer security. And it doesn't have to be honeypots, don't stand up Windows server 2003 boxes set up things, like I said, canary users. If somebody ever tries to log into that user account, post exploitation, trust me, they will.

You'll be able to get an alert notification, put document files with names like passwords dot doc X on file servers. An attacker will automatically find those files and open them. There are ways that you can actually put deception through the actual kill chain. God help me for saying cyber kill chain, where you can put it in the kill chain where the attackers always go. It's like chess. Almost always.

People play the Sicilian defense pen testers and adversaries almost always do the exact same steps. Internal password spray group policy preference file, curb roasting service accounts. These are part and parcel to what we do. And you can put deception in every single one of those places to detect the adversaries. And that's one of the things we do with BHIS with our SOC services. It's absolutely essential and it's absolutely key. Yeah.

So you brought up Kerber roasting, I'm sure mimic KAZ is in there too as a favorite tech. Oh yeah, Absolutely. And you, if you get administrator, remember you gotta be admin first for mini KA to Fire, right? Um, so in, in a recent podcast, I think, you know, if I'm quoting you correctly, you said passwords are one of the number one things to focus on. Um, mm-Hmm. Help us understand why, why this choice and why you advocate for, um, passwordless solutions or at a minimum multifactor solutions.

Um, multifactor is absolutely key. Um, like I said, one of the things I can talk about Colonial Pipeline is they were very open. Oopsie. Oh, we had a VPN didn't have two factor authentication. And I, I, I wish we could sit here and be like, well, they're clearly the exception to the rule. No, no, no, no, no, they aren't. Uh, we do about 650 assessments per year, and a huge percentage of our customers don't have multifactor application enabled.

They may have it enabled in one place, but it may be disabled. Other parts of their infrastructure, like they may have it enabled for their VPN, they may have it enabled for their, let's say their, their email. But yeah, the rooster is priceless, but it's not enabled in, it's not enabled in other different portals that customers and employees use.

So it's about enabling that two factor authentication absolutely everywhere throughout an organization and testing to make sure that it's enabled throughout the entire organization. So that's key. Also passwords. It, it blows my mind to this day. I'm gonna rip on PCI because they deserve it. And PCI, they still maintain that their minimum password complexity requirements are seven fricking characters. That is not okay. Right?

And I've had people come up to me and they're like, well, you better be careful with PCI. Those guys will come after you. It's like, good luck. I'm hearing Costa Rica with a rooster. What are you gonna do? PCI? Um, but that password complexity requirement of seven characters is incredibly stupid. If you go back to the NIST Green Book series back in 1985, NIST back then was recommending eight characters.

So PCI, modern PCI, protecting credit card information, looked at that and said, Hmm, 1985 is a bit too onerous, guys, let's step it back a little bit to seven characters. And that's ridiculous. All right. All you want a go long, fast, quick two factor. Go ahead. You, you, You definitely gotta come back. You just, with that statement, you're earned your way back. You're definitely cyber call material. Thank you. Thank you. Yeah, I'll, uh, I'll hand it over to Wes.

So I wanna, I want Wes to get into threat modeling stuff, but Wes, Wes, yeah. Is it okay if I just ask John, share something real quick? John, you know, you mentioned two, you know, multi-factor two factor across 600 studies. You know, we hear it day in and day out. You know, there's, there arguably 75,000 MSPs globally. Not as many roosters but close. But, um, you know, so these, you know, a typical MS P might, let's just pick a number, 75 to a hundred customers, bell curve, whatever it is.

And you know, so many of 'em still can't get their customers to implement multifactor. Like if you were an MSP, what would you, what would you say to the customer? Would you fire the customer? Like give, give us your Wouldn't Worry. Wouldn't worry. Don't let your heart be troubled. I've heard some people say it's not a problem, barely an inconvenience. And the reason why it's barely an inconvenience is one way or the other, they're going to find religion.

And I, i, I always look at this from the respective of, you can push people so far, right? Right. But they've gotta jump off the cliff on their own. Um, but you know, you, you, you can't force people. I used to get really worked up about 15, 20 years ago in computer security if people did dumb things right?

And I would be like, oh my God, I can't now, I, I don't care if I've sent them an email, I've notified them, I've given them emails, I've given them PowerPoint presentations, and they still decide that they don't wanna do MFA, that's fine, because I'm not the one that's going to convince them. The hackers are going to convince them. And the hackers don't care about your legacy technology.

The hackers don't care about your users and how, how long they've been with the company and trying to get them to change something is really super dupe or hard hackers don't care. And I can push as hard as I want to, but at the end of the day, the attackers are gonna show them the light. Okay? And I've actually told that to some of my customers point blank. I've been like, that's not a problem. That's great. Fantastic. That's good for you. Rock on. Um, by the way, the attackers don't care.

You can argue with me all day about how you don't think doctors are willing to implement two-factor authentication. You can tell me how the CEO read an article about how SMS push notifications are not that secure. That's great, fine, whatever. I do this for a living. I break into companies for a living and on behalf of the offensive community everywhere, I wanna say thank you.

And, and the Chinese are gonna say, waiing, the Russians are gonna say Dasia, but they're going to come in and they're not going to hesitate to take advantage of these vulnerabilities. And they don't care about the, the types of problems that you have. And the final thing that's always blowing my mind is every one of these organizations have absolute like, oh no, this can never be done. They get hacked once and it's enabled within 48 hours. It's amazing. It's a miracle.

I don't know how Fair, no. Alright, Wes, thank you for that. John, One of my, uh, good friends who's been a mentor for me says very similar things. I remember I've said this on the cyber call before John, but I'll say it for you. He said, uh, he's like, Hey, the best way I found to get my board to even care at all about cybersecurity is to have a breach. He's like, trust me. I know it's, I know what I'm talking about. It's happened three times in a row. And, uh, there's truth to that for sure.

And, uh, I, you know, it's almost like, you know, the analogy of like, look, if I'm gonna design some highway road on a super curvy road, like out in California or something like that, and you know, they don't wanna have like, guardrails along the side of it, fine, that's, you're gonna be the one that's gonna drive off the cliff or you're whoever, you know.

And I think cybersecurity comes down the same way, is so, so often we wanna put those roadblocks on the road and like, I gotta stop this and gotta prevent this. But in reality, we should be thinking about how do we design guardrails around this? And if they don't want to put 'em up, uh, it's a risk decision that they're gonna have to accept. And that's one of the things that we're really lucky about.

You know, at BHIS, like almost all of our customers came from teaching at the SANS Institute or on Security Weekly or ions events or from our webcasts and all the things that we do for the community. So they know what they're getting into with us and they know they've got an idea about security. We don't get a lot of just people like OH'S googling around and just fell into you guys and what's the security thing that doesn't happen?

Um, so luckily we have a very high percentage of people that want to do the right thing. It's very rare that we get a customer that doesn't, and somebody mentioned firing the customers. Yeah. We've had customers that we've tested two years in a row, fixed absolutely nothing, and they come back for a third year. We just no bid. And the reason is, I really feel uncomfortable with my name being a test report for a customer that got popped in the past that that makes me really uncomfortable.

Exactly. And, you know, and to some of that, I want to shift gears a little bit to most of our audience here, our managed IT providers and they are serving small and mid-size that don't have the budgets. And it's probably not a stretch to say, John, and maybe I'm incorrect, but you guys probably, you probably have a lot of thoughts for s and b, but may not actually work with him as heavily just because of who you guys are.

And, um, by the way, I remember I wanted to say this, I remember the first time I found out about you guys. It, it was still a professor of security like way back in the day. I think it was like 2011, and I saw a YouTube video by one of you guys. It may have been you, I don't remember about, um, how to exploit a company with nothing but like medium and low CVEs and I, it was probably a black hat video or something. I was blown away. I was like, who are these guys?

This is, it just really changed my view on a lot of things. And so I wanted to say thank you for the content you guys pump out. Um, so here's my question I wanna start with is thinking.

So we just got done going through like a threat modeling workshop that Ryan Weeks and myself and a few others kind of went through and really teaching and, and getting across what does it look like to understand how to build the threat model so we understand who our adversaries are, so we better understand ourselves, our enemy, the battlefield that we're on.

How can we use that to drive, um, you know, building information security programs built upon that knowledge and that kind of maturity rather than just throwing stuff at walls and hoping this vendor stops this and this vendor stops that. We're really trying to get past that, right? And so getting into as well, like adversary emulation, can you, we, we, we touched on that, we brought it out a little bit.

We kind of showed red canary Caldera, you know, how large enterprise are really trying to understand TTPs and attacker, um, behavior and operations. Is that even possible for MSPs and small business? Give us some feedback on that. It should be, and and, and I know for, for our soc this is what we're doing. We're doing adversarial emulation is baked into what we're doing. And I, I'll explain why here in just a couple minutes.

So the first thing is, um, whenever you're looking at reports of the techniques that are used by like APT one or APT 29, that's garbage for trying to develop a future security posture for your company. And I wanna explain why. So let's say an attacker chained together 15 different techniques to break into an organization. Then people look at that and they say, well those are the 15 techniques that APT 29 uses. No, that's not how offensive security works. Offensive security is like water.

We're going to find the techniques that are gonna work to allow us to meet our specific objective. It goes back to the blind philosophers describing an elephant. They're only touching a part of the elephant and they're trying to describe an entire elephant. And many of these different groups that give up these reports and these write-ups of these different a PT actors, they definitively state, hey, these are the techniques they're going to use. Which what's again, is garbage.

An attacker is gonna use any and all vulnerabilities that are at their disposal to break into your organization. Full stop. Alright? So don't look at it like we're gonna find and detect these things and then we can stop this group. So what the beauty is of Mitre and uh, atomic Red team and tools like side that are out there is they allow you to emulate a large number of attacks. So if you look at Atomic Red team, there's a huge number of attacks that it actually goes through and it emulates.

So instead of just emulating 15, run the mole, basically put 'em on a, a workstation and name it, test something with a test domain user. Make sure it's domain join, put atomic red team into a directory and then basically allow your endpoint product to alert but not block. Then you just let that puppy run, just let it scream on through all of its different techniques and it kicks out it's an output, which techniques from Mitre it fired, which ones failed?

Then you can take all those techniques and then you can look up in your sim which of those techniques were detected. An e sim worth its salt today is going to come and say, I've detected this technique and it trace, uh, traces back to Mitre. So you can kind of do a compare contrast of those two things. This allows you to emulate a large number of the different techniques that adversaries tend to use to identify those gaps. Yes, it does take work, but you have all the tools to do it.

You can actually do that today and that's awesome that you can do that today. So work at it from that perspective. That's why that adversary emulation is so important. Just let me give you an example, sorry, I'm monologuing and I apologize.

But if we look at vulnerabilities, people run Nessus, they run Qualys, they run rapid seven and they're looking for remotely exploitable vulnerabilities that if you look at Mitre is two, maybe three of the boxes of all the techniques that are available to an adversary. The vast majority of the techniques that we use are below the surface. And those adversary emulation tools should be part of your vulnerability management portfolio of what you do every day.

If you're a vulnerability management company, please do me a favor, put the crack pipe of vulnerability prioritization down your friend. It was never your friend, it doesn't work, it, it's failed. And I'd love to get into that some more, but vulnerability prioritization from your vendor is an incredibly stupid idea. We can get into that in more detail and focus on adversarial alation. Go buy a company that's doing adversarial emulation.

Do it as part of your own company because it is part of your vulnerability landscape. It is not a missing patch. It's not a CDE, it is a domain misconfiguration, it's an inability for an EDR to detect and attack focus on that and we're gonna start improving security much faster and coming up with another way to automatically prioritize vulnerabilities. Once again, automatic vulnerability prioritization is total complete garbage. Uh, please don't do that anymore.

You're not helping put the hammer down. So Houston, we have a problem.

Uh, one thing I wanted to come back to what you said, John, it's a huge problem for our industry is you, when you're talking about like letting something like, uh, atomic Red run, having the EDR or whatever the endpoint is, just alert, only have it go into a sim, we're challenged because I would say 90%, maybe 95% of our, our industry and probably 60% of those on this call, not only have a capable EDR of doing that as a capability for that, but also have no sim and no place to even see those logs.

Okay? And, and anytime you get to see that, that's an opportunity folks. You're gonna have some customers that'll look at your total technology portfolio and they're, they think that they're spending too much on security. And there was some beautiful person that was here earlier in the chat and they're way far up, but uh, they basically said, insurance is going to also fix this for us.

So my point on that is if you're in the MS MSP game and you're looking at it from the perspective of we're gonna continue doing what we're doing and everything's going to be okay, no, the world is going to change. MFA is going to be become part of your insurance requirements to get your cyber liability insurance full stop. It's coming.

If you're looking at a SIM and detective capability that's coming, and if you're an MSSP or an MSP you have or an MDR, um, if you're looking at any of these different things, this is a point of inflection in the industry. This is a point of change in the industry.

This is an opportunity for your companies that you mentioned, like the 30 40% that are on that vanguard to push that further forward and be in the position where you are ready to take on that challenge and actually bring solutions to your customers. Now, right now, some of your customers aren't gonna spend money on it, but I know that in almost all of your technology portfolios, you guys have that capability. Most of your customers choose not to pay for that capability.

And a lot of times it's very easy to just simply turn that on. Somebody was mentioning Azure Sentinel. You have things like Sisson, you have Elastic, uh, which has an amazing, uh, detection capability built into it right outta the box. And the pricing isn't that bad either. So there's a lot of amazing things that are coming down the pipe and I I, I hope, like I've talked to some customer, uh, some MSPs like info aggressive, uh, Justin and I have had conversations.

They're definitely looking forward and they're pushing forward in the industry and they're not gonna allow this to happen to them. They want to be on top of it. So this is an opportunity, folks, you know, if we look at this, we can actually grapple with it and we can actually start selling additional services.

'cause one way or the other, our, their insurance companies are going to want to have these services and we have to be right there at the front line with those services ready for us to sell it to them. That's so well said. I wish we could just clip that Andrew and just kind of use that. That is so well said. Go for it. Go for it. One, one thing. Thank you.

And John, one thing that you also said that, you know, earlier that I want to come back to is talking about end user clients and don't want x, y, z control and just don't think they need it one way or another. They're gonna find religion. Uh, how many of you guys in chat, this will take about 10 or 15 seconds probably, but give me a yes or a me if the past year or two you've found religion, you, that's why you're on cyber call.

That's why you've, you've realized cybersecurity for MSPs, like the game has totally changed. Um, I wanna get some feedback from you guys in the audience because I do think we've found religion here and yet we're still stuck with some legacy tools. We're stuck with some of the fear of what we, what Jennifer had mentioned earlier in chat of like legacy software that we've talked about with you and Ryan. Wes, these are big challenges. We we're Stuck with something worse.

A legacy business model, Dude. Yeah, that's right. Then we're trying to figure out how do we ram cybersecurity into that. That's exactly right, Gary. Okay. I will agree though. I will absolutely agree 110% with what Wes said because I know how many of you try to go out and you try to sell the right thing and the customers are like, yeah, but your, your services are 25 more, 25% more expensive than this other person. And they call it the magic sock in a box. I'm like, what does it do?

Well, they sent us this box and then the lights blink and we send our logs at it. Does it work? I don't know. The lights are blinking and it's cheaper than you, so you're gonna be fighting that, right? You're Absolutely fighting that. And marketing said it stops threats before they happen And marketing said it stop threats through artificial intelligence, right? Um, and you're like, it's gotta cell run processor from 10 years ago.

I don't, I don't think there's artificial intelligence in that box at all. It's probably the exact opposite of intelligence, but that's okay. You know, that's okay. You know, you fight the good, fight you lose, you fight the good, fight you lose. And this is one of those key things for, for companies to be successful. You don't wanna be a vendor that just goes off into the ether and you're never seen before, uh, seen again. Right?

You know, the, the logs come in, we collect paychecks and hope to god nothing bad happens. You wanna be an active advocate and you want to be working with your customer. So whenever they do get the insurance company that says, look, you need to have two factor authentication. You need to be having a sim solution, you need to have an MDR you, you're gonna be sitting right next to them and you're gonna be like, they're there.

Hey, let's talk about some things that we've been talking about and we can help you. You're already in the door. Don't allow somebody else to actually come in and take that from you. 'cause somebody said the legacy business model prevented you from doing so. Ah, so good. Uh, my last question is I wanna make sure we leave some time for Gary in, in questions from the audience. So, uh, as well is, uh, so you're one to pontificate, you're one to go on your soapbox and I appreciate that about you.

Um, so I want you to do that for a minute. When you think about what, you know, in your years of experience and you think about MSPs and SMBs really, 'cause 99% of all SMEs are served by an MSP, right? They, they're not big enough to do it on their own. So are we, are we completely failing here? Uh, is there any chance of success? Is there any light at the end of the tunnel? We see all of these attacks that are happening. Ransomware is systemic. Like are we just totally backwards here?

I would love for you to step back on that soapbox and, and give us some thoughts on where we're at here and what it takes to even have a chance. Alright, so I'm gonna let, let's back into this. So do you remember, if we go back about seven years ago, what was the average dwell time you would see a, the numbers would vary, but what was the average dwell time about 78 years ago for an attacker sitting on a network? Does anyone remember what that number was? It's like 180 And it varied.

It varied, but go Ahead. Yeah, it's a long time. Way over 30 days. Yeah. So way, way, way over 30 days. For many of the reports that came out, they said the average dwell time for an attacker was anywhere between a year to two years. That was seven, eight years ago where the attackers would sit on that network for hundreds of days.

Right now you're seeing this number come down and I have seen a lot of presentations at like RSA and at Black Hat and different different conferences where they say, look, everybody dwell time's coming down. We're down to like two months now. Look at us. We're doing so well. We're getting better. Unfortunately, whenever you look at ransomware, those ransomware attacks are baked into those calculations.

So if an attacker breaks into a network, they're there for five days and then they notify you that they have hacked your network, then the dwell time for that situation was five days. So you're seeing the nature of what an attacker is doing actually heavily skewing. Yep. Those dwell time calculations back down way further than where they actually should be. So that's depressing. Um, that is really, really, really super depressing.

Um, the other thing that's depressing is I believe whenever I was young and up and coming security professional that the entire security industry would progress. And it isn't. It's spreading. You have organizations that have little to no security whatsoever. And then you have organizations that are very, very, very, um, they are very much at the cutting edge of computer security, like light years ahead of the people that are lagging.

So what, what I'm talking about with this is things are getting better, but for very small percentage of the overall IT industry, the percentage that has been breached and decided that they didn't like it, that frying pan was hot. The percentage of organizations that have gotten CTOs that get computer security, the percentage of companies that understand the risk is far too great for their organization. That is probably 5% of the entire IT industry.

And then there's this huge spread all the way across. So it's very difficult for me to say that we are in fact getting better. What we're breaking into is first world, second world, and third world IT security organizations. Yeah. Okay. That's helpful. Um, John, thank you for joining us today. It's been been awesome. Just wanna say thank you personally, Gary. Yeah, John, this has really great.

Um, although, you know, I hear everything through my years as running two MSPs and working every day, right? With MSPs and peer groups and coaching and you are highlighting why we are where we are, which is you rattled off a bunch of tools, right? You can go out this, it's free. Well, it's free if you are, uh, an IT shop if you're, if you're running an IT shop for a a thousand person company, but it's not free.

If I'm an MSP, my first MSP had 180 customers to be able to take that free technology and use it at 180 customers is super expensive. So I just jotting down like everything you mentioned, thinking okay, estimating, what if we did that for every customer? What if we did that for every customer? I'm up between three and $400 a seat, right?

For us to be able to go do that, which I think is where we have to end up, like we have to get where, where you're talking about, um, it's just a really hard place to get to. Let's, but Let's, let's take a couple of things though. Let's, first, let's go back to context. Whenever we are talking about free tools, I was talking about evaluation of companies, third parties that you work with to make sure that they're actually securing their stuff. So that was the context associated with that.

And it is free. Whenever you talk about these things in these services to your customers, one of the things that you have is your foot is in the door to 180 customers, or a thousand customers or 2000 customers. The hardest thing that you have in actually selling these services to your customers is training your sales team to be able to adequately sell that to their customers and do it as a value add.

So this is, this is like one of the things I've learned from some friends of mine that are Harvard grads, which is this weird kind of dark cabal of men that wear rings and uh, lets you know that they graduated Harvard within five minutes, but they speak the truth. One of the things they say is you never want to be in a position where a competitor can get their foot in the door to disrupt you.

So an example would be if you have another company that comes in and starts offering these services to your customers, that is a door that opens up for them, that allows them to come in and possibly start stealing work from your company. So you don't necessarily have to provide this as part of the fixed price for all of your customers, but your customers damn well should know that it is a service that your company can in fact offer. Yeah. And you're going to charge for it.

And that allows you to actually hunt down those customers that are in that 5%. And you wanna focus on those people. You want them to be your buddies, and then you wanna wait for your other customers. They may not decide to purchase that and that's fine, but you want them to be like, to have your company right at the tip top of their brain when something starts going sideways. Yeah, that's a really, really great point.

And, and to kind of, in the real world, what I'm seeing across our customer base, the MSPs that are following your lead on that, they're the ones that are educating their customers, um, changing their security posture, increasing their prices so that they can do it. Are the MSPs that are selling the most recurring revenue, they're adding the most customers. So they are weaponizing it just as you said, against competitors when they're asking questions that the prospect can't answer.

So that's a huge, that's a huge takeaway for this. Um, so I wanted to ask you, can you give some perspective on like, people can get into cybersecurity, like, you know, if I want to, you know, be a doctor, I go to med school or a lawyer, uh, you know, I, I go to law school. I guess now's a bad time to say that my daughter has a Harvard law degree. Nope. It's not a bad thing at all because remember I said that good advice came from a Harvard BA. So there you go.

So How, how, how do we build this? We all need talent. Like where, where do people go to get going on this? So whenever we started getting into SOC services at BHIS, um, it actually started, I would say 12 to 18 months before I had decided, Hey, I'm, I'm gonna do this. And we would work with our customers and help them choose SOC vendors. And I was absolutely shocked at how tight the profit margins are for MSPs and for managed SOCs and all these different things that are out there.

And the thing that is really, really difficult is understanding that limitation and saying, well, everybody that works for this MSSP or the SOC or whatever, they should be getting trained constantly in security. That's not feasible for a lot of these companies. They don't have the profit margins to actually support that at all. So that, that's problem number one. Problem number two, y'all are the front lines, right?

And whenever I started coupling these two things together with the fact that we'd pen test and a lot of these MSPs would not detect us, I realized that we were making fun of MSPs and MSPs and I started feeling really horrible about it. And I'm not joking about that.

I started feeling horrible about it because I realized that that poor analyst that's working, that particular customer is gonna get beat up because they weren't able to detect me who's been doing this for 20 years and they had zero training and they had zero background. So this coupled with a lot of other things, uh, led us to move into the pay what you can training. Um, there's a link down there, soc core skills.

And I, I realized that by and large, um, almost every single problem we have in security can be solved with education. It absolutely can be. Um, so it is pay what you can, which means if you can pay $500, great. If you can pay 200, awesome. If you can pay 50, great $5, no problem. If you can't pay a dime, I don't care.

And I have had entire corporations send their entire team through and the owners of those corporations contact me because they're caught in this inflection point, this t shift that's happening in the industry right now. And they're scared. I'm a business owner. There's a bunch of other people that are on this call that are business owners and they know like, oh, John wants us to do all of these things. How the hell am I supposed to do all of these things?

It's like going to someone on the street and saying, Hey, have you tried not being poor? That doesn't work. So with the pay what you can training, we basically tried to break down the gates of computer security. Um, I, I trained for years with other organizations and that was great and it was very fulfilling, but I realized there was no way we were gonna fill this bottomless void of security IT skills unless something changed the game fundamentally, dramatically and immediately.

So the pay what you can is what we did the first time out. It blew my mind away because we got 5,000 people, 5,000 people that registered for our first training out of the gate. That was insane. And the other thing that kind of surprised me is I made as much money. You know, we have these shirts at BHIS that say proudly sucking at capitalism. And it doesn't mean that we hate capitalism. It just means that we've suck at traditional definition of capitalism.

So what happened was we actually made more money off of that. Pay what you can class than I made from teaching, just teaching for an entire year at a previous organization in four days. So there was money to be made there, which was weird, but we had the capability to fundamentally change the lives of thousands of people. Right off the bat.

Our last conference way West, hack Infest in Reno, I had four students or four attendees come up to me and they said, look, man, I took your class and I'm now working in the security industry. I'm working in a soc. They paid for my training to get here. I have four. And that's huge for me to see that many people. We see it on LinkedIn all the time. Tell us, Can you tell us more about that event, what it looks like, what it is?

So it's a, it's like, uh, in security we have these conferences, um, like defcon, black Hat Derby Con was one that doesn't exist anymore, unfortunately. But we have these cons where security professionals come, all come together. We talk about security issues. We have tons of hands-on labs for everybody to be able to, uh, play around with, like cloning, RFID, ca, uh, cards breaking into web applications, um, software defined radio, all this stuff. It's all there.

It's like a hacker playground, uh, to truly learn the issues that we're confronting in security, and then how to actually secure systems as well. So that's, that's my thing, is we really need to be at the point where we can get these people training and, and as a security professional and being a red teamer, I realized the solution wasn't me gloating and saying, ha, I broke a blue team.

Again, the solution was to basically make my training as accessible as possible for as many people as possible. Yeah, I mean, there's really no other way, right? To, to move this forward. You don't even get, I talk about the, the business model things, the constraints of how MSPs have been set up for 20 years, but you don't even get to the point to fix it if you don't raise knowledge. I, um, I, I loved your comment.

It was like going up to someone who's poor and say, did you ever try not being poor? Yeah. Uh, that's a really good, uh, analogy. I think, you know, right. As, as to, as to where we are today. Like, did you ever try being more secure? Uh, yeah. You know, And I say, and I said things at the beginning, it's the easy stuff. People, and people are great. What are those? Specter, meltdown, row hammer. Not even close.

You're not even in the ballpark, but that's what's in the news and that's all that they know. Yeah. So I mean, from some of the takeaways that I'm taking away today are, uh, on one side of it, you talked about a lot of stuff that people can do. Uh, a lot of tools, they can use a different way of thinking about adversaries. But if you go back to where you started, which is how much of it is things that everybody with some focus and discipline, uh, can take care of.

To me, that's the hope that came out of, uh, today, is that there's a lot of work to do. But I think if people can start at the beginning and get that right, we can put our finger in the d**e while we get that work done. See, and, and this is one of those reasons why I was, I was very excited to come on here, is, you know, MSPs, MSPs, MDRs, you're the front lines like this battle is gonna be waged by your people, not my people.

Not, not, not the people that are red teamers, not by people that have corporate IT budgets in the hundreds of millions of dollars. Like, we're not gonna be anywhere near the front lines. We'll be sitting in our gilded tower sipping my ties with no salt, no salt watching this all go down because the wave is coming to small and medium sized businesses, the Russians, the Chinese ransomware, there's money there.

And I, I am absolutely terrified because I know the capabilities of the adversaries that you are all gonna be going up against. And I really, really want to make sure that we can position as many people in your part of the industry as possible to be ready for that wave. 'cause I, I think it's already here. It's already happening. And, you know, you look at a lot of companies, even like BHIS, I don't need any more sales in my sales pipeline. I don't, I just don't.

We're completely booked up on sales calls for the next three weeks. We can't keep up. But once again, if we're going back to who's gonna be on the front lines, who's gonna be in the trenches, it's the people that are listening to this podcast. It's the people that are gonna be on that front lines that we need to reach out to and we need to reach out to. Right. Freaking now.

So the last thing I'm gonna ask you, and I'll turn it back over to Andrew for a few minutes to kind of start summing things up, but, um, and this will be less factual. I would just like your opinion if, um, you know, dealing with cybersecurity with MSPs and, and SMBs was a baseball game, what inning do you think we're in? Oh, oh, the baseball game? We haven't gotten done with the national Anthem yet. I'm sorry.

Like, I know that that's a horrible analogy, but I'm gonna use a structural engineering. If you can see trusses behind me, I always use the structural engineering analogy. If we go back thousands of years ago, structural engineering was basically building a mud hut with better mud. Some mud was better than others. We didn't know why. And then we'd build it with rocks and then Ugg would show up and he'd push the house down and be like, UG rocks.

And then your engineers would be like, well, we gotta reinforce it against Ugg. And over thousands of years we encountered h earthquakes, hurricanes, tornadoes, other jackasses, and we constantly started building our buildings better. And it was all about architecture and it was all about the study of failure. If you look at it, we're still in mud huts. There's a lot of technology that's out there, but it's changing so quickly. New technology is coming out.

AWS is coming out with like new technologies faster than people can even recognize what the new technology is. So we aren't even at the point yet where I look at the history of it and where we're going, that we're going to be ready to start developing solid infrastructures for security because we're right now tearing down active directory, tearing down customer firewall, DMZs, and we're shoving everything in the cloud in mass. So that means we're starting over Really good.

Uh, I think my next podcast might start with, might be called the National Anthem. Uh, so thank you. And, and, and I'll give you, and I will give you a mention. Uh, I'll give you a mention on it. So, uh, before I hand it back to Andrew, I just wanna say thank you so much. Like this was, um, not just informative, but this was a really entertaining hour. It's probably one of the fastest hours, uh, that we've had. So thank you very much and thank you to the rooster. Still There staring at me.

Is that gonna be dinner tonight, John? Uh, so, um, John, uh, before we wrap up again, huge, huge thank you to you for coming on doing this while you're on vacation with your family. I, I just can't say enough. Um, can you close us out with anything you'd like to share about education? What could I do our community do to get, you know, you more students? 'cause you're, you know, pay what you can model.

Like again, we, we need to up the game of security across the board and you know, you Tet tell us what, what we can do here with the community. We have access to a lot of MSPs. So the floor is your, so, Okay, so first and foremost we have the, we have the pay what you can class, sign up for it, send it to a friend, do me a favor and send it to somebody that's not even in it. I want to get people that are washing dishes. I wanna get the waitress that's working two jobs and has three kids.

I want to get the people that are looking for a pathway to a better life that pays well. Um, get it out to those people, get it out to your friends, get it out to your family. They have any, any type of tech like inclination, get it out. The other thing I'm gonna ask all of you is you need to start sharing with each other, anything and everything that you possibly can. Let, let me give an example.

In security, we have imposter syndrome where people are like, if I haven't written a zero day or I haven't come up with a novel technique or something, then I don't wanna present of all the presentations that I've ever done. The ones that are the basic core fundamental skills are the ones that are always the best viewed on YouTube and the best attended.

So please do me a favor and don't worry about if it's technically like top shelf, the best stuff that's out there, just get out and share because I share it with so many people. But you may be able to share it with somebody else in a different way. So get out there and share because in security we have this problem where we're all wizards trying to impress other wizards and that's boring.

Get out there and share the simple things because right now, as I said, the MSP market is going to change dramatically over the next, I wouldn't even say five years. I'm gonna say the next like six, seven months. The game is gonna change and you all need to work with each other. And you all need to realize it's a game of hungry, hungry hippo with hundreds of marbles. It's not like a zero sum where I win, you lose, you lose I win.

You need to be sharing with each other what you've learned, what works, working with customers, what doesn't work with working customers. I don't know, start a podcast. Like guess what, you know, get out there and share as much as you possibly can. 'cause if we don't share and we don't work together, we're screwed. Yeah, Ben, fantastic way to close us out, John. So I'll be in touch with you.

We're gonna figure out some ways to get, you know, like I said, you in touch with Joe Pan Terry, as soon as he's back, he's got a massive, uh, go ahead. What were you gonna say, John? I was gonna say, I'm also gonna invite you all on to come play a game of back doors and breaches. Uh, with me. We're, I'll set up an incident. You guys are the defenders. Um, so we're gonna invite you guys on our webcast. That would be, uh, because I figured that that's just fair. Yeah, that would be great.

So, uh, again, on behalf of everybody here, John, and, and the comments and everything, like I said, we're gonna do everything we can to get what you're doing to help anybody that wants to get into security field out there. Uh, but again, make make it a fantastic day. Have a great rest of your vacation and we wish everybody the best. Take care Later, everybody.