Cleo Vulnerability and it’s Impact on MSP’s Customers

All right. Again, I'll give it. Uh, so what I was, what I was told everybody was we're gonna, even though they're probably hearing me in about 10 seconds, is we have to kind of wait just a little bit, um, because there's a delay. So, um, right about now, um, they'll, uh, they're, they're starting to come in. All right. So, uh, we'll kick it on off here. We'll give some time for people to come in. Uh, welcome everybody.

Um, I think Phyllis, you know, you and I were talking early on, uh, early in the day, but I think this will be the last cyber call of the year. Bob, let me ask you, from an MSP's perspective, next week's, a few days before thank, uh, before Christmas, right? Following ones a few days before New Year's Eve, I think it makes sense, wouldn't you say to, uh, a hundred percent. Yeah. PI don't think people are gonna be people. Yeah.

So, so it, I I'm couldn't think of a better way to end it, uh, with our special guests, who I'll introduce soon that a lot of you guys all know. Um, so, um, kind of kicking things off here, um, last week, you know, or the last two weeks, right?

We did, you know, business planning with Gary two weeks ago last week, you know, we, we really focused in on automation because it was one of the things that Garry said was what he was starting to see as one of the big differentiators of companies, both operationally, you know, from, from an efficiency perspective. But he sees it as the next frontier in M-S-P-M-R-R.

And so we focused in on, um, uh, uh, we had Tim Fornet, who's in the audience here, and, um, uh, one of the folks over at, uh, source Path. Um, and, uh, it was a great, great show. So, um, today, um, you know, one of the things that caught my eye last week and always catches my eye of John Hammond, uh, is publishing something, um, is the, uh, this Cleo vulnerability, uh, which we're gonna talk about today.

Um, and so a little backdrop, you know, um, John, I'm gonna kind of let you intro in a moment, but like, just come to you first. Um, you know, last year we had move it, which, you know, arguably even the threat reports talked about even skewing some of the data, right? Meaning how it really rose in terms of that, that exploit in certain areas. Uh, I think even, uh, Verizon called it out, uh, Phyllis to a degree that it skewed their data.

Uh, and a few of the other threat reports did, I think, Microsoft's that. But these file transfer protocols, or these file transfer softwares, John, this one's a little different where, you know, move it, you know, was, you know, exploited, right? In terms of there wasn't necessarily a vulnerability per se that was just, you know, exploited by a threat actor. Um, this one was a theoretical patch that wasn't really patched, but again, big on supply chain.

Is it, is it, do you feel, you know, whether there's a vulnerability or not, this is just an area threat. Actors are like technology leading the way as far as threat actors go. Um, just an area they're just gonna continue to focus in on because of the ripple effect that these types of softwares have. Yeah, I mean, undoubtedly, uh, without a doubt, and I'll color that picture, uh, a little bit more, but, hey, first of all, if I may, hi, yeah, please. Hi. Hello, everyone.

No, uh, look, my name is John Hammond. I'm a, uh, security researcher over at Huntress for my day job. Have a whole lot of fun with them doing really fulfilling work, you know, making hackers earn their access and trying to cut up malware, trace, hacker trade craft, a lot of fun on the side. When I can fit it in.

I have a silly YouTube channel that tries to showcase a lot more of that cybersecurity education, either looking at malware, hey, cruising through the dark web, weird, nerdy and geeky stuff. So I do try to stay close to the keyboard. Um, but Clio has been a wild ride, uh, by no means the scale and severity of, uh, move it, move it transfer exploitation, just as you were alluding to. We've seen others, uh, go anywhere, go anywhere. MFT was a long time ago.

But that acronym, MFT for file transfer is just, as you mentioned, a pretty attractive target for hackers if, just because it means there's gonna be data there, there's gonna be info, there's gonna be files, of course, and a file transfer application.

And in a lot of cases, that may very well mean sort of business to business relationships, because if it's for one thing, your employees, your workers, hey members of the team to have a place to put their files, no matter where they are in the world, it's also a place for collaboration between other organizations.

Um, and for that reason, I think a lot of times, not to say the easiest, but like the most default knee-jerk reaction from maybe a lazy cis admin is, Hey, that thing's gonna be open to the internet. It, it's gonna be exposed, it's gonna be out in the wind. Um, and whatever ports or services, it's just hosting, it's hosting them for everyone. Good, bad, and ugly. Uh, yeah. One opportunity for adversaries to take advantage of. Yeah, absolutely.

And John, um, your silly little YouTube channel, I think has over like 1.5 million followers, right? Yeah. We're inching up to, uh, the big old 2 million. So I'm very flattered, uh, but very, very, it's a, it's a blessing, and I'm grateful for all the support. Yeah, absolutely. So if you are out there, you, you know, if you're one of those people that really want to get, you know, into the weeds in terms of, and in a good way, so I don't take that negatively.

I really wanna understand, you know, malware and, and reverse engineering, and John's constantly, you know, out there teaching. Um, this is something that, you know, I highly encourage you to let your security teams know, or your technical folks know about. Um, you can subscribe to. And John, feel free to, um, I, I'll actually you, I'll, I'll put this, this, unlike Crowdcast doesn't have the, everyone can chat, but I'll put your URL in there for, for folks.

So Phyllis, let me hand it to you, um, kicking things off, uh, with Mr. Hyman. Yeah, well, it's great to see you, John, and great to see you again, Bob. Um, so John, can you, um, give us the scoop on the Clio vulnerability, and, um, how did you stumble upon it and what made you decide to dig deeper? Oh, well, thank you so much. Uh, and there is a whole lot to unpack, so please feel free to, I don't know, start and stop me.

I don't mean to ramble, uh, for too long, but no, this really got started last week, and then a bit of the week before, kind of bleeding of the weekend. It was, I think December 7th when there was a, a Slack message. And I can share screen if we'd like. I dunno, it'd be fun to do some show and tell, certainly don't have to. Um, but our, one of our security operations and our analysts Oh, awesome. Thank you so much. Yeah, You not, you can, can you see it down there in the, in the, um, restream?

Yeah. Fingers crossed. I'm not leaking anything too sensitive, but I think it's fun when we get to go a little bit behind the scenes. Yeah, Absolutely. And not, well, actually, Forgive me, let me, uh, here, stop that.

Now, um, one thing I think is worth noting, our security operation center analyst was tracking this because he saw what looked like multiple different intrusions, uh, hey, compromises and infections that were all stemming off of, uh, a trend, uh, one specific piece of software, at least one name that was kind of all centering around them. Uh, there were three different software suites kind of at play. There's Harmony, there's VL Trader, and there's Lexicon.

Uh, these are all part of Clio and their offering. But Tanner, and I gotta give credit where credit is due, one of our security operations center analysts was saying like, something doesn't feel right here. Uh, something really weird is going on with the signals that we're seeing. And we're seeing some malicious power shell dropping what looks like JAR files, Hey, things that could be used and later leverage all around Clio.

Um, but this is December 7th, and this is the timeline where, okay, things are now starting to grow. Uh, the number of infections we started with was maybe 10, just, hey, touching the double digits. Grew to 14, grew to 18, grew to 24, and then, uh, we wanted to really see, you know, what, this is gonna be something bigger that we should chase And John, so that's just Go ahead. Oh, sorry. I say one thing when I looked at, at Cleo's suite, by the way, Phyllis, this is for you too.

The three that John mentioned, John, it looks like it's like, kind of like, I'm joking around, but it's like they're good, better, best. It's like single user, multi. It's, it's this, it seems like they, they all do this a similar thing, but it's size and scale based on the organization, the complexity, they, they, they need, uh, fair, you know, just for those out there that may not have investigated what this, this software does, I believe so.

And I, I would love to have anyone else kind of chime in. Uh, it's, it's wild, it's silly. It's funny, whenever we try to spring into action, oftentimes it's with software that we aren't too familiar with. Uh, we're just seeing Uhoh, it's being beaten up. It's broken some way somehow. Uh, let's learn more about it. But from what I understand, right, harmony, vl, uh, trader and Lexicom are different tiers and size of, of, uh, yeah.

Power and usage given to you for multifactor, or excuse me, uh, uh, file transfer without a doubt. And, and Phyllis to hand back to you about, you know, know, your, know your inventory, right? That's right. Knows what's on. Like this, this was one where, you know, uh, you know, Bob can certainly chime in, but you know, you sit there and go, I've been in the space 25 years now, and I don't, I, I don't remember seeing it. Like, yeah, I remember when a client, Iran Clio.

But when you start to realize, right, if you're not doing inventory, that this is a pretty widely used solution when it comes to organizations for the logistics and, you know, manufacturing and, and said things like that. So go, it always comes back to right, know your inventories, whether it's your data inventory, your hardware inventory, or your software inventory, right, Phyllis. Exactly. It's always back to the basics, right? Yeah.

And so that's job number one is are you using this piece of software? If so, what version are you using? And did you patch? Now, of course, we always wanna say patch, patch, patch, patch, patch. Which leads me to my second question for you, John. Um, you realize that the patch actually didn't mitigate the issue. I'm just kind of curious, a little off script, like, were you just seeing the same attacks even after people were applying the patch?

Or is it that you are, I mean, I'm assuming your organization doesn't, Um, patch, and, and, and John, can you give a little history here for those out there that don't unders, you know, are coming into this, that may not have researched this whole thing with Clio. Can you maybe go back to October when there's a vulnerability in Clio? Right? Totally. So if I may, let me add the asterisk and the little disclaimer here. This is when the story kind of starts to get a little bit murky. Yeah.

Uh, there's a little bit of nuance, uh, but I'll do my best to, to cut through it. Uh, 'cause you're right, uh, Phyllis Andrew, exactly as you mentioned, hey, we were seeing, uh, hosts affected and organizations compromised with what we believe to be the patch. And I'll put that in air quotes because, uh, way back in October, a little bit back in time, there was the CVE 20 24, 5 0 6, 2, 3, whatever numbers, blah, blah, blah.

A vulnerability in October that was disclosed, identified, but there was no technical information. And really, hey, nitty gritty details as to how this vulnerability came to light. What it was, the, what was the flaw, what was taken advantage of. And maybe a, a proof of concept to see that publicly. There just wasn't information out and about. So put that in the back of your mind for October. But now let's bring us back to December where we are now.

Now we're seeing in the wild exploitation, now we're seeing servers and Clio software get beat up, but it looks like it's from even affecting the patch that came from that October CVE. So in the early days, a lot of us are wondering, kind of scratching our head, does this mean the patch was not effective? Was it the same? Vulnerability is an entirely new vulnerability. What is it that the threat actors are leveraging? Especially because we don't quite know what that old CVE was all about.

So when we, and I'll say hunters, hey, we got to dig into this. We kind of wanted to understand a little bit more of what the exploit really was, or at the very least, what threat actors were leveraging. Um, what we could uncover may or may not be the October CVE, or it could be something new, or it could be something maybe entirely separate of them. What threat actors were beaten up just 'cause there was a lot of uncertainty.

Um, if I may say, Hey, we did kind of come through the end of it understanding, oh, there is a weakness, there is a flaw, we can abuse and exploit this. And suddenly we have our own proof of concept to recreate what we think the threat actors are, are using for in the wild exploitation when we get a chance to communicate with Cleo. And I'm very, very grateful. We did get ahold of them. We were able to kind of talk this through.

They had confirmed with us, yes, what you are seeing and what you've recreated is what we know hackers are using right now. And for that reason, we'll be cutting a new CVE and releasing a new patch. So suddenly, oh, this is a new vulnerability, not quite what was in October. It would also means it requires a subsequent secondary patch. So suddenly there's more rope to pull on here.

Yeah, That's, you know, I love the fact that, um, hunters as an organization, you always reach out to, you know, the software vendor first. And hopefully, um, you know, it sounds like in this instance they were cooperative, which is also, um, really great to hear Phil's kinda add one thing, John, I did, you know, just some back door stuff, you know, when you were starting to post your blog and it was interesting on their site.

They, you know, prior to you guys really stirring things up, it was like, and I forget the dot, it was like, Hey, if you are running these, this is, you know, you do need to patch this and everything will be good. That's the way Cleo portrayed it though, on their security site. You did probably see that fair. Oh, yes. Yeah. So we'll be talking about, you know, some secure by design thoughts with you later on this. Yeah, yeah.

So, um, you know, one of the reasons why we're talking about this is because obviously it's being exploited, um, and still today being exploited, um, by ransomware groups like Termite. And so, you know, can you explain to us how does this happen? How does it happen so quickly? Like, you know, we often talk about, um, threat actors, you know, monetizing using tech faster than us. Are they also, um, monitoring CBEs and doing a better job than we are at that? Ooh, super good question.

Um, without a doubt, I think hackers, threat actors are kind of keeping an eye on, or maybe a finger on the pulse of, look, what are new vulnerabilities and software that maybe they haven't seen or heard of before? Just to add to their own arsenal stuff that isn't patched and would be maybe vulnerable or susceptible to a previous CVE and their vulnerability. Or are we finding a whole new software suite that if we shake the trees more stuff will fall out?

Uh, are we gonna find new issues or things that we could leverage if they were to take a, a piece and take apart that software and just uncover more? Um, I think I, I, I have to think that played a part in this. When we start to talk about the attribution, when we start to kinda wonder, okay, who done it? Um, I know the word on the street for a little bit has been termite. Uh, I think I saw that in a masted on post from Kevin Beaumont. But truth be told, that's all I had seen.

Uh, kind of, even just mentioning the name termite, it's very strange in the activity that we observed in the actual, in the wild hacks and exploitation, um, they would leverage and set up a little bit of persistence, get their comms back and forth with their C two server, get ready to do more post exploitation, but then they just kind of started to look around, like they were just enumerating the environment, seeing what was available in active directory domain.

And truth be told, that's what got them caught. But there wasn't any ransomware. There may very well have been data thefts, uh, potentially. Uh, so there was just, again, kind of still scratching our head wondering. But I think most recently, whether it was yesterday or just a few days before, bleeping computer, I guess, had an interview with Clop, the same ransomware actor that had compromised move it in the days prior.

And it sounds like in bleeping computers article LOP is taking responsibility. They're throwing the flag in the ground and saying, this was us. Mm-hmm. But their leak site, if you look on the dark web and the tour onion sites, they've been a little bit cryptic. They just said, Hey, we're gonna remove all of the old data leak data from past compromises, and now we're gonna be working with new ones, given the recent events of Clio. But they never particularly say outright, we did the Clio attack.

So it's weird, it's wild. I I always try to add as many warnings and disclaimers as I can with attribution. 'cause we never quite know for sure. Right. Phyllis, um, I'm just, you know, your take, let me ask John this first, but coincidence, John, like, I mean, a few weeks prior we had blue yonder. Do you remember that whole thing? Oh, yes. Um, and again, you know, those are, that affected big companies. Again, supply chain termite takes attribution there for sure, right. In blue yonder.

And again, just, you know, a week or two later, you know, they're, they're claiming this is is that, you know, a threat actor's mo maybe when they're getting the limelight or, you know, uh, hey, we did this and then we also did this. Um, but, but, um, and, but again, the, the similarity is, is kind of striking that this is, you know, again, it has to do with supply chain. Yes.

Uh, it's strange, I think, and it's not to say, oh, it's an attack of opportunity, um, but sometimes it, it is, sometimes it is just, Hey, what else can we leverage? What more can we do? What other damage can be done? Um, and whether that's from one, uh, vector to the next, I think they'll kind of keep going. They, they won't slow down. Yeah. Yeah. I do.

And I think you bring up a good point, Andrew, by just even having this call is, um, you know, it's threat actors wanna go after, um, software that's, you know, has a big footprint, right. You know, because it's like the more bang for your buck. That's what we say about MSPs. That's why we believe MSPs are targets prime for the picking, right? Because you hit one Ms. P, then, you know, you, you can get a number of organizations, um, possibly compromised. Yeah. I mean, we're seeing results.

Oh, go, go Bob, go. Please. No, I was just gonna say, I mean, to me it's really clear why they're picking these particular targets. I don't think it's any, I don't think it's random or, or, or at all. I think in the end, they know they're, they're looking at, at strategically they're saying, okay, if I can compromise a package that is being used by lots of other packages, in this case, there are lots of other pieces of software that use Clio to actually do the, the, the secure file transfer.

That's their whole mo that's what they do, right? Is, is basically, yep. Secure file transfer. Well, so you look at that, that means there are part of the infrastructure of a lot of other software that just happens to play in the supply chain. So, I mean, if you're, it, it's the trifecta of how much s**t, how much, how much bad s**t can I cost if I can get into the supply chain of the supply chain. You know what I mean? Right.

Which is, to me, it, it gets to be pretty simple when you start thinking about it in terms of that. Right. Well, you unders a good example. Somebody who does something that if you could get in the middle of, you have access, just like Phyllis just point out, you've got access to a lot of much bigger clients because they use that as kind of a core element in the process of doing secure file transfer. Right?

I mean, so it, to me, it, it's a, I would say that every, if you own a file transfer or software company, now's the time to quit goofing off and go make sure that you got your stuff wired tight. Because whoever you are, you're gonna be next in this parade of people like it. And now Clio, right?

I mean, I think it's pretty obvious that they're, they're, they're targeting those things because they get access to a wide range of types of companies, healthcare, finance, all of that by just tackling one, right? Yeah. And, and it's easier to tackle one than it is to go to try tackle all those individually. So if they can make that work, then the, the doors are open for, for whatever they want to go after at that point.

So to me, this seems like more of the strategy of move it, I mean, move, it was successful. We can all agree, move. It was wildly successful from a bad guy perspective, this is moving on, this is the locust moving on to the next tree to chew. You know what I mean? They just see this as another opportunity to get into that. Right. Yeah. Really good, really good point, Bob. Very good.

Now, John, um, for those of us or those organizations who are curious, can you, um, give a little intro to how is it that you actually start reverse engineering a piece of software? What would you recommend to an organization? If, I mean, really not, not very many organizations have the resources, but let's just say, Well, no, thank you. Thank you. And I, I hope this is a part of the fun that we get to have.

Uh, and it's a labor of love, but keyword there being labor, uh, when we're chasing the ambulance for, hey, some new thing, hitting the streets, vulnerabilities flying outta the sky and exploits kind of all over the place. A lot of time it's, uh, energy drinks and pizza and some of our team kind of staying up late trying to chase this thing. So by no means feel like you gotta run through it, but that's why we'd like to do the work for you. Um, right.

We have a couple different approaches when we get to do what looks like a lot of reverse engineering to find an end day vulnerability, maybe what was the first CVE from October, but now realistically a zero day, uh, vulnerability because it was a whole new threat that needs a new patch and needed as a new vulnerability, CVE. So what we had done is try to deconflict and find differences or benchmark, compare and contrast the old patch and then prior software.

So that kind of at least puts them side by side and you can look and see what's different, what's changed, what was modified in that patch. But then we realized, well, that's not super duper helpful if this is entirely new, but it might still show us, oh, some vulnerable or some weak parts of the code. What you can also use as sort of a map and a compass during that work. It's a little unfortunate, right?

But because there is in the wild exploitation and we've got some of the purview and visibility of the already affected and compromised hosts, well, we can look in the logs, we can look and see what other artifacts, whether it's debug information from the program running and kind of just clue us in, okay, what are the same sort of strings or messages or what data does it show when one specific PO code path is followed through?

That helps you really, really narrow the search when you're trying to find what could be weak and vulnerable code that's being leveraged. Um, so truth be told, that was how we uncovered the, in the wild exploitation vector. The proof of concept kind of puzzle pieces for us to start with was really looking at a lot of the debug messages from affected hosts that gave us the most mm point in the right direction. Awesome.

Now, we often talk about, you know, building your technical res res, uh, workforce, um, and things like that, um, on this call. And so for you, um, obviously you do SRE on your own, what do you think, um, makes a good software reverse engineer? Is it, you know, just the skill building up your skills?

Some of it is intuition or, I mean, I would say I, having witnessed some of it perhaps in my, in my past, you know, sometimes people really, you wonder the people who are really good at it, and you're like, did you, sometimes, you know, I have to believe you make your own luck. Sometimes people are like, oh, I'm just gonna look here in memory, or I'm gonna do this here. And then it's like, bam, there it is. And it's like, what? You know, how did you do that so quickly?

So I'm curious what your thoughts are on that. Oh, okay. Huh. Well, maybe this is kind of a fun one. You're totally right in that it is, uh, a lot of competence. A lot of, you know, your own skills that you've acquired, you've accumulated, you've practiced, you've gotten, you've learned as much as you can, blah, blah, blah. We all kind of know putting in that hard work.

But what I would really harp on, and I think pays and dividends is a certain amount of grit, uh, really like stubborn determination that like, my goodness, there is something wrong here. Some way. Somehow I'm just either gonna keep scrolling through lines of code or keep reviewing or reworking through some function or trying to understand one piece, look at all the options that I have in front of me.

Uh, really just that, uh, ambition or determination to like, not give up, even though you're banging your head against the wall, that will end up, Hey, something will fall out. Hey, the shoe will drop at some point. Um, when you do that over and over and over again, maybe that's what's built enough experience where you're like, oh, I, I know kind of what to check even more so now. So this process will be faster. Uh, I, I really think that's a testament.

I hope to, again, I'm not gonna lie, I feel like this was a grand slam. A lot of LA hunter's team working together. I feel like we kind of hit a home run with this one is like super quick turnaround, get that actionable intelligence out, um, both on the soc detection work, both on the recreating and understanding the threat intel, uh, all around. I just, I hope that was a another, Hey, good green light. That's awesome. I, I agree. I think that's great.

Um, and a prime example of, you know, we always say cybersecurity is a team sport. I really appreciate you, you know, um, talking about the whole entire team contributing to, um, to, to get the awareness out there so quickly. Um, and off to you, Bob. Yeah. Okay. I can tell you in my opinion, and what it takes is people who are just hardheaded as hell and will not take no for an answer to really work that job, right?

I mean, because I can remember being younger, looking for patterns and log data and, and you just have to have a drive internally. It says, I'm not letting this go down without me understanding what's gonna happen, right? So, but it's a game for younger, it's a game for younger people that have a lot more patience than I do anymore. I, I used to do that, but I'm, I'm well past my ability to, for that, to hold my attention long enough to Do. That's right, Bob. So, John, you're old now.

That's Yeah. I, I think it is, it's ageist, but I'm old, so I can say that. I just think it's one of those things you got, you're better at when you're younger and can focus much more. And like, like John was alluding to just eat pizza and energy drinks, and that's all you need for like 48 hours straight, you know, working with things. Well, I'm Teasing. I know I'll make light of it, but it's, uh, It's a good, it's a fact. Yes, it's a fact.

So, I mean, so there's a couple things about this that I thought was interesting to kind of discuss, um, John, especially with you, right? Because I, I've got a, I I've, this coming year, so I already have a, something set aside that I wanna try and tackle this coming year. And it's all about vulnerability management in general, because I think, um, I think there's a lot that we need to discuss about what's the best in class methodology for doing that sort of thing.

So I wanna, I'm gonna layer that in on top of this because I think it, I think it goes together. First of all, the, the real first question is, we all know it's very difficult to get information, you know, at, at timely information sometimes, you know, from vendors, right? And, and I know that there's a, there is a couple reasons for it.

I mean, one of the logics, one of the logical arguments is, well, if we tell everybody, you know, that we're working on it, then that's gonna cause more people to run and try to exploit it, right? So there's that, that part of the bell curve where, okay, we gotta be careful because we give them too much information that something successful, then it's gonna create more, it's gonna create more threat actors flooding into that zone, trying to take advantage of it.

But then there's the other side of the bell curve, which is all of these companies who are trying to protect themselves. And when it's a credible risk, you know, when the information changes hands gets to be really pretty important as, you know, as we point out, right?

I mean, I, I would say that Andrew pointed to a perfect example back in October, what was coming out related to that, um, that CVE at the time really wasn't enough information for anybody who's in our universe to be able to be completely actionable on, other than knowing that there was a software package with something going on that needed a, a patch, right?

So why don't, why don't you talk about, I mean, if you will give kind of your angle on, you know, what's the, what's the challenge with working on these issues and how you disclose it or when you disclose it, and how that, because that, that impacts everybody who, who's on the call. So, Oh, asking kind of from the hunter's perspective, from our perspective, is that right? Yeah, I, well, I mean, your, your perspective should be the perspective of mooch MSPs to begin with, right?

I mean, because we have the same concerns you do. So that should match up right ahead. No, without a doubt. And thank you. Uh, when we are seeing in the wild exploitation, that is kind of the biggest indicator of like, okay, we need to get the word out. Like we need to start talking about this because it's kind of already in action, right?

Um, with that said, we, I'm acknowledging, hey, so sometimes there can just be a little bit too much that is easily given away and too much that will kind of tip the hand to threat actors, or, hey, even some actionable info that might be, uh, a little bit risky to have shared. Um, if I may, I think this harkens kind of, well, back to screen connect. Uh, the slash and grab incident was really something where we were like, oh, this is a rock and a hard place.

I wanna talk about this, but it's just something that we can't talk about yet. Um, when we get back to Cleo current day, uh, in modern time exploit is out already out in the wild, uh, we can recreate a proof of concept to validate, prove we understand this from cradle to grave. And that's, I think, I hope part of our strength in that. Like, look, we got a strong arm to make sure we know what we're talking about. 'cause we, we see it through the lens of the hackers.

With that said, we're not gonna release this exploit that, that, that feels like a, a bad call that feels like we're handing a gun to the hackers. Um, so with that, we will do a cutesy little video of some cooking show magic light off some fireworks to show, look at the impact of this. I want to get your attention and make sure folks can do the due diligence patch, keep their ear to the ground on this thing, but we're not going to throw that out to the wind.

Um, that is normally the, uh, questions that we'll either ask ourselves when we're going through this rapid response process is kind of what, what we call internally, and then even just the decision, do we enact a rapid response? Are we getting all the Avengers together to go fight this thing? Are we, uh, gonna rally the troops? That is usually a matter of, is it in the wild exploited already? Does it affect a large number of people? Is it ubiquitous software at least enough to have an impact?

Um, all those things really kind of weigh in, but it's just really just a matter of trying to carry that same grace and ethics of, okay, the damage is done, is already done. We don't wanna make it any worse. We just want to help and make sure we get the actionable info out. Yeah. In my mind though, I still feel like the vendors have at least a responsibility to give us a little bit more than was the case before, right?

Because it didn't give us any idea what we really should be looking for on what front. And I think that's really what it gets down to. And now, along with those lines, so here's the interesting thing that I found out about the CBE, right? From a, from A-E-P-S-S score standpoint, it was actually a 0.04 per, it was a 9.8 percentile. In other words, so low, the EPS score was so low that anybody evaluating it based on EPSS score wouldn't even probably float this to the top, right?

I mean, if your process is to look at the things numerically or, or analytically, then in this case, it wouldn't arose to the level of something you should be stopping and doing. My, my, so this is a curve ball question, Andrew. I'm, I'm, I'm all, I'm, I'm as bad as Phyllis about getting off script, but here's the deal.

So my point is, is we sh in your opinion, do you think we should be looking at a way that when and as vulnerability is actually actively being exploited, that there is a modifier on the EPS score to say, yes, you know what, this is priority wise in the universe, this is not that big a deal to, to worry about, but in reality, it's being exploited today currently. Shouldn't we all be knowing about that at a much higher level?

Especially if all the, uh, you know, everybody's software is keying off of things like, um, EPSS scores, right? Because that's always come down, it's the vulnerability thing, right? Which one should you work first and in what order? And then what tool the tools work very specifically in a certain way. So, um, what's your opinion about that?

I mean, in my mind, it seems to me that when it's being actively exploited, there should be a way for there to be a modifier applied to the EPSS score based on just what you talked about, you know, the forensic information being determined that, look, it's really being exploited, it's being exploited in more than one place. Therefore, that should have some sort of modification to the score on the EPSS level. Yeah. I, I don't think I'm super smart on the EPSS score end of it.

I would, I, I kind of wonder, you know, just as you mentioned, hey, we're trying to add a whole lot more formality and process and streamline stuff, so we're smart with CVEs and identifying can help track, right, and prioritize and all. Uh, but I know, uh, oh, when we look on the internet, whether it's socials on Reddits r slash MSP or Twitter, or Mastodon, blue sky, whatever the heck, LinkedIn even, yeah.

And when people start kind of running around chicken with their head cut off, like, Hey, we're seeing this vulnerability make some waves, then even internally, that makes, I hope the alarms and bells go off. Like, okay, we should be chasing this thing. It doesn't matter the EPS s score. I know it's in my environment, we gotta spring it to action. Uh, maybe that plays a part, But that depends on the vendor telling you how bad it is, right?

Because unless you figured it out yourself, you're not gonna know. Go ahead. Right, Andrew? Yeah, yeah. Can I say something? So, so John, interesting you said that. So, um, I think of two people that I, i, you know, I respect greatly in this industry. One is Brian Blakely, who's been on a bunch, and the other is, uh, Steve Carter. Uh, Steve is the CEO of nucleus, which I think you know of him as well.

And you know, both of them, Phyllis, this'll be, you know, kind of, I think new I, I, I kind of, um, you'll, you'll appreciate this, but you know, they, they, you know, they say, yeah, okay, you have EPSS and you know, CVSS, but it really, you know, what is your policy like? And, and what they mean by that is, you know, starting there is like, okay, if you have a critical vulner, if you have, you know, what are your critical systems, right?

Again, Brian always talks about what, what makes the money in the company? And, and so what's, what's the policy if something's being exploited in the wild, forget the EPSS and CVSS and if it's impacting your most critical systems and processes in your organization, who cares what the score is? It's being actively exploited. Um, what's your policy say if you know the, the, the software that runs your entire, you know, manufacturing department is being exploited, um, what is your policy?

Is it, you know, stop, shut down, mitigate, you know, but you, you know what I'm saying, fellas, like, to me, that's, that's way more important, is what is the action? What is as man, as Brian Blakely says, what is management's intent when bad stuff happens? You know? That's right. 'cause numbers are numbers. That's right. I mean, that's, that's where that business impact analysis comes into play. It's like, what, what are the applications that are critical to your business?

What happens if those applications go down? Or perhaps you lose data in those applications, right? And so that is a way also, um, in which you should be prioritizing what you're looking at and what you're going to fix first, right? Yeah, yeah. Yeah. I mean, and, and by the way, it's what, you know, one of the things that Nucleus, you know, they're, they, they don't really play much in our space. They have some bigger MSPs that certainly use them. Mm-hmm.

But they're an amalgamator of, of, um, different vulnerability systems And that, But, but, but one of their most critical point of using their software is, and prioritization is, is it exploited in the wild, right? Mm-hmm. Above and beyond all the other stuff, right? Because once it's exploited in the wild, right, John, it's fair game. No, that, that's exactly my point, Andrew, that, that, you know, because you got, I am from the MSP perspective.

So to be clear, we're watching hundreds of clients with thousands of endpoints out there, not unlike what John and them do for a living. And at some point when, when something rises, but in that situation, you, you're not man to man with the reports coming in, right? So you, you don't have enough personnel to watch the big data problem, which is this vulnerability management process, right? Which is part of the CIS controls.

When you're really managing your vulnerabilities across that kind of a span of control, there are thousands of things that are happening simultaneously. So you need some mechanism to, to make things stick out, right? And so we do wind up using things like CV and EPSS to help us be able to, from a sys systematic standpoint, be able to prioritize in a lot of cases, right? And so that's my point. And even the CVE was a low risk CVE reported out, right?

I mean, it wasn't like it was a critical risk. I thought that's my point to John is I think the fact that, you know, that it's being exploited in the wild changes, changes ultimately changes your opinion about what you should be doing about that particular vulnerability. And right now, neither system has a way for that to be taken into account, right? And I feel like that's a gap in, in what it is that we're using this tool.

I, I, I agree, Bob, but, but again, I'll come back to, I'll double down. You know, it's always about, you know, you interact firsthand with your customers, right? Are they gonna let you shut down systems so that you can patch something? Let's just say there was this, you know, this patch out.

You know, what, what's, what's the policy say, you know, 'cause when, when the, when the rubber hits the road, and you're gonna tell 'em, Hey guys, I know you know, this thing is responsible for x millions of dollars a day, but if we don't take this thing offline for X amount of time and take care of it, really bad stuff can happen. And you said in policy, if something's exploited in the wild, we can do so.

So that's, that's why I come back to, you know, but, But to be able to policy a hundred, a hundred percent, and you're right, you're all you're doing is illuminating the complexity of the problem, right? The problem is, is they may not, because they don't have an emotional attachment to what that means, they don't have a technical understanding of what that means. They might not be able to make a good business decision. Absolutely. Right? And you can't fix that because it's their systems.

And even if you pointed out the point is though, if we, if we point it out and they decide to not take action, then that's the best we can do as an MSP, right? Is to make sure that they're notified and say, Hey, look, our recommendation is that you deal with this and you deal with it. Now, if they choose not to do that, then, then that's one thing.

If we don't tell 'em that there's something being exploited in the wild, because it doesn't rise to the level of us seeing it and paying more attention to it, then in the end they'll go, wait a minute, you guys are supposed to be helping manage the vulnerabilities on our network.

And it turns out this one's being used by a bunch of people, and they're not gonna listen to our argument that, oh, yes, but the scores were so low that it, you know, it didn't really rise to the level of being dealt with. That's, that's not gonna be an argument you're gonna be able to make. Right? That's The last, last thing I'll say, and I'll let you continue with John, because John's our guest. But again, the importance of two things we talk about all the time, right?

You know, Eric, till well, right? Is how generic is your MSA I'm patching. If it's generic, you're in big trouble. We patch your A hundred percent. Yeah, hundred percent. Big. The second thing right, is again, if, if why the policy? Because if they're deciding to go against their own policy, at least then you can say, Hey, here's the MSA, here's what we, you know, we agreed to who patches what.

And oh, by the way, on top of it, you know, we agreed the policy, you know, you're, you're, you said this was the policy. So again, coming back to the criticality of those two things. So back To you. Yeah, absolutely. So I mean, the bottom line though, John, what, what do you really feel like this vulnerability, what kind of impact does it have, you know, for, for the clients of MSPs in general, right? I mean, because it, let's, let's assu make some assumptions.

Let's make some assumptions that the customer base that most MSPs has, have at least one or more of these, one or more of these types of either retail, healthcare, finance, they're involved. What kind of risk does it present? You know, with that, having that, having the knowledge that that's a, a vulnerable system, what kind of risk does that really represent from an Ms P standpoint? If you have an opinion on that?

Well, I think I would kind of lay the cards out on the table and then maybe ask the listener, the, the MSP, the business, the owner, to think like, okay, which of these sound like a threat or a risk or a concern to you? Because this software, uh, after it's exploited, and what, from what we've seen, uh, has been kind of a slow burn for potential damage or havoc that could have been done. Uh, we hadn't seen ransomware, so it wasn't an immediate detonation explosion thing, right?

No business going or systems offline immediately. Um, maybe that's potential because they've now got their hooks and claws and persistence and malware and in CTU to be able to do stuff later down the line. So maybe that's a thought. Maybe there's the concern, what happens later. Um, but know that, okay, this is now in the system that may very well have sensitive data, sensitive files, important confidential things or work in trading and relationships with other businesses.

Those probably kind of want to be kept preserved and confidential. So that's the thought of, okay, a little bit, even just the stealing of your own data. Um, but it's the matter of wondering and waiting. Okay, I've got maybe an actor lurking in my network. What will they do next later afterwards? Um, I think at the, at its core, this is kind of awesome that, hey, we were able to get out in front of it as fast as we were, and we had the whole industry kind of catch onto this.

So we haven't been sending any new infection reports, we haven't been sending any uhoh, any other compromise hosts. It feels like threat actors kind of caught on that we caught on and now they're done. Well, that's good, right? I mean, I think that's the biggest deterrent is when, when you shine light on it, right? I mean, I think that does Oh yeah.

That because everybody at least is awake and aware, and therefore they can kind of, the bad guys know that, so therefore they're gonna try and pick a different path, right? Just making it more difficult than not. Yeah. So the, um, um, John, what do you think the best practice would be for an MSP to try and stay ahead of these types of situations where there's CVEs and vulnerabilities? I mean, it, it comes down to the field intelligence necessary to be as proactive as you possibly can be.

And I do not know that I, I don't really have a place that I could point someone and say, Hey, these are the three things you really need to do to try and, you know, keep up and or stay ahead as much as possible as an MSP. Do you have a, do you have any observations related to that or vulnerability? I'll, I'll try my best. I know this is hard because especially it's like, I can't see the future, don't have a crystal ball, don't nostradamus.

Like, it's very, very difficult to say, oh, we could see this coming. It's very difficult to say, oh, how could we prepare for something that we have no idea what we're preparing for, et cetera, et cetera. Um, I know we talked about a couple pedestals already. Hey, identifying your network, knowing your environments, just having the application inventory, asset inventory, blah, blah, blah.

Uh, the big one that I think I would harp on here is network segmentation for one thing of, hey, could we keep this away from open naked to the internet? Uh, and really a lot of how can we limit the attack surface? How can we limit our attack surface? That's the question I would just have you keep asking yourself over and over and over again and whether or not you have some, I don't know, benchmark guide compliance checklist thing to walk through with that or not.

If you just genuinely keep thinking about how do I limit my attack surface, whatever server, whatever workstation, whatever device, whatever user you see, one way or the other, you could be thinking, can I tighten up the settings? Can I take that out of the danger zone? Can I decrease risk appropriately? As you are working through stuff? That's the question I'd keep asking. How can I limit attack surface? Yeah. Okay. All right.

And so, I mean, this is a, and honestly feel like I know the answer to this question, but I'm gonna ask it anyway because to me, I feel like there is a, there's a gap, but I'm gonna get your opinion on it because, um, do you think it, do you think it highlights that the problem that we have with, in general, with suppliers who are vendors and how they manage the information flow as it relates to vulnerabilities they're either told about and fixed and or find themselves?

I mean, because like I pointed out in my mind, there was not enough information given out in October for someone to do a thorough job of, of getting ahead of this. Right? Um, so to me, now, they may not have known how to do that. I'm, I'm not arguing the fact that they did what they thought was right. I'm sure they did what they thought was right. But to me it highlights that I think there's, we need a better prescription for vendors to follow as it relates to things like this.

So that can be adequately dealt with by people like us who are managing lots of people's networks. What do you, what do you think about that? Uh, short answer, yes. Uh, long answer while I try to be as polite and, uh, courteous as I can be. Yeah, I, it's, it's interesting with this one, 'cause you know, I try to just be John and be as understanding and as sympathetic as I can.

Um, and I do want to give folks the benefit of the doubt, but I think a lot of the confusion, a lot of the misunderstanding and a lot of the heartache that we're trying to now diffuse or demystify kind of came from not having the clarification that the October CVE is not what we're seeing right now. When we communicated with Cleo on Monday, um, that was already the consensus.

Um, but the notion of we're gonna have a new CVE and a new patch, if that doesn't come till Wednesday or Thursday, exploitation already happened on Saturday. Yeah. And Sunday, uh, we're just a little bit lagging behind. And I would love to see megaphones and microphones, not just from us, but from everyone that can hold that and, and speak loudly. I, I tell you what, John, I'll make a deal with you.

Any, when, when you've, you've done it as nice as you can, you need to give me a call and I'll get involved because this is, this is dangerous. I mean, these are people's lives we're fooling with, right? If we let these exploits go on over the, you know, over days, right, because of non-information, then I know what the knock on effects of that is to the, to businesses and the people who work in them.

And some, sometimes you have to be more firm, and you're a super nice guy, but I, I will be happy to help fill that role for you because I, we need it, right? We need some mechanism for, you know, for dealing with these things, you know, you know, like they need to be dealt with. So anyway, Andrew, that was my last question, so back to you. Yeah. So, so Phyllis, I was gonna ask you this at the end, but I think it's better timing right now with what Bob just asked John.

I mean, this, you know, look, you, you and Kurt, and I know CIS is, is kind of starting to beat the drum of secure by design and, you know, so obviously that was an executive order and this and that, but is that, so one, is it gonna get any teeth? You know, when should, should vendors be held accountable?

You know, like John's saying, you know, and you know, you read between what John's really, you know, being, again, being nice and saying, but he's saying, Hey, look, you know, you had a, you had a pretty, you know, bogus security release out there about what was going on.

And then, you know, obviously that that book, you know, again, uh, and I forget who said something about, oh, uh, Eric's son, like, Hey, if you're a, if you're a not a, if you're a a real threat actor, you're tracking CBEs, well, of course they're gonna go look at that kind of stuff. They're not, you know, they're not ignorant.

Um, and so, you know, maybe shed some light on how not only this secure by design, what's happening in that area, uh, Phyllis, but, you know, should there be some skin in the game here when this kind of stuff happens? What, where do you think we're headed? Yeah, so the 2023 National Cybersecurity, um, strategy, um, you know, what we're seeing is the kind of wind tilting towards, um, the, the liability liability being pushed onto software vendors. So, um, I pulled up like strategic objective 3.

3, shift liability for insecure software products and services. So they wanna, the national policy is to shift liability onto those entities that fail to take reasonable precautions to secure their software. The UK government also has a voluntary code of practice, and then of course, cis a just this year, um, has a secure by design pledge. And that pledge was created with, um, multiples of other nations. And so, um, while it's not cemented in policy yet, it could be just a matter of time.

So what you're seeing is because of things like move it because of, you know, um, something like Clio, uh, it, it has to be that who's in the best position to actually fix what's happening? And who's in the best position to actually prevent for decades now, it's been on the end organization with the cv. We're gonna tell you how to patch, we're gonna tell you what's the highest we're gonna give you that CVSS score.

It's always putting the liability on the end organization to patch after the fact. Right? And that's not working right. So Bob, Bob made a few points. So, so what do we need to do? You're gonna take that back and it's really going to be the software vendors who are really best positioned. And so, um, we see the winds going in that way. And so what we wanna do is, um, at CIS is try to bring some levity, try to bring some practicality to it.

So there's a secure by design document, it references the SSDF, um, written by nist. And so what we wanna say is, okay, um, government's asking software, um, developers and software vendors to take this pledge. What we wanna do is be able to say, okay, what does it really mean to take this pledge? How do you measure, how do you attest and kind of create some, um, kind of metrics for organizations who are going to take that pledge.

So consumers or buyers know, this is what I'm getting when an organization takes this pledge. And then of course, those organizations who are taking the pledge, you know, put some rigor behind it, put some standards behind it. So it's, you know, but company a companies a pledge and company B'S pledge, you know, you're gonna kind of get an equivalent, hopefully an equivalence of what does that really mean when someone says yes, I, you know, whatever one of the, one of the pledges is.

And, um, that's what we're hoping to do. That's what we wanna bring to, to the table. Of course, we'll have a community around that and try to gather, um, a bunch of folks to help with that. So, Really interesting. Really good, good perspective.

So John, you know, any, you know, if there was a top recommendation coming out of this for an MSP, Bob asked a little bit about, but you know, you know, aside from hey, know your inventory, that that's, you know, again, we come back to that one a, a business impact assessment. You, again, we're trying to instill MSPs, cajole MSPs, right? Figure out, you know, what runs and drives revenue. Right?

Um, in, in, as you guys interacted with MSPs on this one, was there a number one takeaway that you go, wow, that that was really the, the top lesson here that we, that you should have taken away from this? Uh, um, did any anything bubble to the top knowing, you know, the, kind of the backstory as you talked about what happened in October and you know, with, if you could be a Monday morning quarterback, you know what I mean?

And, and have your mind in security research, you know, if you were an MSP and go, wow, this something's not quite right here in the security patch release and, you know, just love your take. Yeah, no right or wrong answer. No, I think I got one for you. I don't know, I, if I may a little bit of an anecdote, uh, as we were going through, you know, the things that we tend to say, oh, hey, no, you're, uh, uh, inventory limited tax efforts, blah, blah, blah.

The big one that I like is keep your ear to the ground so that you kind of, hey, get all the information, can get that tactical, actionable intel. Um, but I, I know even that's a little bit trite.

I think the one that I really, really, really want to emphasize is that, um, when we say it takes a village, when we say it takes a community, and as you mentioned, we're communicating and working hand in hand with a lot of our partners, um, I gotta say the technical aspect even that I got to be a little bit more a part of.

We weren't really able to readily test the full cradle to grave proof of concept unless we had a genuine, dedicated, legitimate and official installation, um, with a license and set up in a real environment. So we got hand in hand with one of our partners and had a cool, genuine conversation like, Hey, can we work on this together? Are you comfortable with us using little bit of your environment to make sure we have the full understanding of this?

And we got on the line with them, we got to call communicate, and hey, get, start throwing code back together and do that back and forth. That was incredible because I think it was genuine collaboration of partner business, the community to a certain extent. Um, so while I say keep your ear to the ground while I say, hey, work hand in hand, I, I really do mean it.

I think that's been a huge mark of the success and how we've been able to get in front of this is because we had a team, not just our own team, but the people that we're trying to help and be a part of and serve altogether. So, Really good stuff. Yeah, that's, that's cool that your partner was that open to doing it. Um, John, we have a few minutes left.

Share the floor, like te tell us a little bit about, you know, I put your URL in, you know what, what, talk to us a little about your channel, you know, up to two, congratulations, you know, we'll, we'll, you know, give you some time here with, at the end of the year and another amazing year for you, I think. What, what did you start the year with in terms of subscribers? Oh wow. What, what you're, you're, you're coming up to 2 million.

What are, what are, what's the type of person that's coming to you? Is it a, you know, are they all across the board from MSPs to, to researchers? What are people learning? Because that you have a community in and of itself. So give us a little background on that as you close this out. Thank you so much. I'm super flattered. No, the YouTube channel has been a, another passion project for over a decade now, which is wild.

Um, but the early days started with a whole lot of like learning the ropes, trying to get smart on the keyboard to do some of the technical work. So think like programming tutorials, think, capture the flag exercises and war games. And now as I've grown more and more into the industry, it's been, oh, chasing real malware or looking at real threats on the dark web or having those tactical, like even TTPs conversations.

What's the trade craft that we see in the real world across the cybersecurity industry and the landscape? Um, it's been a lot of fun to chase that. Maybe sometimes I think people are probably a little bit tired about hearing vulnerabilities 'cause they know it already. They're living it day in and day out. So I might honestly for 2025 be looking for some new explorations while I'm trying to play in different directions that are all cybersecurity tangential.

Um, a lot of folks really, really love when it can interact with, uh, a topic or a way that bridges the gap for even folks that aren't super sharp in security. Log four J was a really big prime example because everyone was thinking, oh, this affects everything. Even like Minecraft. My kids are playing on their computer video game influence to a certain extent that still gets people educated in awareness about this.

So I'm curious if we'll explore, oh, some silly game hacking, or we do more of that scam baiting work where, hey, you've got someone that's on the, the fake call centers trying to get you to get another Lorton LifeLock description or Best Buy Geek Squad scam. Uh, I think shining the light on those will be something that really excited about and hopes to do more in 2025. That's awesome. John, keep up the, the fantastic work and, and I know there's big things on the horizon here for you in 2025.

We're excited to hear about that. Um, so, so I'll give you an early congratulations to that. Um, Bob, um, really appreciate you filling in as our co-host today. Um, Phyllis is always wishing you everybody a, a happy, uh, holiday season. Merry Christmas, happy Hanukkah, happy New Year, whatever you celebrate. Uh, I want to be Phyllis so I don't get myself a band off the internet here. I actually, I think this is the first time I'm gonna say this, this year.

Are we gonna say to everyone since we're not gonna have a call the next couple weeks? See you next year? Yes. See you next year. Wow. That's right, Phyllis, but I will close with. Hey, stay vigilant. This is the time Bob and I were talking offline, man, this is the time that threat actors love doing stuff, don't they? John? Uh, this is, uh, you know, they're gonna look for, um, you know, you know, any types of, um, of, uh, weakness. So wishing everybody a, a great one. We'll see you in 2025.

Take care, John. Thanks a million. Thank you all. Take care everybody. Thanks.