MOVEit Vulnerability and the aftermath!

Uh, thing that came up for her, Gary is speaking at Da Ocon, and, um, and as always, you got me and Wes, uh, in the seat. Um, so let me kind of just, um, set the stage for everybody. Um, move. It isn't necessarily a new vulnerability that we've heard about, however, um, the ramifications of this vulnerability are really starting to percolate and, you know, come to fruition. Not more. So.

Yes, we've heard a lot already, but I think, um, I think we're in the early innings, but I'm gonna ask Corey and John about today, just what they think and what it means to MSPs because I think, um, we will feel some effects, um, uh, of, of, of this. Um, so far, uh, MC Soft, which is an incident response firm, they also do some EDR work has the count. When I was talking to John, they have an article published with about 2,500 companies and 62 million records.

Um, and John, you felt, um, that might be a little light based on your research and Corey's research that we're gonna get into. Um, alright, so go ahead. So, Yeah, so just to kind of set it up, let's, let's talk about whenever you get a data breach, kind of how that's researched, right?

And, you know, traditionally a lot of the data breach stuff that was dumped, if we go all the way back to kind of the heyday of anonymous and, you know, doing data dumps even like payin and things of that nature, it was relatively easy to go through and do a research, uh, kind of just doing string searches across those data breaches, because a lot of times it was documents and you could go through and you could do a search of documents.

Um, as these data breaches got larger and larger, you can start to use specialized tools to do that type of analysis. And Corey, what's the main one that you said that reporters are using? Before I kick it over to you, Corey, just freeze. Yeah, That figures, um, Yeah, Corey froze, But it, it, you know, I do webcasts with him like every week and, uh, this is the week he decides to poop out, but they have like search tools, right?

That they can go through and they can search through for specific strings within the data set. So whenever you're looking at, um, just a simple string search for text, that works really, really well for a lot of tools because they have the capability of searching through Word documents, PDFs, things of that nature, right? However, what happens whenever the data is zipped or compressed, what happens whenever the data is in a P cap file?

And when you're looking at the clock, ransomware dump, it's absolutely monstrous. A very, very, very large dump. And I don't know why I'm, uh, but there's Corey. Uh, so Corey, what was, he's gotta unmute himself. Cool. Um, so whenever you're looking at all of this, and he'll just come in and start talking once he gets back. All right, dude, we got you back. Just not voice.

You have no voice, but what he can do, John, is Corey continues to, uh, he's got great hand gestures, Alright, I'm gonna ask Cory questions and he's gonna answer inside of chat. Cory, what is the name of the software that a lot of reporters are using, um, to do analysis of data dumps? Oh, right. And actually as a GitHub page that Corey is gonna be kind enough to share with everybody that allows you to do searching of large data sets.

Um, but once again, the actual file types that you can do analysis of, um, are very limited. Now, we shouldn't say very limited. It's a, it's a fine product, but once again, once you get into compression, once you actually get into, like I said, PCAP files, um, this becomes something that you gotta have some specialized knowledge on how to actually do a search. So I'll give you a, let's go ahead and let's do a, um, let's do a quick quiz. Uh, we've got 147 people live.

If you wanted to go through and do a search through a, like a capture file on network traffic, what would be the tool that you would use? And while that's going, I'm gonna answer Sandy's question. Go ahead Andrew. Yeah, Corey, um, two things. One, I'm assuming you're using Chrome. Uh, two, I'm gonna ask if he leaves. And then, so I'm gonna, I'm gonna, if you can hear me, John, I'm gonna boot Corey and then, um, have him, I'm gonna invite him back up to see if we can get him working correctly.

That sounds good. All right. So we've got, we've got a lot of answers and I want to go through these, Andrew, while you're working it. Yeah. Um, we also had a question from Sandy. Uh, does integration of zip and ROAR on Windows 11 make this, um, even worse? It can, um, once again, whenever you're dealing with that type of compression, because from a computer's perspective, compression and encryption look awfully similar to each other.

Now the mechanisms and how they actually work are very, very, very different. But actually trying to find specific search strings can be somewhat difficult. However, there are tools that can start inside of zip files as well. A lot of people mentioned Wireshark. Alright, we've got thar and we've got Wire Shark. What are some of the problems that you run into with Wire Shark? Let's say we wanna find specific strings in a capture file.

What could be something that would be very problematic, uh, when working with Wireshark? Because Wireshark absolutely. Well, you can go into Wireshark, you can do a search, uh, for specific strings in Wireshark, and you can pull out those strings. Problem is size. Whenever you get to very, very large capture files, Wireshark starts to have issues in actually parsing those very large capture files.

And if we're dealing with data dumps, like we're dealing with lop, a lot of the data, as I said, is in Pcap files that are monstrous. This data dump is seriously no joke at all. It's not something to just be, you know, taken. We're gonna drop it into a drive and just search through it. It's not going to work. Um, it's absolutely huge. Eric is getting really close. Um, he got, he said you could gr it. Unfortunately, GR in and of itself does not have the capability to actually gr for strings.

GREP is a very powerful tool, predominantly used in Linux. Things that are out there, Hadoop, absolutely. But once again, Hadoop has trouble with Pcap files, but Hadoop can handle the data size, the tool of choice for doing search whenever you're looking inside of, um, PCAP files is a tool called nrep, um, network GRP that allows you to go through in gr four strings within capture files themselves.

And that is one of the tools that we've been doing to do a lot of research and it's made Corey very, very uncomfortable, um, with what he has actually seen, um, so far. So let's do a quick audio test from Corey. How We doing? Corey? Am I here? Does computers work? Thank you. Okay. I, I'm sorry, I don't, I really don't know how to computer. I, I will learn someday. Yeah, if you wanna learn more about computing, um, not from us. So golf clap. You got a golf clap from Sandy.

So I was your warmup and hype. Uh, I was talking about the data size and the different types of data sets and Andrew has something to add as well. Go ahead. Yeah, Before we just jump in, let, let, let's just take a quick step back, John. Not everybody knows, certainly Corey's first time on the cyber call and thank you so much for joining Corey. Um, really easy platform to use as you can see.

Um, uh, so if you could, um, share a little bit about yourself and then I'll introduce Bob Miller for those that dunno, Bob, who's, Oh, that sounds good. Sure. And then we'll get into it here. You bet. Yeah. So now we've got everyone here. So my name's Cory Ham, I'm a red team at Black Hills InfoSec. Um, been pen testing for about 10 years now and I am a data hoarder at, on the side I guess.

Um, so I just have been for years always curious as to the sort of data that's out there, the impact of the data. And I think like my, you know, five minute pitch for why it matters is that attackers use it. Uh, attackers use data breaches, whether it be move it, whether it be some SQL database that gets compromised, whatever it is, attackers use it. So if you're a defender or a white hat hacker and you're not using it, then you are missing out.

And the other thing is, every time we use data breaches, uh, invalidates the data, we're basically taking the value away from the attackers. So if they were going to sell this data breach for, you know, a thousand dollars and I get access to it and I tell them to reset all the credentials, then it's not, not worth anything anymore because all the credentials are invalid. So that's my, that's why I do it.

Um, but yeah, I'm really a pen tester so there are people who are way better at it than I am, but I'm here so I have that going for me. And you're awesome on the news with John, so it's good to have you with us, Corey. Well, thank you. Yeah, I'm on, I'm, I'm on many podcasts. I'll go to your podcast, whatever it is. And then Bob, um, filling in on the MSP side, great to have you with us and, uh, co-hosting.

Uh, we've had, uh, I think, uh, Keith Bartol on as Phyllis a number of times and, and now you're filling in and you're acting as Phyllis, so it's great to have you. Yeah, thanks for having me. It's, yeah, I got the best spot in the house. It's phyllis's spot, so I thought felt really privileged to be there. So yeah, my name is Bob Miller, I'm the Chief Operating Officer Global Data Systems and we're A-M-S-P-M-S-S-P.

Um, and I'm also a software developer since I was a kid and probably one of the few cos you're gonna find it has two or three compilers on his machine, knows how to write code and knows how to use a and grip and yak and Lex and all the things that are down at the language level and binary level. So John and I know this speak the same ty on a lot of that and um, Corey as well. So that's, um, and I'm glad to be here. So happy to, happy to be a part of. Thanks for joining Bob.

Hey John, before we jump into move it, 'cause Corey was alluding to why this matters to MSPs and Bob, uh, being in Louisiana has got a personal, you know, knowing how this matters to MSPs. But we've got now John, this, uh, small issue, um, with um, WebP. Yes. Um, can you tell us a little bit about, you know, I know people have read and, and understand how big this is, but can you, can you give us some insights and thoughts and, and what this means? So Can somebody do me a favor?

Uh, can they pull up a screenshot of, uh, XKCD? We have, I'll ask the attendees 'cause they can find a link. 'cause we got people that are fans of XKCD and Randall. Um, there's an X KC D comic that shows all of the different technologies, uh, that the internet uses. And it's like this really horrific JINGGA stack and it's all held together by like a handful of technologies at the bottom.

And we have seen issues over the years, um, where like some issues in SSL, um, we had some issues in, uh, flat decode and, um, a couple of other encoding and decoding modules that were used in Adobe Acrobat. Um, and then we also have seen, you know, more recently some libraries, uh, that have been used that really were heavily used like Log four J right? That was heavily used on a wide variety of different platforms, right?

So turns out there's this library called Lib Web p and Lib Web P ironically, whenever I'm talking like, uh, FLATE D code, uh, for Adobe, um, it's used for compressing and decompressing image files. And I wanna talk a little bit about why you see a lot of vulnerabilities in compression software and decompression software. Um, so this actually gets into zip Whenever you go through and you compress a file, um, you go through and you find strings of data that repeat, okay.

And then you replace those strings of data that repeat with a placeholder. Like we can have Supercalifragilistic osis, even though the side of it is somewhat quite atrocious and it shows up in a document no less than 150 times, we can replace that with Cat. And I'm just using a really, really basic example. Sure. Well, what happens with that is whenever you're going through and searching for strings in a file, it's taking that data and it's loading it into a buffer in memory.

And the difficulty is you don't know what size that buffer needs to be in memory, okay? And that means that there's, there's right ways to do this. Trust me as Bob, I'm sure absolutely knows there's right ways and there's wrong ways to do this. And you really run into a wrong way if you're not checking the buffer sizes properly or at least checking before you're loading it into the buffer.

So traditionally, whenever somebody mentioned Wireshark, uh, traditionally there were a lot of vulnerabilities in Wireshark because of its ability to parse compressed data. And that's where a lot of vulnerabilities arise. So we had issues in zip, we had issues in Wireshark, we had issues actually in snort signatures years ago. Um, Adobe Acrobat is another example.

And also this library, which is utilized for compressing and decompressing images, there's a vulnerability within that parser and how it compresses and decompresses images that allows an attacker to do remote code execution on quote unquote Chrome. But here's where this gets really, really interesting. It isn't just Chrome.

Um, you're actually seeing this, uh, like any type of electron app, um, anybody that's using Chrome in any fashion whatsoever, like if we're looking at like Microsoft Edge is just basically a a a, like I think Bob made a joke and he was right. We're all using Chrome and I think zero, uh, zero click for like iMessages. There's been a number of different applications that we realized are using this library.

And it isn't necessarily that they're using Chrome, it's that they're using this lib web P library and this library is very wildly used. Now the problem that we're going to run into is just like Log four J, it isn't an issue of just we need to patch Chrome and then we're done.

We don't know the extent of how many applications that are used and are created out there that are working with images, processing and parsing of images are using this exact same library and the range of vulnerable versions, it's like something 0.5 all the way up to like 1.5, like huge version spread, have the exact same vulnerability again and again and again showing up. So this is gonna be a zombie vulnerability.

A lot of people are gonna patch Chrome, Firefox edge, it's all gonna get patched. Life is good, but there's gonna be tons of applications in the land that just gets ignored. The island of misfit toys that don't get patched and don't get updated. So hopefully that helped explain, and I think Ken actually gets a gold star for the XKCD comic. Um, I think he got the right one. Let's see if he did. Yeah, he did. Yep, he did. Ken, uh, Ken nailed it. Great job Ken. Thumbs up sir.

Um, but that kind of highlights as we move forward in the future, a lot of our technology is based on older technology stacks, John. So is this gonna be a air quoting a log four J issue? Are we gonna see people go, yeah, we fixed it and then all of a sudden get hit through some other mechanism? Possibly. Um, so this gets into the definition of remote code execution. Whenever you're looking at Chrome, I have to get somebody to click a link for it to actually exploit, right?

If we were looking at log four J, you could literally send an attack string to an application that was using the Log four J facility to parse logs to exploit it. That this is not that yet. However, once again, we don't know how many applications are actually utilizing this library. So it is entirely possible. Um, this is not true. I'm just making this up, but I'm using it as a case study so no one freak out.

Let's imagine that WordPress uses this library and somebody can upload an avatar to a website where you can upload your picture. Like a lot of people here I can see Tracy's picture, Wes' picture David's picture. Imagine that we have an application that allows you to upload images and that application has this vulnerability, then you're absolutely looking at a log port J situation, The buffer overruns of the over oldest trick in the book. I mean, Corey will tell you, right?

I mean since the beginning of time you've been able to run stuff past, you know, buffer links. And when you do that, you get inside an executable code that's outside the scope of that object you were working on as a programmer. And then all, all hell breaks loose. But John's right, a lot of those libraries are legacy libraries and they get compiled into sub products that are compiled into other sub products that are compiled into other sub products.

And unless you're able to kind of track that programmatic DNA all the way back to that core library, then it's very difficult to kind of pick up on those things. And especially if you're three generations into a piece of software and you got four different programmer groups that have worked over it all that whole time, you may yourself not know that you've got that way down the list of, um, embedded libraries in your systems because by the way, they don't always come as source code.

Sometimes they come as object code, which means they're binary when they're compiled into the, the system to begin with. So you wouldn't actually be able to see that physically from a source code standpoint. So he's right, we're gonna be watching for this, you know, for quite a while.

But by the way, this is not the only instance of programs that are susceptible to remote, you know, remote command processing or buffer overruns and being able to, to do this, this is one of the older tricks in the book. So it's, that's the reason it's, uh, in so many other libraries. So. Sounds good. And I did wanna clarify, uh, Bob, just as a distinction without a difference to most people, but some people as you know, are, are kind of sticklers about this.

This one is technically a heap overflow. Okay? Yes, You're Right. The buffer is closer to the stack heap. It's, it's an esoteric thing, but it's the exact same concept. Um, that's right. It's just more in the implementation just to be, the Programmers will care about that though. That's true. It is a heap. Yeah, Exactly. A lot of times the same problem that starts it actually. So it's funny, Bob's talking about buffer overflows.

There's a bunch of protections that your operating system have, like, uh, Terminator Canaries that protect the return pointer. Lots of protection on the stack and there isn't as much protection in the heap. That's right. So yes, a lot of these issues can manifest in the heap. It's the same, not the exact same, but many times the same type of root cause. Got it.

So, okay, we're Going this monster list of electron apps and it is frightening how many applications are using, um, this version right now. Alright, so Corey's our guest, Bob, I'm gonna have you pick up and kick things off and let's, let's, let's dig into this cop ransomware group. Um, I think it's a pretty fascinating story. Um, and we're gonna be talking about why a notorious ransomware group isn't using ransomware. And, um, so let's talk, uh, let's talk to Mr. Corey. Bob.

Yeah, Corey, I mean, I mean, as, uh, Andrew was talking about, can you kind of get us up to speed because I don't know if everybody really understands who LOP is, you know, and then, you know, talk a little bit about the, the move issue and how you, how big you think that really is and what the vulnerability was. Try to get us all clued in if you can. Sure. Uh, and I, I think with most ransomware groups, they don't really know who they are either, so it's okay.

But um, yeah, basically Pop Ransomware group is sort of a long running group. I think their other code name is like TA 5 0 5 or something like that. Um, there's a really good CISA breach that pretty, or sorry, brief that runs through all, yeah, I don't wanna say CSA breach.

Uh, there's a, a good brief document that runs through sort of their history as a group and, uh, you know, their tactics specifically covering the MoveIt breach and, you know, other file transfer appliance breaches, it seems like they really just have it out for file transfer appliances for whatever reason they went after Excel in the past, they went after move it. Um, so, um, and yeah, as far as like their motives, I mean their motives are to make a giant stack of money.

Um, just like every other ransomware group, um, they do sort of the, I guess what we would call double extortion ransomware, which is essentially not only do they potentially prevent access to critical systems and try to extract payment from that, but they also exfiltrate data and then try to extract, uh, you know, a ransom payment so that the data is not released.

Um, the, in this case, you know, if we're talking about the MoveIt breach, which is sort of the, you know, it's, it's right there in the title. Um, the scale of the data is significant. I think that recent estimates are around like 2000 companies are currently in the list of breached companies. Uh, and when I say that, um, that's sort of the, like, best guess, ultimately they've only actually released the data of about 230 ish companies.

So, um, how many of those, it's kind of funny 'cause if you go on their webpage, their ransomware page, they have, you know, these ridiculous ethics and it's, it's actually pretty funny, but they're, they're basically like governments, you don't need to worry. We already deleted your data healthcare companies, you don't need to worry. We already deleted your data. Everyone else pay us. Um, so basically like, you know, do, do they honor among thieves? I don't know.

Um, but yeah, like that, that's sort of my overview. The, uh, cloud group they've gone after other file transfer appliances they've got, that's sort of their, it seems like that's their mo we find a zero day or a vulnerability. Um, we exploit it and then we try to extract payment, um, specifically from file transfer appliances. They've also been involved with like ransomware as a service. They've done initial access brokering, they've done all that good stuff as well.

But that seems to be sort of their bread and butter is the, we find a zero day in your file transfer and we make your life really bad after that. Hey Corey, can you talk about initial access brokering, because that's something I don't think a lot of people know is going on. What, what is that and how does that work? Yeah, so initial access brokering is essentially, um, as you'd imagine, uh, black hat hackers are lazy and they wanna make money as soon as possible.

Maybe they're trying to make rent or maybe they're trying to keep the KGB outta their front yard. We don't know. But basically they, the initial access broker thing is they sell access. So let's say I'm a threat actor doing a bunch of password guessing. Um, you know, 'cause that's what I wrote this really awesome program. Once I get a valid credential from password guessing I can use it and maybe get, you know, I, it's a lot of work.

I have to go in, you know, maybe steal some emails, mess with some invoices, email 'em to other people. It's a whole lot of work. So instead, you know what I'm gonna do? I'm just gonna sell this valid credential to someone else. I'm just gonna sell this, you know, maybe it's worth five bucks, maybe it's worth 10 bucks. That's money in my pocket right now. Versus me having to do the work of, oh, I have to go out and, you know, I guess I have to edit this email between the CEO and his assistant.

Now I have to do all this. You know, so that's sort of the economy of it is threat actor groups like klo either sell or purchase initial access from other threat actors because it's sometimes it's easier to just purchase it versus finding it yourself. Or sometimes you're good at finding it, but you're not good at using it. So it kind of is like, ironically similar to how business processes work within like penetration testing teams or like it, there's handoffs.

Um, it's just when the handoff is a criminal handoff, we, we give it this fancy term, we call it initial access brokering. But that's in a nutshell what it is. Well, Corey, you talked about the number of companies that were affected, but from a, just to kind of have people get a feel for scale, what do you, what's the, what's the dimensions on the amount of data, um, that they've exfiltrated? I mean, I'm sure there's some guesses out there about how big that is, right? Yeah.

So at least of the data that I've, that's on my radar, which is what's public been publicly disclosed, it's about 24 to 30 terabytes compressed, which probably expands to about, I don't know, 62 a hundred terabytes uncompressed, assuming at least a two x compression ratio, which is, and and it's tough because a lot of the archives are like multiple layers deep.

So you have like a compressed archive of this company that was breached, but then there's, you know, a thousand more archives in that archive. So I've seen about a two x inflation ratio on the data sets that I've analyzed either for clients or just for, you know, research purposes. Um, but so it's large, but it's not that large. I guess it depends on yours, it depends on like your sense of scale, and that's just what's disclosed by the way, right?

If we're looking at what they have, I mean, it's probably at least 10 times more. So let's say maybe 300 to 500 terabytes, because we know there's, they've disclosed about 230 companies where there's probably at least 10 times more that, that they've breached and haven't disclosed. So let's guess a 10 x increase in size. So probably around 300 terabytes, I would guess. But it's, that's really a guess.

I just wanna Say, as a business owner, I got this text message from Corey one night, and he is basically like, I need a server with a hard drive this big. And I, I, I love hitting approve on those types of things. So Yeah, and analyzing it legitimately, like, I mean, obviously if we're just talking about one customer at a time, it's not, it hasn't, I mean, the technical challenges are still there.

Like I talked about recursive, uh, you know, archiving is actually kind of annoying to like recursively extract everything down to, you know, random levels. But, um, when you're talking about analyzing this data at scale, like I'm already running into like, I'm, like, yesterday I googled like, what is the theoretical max size for a Postgres SQL database? Like what, how, how big can that get? Um, turns out it's about 32 terabytes, which, which is like, we're just about at the limit.

So it's like, um, you know, even just analyzing it. Um, and yeah, it's, it's, it's kind of terrifying. The data is all over the place. Um, you have data that, like we were talking about P caps, that's an example data. Um, there could be everything from emails to documents to software packages to, um, I mean, tell yeah, anything, anything. Basically, I will say it's funny from a business perspective to see how different companies are using their file transfer appliances, right?

Some companies use this for, like, there was a theater that was breached. They use it almost only for HR purposes. That's one use case. There's another company that's using it almost only to distribute information to their clients, like troubleshooting. You know, if, if you have a problem, create a debug archive and upload it here, and then other companies are using it internally or it's just all over the place.

As far as like, file transfer appliances are pretty integral to most businesses, right? Um, that is a required use case for most companies. It's just they use it for different things. So it's, it's kind of up to that impacts the breach for each company, right? Well, I mean, clock was kind of known as a ransomware group, you know, I mean, for the most part, right? I mean, it seems like they've kind of shifted gears and, um, taken on some new tactics.

So, I mean, I've, I've heard rumors that they may even have bought this kind of zero day exploit and just started running the ball with it, you know, once they kind of got their hands on it and saw what it was. Because like you said, they kind of are getting an MO now about hitting these, um, you know, fall transfer mechanisms that a lot of people are using. And like, we all know F-T-P-S-F-T-V, these have all been vulnerable for a lot of years for a lot of reasons.

So why the, um, so, but I mean, why are they, is exfiltration over ransomware just a better, just a better use case for them? I mean, are they having better success from extorting money as it relates to that? Do you have any kind of feel? I would say I, I mean, yeah, I mean, I would say it's, my book still counts as ransomware. It's just more extortion for data than it is extortion for access. Um, you know, we, there's different types of ransomware tactics, right?

So the, when you say traditional ransomware, I guess you could, you're talking about preventing access to IT systems, right? I, you know, what happened to MGM recently? I'm in all your SXI servers, I'm gonna type RM dash RF unless you, you know, give me the ransom payment of $50 million or whatever. Um, but yeah, I would guess for as, as to the reason why, I mean, who can, who knows, right? They're, at the end of the day, they're criminals. So why do they crime? We don't know.

But I think that yeah, there is better success if you've exfiltrated the data already. They can't just restore from a backup. They can't, there's no disaster, uh, response plan that can get you out of that situation where someone has stolen your data. So I would guess just from a success perspective, I can't just as the CSO say, oh, well we have backups, don't negotiate. Um, that doesn't get me out of this situation. I still have to notify clients.

I still have to examine the scope of what was breached and potentially invalidate credentials or invalidate certificates or whatever it is. Um, so that's, I think that evolution of ransomware. And, and by the way, they would do both if they could, right? This v this particularly, this particular vulnerability didn't really give them, um, a good, I mean, I think also they probably had, were getting discovered because their tactics were not particularly quiet.

And so I'm guessing they just took what they could and left before sort of getting to that stage two of ransomware, right? So this is how we see ransomware actors now. They go, we get in, take everything out, then we start moving laterally and trying to actually compromise credentials. Like the traditional, you like using cobalt strike, all that good stuff. So that's like, they do that first, I would guess someone probably caught onto them and was like, that's weird.

Why do we have a health check user in our move it, I don't remember that. Or, you know, why do we have human two aspx generating all this traffic, whatever it was. And then they're like, ah, crap, we gotta get out.

So, But Corey isn't, isn't part of it, just like asked Bob they as quickly as they possibly could knowing like, you know, the showdown scan shows 2000 plus, let alone we know the web was also also vulnerable, but wasn't, if you're, if you're plot, don't you want to hit as many as possible? And if I have to deal with ransomware simultaneously, it's slowing me down from exfil. Yeah. Was it, was it a race against time? If you had to imagine?

I would, I would say that's a really solid assumption because it, as we know, these things are monitored, right? Not, their tactics weren't particularly stealthy. They were literally adding web shells and users to the people's file transfer appliances, right? So even beyond that, they weren't really, they didn't really have capabilities to be that stealthy.

Like if I'm an IT admin looking through my MoveIt users and there's one called Health Check that was made, you know, last night and it has, you know, a last login from Belarus or whatever, I'm gonna be like, that's weird. Um, but you know, I, I think that's a solid call.

And I think it's also, even if they, even if their attacks were steal, even if they weren't adding users and web shells and things like that, when a zero day gets used, things, you know, data is mind for everything, including threat protection. So when a zero day gets used, there are companies that sit in the middle or sit on the network appliance level and are looking for that kind of traffic.

Um, you know, if you happen to have a WAF in front of your movement appliance, you probably saw something, right? And those are gonna get investigated. So I think your sort of guess that it was time-based is a solid guess because it's like, get in, get all the data as much as you can out as quickly as possible and then go after lateral movement, DMZ bypasses getting in and then preventing access to other systems. Yeah. I wanted to hit David's question, if that's okay, Andrew. Yeah, sure.

Um, so part of the issue is the size of the data here, doubtful the encryption would even finish in time to cover much before being discovered, and not necessarily on the encryption side, but how long does it take this to actually decrypt? Like you said that it's, sorry, decompress. You said that it's got compression. Is it a bunch of compressed files that are all different sizes, or is it just a mons huge compressed file and are you running any issues processing the data with compression?

So I mean it, to fully analyze a data set, it probably takes about, so talking what, what I mean by fully analyze, I mean take the archive provided by the threat actors, which varies in size from, you know, a gig to, there's one zip file that's like 950 gigs or something like, it's like a stupidly large zip file, taking that file, extracting the file, then running that through a left, which is the tool we were talking about at the beginning, which is a tool designed for journalists, but it's essentially like an unstructured data analysis tool.

It, it will recursively extract archives, it will try to analyze business documents. It'll do OCR, um, it tries to look for like, it's actually really cool. Um, but yeah, just based on the scale of the data, I'd love to look through it manually. Like I'd lo I'd love to look at every single P cap and run those through Zeke or whatever and or bro, and find some, you know, look at the file log and see, oh, look, there's a file in the pcap. Great problem is I'm doing this for 30 terabytes of data.

So it's not really feasible. Um, yeah, as far as compression and decompress, yeah, it would take, um, it would take a large amount of time to sort of encrypt that, uh, data. I would also guess that they probably didn't necessarily have, um, access or permissions to do that on within the scope of a file transfer appliance.

It doesn't have like a, you know, encrypt, like they didn't, you know, it's, they're working with what they've got and they actually didn't have that deep of access to, you know, they basically had like web shells, right? So as a, you know, as an attacker, I love web shells, but when I get them, the next thing I'm doing is I'm getting a real C two. I'm not gonna use web shells for everything, right?

So, And that gets into, like one of my pet peeves about a lot of people that are analyzing these groups. They say the group didn't do X, Y, and Z. The group didn't, you know, use elite malware to actually do what they were doing. And a lot of times they get called like the not so advanced persistent threat.

And kind of my take on that is the, the attackers, like if you're looking at clop or any of the different shadow groups or tiger groups or whatever, they'll be just as advanced as they need to be to achieve their specific goals and objectives. Their goal and objective is not to impress you. And whenever they have all of these servers, like if we're looking at the MoveIt vulnerability, I can only imagine just how busy they were, right?

Trying to exfiltrate and trying to break it like that many systems and trying to pull down that much data. So if you ever have somebody that's a researcher that's like, well, this malware's crap, they got this from Cobalt strike. Yeah. Because it fricking worked and, and you know, they use this exploit, they're gonna use it until they just don't have it anymore out there. Well, hey John Corey, just let me ask Wes and Bob to chat just for a second.

From the MSP side, Bob, you personally are aware, right? What happened to the can, can you un unpack the DMV in Louisiana? Sure, sure. I mean, that was, um, yeah, sure. And then, and then let's think about this as MSPs. A lot of MSPs, when they heard move it probably, nope, not casting dispersion, but probably Wes probably said, Hey, we don't use this kind of stuff, our clients don't use this kind of stuff. But when we look back, I'm gonna save Wes on this one. I totally said that.

I was like, I think it was on of the news. I'm like, no, like real enterprise is gonna use this software. It turns out they did. Yeah, The government did too, right? Right. So 900 plus universities know that as, as John said, well, why is initial access important?

So Bob, maybe just take this away from what you know personally, what happened in L and then Wes maybe just unpack how you could foresee this all falling back into the MSP ecosystem because they have a, just a tr trove of fishing arsenal now, uh, at their fingertips. Yeah, this is, this is gonna be easy. This is gonna be the best softball, this is the biggest softball west has ever clocked out of the park, right?

So in, in Louisiana, so we, early on our office of motor vehicles got compromised, right? So that's about, it was about 6 million records. Um, which if you're from Louisiana, you do the mindset, wait a minute, there's only 4 million residents, but vehicles were registered along with human beings, right? So ROMV actually had a compound database.

So, but the thing that our, the thing that's important about that from an MSP standpoint, which Wes is gonna, um, Wes is gonna talk about in just a second, is that what, what happened in this case is, is this is the first data set that I'm aware of personally, where our driver's license were connected to social security numbers, were connected to addresses, which were connected to names, right?

In a lot, in a lot of the, in a lot of the, uh, exfiltration in other places, you get parts of those pieces, but rarely do you have full linkage between all four of those really important ID numbers, you know, and, and your, what identifies you all in one place. So I'm gonna hand that over to Wes 'cause he is gonna explain, he's gonna explain how much ammunition that is. People who, who, you know, attack human beings and do social engineering. So, Wes it away, You're on mute, bud.

You're on mute. We, One sec. Wes, you're on mute still. No joy. I think I helped set up his mic. I think he's, You guys are right next to each other in, in, He's gotta be watching this and just going, yes, I'm infecting other people. I'm sorry. I mean, I can speak a little bit to it if you want. No, but yeah, Bob, so, so yeah, I mean, this is interesting because I mean, think about like, if you're an, as an example, like I'm just thinking an MSP that has cars registered, right?

You know, uh, vehicles registered, uh, you have customer, you know, go ahead, Bob. What, what Are your thoughts? That's the thing. So, so what it does, right? I mean, we've got 300 or so customers and they've got employees and you know, a lot of 'em are in Louisiana. So now all of a sudden, all of their critical telemetry is in the wind, right?

So all of those things that are already exploits for these guys, phishing, business, email compromise, social engineering, they have more data to work with. So the risk profile has gone up, right? Because now those people who are customers of ours, their probability of being exploited in some way has gone up probability wise, and they're, therefore us as a service provider than them connected to their network have to be even more observant about what's going on, right?

Not only our employees, but also their, you know, their people. So the whole ecosystem in general, right? An order of magnitude more complexity has been introduced into the, into the, into the system because there's just that telemetry gives them the ability to make their phishing attacks and their social engineering attacks look a lot more real, right? Because the way you usually detect those things is there's something not in line with what it is they're telling you, right?

Either they got your name slightly wrong or they got, you know, your address is not right, or God forbid, your social security number suffering the wind. So you're always kind of deck, you know, humans can detect that a little bit, but when they got that much real valid information, it just makes it a lot harder to detect. Um, so therefore we're, you know, our risk profile has gone up as it relates to that. That was very critical information to be put into the wind.

I mean, all of that together creates too many linkages. Yeah. Wes, are you with us? I Think I'm back in my back. I, I don't know what I did different Or is gone, but you're back. So Yeah. Bob, Bob, you nailed it. The only thing I'll harken back to a little bit is, does anyone remember the OPM breach from way back? Yeah. Um, you know, that was significant because of the amount of data that that, that China now had access to and, and very, very, very highly controlled data.

And then also Equifax, you look back at what happened with equ Equifax. Oh yeah. I think both of those in particular, there's a lot of chatter about what's gonna happen now that China in particular has so much of our information. Maybe we didn't see a lot of it play out, but it doesn't mean that it's not sitting there in the same way as nuclear weapons sitting in an arsenal ready to be launched, right?

And so, you know, I think part of this is gonna be what all can and will bad guys do to leverage all of this. And as you know, data gets better and correlations get better. You know, you see, I, I hate to be the, the person that talks about the marketing buzzwords, but is, you know, is natural language processing gets better. Um, you know, and you have models like GPT that allow people to do things better at scale. These things do get more significant.

I think clients need to understand they're not less of an attack, they're more, they're more of a subject and int attack that's going to be more effective against them because they're gonna be more willing to open and and click on these things that seem relevant and real to them. And, you know, I kind of wanna piggyback on that. 'cause what Wes said I think was really, really, really important.

Whenever you're looking at the, the Office of Personnel Management, like all my classified SF 86 was completely dumped in that. So they have information on me, my wife, all the people that I filled information out on. But whenever you're looking at like DOD and law enforcement in the United States right now, when you're, whenever you're sitting around with them and they're having beer and they're drinking and they're being completely honest, they're not worried about ransomware.

And the reason why is ransomware is a thing. Yes, it is destructive. Yes, it is absolutely immediate, no question. It hurts. Absolutely. It's putting companies out of business. None of that is that question. But what is terrifying to the United States government and law enforcement and DOD are the attackers that are not destructive.

Whenever you're looking at like the Chinese, you're looking at the Russians, you're looking at any organizations that actively wanna hurt, um, the United States or anybody that's just trying to do their job. We are not in the industry as, as a whole having a really good conversation about the attackers that aren't there just to encrypt your data about the attackers that aren't there just to steal your data and then try to get money out of you.

We need to somehow open this conversation up, especially with MSPs, but what the hell are we going to do to detect the attackers that are in the power plant that you're responsible for protecting? They're in the water processing plant that you're protecting in the critical infrastructure, which MSPs are responsible for 56, I think I remember that at write a boom, percent of the critical infrastructure in America is protected by MSPs who can barely keep up with ransomware.

We need to have an open conversation about what we're going to be doing about the attackers that are dwelling and siphoning off data. And right now, I really feel like the MSPs with what they're doing and their resources that they have, they can barely keep up with a destructive ransomware without even having a conversation about the dwell attackers. Yeah, makes sense. John, John, can I just ask you a question?

And maybe you can answer this to whatever degree you feel comfortable, but you know, that's part of what you just talked about is the purpose of federal government recognizes that that's where CMMC is trying to come out and say, we've gotta, we've gotta bring privacy into this whole thing. Any thoughts or opinions on that whole thing? And, um, its ramifications and effect that it'll have, especially down market. Hey, let's p**s off some people. Um, so if we're looking at CMMC, right?

Um, a lot of the people in the government look at it as a failed project. And there's gonna be some people from CMMC that are gonna contact me and they're gonna yell at me, and that's fine. Um, what is the level of adoption and did it just become another compliance check kind of compliance chase? Like we just have to do the checkbox. So whenever you're looking at it, a lot of is what is the absolute minimum that I have to do to become compliant?

And this is not an issue in CMMC and saying that it's inherently flawed in its design. It's just that's the flaw of every single compliance framework that's out there. It just is because people are looking at how they can meet the absolute minimum. That's what we're going to have to deal with if we're actually looking at it. Um, going further, if we're looking at cisa, uh, once again, these are well-intentioned people that are trying to do their best.

CI is literally coming out with lists and saying, here's the top hundred or so vulnerabilities the attackers are going after. And there are vendors out there who are literally just creating vulnerability scanners that are going through and just checking for those vulner vulnerabilities. That's horrible, right? We're missing the point here. Like we're literally going through and trying to fix the things that are already being patched without actually being proactive in what we need to be doing.

Um, when we're looking at this as an issue, and like I said, I know I've just p****d off people in the government, um, and once again, they're trying to do the absolute best that they possibly can in almost every single framework that's out there. It's a race to the bottom. How do I run a scanner?

And even going back further with cisa, CISA is now offering free penetration tests that are nothing more than a vulnerability scan with a handful of tools so that people can actually get their checkbox. This isn't helping a g*****n thing. It's actually normalizing mediocrity across the entire space.

And Frank, I think looking in the comments, Frank is going through and it's saying six class action lawsuits and the Nevada District Court claims against MGM resorts and Caesars and attainment failed to protect the personal identifiable information that's out there. And boy, the comments came through really, really fast. And I think what the United States honestly needs is A-G-D-P-R framework.

And I know that a lot of people in the MSP space are like, oh God, not another compliance framework, please no. But GDPR is a penalty framework. If you get compromised, you're never in compliance with GDPR. If you get compromised, it's gonna get reported. And once it gets reported, people will see were you actually doing acceptable levels of due diligence to try to protect your networks and then they will punish you based on how much you did or did not do to prepare for it.

So trying to come up with a proactive framework, I just don't see that actually working. And we work with customers all the time. They're like, we had an audit from X company, they said that we were completely good and you came in and wrecked our environment. And they get mad at us. They're like, how is it possible that they said that we were fine and you came in and you hacked our network? I'm like, I don't know what to tell you other than they probably sucked at their jobs.

So I think that we need some type of an accountability framework, and this helps MSPs because, or MSPs as well, because it puts a lot of, lot of that responsibility on the companies and the organizations that MSPs and MSP MSPs are protecting. And if that responsibility for not implementing proper security controls falls on their shoulders and they get compromised with a crappy MSP or crappy MSSP, then you're gonna have people that are starting to really seriously look.

And if I'm hiring an MSP or an MSSP, do they actually have a good security program that will help protect my organization rather than just being a simple checkbox exercise. Sorry for the long rant, Wes, I apologize. It's awesome. It's good, you know, in, in, in for time. John, I would like you to, if you could maybe just chat with Corey and then we'll get into Wes, um, on a few.

But, you know, for for can, can you ask, you know, for Corey's perspective on, you know, if you did have MoveIt or if you are an Ms P, you know, 'cause again, we talk a lot about MSPs moving up. I know Bob's company specifically is hard right? Heart of mid market government, et cetera, that do have things like this. Um, so, you know, I'm, I'm just wondering your John really Corey's perspective, like, is this, you know, if you have it, is it rip and replace?

Is it, you know, you know, if I had move it, you know, what, what are your maybe your, your context on that, John and over to Corey, his thoughts. I'm gonna, since I've already ranted enough, I'm just gonna say patch it. And if you think that this software is any more or less secure than all the other software platforms that are out there, I am sorry they all have these issues. This is just what they're actually the attackers are focusing on right now.

Yeah, I'm gonna come in with a slightly different perspective, which is, I I would approach this similar to a LastPass scenario. Um, so for those that don't know, LastPass had a few breaches. Um, and you know, a breach is a breach. I wouldn't necessarily hold it against the company for being breached. It can happen to anyone. Um, but the breaches were multiple, they disclosed it in a sort of, let's just say, uh, slow manner and investigated things slowly.

It it, and also it revealed sort of some weaknesses with sort of outlined some of their business plan or security plan, which wasn't really up to a lot of people's standards. Essentially, the, the long story short being LastPass should have assumed that if these things are compromised, they should have planned that into their meaning the vaults.

They should have planned for these to be compromised, they should be secure and un like there should be no value in compromising someone's LastPass fault. That wasn't the case. And so it led to a long discussion from various companies, do we ditch LastPass? I would say that same conversation that companies had should be the same way that you should approach move it. And my, the reason why I say this is this wasn't the first file transfer appliance to get targeted by lop.

We know that file transfer appliances, it happened to excelon, they also got zero date and they also got ransomware. So if I'm the CSO or whoever running move it, I am like, well, there's a ransomware group that's targeting file transfer appliances right now.

Maybe we should get a pen test guys, or maybe we should, and I'm not saying they didn't get a pen test, but these flaws SQL injection in 2023, I mean, don't get me wrong, it happens, but, um, the fact that their cloud product wasn't vulnerable and only the on-prem client was vulnerable, um, the fact that it was SQL injection, the fact that I can just deploy a web shell, um, essentially like you have a similar discussion with your people that you had about LastPass.

How do we feel about the fact that they're using A-S-E-C-B instead of a s like end-to-end encrypted real good crypto. Um, how do we feel about the fact that the, you know, cloud product wasn't vulnerable, which means they fixed things or it was a different platform or whatever and our product was vulnerable, right? This as me as a customer, I'd be like, where's my patch? How come like, I, you know, like how come this was left in production?

Um, so I would say it is sort of a, it's a discussion to have at a company between the, you know, stakeholders that are purchasing or maintaining these products. Um, but I guess what I would say is there was before, long before move it, there was a ransomware group going after file transfer appliances and it was SQL injection. Um, so looking at the vulnerabilities, looking at the disclosures, like reading through things.

I'm not gonna say it was the most complex thing in the world, but it's not the simplest thing in the world either. Um, it's kind of right down the middle of being, but really a sequel injection. I mean, come on. Um, you know, have a waf have a, you know, have effective logging have, you know, like why isn't there an alert that fires that says new user created, I don't know. Anyway. Yeah. Or decent long pile. Yeah. Anything. Yeah. Yeah.

So that's basically, it's a long answer, but that would be my answer is I would approach it as, yes, anyone can get breached, anyone can have a vulnerability. This is absolutely true. We know this. And it's more about how the company reacts than it is about how the, you know, and I will say they reacted, in my opinion, just absolutely fine. They reacted properly, they disclosed, they had a patch quickly. They like, they reacted effectively.

But it brings into question the, they knew this, they had a target painted down their backs. They didn't necessarily do the right security testing to identify this vulnerability. And, you know, it wasn't fixed before the, you know, it should be fixed in dev, right? Not, not to mention fixed before, after a breach. So your decision, that's my personal take. I, um, you know, I don't think, I think at a higher level, if you're sitting there wondering what can I do about it?

How can I not be the next to move it? Basically, uh, from a, from a customer perspective, if you're purchasing these products, you need to put the security controls that you're comfortable with in place on top of them. So like the Office 365, uh, storm 0 5, 5 9, recent compromise, great example. You can't ensure or maintain the security of your products that you subscribe to, but you can make sure that they have logging. You can make sure that you have effective alerting based on those logs.

You can make sure that the customer, the, the vendor that you're purchasing services from has your back to some level, right? Like there you should have a log of who's logging in to your file transfer appliance or who is, you know, a WAF in front of it or things like that. Like you put this in place place, um, and that's how you not, that's how you avoid being the next move it or the next customer of move it. So, so, great. So Wes, um, want you to ask a question.

I know we got a few minutes here, but Wes, Corey mentioned logging. I'm still waiting. I haven't, I've asked MSPs, right? September, speaking of storm, um, we were supposed to get logging free. Um, does have you, I haven't heard of one SP that can validate Microsoft can says, Hey, yeah, this is where to get it. Um, will we, or is that lip service? Wes? It's lip service until it's here, right? Yeah. Is that the pessimist to me speaking? Yeah. No, fair, fair.

Wes, you wanna wrap us up with a question or two? Yeah, for sure. How about one last question. Um, John, I want to get real practical for a minute. So let's say this is, move it, let's say this is the web p was to say something new that comes out and as an MSP, you just heard about this today on Reddit, bleeping computer cyber call, something, what should the runbook be for an MSP the instant they hear about something like this?

What's the motions that they need to be doing and thinking about Planting trees? The best time to plant a tree was 20 years ago. The second best time is today. Um, the best time to do a software inventory or start trying to figure out how to do software inventory, um, was last year. The second best time is today.

Um, it's not about what you're gonna do whenever this happens, what you're gonna do, you're listening to the show, you need to answer the question, how do we go about inventorying our systems? Um, whether you're using Microsoft Tools, whether you're using your, uh, RMM tools, how do you get an inventory of your customer's environments? So when this happens, you can quickly and efficiently answer that question. 'cause it's not about what you're gonna do for the next one.

That's, that's, I can't even predict what that's going to be. But if we're playing a game of chess and I know what your first gambits gonna be, it's gonna be searching across your customer's environments to find that, learn how to do that software inventory right now. And oh, by the way, it's part of the, uh, critical security controls. Um, so you're gonna be doing good there as well. Yeah. Good. Really good stuff. Right on.

Well, uh, first, um, to the audience, thank you for, um, hanging in there with us. Uh, it was off to a little bit of a rocky start today. Um, on the flip side, Corey, I want to thank you so much for coming on and, um, you know, sharing your perspective and the research you're doing, John, always appreciate the support. Um, and, uh, stay tuned for some John, John might have some big announcements coming up here with, uh, soon and we'll, we'll, we'll, we'll let, let everybody know what those are.

Um, Wes, always thanks for joining and, uh, Mr. Miller, awesome. Having you on the host side. Wishing everybody a fantastic day and we'll look forward to, to seeing you all next Monday. Take care. Alright, thanks everyone. Thanks everyone. Bye Later, everybody. Take care. Take Care.