Adversary Emulation demo

All righty. Welcome back. And I am getting all the, uh, important folks here for this. Matt, we for and Ryan. All right. Let's see if we get everybody back here. Wes, email everybody back, Ryan. And then Four outta five Five. Let's see if forest is coming. I dunno what it is. All the, all the good M-D-R-M-S-S-P vendors are are Bird themed. It's right, baby. All birds. All right. The force may not be back yet, but, um, oh, he's coming.

Hey, and while we're waiting, um, Andrew, can you pop out a poll question that says something to the effect of, have you ever seen an adversary emulation tool or something to that effect. I'm just curious to get a gauge on our audience today. Um, do they have, have they ever even seen what we're about to show them today Or not For where at my friend Wendy's. Uh, so that, that Got a little complicated. I apologize. I'm a few minutes getting back.

My wife is now gluten free for some experimental medical reasons, and that whole thing kind of went to crap at dinner. So I've got the 2-year-old, um, he'll be my, my, uh, popup assistant intern here a minute. Sounds Good. Okay. So with that, um, Matt, should we let you kind of set the stage a little bit here, maybe with Brian in terms of what we're about to see? Yeah, certainly. And welcome, by the way, Matt, by the way, thank you for joining us.

This I know is something that we requested of Red Canary and really appreciate it. Matt, can you just give a quick overview of who you are, what you do, uh, and then we'll get right on into it? Yeah, certainly. My name's Matt Grabber. I'm the Director of Threat Research at Red Canary. My background in InfoSec started off, um, in the offensive sphere, um, then eventually transitioned into, um, a couple years of dedicated malware analysis and reverse engineering.

Uh, since then the past few years, I've dedicated myself to pretty much exclusively, uh, the defensive side of things while still always wearing my, my attacker's hat on in the interest of, um, creating new and improving existing detection logic.

Um, 'cause really, like, the only way that I know, like with my background, um, in how, um, myself and my team can go about being confident in, do we detect M-S-H-T-A, do we detect technique X, Y, and Z, um, is to really think like an adversary and understand their process and really emulate how they would go about doing their research to identify as many ways as we reasonably can to bypass the, the logic that we ourselves develop.

So it's very much a, a cyclical process, like rinse and repeat this whole thing, like develop logic, validate it, try to bypass it, build resilience against those bypasses, um, until you're just increasingly confident and then can move on to the next thing. But detection is, is never done. Best thing you can ever hope to do is, is, is manage it. And, um, that's, that's part of my role.

And the, the deep dives that, uh, me and my team get to do is do that deep dive research, um, and just continually respond to threats and attack techniques as they come out so that we can be as confident as we possibly can that when we do develop detection coverage, um, that we actually have something good to, to show for ourselves. That's awesome. Thank you for that.

And Forst, maybe if you could just set the stage in terms of how you work with adversarial in elation and how you're gonna be interacting with Matt, because again, this is new probably for most MSPs. Um, in fact, uh, you know, when I look at the poll west, you know, 90% I've never seen adversarial animation tool. So force would be awesome if you could maybe just set a little bit of the stage there. Yeah, everybody. So, um, and again, apologies up front.

My intern jumps in here, but the, the whole concept, right, as we talk about adversary emulation, um, if that doesn't translate really well, it's just like Matt just said, you wanna understand what the adversary does. So he said he wears uhhuh. Okay, can you wait one minute? Um, so you wanna wait what the, and, and you wanna actually look at what the adversaries are gonna do. So Matt said he wears his attacker's hat, right?

So there, a lot of us that do this do come from pen testing, um, adversary emulation, red teaming, kind of background attacking networks, but really for the good guys, right? You're, you're doing it to try to find the problems first and try to help organizations mitigate those problems. And so at Mitre, you know, we, you just heard a lot about shield, you heard a lot about the attack framework. So we certainly start by first trying to understand what real adversaries are doing, right?

We want to understand the techniques that they use, we wanna understand who they're targeting, why they're targeting you. And you know, and once you understand that, because there's, you know, basically every, oh, I see the streams locking up it looks like in the chat. How No, you're fine. You're Fine first, you're fine. Okay, good to go. Awesome.

So yeah, so once you understand all that, right, if you were to look at the attack framework, like you just heard from Ryan and Wes, there's a lot of things on there. Um, and if you wanted to suddenly secure your entire environment against every single technique or sub technique, that would probably be a bridge too far. So really the goal for most organizations, if you're an MSP, you might specifically be interested in organ, in APTs targeting MSPs.

But again, if you're doing managed security for a variety of clients, equally interesting is for you to understand the specific adversaries going after those clients or those client industry verticals. And so at Mitre, we, we do a lot with that, right? We start with the attack framework. We do a lot of open source mapping of real apps and real attacks, right?

To the attack framework so that it's very clear if you want to go to, um, a PT one, AP T three, APT 28 or something more like Menu Pass or F six, that, that these groups are there, they're mapped and someone's already done the homework for you to understand the TTPs they use, but okay, but that you, you could say knowing's half the battle.

And maybe that's literally true because even if you know the, the actual TTPs those adversaries are gonna use, and you have a high confidence they're coming after you or your clients, that still doesn't mean you know how to detect them. That still doesn't mean you know how to defeat them.

And so I think what, what Matt's about to show you, uh, in terms of adversarial emulation is basically instead of, you know, paying for a more boutique firm to come in every single time and kinda replicate a, a, an attacker group, if you already know the ones coming after you and you already know the techniques they use, there are mechanisms to go ahead and if you will, play those attacks against yourself. And, and especially to do that via automation so that it's repeatable.

So you can attack, try to detect, and if you miss it, try again, try again. And you're not, again, bringing in external firms every single time you want to do that. You can build a program in house to automate that yourself for the subset of adversaries coming after, again, you or your client base. So I think this is huge.

Uh, MITRE Hass been doing a lot in this out of the attack team and others, and out recently out of our Center for Threat Informed Defense publishing, quite a few of these emulation plans to help you either manually walk through or automate an adversary. And a lot of the work that we did, if you look at our automation, um, format, was wholly founded and related to what you're gonna see in Atomic Red that Matt's about to show you.

So, uh, really looking forward to the demo, uh, again, what Mitre does, sometimes a little more academic, I think Matt's about to show you the hands-on and how you can really get a program like that up and running for yourself. Yeah. So two, two things there. One, MITRE has its own emulation tool called Caldera, right?

And so it, one, one thing I want you and Matt to do is I want you to compare and contrast the two and like, what the difference is, why you might use one versus the other in what scenarios. And then two, hot off the presses. You, uh, you mire recently released a beta of the Defend framework. Help us understand how we can use Defend in relation to the attack framework and how Defend is different than shield. Awesome.

Matt, the floor is yours and, uh, if you wanna share things out or take us away. Yeah, Yeah, great. Um, just comment on what Forres was saying. Uh, I think it was, uh, selling mit, uh, MITRE, uh, perhaps a little too short. Um, the Mitre attack framework is like the foundation of re canary's detection ability, um, and so many other organizations that I'm aware of, like use that as the foundation for, um, from which all of their detections are built.

And so it allows us to, um, to track adversaries, to track threats, to, um, attempt to establish a baseline of detection coverage of techniques. And so it really is core to everything that we do, um, in, uh, in our role at ary. Um, and so we, we take that foundation and we go and apply it, um, in this case, um, in reference to adversary emulation. So let me pull up my VM here. I appreciate those comments, Matt.

It's way better when someone else says it than if you're from Mitre and you're like, Hey, what we do is really a big deal, you know? Yeah, It really is. Okay, so let me walk you through the, uh, the scenario that, uh, we're, we're gonna go through here. So trying to make this as realistic as possible. Now, mind you, some of the testing that we'll be doing and some of the, um, the weeds we'll be getting into, uh, could be viewed as somewhat contrived.

Um, but, uh, it is at the end of the day, adversary emulation. Um, we want to be emulating the adversary and testing, uh, adversary, uh, techniques and, uh, procedures to the best of our ability, ideally before an actual adversary does it for us. Um, so just by their nature of some of these, uh, examples may be con contrived, but just work, work with us through that and understand that there's always gonna be a certain level of artificiality in any testing that we do.

So the scenario that I wanted to introduce was, um, this article, um, uh, I was reminded of this article as I was preparing for this, uh, in thinking about M-S-H-T-A, um, so like fin seven using that, uh, and also just with the advent of ransomware and all of the techniques associated with ransomware activity. Um, so there's quite a few, um, but one that really came to mind for me in regards to ransomware was, uh, volume shadow copy deletion.

So, um, just at a semi basic level, um, what this is, is it's a built-in Windows feature that can kick in automatically to auto back up your files. So it's, it's sort of like a snapshotting mechanism for windows, um, that, uh, works automatically.

And one of the things that the bad guys do, um, so that, um, you know, because obviously they're, they're all jerks and they want to make recovery as difficult or, uh, as difficult as possible or impossible, is do everything in their power to make it impossible or improbable to recover from backup. So one way that could have been done that recovery is through volume shadow copies. And so there's, um, series of commands that they can use to delete any existing, uh, backup copies.

And so that's gonna be one of the atomic red tests that we're going to emulate in this safe environment in, in this, uh, in this vm. Um, so that on the other side of that, we could then, uh, try to gauge, uh, what events might be available to us and what data could be available to us to form the foundation for potential detection logic. Now, can we detect all volume shadow copy deletion in the moments? Um, I'm not confident enough to say yes or no in that.

Um, but what I consider atomic red team to be really good at is taking that initial step towards answering that question. And so having a framework to allow you to execute something in a safe and repeatable fashion would at least at a minimum allow me to answer the question, do I see any events or data be surfaced as a result of this very specific controlled test?

And if the answer is no, then I need to address that first before I can even, uh, think about going into detecting, uh, a technique as broad as say, volume shadow copy deletion or M-S-H-T-A.

Um, so with that quick, Quick, maybe a quick comment, quick question, based on what Ryan had said about, you know, atomic red versus Caldera, I know at the top, so you talked through HTA, which you know, we're gonna get into in terms of a specific technique for how phishing email works for that initial access bit. Yep. And then you talked about volume shadow copy. So kind of a very dangerous technique the adversary can use in a ransomware attack, right?

To really prevent you, the host organization from recovering your data on your own right forces, you kind of into their, where, where they want you to be to pay that ransom. Yep. But at the top of the screen that you've got shown there, you've got kind of this whole kill chain. So this concept of a phishing email runs some code specifically in PowerShell that PowerShell pulls down emote, kind of that malware or Trojan. And then from there you begin to execute some other activities.

And since Ryan had asked about Caldera versus Atomic red, if people on the call aren't really familiar with either one, the number one that that's, that's a very, like, makes sense, adversary kill chain, they have to do initial access, they have to gain kind of persistent access to the target. And, you know, by dropping their tool set and so forth, then they're gonna laterally move through your enterprise. They're looking for your servers, your valuable data, right?

What, again, you've gotta understand who that adversary is and what they're trying to do. In this case, they're looking for that valuable data so they can then do data exfiltration at the end. So you could think of those as trade secrets, financial information, whatever you may have that, that adversary wants.

And so you gotta get in the head of that adversary, understand why they're aggressing you what they want, because all of those things in their mindset are gonna totally change the techniques like Matt's talking about in your enterprise. Some are loud, some are quiet, um, some have massive effects against you, some don't. If they just want your data and they wanna run, you may never know they were there, right?

So, so really understanding who's coming after you so you can tailor these tests and just, like Matt said, tailor the data you're collecting, tailor your detections to really be focused on kind of the threats against your enterprise or your clients, I think is huge. And then I guess I'd ask a question, uh, Matt to you and, and let you answer it.

So Caldera anatomic radar difference in, in this right where you, here you see this entire kind of adversary kill chain, and you could think of it as one total kill chain, or you could think of it as a series of individual techniques you could test, right? Like, can I detect phishing? Can I detect PowerShell? Can I detect lateral movement? So in your kind of perspective, how do Caldera and Atomic Red differ? I know, I know my perspective, but I've teed up for you first.

Yeah, I, I would love to get more of your perspective on the Cloudera side. Um, I, I have some like rough background information of it. Um, uh, but yeah, you, you would definitely do it much more justice. But the way that I think about Atomic Red team is now this is not gonna be the prettiest, um, but this is like a raw set of atomic tests. Uh, it's in YAML format, so it's a text-based format, um, you know, it's semi quote readable.

Um, it's meant to be consumed by a machine, like through some automated process. Um, but you can kind of get the, the gist here of, uh, maybe what's going on. So like the, the key thing to focus on here, I, I think for starters is, um, what is going to be executed. And so that's described in this executor section. Um, and there's this command. So literally upon running this test, now, what is this test that's defined by this name here with its associated Mitre attack technique id?

Um, all of these atomic tests will always have an associated, um, uh, technique id, right? So what is this executing? Well, it's executing this, this particular test tests, uh, the M-S-H-T-A, um, attack technique, which corresponds to this, uh, technique id. Um, and in this particular instance, the means by which M-S-H-T-A script code is executed is via the command line via the built-in Mshda XE executable.

Um, but then even more specifically there, there's a, a host of different ways attackers can execute their arbitrary, uh, malicious code. And one of those is by embedding script code into the command line itself.

Um, and then even more specifically, uh, when you're embedding attacker code in line in the command line, you can, uh, use this one like scripting trick to directly download a follow on payload and then execute that directly, um, as a means of trying to, uh, evade certain types of detection where in this case, the actual, um, attacker script code would not be available clearly in the command line.

Um, so other investigation or data sources would be necessary to actually see what script code executed for the purposes of, uh, of an investigation. So to go back to your question for us about like comparing and contrasting Atomic Red Team, um, atomic Red team is, uh, as its name implies, atomic by nature, it tests a single thing and it tests a single thing really well. Um, now this YAML document doesn't do anything by itself.

It still requires a, a test framework to read this YAML format, read in that content, and actually do something useful with it. And like in this case, actually execute this thing, um, and then supply some level of meaningful output, um, so that the user can interpret like, did this test actually pass or fail? And what does a passing test versus a failing test actually mean to me as the user?

Um, so again, going back to the notion of it being atomic testing that one thing and that one thing really well, so how would you contrast that for us against Calera? Yeah, no, that's outstanding. So I think, again, when we started out in this journey, although you're right, in many cases, like the attack framework became this common foundation. So we can all talk about these things in a, in a set way. We all know what we mean.

I think others like Red Canary probably jumped out in front in, in terms of, okay, how do you automate testing against this? Like with atomic red, right? So Atomic Red totally came before Caldera. And, uh, so I guess just for kind of the audience, so Caldera, like Matt just said, so you've got this machine readable format, it tell, this is the exact attack technique we wanna test, this is exactly how we want to test it.

And then you've got the, the exact no kidding string that you're gonna run to, and then you're gonna check yourself, right? You run it, you know, that's the exact thing you did, and now you're looking to see if you collected that log data or if you detected that thing.

Um, caldera can be used the same way, but, um, you know, if you think about that kill chain we talked about first Caldera, um, it does try to do a lot of things, but it, it's not just the way to describe the test, but it's also the tool and it's, I think you made that differentiation with Atomic Red, right? Like, so you've got the test and now you've gotta figure out the tool to kind of run it Caldera, you have, um, kind of a server side front end, uh, that with a user interface, right?

So for me, the user, and it's, it's almost like a piece of malware a little bit, but, but for you, the, the good guy to control, so it lets you drop agents out in your environment, um, just like this adversary would, right? You, you kind of give yourself initial access to yourself, and then from the the server side, you can control these different implants and give them instructions on what to do.

So it's kind of a server side and a client side tool to actually conduct some adversary emulation yourself. And you can do it manually, you know, manually moving from machine to machine, manually running, um, activities on those machines. But Caldera does, it's also linked to the attack framework. And so you can pre-build out these kill chains, much like Matt and I talked about already. You know, here's what I've got, step one, step two, step three is step four, step five. Here.

You can let it automate that all the way through that entire kill chain. You can have breaks where it pauses and you can give it new instructions, or you can go completely manual, like I said, where the human just kind of like live is driving through, um, an environment. But Caldera was on our end just, you know, truth and linear on the Mitre side started as a research effort internally exploring how we could use AI to drive those implants.

And as they learned more things from the different machines, would that artificial right? AI adversary make different decisions? So it it, it was born out of adversary emulation meets AI as a research project, but when we started publishing these adversary emulation plans, um, caldera can import plans like, uh, that are in ya, uh, in yamo format. And so we built out our own kind of format that was more tailored to what Caldera was ready to ingest. And we started with the atomic red format.

And then we had, I think some small modifications in there. But the, the same premise comes through, right? You could test a single thing or in the more caldera centric way of the world, you would almost imagine this entire kill chain from beginning to end. And Cal can automate all the way through that whole thing.

So sometimes you may want to do that, but I think, like you already said, Matt, like if you're really just trying to really zero in on specific log collection, specific detection, you may only be interested in, in, like you said, testing one thing very, very well, um, which is a cald probably could do, but wasn't as tailored to do as what atomic red is. Yeah.

So what, what I'm hearing is like ultimately at the end of the day, there's a million different use cases for how you'd want to think about and ultimately apply adversary emulation in your environment.

Um, like from my daily perspective as I'm researching, say like whatever the, the, the latest and greatest attack technique is that like we're trying to gauge what our detection coverage for that may be, um, writing a simple atomic red team test, um, really just enables us to sort of get that like ground truth of can we even see this thing executing in the first place? Um, 'cause obviously like we don't want to just rely on like customer threat data alone.

Like, I mean, obviously it's unfortunate if, if a customer's gonna get popped with something that, um, that we may not even have, um, proper detection coverage for like that day. Um, so really like this can help give us that ground truth of having something that, um, we control and is repeatable, um, and just allows us to just get better insight into like what is actually going on and what might pop out on the other end that we might be able to, to detect.

But at the end of the day, atomic Red team, like this YAML document that you're seeing here, atomic Red Team is really just a schema that describes, um, a thing and that thing that it's describing is a specific, uh, procedure, right?

So we, um, you, you probably think in terms of like tactics, techniques and procedures, um, the technique that's being tested is this Mitre attack technique, M-S-H-T-A, but think of all of these individual tests as procedures, implementations of those particular techniques.

And, um, I think as like, um, as a detection engineer, I think a lot about coverage and, um, I mean one way that you could arbitrarily measure coverage is like the more atomic tests that you have to exercise unique functionality for a given attack technique, then you've arbitrarily increased your detection coverage hypothetically. Um, there's a lot more discussion around that that, that we could have.

But in the interest of time, um, that's just kind of how I roughly think about like how a topic red team can be used in the interest of increasing, uh, potential, uh, detection coverage and how we use it internally to, um, to facilitate repeatable testing in a safe environment. Um, so what I'd like to do now is just dive right into this. Um, and so I meant Jump into Matt, is msh TA do you want to give everyone like a quick summary of what that specific technique is?

I know we're talking about phishing email, but what's s ht Yeah, certainly. So, uh, M-S-H-T-A, I don't even know if there's like built in help for this thing. There's not, um, yeah, good question. So I happen to have the Mitre documentation up for this Outstanding, There we go.

Um, so if you're not a subject matter expert, which God forbid, like no one should be a subject matter expert in M-S-H-T-A, um, but, uh, if, if you're investigating something or like are responsible for developing coverage for this, um, first place you should always go is the, the Mitre attack site to get some context about what this thing is in the first place. So MIT's great at giving you some very high level information about what this technique is.

It's a built-in utility in Windows that executes these Microsoft H TM L application files. And what you need to know about these, if you're trying to detect it, like from the attacker's perspective, like why do they care about this? Well, one, it's a built-in utility, it's signed by Microsoft, and two, it will execute arbitrary code that they supply the code that they supply is go, uh, that M-S-H-T-A supports is JavaScript or JS script, uh, and, and VV script.

Um, so these are powerful, um, albeit antiquated, uh, scripting engines built in to Windows. Um, they're extremely flexible with what they would allow an attacker to do. And again, this utilities built in. Um, and there's a whole host of variations, like different ways in which M-S-H-T-A can be executed. And so if I'm thinking about like, do we have coverage for M-S-H-T-A, uh, MITRE attack is the first place we'll go, but that is not the end all be all for assessing coverage.

Um, because I know like having investigated all this, like there are so many variations in which M-S-H-T-A code can be executed that Mitre attack doesn't cover. Um, but that's not the point. Um, but, um, feel free to correct me if I'm wrong, but like that, that's not the point to establish, like to use Mitre attack, to check the box to say, yes, I have a hundred percent coverage.

No, it's just the framework, um, and a common point of identification for a technique that, uh, it's, it's a language that we as an industry can, can speak collectively, but again, it's not an end all be all. Um, so I'll use this to learn about in this case, like what the heck is NM SHT in the first place? Then I might go to Atomic Red team, consult it and say, uh, and ask, is there any coverage in Atomic Red team?

Um, so that me as the nons subject matter expert, I can just go emulate a few things and see what comes out on the other end. Um, so what I'd like to show you here is, um, we, we've talked a little bit about like the distinction between the Atomic Red Team YAML document, and then there still needs to be something to execute it, to actually emulate and consume that, uh, that document.

And in the case of Atomic Red Team, the supported means of doing that is through PowerShell using the, um, the Invoke atomic, uh, PowerShell module. Um, so with that module, you get some, uh, built in functionality to allow you to work with Atomic Red team tests. And so the first thing that I'd like to do is copy this command, go and paste it into here. And I'm just using this to, um, to enumerate.

Um, I've already downloaded the Atomic Red Team framework to, to my desktop, and I'm just asking what attack techniques are implemented already out of the box and Atomic Red Team. Um, by default you'll see many, many, many more attack techniques. Um, I've taken all of those extraneous ones out, um, just so we can, you know, have a, have a laser focus on, on the two ones that we're interested in, but otherwise you, you'd see a lot more.

Um, so this, I just wanna point out like, here is the built-in test coverage for these two relevant techniques. Now, next, um, so let's just focus on M-S-H-T-A. I mentioned previously the Mitre attack technique ID for that is T 1 2 1 8 2 0 5. Um, so the way in which I would ask invo atomic that PowerShell module, um, show me all of these specific tests you have for a given, um, attack technique would be to run this command here.

And so what this is doing is it's going through that YAML document in an automated fashion, and it's just giving me a list of all of the test names that are available. And so one option that I would have available to me is I could say, Hey, invo atomic tests, just run all of them and let me know what pops out on the other end, whether it's a detection, whether it's my, uh, endpoint AV firing on one of those, or if it's, um, you know, what have you.

Um, MSP like any, any service or like EDR uh, platform detecting this sort of thing. Um, but I'd like to retain our laser focus and one of the ones that I saw in the wild recently was, was this variation where the JavaScript or JS script command was embedded, um, in the command line. And so I, I'd like to actually emulate that in, in a safe environment.

And, And Matt, before you jump off that, I think it'd be interesting, like you, you already, um, well actually back on your PowerShell window just for everybody, all this thing. So I know we said this isn't the end all be all, and you're not just trying to just check a box to say, Hey, you know, blah, blah, blah, I'm, I'm good to go on Mitre attack, uh, either across the board or one specific thing. I think what Matt just showed you is that's absolutely true, right?

Even for this, if you really want to oversimplify what the attack framework did to us, is it, it lets you get away from saying, Hey, I, I, I think some adversary is doing a thing and maybe they're doing a thing using email to get us to be able to talk about it in a very specific way. Like they're, they're very precisely doing this thing, but even for 12 18 0 0 5, you saw Matt showed you there were several tests, were there seven variants, Matt, nine variants, right? Yeah.

So just in atomic red, nine different variants, and they all mean different things, right? Like you said, M-S-H-T-A could use VB script, M mss, HT A could use JavaScript and those may call different engines under the hood. Um, you may directly, you know, you may get that on a phishing email and it may need to download the script to run. That's one variant. Or it may simulate a double click.

So, you know, if you're really monitoring your endpoints, well it may look like a user did click something. So there's all these different ways to do it. And under the hood of an endpoint, that's gonna create different data, right? Different evidence. 'cause it's gonna run different programs really at the, at the lower levels of windows. I'm talking about windows specifically. So it, this is definitely not meant to be one and done.

Um, don't, um, don't oversimplify it to yourself that if you do it once and you detect it that you're good to go. But it, but I just want to give you all some of that more, if you will, a little deeper background that this is meant to really be an iterative process. And I know, Matt, you probably have teams that do this full time all year, right?

That this concept of detection engineering and really going through this step by step by step to have these very precise detections just because, you know, as soon as we do all nine of these, the adversaries will find variant 10, 11, and 12, right? So back to you. Guaranteed. Yeah, guaranteed. Um, yeah, I've, I have absolutely nothing to add to that, that, that, that was perfect. And then I, there's a quick question in the chat. Matt.

Can you run atomic red with user level permissions or do you need to have elevated permissions? Yes, Absolutely. Um, one, one of the cool things, um, that you could define if, if you are, uh, writing these atomic tests is the YAML schema lets you specify if a test requires admin or not. Um, you see that I, the only reason I I have an admin, uh, window up is for the volume shadow copy tests. Um, so that is specified in the atomic tests as requiring admin.

These M-S-H-T-A tests, um, do not require, um, elevation. So I, I I just have the admin window open for, uh, convenience sake for, for the M-S-H-T-A portion of the testing. Good question. Okay. Um, so next thing I'd like to do is, let's say I've identified test number eight as the test I would potentially like to execute.

So I'm o only thing I'm changing now is I'm just changing it to show details and I'm specifying specifying a specific test number here so I get a little bit more detail here instead of just those summaries. Um, and so what is actually going to be executed will be this. So anything in red here, like this is a template that will be filled out dynamically.

And if, if the user doesn't override any defaults, then it will tell you without even executing it exactly what will be executed when you do run this test. So it's gonna run this PowerShell code, uh, which is, this is a PowerShell wrapper that, um, abstracts a lot of the complexities of automation around, um, HTA payloads. Um, so this is what's going to execute. And assuming I'm okay with what's being executed, um, some atomic tests may have some prerequisites.

So like, if there are any dependencies that have to be met, you can just run check prereqs. And if I don't get any errors, then I have met my prerequisites. And this details, uh, the details view will show you like what code runs to satisfy those dependencies. Uh, but because I didn't get any red, it looks like I'm clear to go to execute my test. So I'm just gonna go ahead and run this thing.

Actually, I want to go into, I, I had, uh, this custom event view ready to go, um, just ignore that first event. We will ignore that 'cause it's not related to this test. So I'm gonna go ahead and run that. And I love it when demos work when we're doing 'em live. Um, so this is the output that I was presented with. Some atomic tests will have better output than others. Um, this is one of those examples of where you get very rich output from the test.

So for example, you get the technique id, you get an indication that the attack technique actually did execute successfully. Um, you get some context about what actually executed in this case, like mss HTA XE executed from this path, which is the, the built-in path with this process identifier. It gives you the full command line of what executed, um, and also some context about what the HTA payload, uh, executed.

Um, this, this will execute a template payload that's used to validate that it successfully executed the HTA code. Um, and so you get some context about that. So what's cool is I can take the output of this test and the context that I get, and then I can ask of whatever my relevant data is on the other end, are you seeing the same thing that I'm seeing?

And so going to our data here, um, on my end point here, um, I've got CISSON running and I, for the sake of this demo, I only have it, uh, capturing events related to M-S-H-T-A and volume shadow copy, uh, deletion, just so that, um, this, this isn't like overly noisy. Um, so I've got that configured and running. Um, and here is the first event that's created. So a process creation event.

Event ID one, um, if you're not familiar with cisson, um, think of this like, um, the native, uh, process logging that you can get via, um, force. Maybe you could chime in. What, what is the, uh, I like, Uh, just Windows event SEC event, those kinds of things. The default windows. Yeah. What is it? 46 88 event? Yeah, I think so. Okay. Yeah, so think of it as similar to the, the native logging that you get with some additional fields.

Um, so let's go into the details view here of what, what we get. And so what Matt's about to show you, everyone right, is, so, I mean, windows is already logging itself. Um, and, and you could use that to detect this also, it would use a different event id. So if you were tailoring a detection, you'd, you'd want to, you know, be precise there. But what Matt's about to show you the value of sysmon and why almost everyone recommends that you do deploy it, even though it collects a lot of data.

So you'd wanna, like, he just even explained, you'd want really tailor what it's collecting, but it gives you a lot of additional details that the baseline windows of event locks don't. Yep. So go, go ahead Matt. Yeah. Yeah. So, um, this should look familiar based on the output of the test. So here's the, the image path of M-S-H-T-A, um, the process ID lines up. We get the full context of the command line, which we were expecting, um, and we get all kinds of extra information as well.

Um, this is really cool information. Um, don't really have time to get into like why it's so useful. But, um, this could be used to detect, um, evasion attempts. For example, if an attacker were to rename MSS HT to say like, notepad do e xe, um, this context would give you an opportunity to detect Notepad XE as actually MS. ht, uh, running. So that's, that's some pretty cool, uh, context that you get there.

Um, in the interest of time, well, uh, I, I also wanted to show you just, uh, another, um, another, uh, source of events that Cisman could supply that could be of value is, uh, image loads or like anytime A DLL loads into a process, we get some context about that as well. So like, this is indicating that JavaScript executed in the context of msta xe. So, um, it's totally going to depend on the technique, um, as to like how, uh, how valuable or relevant that, um, that, that data is going to be.

Um, I, I'm not claiming here that like this event may or may not be specifically valuable, um, but it's always good for me as a detection engineer to be aware of what's available to me so that I can, um, cast start by initially casting a wide net and then narrow that down as as necessary. Um, the last thing I'd like to do, just, um, because we're starting to run a little shorter on time, is just directly run the, um, the volume shadow copy deletion test.

So the syntax is pretty much identical to what we ran for M-S-H-T-A. The only thing I'm switching up is the, uh, MITRE attack technique id. So this corresponds to inhibit system recovery, which is related to, um, volume shadow copy deletion. And I've got that pulled up here. So you get, you get some of that context there. Great documentation of course from Mitre. There's a lot of hours spent on that. It's, it's always good to hear, appreciate it.

'cause I know the team that works at man, they sometimes thankless hours, uh, trying to keep all the research up to date. No joke. It, it is a lot, a lot of work and it's very much appreciated. Um, I've already identified the relevant tests that I want to run, um, which is going to go through and just delete all of my volume shadow copy backups. And here again, we'll just, we'll just see what, uh, what output we get and what pops out on the other end.

Alright, so, um, this test, we don't get quite as much, um, detail contextual information as we did in, in the previous one. That's okay. Um, we get a little bit of stuff here, like it, it looks like shadow copy deletion was successful. Um, so what that might look like on, on the other end of things. Um, uh, this is, this is where the whole investigation process begins. Uh, I may not have all the answers as to like, do we detect volume shadow copy deletion or not?

Um, but this, this is the, this is the series of steps that I would take initially to ask what kind of coverage might we have in the first place. And so I see some relevant events here. WMIC executed what actually ran, okay, WMIC shadow copy delete, huh? That looks like it's probably relevant. So I've got, I've got a starting point here. What other data do I have? I've got this wimy WMI activity event. Hey, let's, so we had WMIC running, uh, another thing related to W-M-I-V-S-S.

So this is related to volume, shadow, copy service. Um, so, you know, we're starting to build out, we're, we're starting to collect, uh, crumbs in, in this potential, uh, uh, trail that could possibly lead us to some form of detection coverage with the WMI. This is the service executable loading, uh, the V-S-S-W-M-I-D-L-L. So this again seems related to volume, shadow, copy, operations. Uh, and then what's the last thing we have?

Okay, we have the volume shadow copy service executables starting what seems like it started in demand in response to the shadow copies being deleted. So that, that's interesting. Um, uh, again, so just, um, the, this is just the beginning of, of the process, um, but we have, in this case atomic Red team, um, a large suite of existing, uh, tests that we can just use, right, right outta the gate.

Um, but also a framework and a schema that we can use to describe new tests as new attack techniques come about. Um, and then we've also got the testing framework in, in the form of invo, atomic test, calera, what have you, these automation frameworks that know how to consume those tests and do something useful with them. Um, and then again, uh, how you interpret those tests upon being executed is really going to be up to you and what your specific use cases are, of which there are many.

So that is all I have for now. Um, and, uh, force any, any comments, questions, or do we have any?

Uh, Yeah, I was gonna say, well, that, that was outstanding and I think, you know, Ryan had a few comments here that for kind of everyone in the audience that probably already saw it in chat, but you know, Matt picked some TTPs that are, you know, you see 'em out in current news, this is what adversaries are really using, it's what certain malware, especially ransomware real uses, and Ryan threw out another couple links, articles where other malware variants like the Rio variant automates this same step of volume shadow copy.

So, so again, you kinda like tie a ribbon on this whole thing. Uh, I don't know to what degree kind of all MSPs, like if you spend time on cyber threat intelligence, right? If you have full-time, staff are not looking at it, but you know, luckily there are others. You've got, you know, here, red Canary, you havemeyer, you have others out there that publish their results, they publish their findings, so you're not having to start from scratch.

But even if you're reading articles about what adversaries you're doing, right? I mean that's, it's hours and hours of trying to understand all of that and then, and then figure out what to do with it. And that's where, so if you put yeah, the attack framework together as kind of the, the dictionary and what's this group do? What techniques do they use, what does that even mean, right? What is volume shadow copy and how could you use it in a, in a malicious way?

You can read about all that and and understand it, but then you wanna start figuring out can you do anything about it? And that's where maybe Matt has potentially under undersold the value a little bit he showed you, but of Atomic Red. If you just read Mitre attack about volume shadow copy deletion, and you're like, oh, that's a really big problem. Now what? Well, you would be kind of on your own to write quite a few lines of code and do some research to try to do what Matt just showed you.

But the fact that just out of the box, you can go through atomic red, you can search it like, oh, I, I know it's this, it's this attack id, oh yeah, there's nine different ways I could test myself against that. And you could then, right. Just use that in whatever way it makes sense whether you, uh, wanna set up like a dummy box in your environment and start testing that way. Um, if you want to try like setting up a new detection a week or a new detection a month, right?

All this is completely tailorable based on what you wanna do, but I think my bumper sticker to you is, you know, coming outta doing cybersecurity first and more like the traditional vulnerability assessment and risk management environment, um, kind of 10 years ago, just patching your systems isn't enough, right? 'cause even fully patched systems, what Matt just showed you, these are real things that would affect a fully patched Windows machine, right? Patches have nothing to do with it.

Um, the adversaries are abusing real capabilities. Microsoft put there for a reason. So the only way to get ahead of that is to, to read the CTI to try to get ahead of your specific threats and then yeah, to, to put something in place where you test yourself before the bad guy gets to you. And that's, you know, atomic red I think is a huge example of why that's valuable. But, um, yeah, I mean, Ryan, I don't know if we, we covered kind of what you wanted to cover there.

We didn't really talk about Defend and Shield. I think that the 32nd blurb I'd give everyone on that, as we've talked a lot about the attack framework, which describes what bad guys do ultimately shield and defend are both trying to set up the same, um, common language, but for defender activities, right? So the fact that, you know, attack id, uh, I should even have committed memory, what wasn't that like 14 something, um, for volume shadow copy deletion, right?

Um, the fact that you can say that and it means something and you can send people a link and they all know exactly the thing you're talking about didn't exist before for describing adversaries. Now it does. And so shield and Defend both are trying to describe that for defenders. And so, um, I think Defend more specifically is trying to exactly match to the attack framework. So if a bad guy does x, what's the very specific thing a defender can do to detect or prevent it, right?

And then SHIELD in general is this way of talking about how could you kind of harden and shape your environment in a way that just makes it harder for the adversaries and kind of guides them, uh, where you want them to go so that you can better detect them or better stop them, right? Yeah. Um, but yeah, outstanding.

And Matt, tha thanks for, uh, the chance really for everybody to see what you showed there, and I don't, do you have any recommendations before we kick it back to Ryan on, is, is it it pretty easy to take Atomic red and spin that up like in an enterprise? Is that even a thing that you do or do you just kinda start on one machine and and build from there? Yeah, that's generally our, our recommendation is to start small on, um, just some, some test systems.

Um, one, one of the challenges with, um, with Atomic Red team and the suite of tests that it comes with based on the awesome community contributions that we're constantly getting is that that stuff's gonna flag av even like those YAML documents that I showed you, like if you just download the, the, uh, the default suite of atomic tests, uh, it it's gonna flag AV almost guaranteed.

And so that's one of the challenges of working with Atomic Red team initially is, uh, you often have to like subvert security controls or create the appropriate exclusions to allow this stuff to run. And so that's one of the problems.

Um, or not even a problem, just one of the challenges associated with scaling this out, um, is, is because we're doing, uh, adversary em emulation, like, uh, security tools are working as they should, but the challenge is that, uh, one security tool might get in the way of the other security stack that we're trying to test.

Whether it's like an EDR platform or anything else, like further upstream like just sim collection if say, uh, defender AV is, is flagging and just preventing it from executing in the first place, then our test isn't running. And so we'd have to, uh, you know, do root cause analysis on why our tests aren't running. And so, um, uh, technically, like you can scale it out, the Invoke Atomic Power PowerShell module is designed to, um, to execute tests remotely as well.

So you can establish multiple PowerShell sessions and, um, execute one test across many systems, but you're still gonna have to, um, uh, clear those hurdles of, um, subverting or creating exclusions necessary for the security solutions that you're not directly testing. No. Awesome. No good advice. I don't think I have anything else. Ryan. Andrew, man, thanks for the time. What, what do you guys have for us? A big thank you man. That was awesome for us. Thank you. I know it's late there.

You got your family with you. Thank you for, uh, for doing this on your vacation. So big thank you to you and, uh, and Mitre support. It's really, really cool. Matt, same. Thanks so much for, uh, taking time out and donating your time and red canary's time, uh, to helping these awesome MSPs. And uh, Ryan, how about any closing comments from you first? Yeah, I mean, I just hope the power of adversary emulation and the atomic tests really landed.

Um, like once you understand your threats, you know how they operate, you can start to emulate the things they do in your environment, and that's when you start to have the threat informed defense, right? So there's, we've walked you through the two things, right? Building your threat profile, understanding what they do, emulating that Wes and I in the next hour are gonna start walking you through how to really apply these threat profiles to an MSP environment.

And that's really where the threat modeling really starts to come in because now we understand our adversary and we can start to take action on it. That's fantastic. Wes, any closing comments? Let's see. There we go. Hopefully, um, that gave you guys some interest in wanting to get into this right? And really test control effectiveness, really understand where gaps may be and really go on on this adventure.

Um, so I, I would just say, you know, forest and Matt, both of you, thank you for the time. I don't think we really do any of this in the channel and uh, there's a lot of reasons for it, but hopefully this was a great introduction. I dare say, as far as I'm aware, this is probably the first time we've ever done any, anyone in the channel has really done adversary emulation showing how it can, you know, the tools are involved and how you go about this.

So yeah, I hope you guys all go down this journey, those that are watching, and, uh, thank you guys. Yeah, fantastic. Again. Um, so next session coming up with Ryan and Wes, I'll end this session. Stay tuned. I'll pull you back in just like the last time. Uh, with that then I'll pull Ryan and Wes back in and see if Gary's there with us as well. Until then, Matt and Forrest, thanks so much. We look forward to having you both back again and, uh, we'll see everybody else momentarily. Thank you.