Endpoint Security Testing – Part I

To NGAV, or EDR, that is the question!

Six months ago I was told that we were looking at replacing our current AV solution. As a security person, in the past 15 years most things have crossed my path, but I had never really looked that deeply into AV, so this was an exciting project to start 2017.

As I started to work on this initiative and begin my initial research, I noticed management and peers interchanging Next-Generation AV and Endpoint Detection and Response (EDR) acronyms like they were the same thing.

Yeh Yeh we all know signatures are dead, let’s bury them now. However, with everybody confusing the definitions in the office, security conferences and webinars overloading us with machine learning terminology, math models, artificial intelligence and other descriptive nouns, I swear I just wanted to grab a double espresso.

Now for anyone that knows me, I’m a real perfectionist and sometimes I think its a weakness as opposed to an asset as I get consumed by the work. I really wanted to understand details, key differences and all the hype. Whilst I think most professionals understand it at a high-level, I noticed many referred to these technologies as if they were the same thing when there were clear lines of distinction.

Over the past 3-4 months I have spent a significant amount of time testing 5 solutions in the market. I hope that these series of blog posts will help others in the industry by sharing what I learnt and the techniques I used in my testing, in particular:

  • what I did (personal research, business requirements, including functional and non-functional)
  • the test use cases (this is a big one!!)
  • results of what I found (some vendors have strong EULAs, so I will be respectful).
  • If AV replacement is on your initiatives list for this year, it’s important you can answer one question, Why?

Why is NOT because you manager thinks its cool idea, or because they went to some conference, had a drink with a vendor (not that anyone would do that!!), heard SaaS is the future or has seen that cloud pizza diagram (you know the one), or that artificial intelligence sounds like fun. No, that’s not how it “should” work.

Why = What’s the problem are we trying to solve? (to quote the great Michael Santarcangelo). As Information Security professionals, we must not forget we are here to help the business operate in a secure manner and help reduce risk. When you have that answer and understand how it will help increase business revenue, shareholders, customers and all that good stuff, then you are ready. Boring Yes, but that’s security reality I’m afraid, otherwise we work in vendor-land!

Once I had the problem statement documented (which I will go into the next blog), its important you do your use case preparation. Like painting a room in your house, you just don’t start applying paint and hope it looks good. This is the same, you just don’t sign NDAs with a bunch of vendors and start throwing Malware at their solutions. What are your measurements for success? How do you know you have met your business objectives? Yeh its fun and you get a bit of a buzz when you detonate your first piece of ransomware, but you end up with half of the paint on your furniture and carpet, no order, no statistics and one big mess and way too many system rebuilds!

In between my BAU work, it took approximately 4-6 weeks to write up business and security requirements, including functional and non-functional requirements. Additionally I wrote over 100 test use cases, comprising of different users with different privileges, cloud vs no cloud reputation, file-based and fileless malware, malware execution using windows native programs such as powershell, cmd.exe, batch scripts, mail and web-based attacks, etc. Yes the doco is boring, but once you start testing, this ground work will help you provide the evidence and more importantly the facts for your business case and problem statement to really shine. I’ll be uploading a sample of my test use cases in blogs to come in the hope to help others.

Other than your own preparation documentation, make sure you talk to a few vendors, peers and people you respect. Get Vendors to demo a few solutions! There are so many in the market right now, Carbon Black, CrowdStrike, Cylance, Symantec, McAfee/Intel, Sentinel One, Cybereason, Invincea/Sophos, this list is growing and fast becoming a saturated market.

So whats really the difference between Next Generation AV and EDR (Endpoint Detection and Response)? Here is the simple and short answer, NGAV = Prevention and EDR = Response.

If we use Crowdstrike as an example, they started off as a pure detection and response solution. CS were all about threat hunting, investigation and detection. They weren’t really a player in the AV space, but their solution has since evolved to include classic AV capabilities such as their prevention policies for malware execution in addition to its other existing capabilities.

Next Generation AV aims to:

  • Stop and prevent on bad behaviour
  • Most should no longer be using static hash values for detection (this was a fun use case)
  • Many of these NGAV solutions still by definition will perform a classic AV on-demand or scheduled scanning, additionally an on-write file scan (pre-execution category)
  • Think one word – PREVENTION
  • Most NGAV tools want to ensure the end-user is impacted as little as possible. Less operational, less engagement so they use can get on with their daily business

EDR aims to:

  • Detect bad behaviour and respond when something is suspicious
  • Many will show the process tree (map) enabling the security investigator to see what happened, what executed and the flow of events
  • Enables hunting and triage of events on endpoints as opposed to classic network traffic or SIEM logs
  • EDR wants to go beyond just malware detection and reponse, it aims to capture all the endpoint data to help visiualise lateral movement, damage caused and other IoC/IoA’s
  • More than an IR tool, think extra bodies and staff training required to understand what they are seeing otherwise ensure the EDR vendor can offer an managed services function
  • Traditionally EDR capabilities did not extend to kill and prevent, this is a changing landscape

Welcome Endpoint Protection Solutions:

  • Many EDR solutions are expanding their capabilities to include prevention technology. Crowdstrike and Sentinel One, for example, were all born as EDR solutions. Their new policies have a prevention option that can run along side their EDR capabilities.
  • Additionally many of these EDR solutions are now including pre-execution modules to scan for dormant files or on-write of a file. In order to compete with traditional AV or NGAV solutions, companies still require pre-execution functionality. Think of those file shares lying round with years and years worth of data. No illegal or malicious software lives on file shares right!?!?!?
  • We are starting to see this hybrid of NGAV and EDR merging to provide a more comprehensive solution

**There are many other features these solutions provide which I will discuss these in later blog posts. The above is not an exhaustive list **

We are now seeing a convergence of NGAV and EDR capabilities, such as CrowdStrike, Carbon Black and Sentinel One as these were traditionally EDR solutions. With many EDR solutions incorporating prevention technology, it’s no longer about detect and response, but an overall Endpoint Protection strategy. To NGAV or to EDR?!?!?! This is not the question anymore. With so many other capabilities on offer, application whitelisting, blacklisting, process baselining, script control, etc, it’s more important than ever we understand our business needs.

In preparation for endpoint testing, these are my takeaways:

  1. Important you start by understanding your business problem, what problem are you trying to solve
  2. By understanding this, the requirements and capabilities become clear
  3. Make a plan, its more fun, easier to execute and follow and communicate the findings

Part II – Business Problem and Preparing for the PoC

One Comment

Add a Comment

Your email address will not be published. Required fields are marked *