Nss Labs Aep Test Report Check Point Software PDF
Nss Labs Aep Test Report Check Point Software PDF
This report is Confidential and is expressly limited to NSS Labs’ licensed users.
Advanced Endpoint Protection Test Report – Check Point Software Technologies Check Point SandBlast
NSS Labs Agent Next Generation AV E80.82.1_030519
Overview
NSS Labs performed an independent test of the Check Point Software Technologies Check Point SandBlast Agent
Next Generation AV E80.82.1. The product was subjected to thorough testing at the NSS facility in Austin, Texas,
based on the Advanced Endpoint Protection (AEP) Test Methodology v3.0, which is available at www.nsslabs.com.
This test was conducted free of charge and NSS did not receive any compensation in return for Check Point’s
inclusion.
This report provides detailed information about this product and its security effectiveness. Additional comparative
information is available at www.nsslabs.com.
As part of the initial AEP group test setup, 96 instances of the endpoint product were deployed on Windows 7 and
Windows 10 operating systems. All product configurations were reviewed, validated, and approved by NSS prior to
the test. Figure 1 presents the overall results of the test.
Block Rate is defined as the percentage of exploits and malware blocked within 15 minutes of attempted
execution. The Additional Detection Rate is defined as the percentage of exploits and malware detected but not
blocked within 15 minutes of attempted execution.
An AEP product with a low block rate will incur less security savings, since additional operational overhead will be
required to remediate the effects of a compromised system and protect the business. For detailed total cost of
ownership (TCO) analysis, please see the TCO Comparative Report at www.nsslabs.com.
This report is Confidential and is expressly limited to NSS Labs’ licensed users. 2
Advanced Endpoint Protection Test Report – Check Point Software Technologies Check Point SandBlast
NSS Labs Agent Next Generation AV E80.82.1_030519
Table of Contents
Overview ............................................................................................................................................................... 2
Test Methodology................................................................................................................................................ 14
Table of Figures
Figure 1 – Overall Test Results.......................................................................................................................................2
Figure 2 – False Positive Rate ........................................................................................................................................5
Figure 3 – Malware (Various Delivery Methods) ...........................................................................................................6
Figure 4 – Exploits ..........................................................................................................................................................7
Figure 5 – Blended Threats ............................................................................................................................................8
Figure 6 – Resistance to Evasions ..................................................................................................................................9
Figure 7 – Threat Event Reporting Capabilities ...........................................................................................................11
Figure 8 – Three-Year Cost (US$) .................................................................................................................................12
Figure 9 – Scorecard ....................................................................................................................................................13
This report is Confidential and is expressly limited to NSS Labs’ licensed users. 3
Advanced Endpoint Protection Test Report – Check Point Software Technologies Check Point SandBlast
NSS Labs Agent Next Generation AV E80.82.1_030519
Security Effectiveness
The aim of this section is to verify that the AEP product is capable of detecting, preventing, and continuously
logging threats accurately, while remaining resistant to false positives. This section utilizes real threats and attack
methods that are being used by cybercriminals and other threat actors, based on attacks collected from NSS’
global threat intelligence network.
The ultimate goal of any attack on a computer system is to gain access to a target host and perform an
unauthorized action that results in the compromise or destruction of an asset or data. Computer systems are
designed with many levels of protection to prevent unauthorized access. However, intruders may use several
techniques to circumvent these protections, such as targeting vulnerable services, invoking privilege escalation, or
replacing key operating system files. AEP products protect against automated and manual threats by leveraging
the following key capabilities:
NSS has created a unique testing infrastructure—the NSS Labs Live Testing™ harness, which incorporates multiple
product combinations, or “stacks,” within the attack chain. Each stack consists of either an operating system alone
or an operating system with additional applications installed (e.g., a browser, Java, and Adobe Acrobat). This test
harness continuously captures suspicious URLs, exploits, and malicious files from threat data generated from NSS
and its customers, as well as data from open-source and commercial threat feeds. Captured malicious samples are
further validated to confirm that they are malicious in nature. During testing, NSS combines its knowledge of a
product’s defensive capabilities with these samples.
An AEP product must be able to detect, prevent, continuously monitor, and take action against threats while
providing end-to-end visibility through event logs generated by the endpoint product. Each type of threat (e.g.,
malware, exploits, blended threats, and evasions) contains unique infection vectors. This test aims to determine
how effectively the AEP product can protect against a threat, regardless of infection vector or method of
obfuscation. Within this report, the term “threat” is used to refer to malware, exploits, or blended threats that are
able to successfully access, download, and execute on a target system, with or without subsequent post-infection
compromise and/or outbound communication attempts.
One of the most common threats to the enterprise is the infection of enterprise systems by malicious software.
Products were tested against threats from the following categories:
• Malware
• Blended threats
• Offline threats
• Unknown threats
• Documents and scripts
• Evasions
• Exploits
• Any combination of the above in addition to follow-on threat actions or behaviors
This report is Confidential and is expressly limited to NSS Labs’ licensed users. 4
Advanced Endpoint Protection Test Report – Check Point Software Technologies Check Point SandBlast
NSS Labs Agent Next Generation AV E80.82.1_030519
Each type of threat is generally deployed via one of the following common infection vectors:
• HTTP: These are web-based attacks where the user is deceived into clicking on a malicious link (on, for
example, a web page or a banner advertisement) to download and execute malware, or where the user
merely needs to visit a web page hosting malicious code in order to be infected via exploits (also known as a
drive-by exploit).
• Email (IMAP4/POP3): These are inbound, email-based attacks where the user is deceived into clicking on a
malicious link within an email to download and execute malware, or where the user is asked to visit a web
page that hosts malicious code in order to be infected via exploit.
This report is Confidential and is expressly limited to NSS Labs’ licensed users. 5
Advanced Endpoint Protection Test Report – Check Point Software Technologies Check Point SandBlast
NSS Labs Agent Next Generation AV E80.82.1_030519
Malware
One of the most common ways in which systems are compromised is through the use of malware. Malware can
infect an endpoint using numerous attack vectors or delivery methods.
Figure 3 depicts test results for malware delivered via HTTP, email, documents and scripts, offline mechanism, and
previously unknown threats.
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
HTTP Malware Email Malware Docs & Scripts Offline Threats Unknown Threats
Additional Detection Rate 0.0% 0.0% 8.1% 0.0% 0.0%
Block Rate 100.0% 100.0% 90.3% 100.0% 81.8%
During testing for malware delivered via HTTP and malware delivered using email, NSS intentionally tested a large
number of the same samples in order to validate whether or not the introduction mechanism had any significant
impact on detection or block results. Test results demonstrate that there was no significant difference in detection
and block rates regardless of introduction mechanism.
This report is Confidential and is expressly limited to NSS Labs’ licensed users. 6
Advanced Endpoint Protection Test Report – Check Point Software Technologies Check Point SandBlast
NSS Labs Agent Next Generation AV E80.82.1_030519
Exploits
Figure 4 depicts the results of exploit testing for the Check Point SandBlast Agent Next Generation AV. Exploits are
defined as malicious software designed to take advantage of existing deficiencies, such as vulnerabilities or bugs, in
hardware or software systems.
Figure 4 does not include results for 138 drive-by exploit test cases. Following testing, it could not be determined
that these drive-by exploits executed consistently across all products in the test. However, additional analysis was
performed to assess a product’s ability to detect or block drive-by exploits.
It was observed that the Check Point SandBlast Agent Next Generation AV can block HTTP drive-by exploit attacks.
Specifically, testing revealed that the product can prevent the attacks using URL reputation and behavior
monitoring. The product’s Anti-Bot URL Reputation module prevented users from visiting blacklisted websites by
displaying a splash page in the browser. In other cases, the product’s Anti-Malware and Anti-Exploitation module
prevented the exploitation through either a known malicious signature or a behavior of the attack. In the event
that a domain was identified as malicious, the splash page displayed information pertaining to the threat
detection. Other technologies may exist to block exploits, but they were not observed. The product’s management
console identified the malicious exploit/URL.
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
Exploits
Additional Detection Rate 5.9%
Block Rate 88.2%
Figure 4 – Exploits
This report is Confidential and is expressly limited to NSS Labs’ licensed users. 7
Advanced Endpoint Protection Test Report – Check Point Software Technologies Check Point SandBlast
NSS Labs Agent Next Generation AV E80.82.1_030519
Blended Threats
Figure 5 depicts the results of blended threats testing for the Check Point SandBlast Agent Next Generation AV.
Blended threats possess the characteristics of socially engineered malware as well as the features of legitimate
applications. In these tests, all blended threats were delivered via deceptive emails. Blended threats attempt to
make it difficult to distinguish between malicious and legitimate activity. Enterprises expect AEP products to be
able to address this type of threat. During this testing, a series of attack techniques was used to execute code using
legitimate functionality in Microsoft Office products. The same series of attack techniques was also used for 20% of
the malware samples delivered via documents and scripts. Ten unique techniques were used across three different
Microsoft Office applications. If the attacks were successful, the test cases deployed custom ransomware payloads.
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
Blended Threats
Additional Detection Rate 0.0%
Block Rate 50.0%
This report is Confidential and is expressly limited to NSS Labs’ licensed users. 8
Advanced Endpoint Protection Test Report – Check Point Software Technologies Check Point SandBlast
NSS Labs Agent Next Generation AV E80.82.1_030519
This test aims to verify that the AEP product is capable of detecting, preventing, and continuously monitoring
threats and that it is able to take action against malware, exploits, and blended threats when subjected to
common evasion techniques. Please contact NSS for information on the evasions utilized.
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
Evasions
Additional Detection Rate 0.0%
Block Rate 100.0%
All of these tampering techniques leverage administrator privileges. In enterprises where users are not granted
administrator privileges, privilege escalation exploits (such as those utilized during this testing) would allow similar
outcomes.
This report is Confidential and is expressly limited to NSS Labs’ licensed users. 9
Advanced Endpoint Protection Test Report – Check Point Software Technologies Check Point SandBlast
NSS Labs Agent Next Generation AV E80.82.1_030519
When tampering techniques were utilized, at least one potential issue or security vulnerability was discovered in
each product tested.
This report is Confidential and is expressly limited to NSS Labs’ licensed users. 10
Advanced Endpoint Protection Test Report – Check Point Software Technologies Check Point SandBlast
NSS Labs Agent Next Generation AV E80.82.1_030519
*Note: Checkpoint has several syslog alternatives that were not tested.
This report is Confidential and is expressly limited to NSS Labs’ licensed users. 11
Advanced Endpoint Protection Test Report – Check Point Software Technologies Check Point SandBlast
NSS Labs Agent Next Generation AV E80.82.1_030519
For the purposes of this report, capital expenditure (capex) items (the cost of acquisition and installation) are
included for only 2,500 agents.
Cost Information
Calculations are based on vendor-provided pricing information. Where possible, the 24/7 maintenance and
support option with 24-hour replacement is used, since this is the option typically selected by enterprise
customers. Prices depicted include the purchase and maintenance costs for 2,500 software agents only; costs for
central management solutions (CMS) may be extra. Please contact NSS for additional detail. Year 1 Cost includes
an additional $600 installation cost that was applied to all products in the test.
For additional TCO analysis, including operational costs, refer to the AEP TCO Comparative Report, which is
available at nsslabs.com.
This report is Confidential and is expressly limited to NSS Labs’ licensed users. 12
Advanced Endpoint Protection Test Report – Check Point Software Technologies Check Point SandBlast
NSS Labs Agent Next Generation AV E80.82.1_030519
Figure 9 – Scorecard
Test Composition
Each product was initially tested against 1,629 unique malicious samples and 1,061 unique false positive samples.
Ultimately, 897 unique malicious samples and 1,053 unique false positive samples met NSS’ validation criteria and
were included as part of the test.
1
No product is able to provide 100% protection against attacks. A single successful attack is often all an attacker needs to gain unauthorized
access, infiltrate an organization, and steal or destroy data.
2
Block Rate is defined as the percentage of exploits and malware blocked within 15 minutes of attempted execution. The Additional Detection
Rate is defined as the percentage of exploits and malware detected but not blocked within 15 minutes of attempted execution.
This report is Confidential and is expressly limited to NSS Labs’ licensed users. 13
Advanced Endpoint Protection Test Report – Check Point Software Technologies Check Point SandBlast
NSS Labs Agent Next Generation AV E80.82.1_030519
Test Methodology
NSS Labs Advanced Endpoint Protection (AEP) Test Methodology v3.0
Contact Information
NSS Labs, Inc.
3711 South Mopac Expressway
Building 1, Suite 400
Austin, TX 78746
[email protected]
www.nsslabs.com
This and other related documents are available at: www.nsslabs.com. To receive a licensed copy or report misuse,
please contact NSS Labs.
© 2019 NSS Labs, Inc. All rights reserved. No part of this publication may be reproduced, copied/scanned, stored on a retrieval
system, e-mailed or otherwise disseminated or transmitted without the express written consent of NSS Labs, Inc. (“us” or “we”).
Please read the disclaimer in this box because it contains important information that binds you. If you do not agree to these
conditions, you should not read the rest of this report but should instead return the report immediately to us. “You” or “your”
means the person who accesses this report and any entity on whose behalf he/she has obtained this report.
1. The information in this report is subject to change by us without notice, and we disclaim any obligation to update it.
2. The information in this report is believed by us to be accurate and reliable at the time of publication, but is not guaranteed. All
use of and reliance on this report are at your sole risk. We are not liable or responsible for any damages, losses, or expenses of
any nature whatsoever arising from any error or omission in this report.
3. NO WARRANTIES, EXPRESS OR IMPLIED ARE GIVEN BY US. ALL IMPLIED WARRANTIES, INCLUDING IMPLIED WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT, ARE HEREBY DISCLAIMED AND EXCLUDED
BY US. IN NO EVENT SHALL WE BE LIABLE FOR ANY DIRECT, CONSEQUENTIAL, INCIDENTAL, PUNITIVE, EXEMPLARY, OR INDIRECT
DAMAGES, OR FOR ANY LOSS OF PROFIT, REVENUE, DATA, COMPUTER PROGRAMS, OR OTHER ASSETS, EVEN IF ADVISED OF THE
POSSIBILITY THEREOF.
4. This report does not constitute an endorsement, recommendation, or guarantee of any of the products (hardware or software)
tested or the hardware and/or software used in testing the products. The testing does not guarantee that there are no errors or
defects in the products or that the products will meet your expectations, requirements, needs, or specifications, or that they will
operate without interruption.
5. This report does not imply any endorsement, sponsorship, affiliation, or verification by or with any organizations mentioned in
this report.
6. All trademarks, service marks, and trade names used in this report are the trademarks, service marks, and trade names of their
respective owners.
This report is Confidential and is expressly limited to NSS Labs’ licensed users. 14