Unveiling the Hunter-Gatherers: Exploring Threat

Hunting Practices and Challenges in Cyber Defense

Priyanka Badva, Kopo M. Ramokapane, Eleonora Pantano,
and Awais Rashid, University of Bristol
https://www.usenix.org/conference/usenixsecurity24/presentation/badva

This paper is included in the Proceedings of the

33rd USENIX Security Symposium.
August 14–16, 2024 • Philadelphia, PA, USA
978-1-939133-44-1

Open access to the Proceedings of the

33rd USENIX Security Symposium
is sponsored by USENIX.
 Unveiling the Hunter-Gatherers: Exploring Threat Hunting Practices and
Challenges in Cyber Defense

Priyanka Badva Kopo M. Ramokapane Eleonora Pantano Awais Rashid

University of Bristol University of Bristol University of Bristol University of Bristol

Abstract sources [17, 29, 40, 49], the SolarWinds or Sunburst attack 3
is believed to be the biggest and most advanced attack to date.
The dynamic landscape of cyber threats constantly adapts its
Its discovery was not straightforward; it was through trial and
attack patterns, successfully evading traditional defense mech-
error and a series of separate and loosely connected activities.
anisms and operating undetected until its objectives are ful-
Security attacks, e.g., Sunburst attack, require the exper-
filled. In response to these elusive threats, threat hunting has
tise to proactively seek them out from an organization’s net-
become a crucial advanced defense technique against sophis-
works or systems before they could cause significant dam-
ticated and concealed cyber adversaries. However, despite its
age or compromise sensitive data. However, their discovery
significance, there remains a lack of deep understanding of the
highlights the complex nature of threat hunting and the chal-
best practices and challenges associated with effective threat
lenges it poses to security experts. Despite an increasing
hunting. To address this gap, we conducted semi-structured
number of threats being detected in the wild, the processes
interviews with 22 experienced threat hunters to gain deeper
and practices of threat hunting remain poorly understood and
insights into their daily practices, challenges, and strategies
undocumented. Prior research on threat hunting has largely
to overcome them. Our findings show that threat hunters de-
focused on understanding attacks [18, 23, 31], improving de-
ploy various approaches, often mixing them. They argue that
tection [4, 14, 47] and mitigation techniques [20, 26, 33], im-
flexibility in their approach helps them identify subtle threat
proved policies around breach disclosures [41], and building
indicators that might otherwise go undetected if using only
effective tools [7, 13, 45], but very limited on the analyst who
one method. Their everyday challenges range from technical
do the job. Another body of research [1, 3, 8, 32, 37–39, 42]
challenges to people and organizational culture challenges.
has primarily focused on Security Operations Centers (SOCs)
Based on these findings, we provide empirical insights for
to improve their functioning and the well-being of experts
improving threat-hunting best practices.
within them. However, despite threat hunting largely falling
under SOCs, there remains a significant lack of understanding
1 Introduction of the daily practices and challenges threat hunters face.
Understanding these factors is essential for establishing
Investigating a security breach can be daunting, complex, best practices, streamlining threat-hunting procedures, en-
and time-consuming for security experts. In 2020, a security hancing tool usability, and identifying skill gaps and areas for
analyst at Mandiant1 responded to what seemed like a routine improvement in the field. To bridge this gap, we interviewed
security alert, unaware of what would unfold in the following twenty-two (22) threat hunters to gain deep insights into their
weeks and months. Soon after, the team discovered that the daily practices, constraints, needs, and experiences with cur-
hack had been active for weeks, undetected by the tools meant rent processes and tools. We asked two primary questions:
to raise alerts. While they could see the intruder’s activities, RQ1: Who performs threat-hunting activities, and what
they could not determine how the attack had occurred. But methods and processes do they use? We aimed to under-
after weeks of intensive investigations, they traced the source stand the requirements for becoming a threat hunter, including
to a tool supplied by SolarWinds 2 [49]. According to multiple the necessary skills and experience in the field. Moreover, we
sought to understand the various approaches used for threat
1 Mandiant is an American cybersecurity firm and a subsidiary of
hunting and the factors influencing their adoption. This analy-
Google. www.mandiant.com
2 SolarWinds Corporation is an American company that develops software sis would enable us to identify skill gaps and areas of focus
for businesses to help manage their networks, systems, and information
technology infrastructure. www.solarwinds.com 3 Sunburst Attack disclosure mandiant.com/sunburst

USENIX Association 33rd USENIX Security Symposium 3313

for recruitment while streamlining threat-hunting efforts by proaches: proactive and reactive. Proactive hunting involves
understanding the preferred methods. the utilization of threat intelligence to formulate hypotheses
RQ2: What challenges do threat hunters face, and what or use cases, enabling security teams to actively search for
strategies do they employ as best practices to overcome potential threats before they can cause harm. On the other
them? Through RQ2, we aimed to explore the challenges hand, reactive hunting focuses on conducting forensic investi-
faced by threat hunters, their resolutions, and the practices gations and responding to alerts indicating threats after they
they found essential and effective. Addressing RQ2 will pro- have been detected [33]. According to SANS 2018 threat
vide valuable insights into the areas where improvement ef- hunting survey4 , 60% of respondents reported engaging in
forts are needed and which practices should be standardized proactive threat hunting, with 43.2% conducting it continu-
across the industry for effective threat hunting. ously and 16.7% performing it at regular intervals. Our study
Our analysis suggests that threat hunting is performed by aims to investigate this further and understand the threat hunt-
various experts with diverse skills and experiences. However, ing practices in the wild.
it also requires tacit knowledge, making training essential Threat hunting is widely recognized as a complex and cog-
for new beginners. We unearthed three broad threat-hunting nitively demanding process. Consequently, ongoing efforts
approaches commonly used in the field: use-case/hypothesis- have been made to automate various aspects of the threat-
based, intel-based, and random-based hunting. The innovation hunting process. Previous works [3, 11, 33, 36] argue that
of threat hunting lies in seamlessly combining these meth- automation in threat hunting can offer numerous benefits,
ods to suit needs, available resources, skills, and organiza- such as reducing response times, resolving repetitive tasks, re-
tional requirements. Though planning might vary, the core quiring less technical knowledge from analysts, and reducing
process remains constant - collecting data, identifying and their cognitive load. However, it is also acknowledged that
validating threats, and instrument remediation and reporting. complete automation may not be feasible, as certain aspects of
Threat hunters encounter various challenges, technical com- the process require human analysts to make critical decisions.
plexities, interpersonal dynamics, and organizational issues. Most analysts rely on their domain knowledge and experience
For instance, they face difficulties in building hunting hypothe- to make decisions, often without deeply considering specific
ses due to the ever-evolving threat landscape, with new tactics cases [19, 22, 50]. Chen et al. [10] have argued that experts’
emerging regularly. Consequently, adaptability is highly em- critical knowledge is often lost and have advocated for tools
phasized in their approach to address these unique challenges. that can retain expert knowledge to reduce their workload
Our results offer valuable insights into the specific daily ex- and address their blind spots. We expand this knowledge area
periences of threat hunters. In summary, our contributions to by investigating the role of automation in threat hunting and
the field are as follows: hunters’ experience influences their efforts.
• Comprehensive Understanding of Threat Hunters’ Prac- While the consensus is that complete automation is not
tices: We provide the first empirical evidence on the daily achievable, significant progress has been made in automat-
practices of threat hunters in the wild. We highlight the ing specific parts of the threat-hunting process. For exam-
prevalent methods and how they are used to tackle cyber ple, hypothesis generation [36], data collection, threat detec-
threats. Additionally, we shed light on the required skills, tion [27, 48], data triaging [50], malware classification [6, 28],
qualifications, and experience needed for threat hunting. and other aspects of the process. Despite the advances in au-
• Addressing Challenges and Suggesting Improvements: tomation, prior studies have not focused on understanding
Our study offers insights into the most common challenges how these tools impact threat-hunting investigations. In ad-
in threat hunting and the strategies that threat hunters em- dition to tools, threat hunting heavily relies on intelligence
ploy to overcome them. By doing so, we identified recom- (e.g., threat reports), analysts depend on it for investigations
mended practices for improving threat-hunting processes. and to ensure the security of their systems. Threat reports
are typically based on analyzed and remediated attacks, offer-
ing reactive advice [12, 34]. Outside academia, vendors such
2 Background and Related work
as Symantec, McAfee, Trend Micro, FireEye, Cyveillance,
The concept of threat hunting has evolved over the years, pri- and Kaspersky regularly publish threat intelligence for multi-
marily driven by the growing sophistication of cyber threats ple government and commercial organizations. Our current
and the realization that traditional security measures alone study provides valuable insights into how threat analysts use
are insufficient to safeguard against these advanced threats. intelligence in their everyday hunting processes.
Threat hunting is typically classified as an essential element Other studies [1,3,8,32,35,37–39,42,46] related to analysts
within Cyber Threat Monitoring and Analysis [33]. It can be focus on SOCs, primarily aiming to understand the well-being
defined as a proactive and iterative process aimed at searching of security analysts. For instance, previous works [35, 37]
for and identifying potential cyber threats and malicious ac- investigated the factors contributing to analysts leaving the
tivities within an organization’s network or systems [28, 36]. 4 https://www.malwarebytes.com/pdf/white-papers/sans_

Threat hunting can be broadly categorized into two main ap- report-the_hunter_strikes_back_2017.pdf

3314 33rd USENIX Security Symposium USENIX Association

field. Another line of work on security workers has primarily particularly in identifying, analyzing, and managing cyber in-
focused on bug bounty hunters [2, 5, 43, 44]. For example, cidents. Once interested, we sent them the information sheet
Votipka et al. [44] compared how testers and hackers discover (PIS), the demographics form, and consent form. The PIS
software vulnerability techniques. They concluded that the explained the purpose of the study, anonymization, what par-
discovery experience, knowledge of underlying systems, avail- ticipation entailed and the withdrawal process. The consent
ability of access to the development process, and motivation form was seeking consent to participate, audio recording of
play a crucial role in each step of vulnerability discovery. Our the session. Respondents who completed and sent the consent
study builds upon these investigations by examining factors form were further contacted for scheduling a session. We did
that are influential in identifying threats. Additionally, we pro- not conduct any further screening of our respondents since
vide insights about other security workers, specifically threat our recruitment material explicitly requested participants with
hunters. over 2 years of experience. However, to ensure suitability, we
asked them to clarify their roles and daily activities during
the interviews.
3 Methodology Similar to other works on security workers (e.g., [4]), tar-
geting such a unique and specialized group presented a sig-
3.1 Study Design nificant challenge. Some potential participants chose not to
participate in the study due to concerns about being recorded
To gain insight into the practices and challenges of threat and directly quoted or about disclosing their company’s op-
hunters in real-world scenarios, we designed and conducted an erational practices. Given the sensitive nature of the subject
online semi-structured interview study. Our interview script matter, participants were allowed to participate without shar-
was divided the script into five sections: introduction, data col- ing their cameras. Moreover, SOC personnel, for example,
lection, threat identification, threat analysis, and conclusion. are typically occupied with responding to severe incidents,
The introductory section aimed to establish rapport and ob- and their willingness to participate in academic studies may
tain information on participants’ role and responsibilities. The not be motivated by financial incentives. To overcome these
subsequent sections, data collection, threat identification, and challenges, we employed a snowball sampling technique to
threat analysis, were designed to delve into the details of the increase our sample size. We targeted professionals leading
threat hunting process. Lastly, the conclusion had questions threat hunting teams and asked them to encourage their col-
that sought to gather suggestions from participants on improv- leagues and other professionals they know to participate. We
ing the process and concluded the interview. Demographics also encouraged those who participated in our study to share
were gathered using a separate form. our research with their colleagues.
To validate the effectiveness of our interview script, we
conducted pilot studies with two members of our research
group who had experience working in SOC. The rationale 3.3 Participants
behind this was twofold: to assess the clarity of our questions
In the end, we successfully recruited 22 participants from
and whether they could elicit responses that addressed our
various countries. The majority were from the UK (7) and the
research questions and ensure that the interview could be
US (9), while the remaining six participants were from Qatar,
concluded within a reasonable time. Based on the outcomes
Australia, Singapore, Germany, India and UAE. These partic-
of the pilot study, we refined our script by reducing the number
ipants were employed by leading multinational companies,
of questions and supplementing follow-up questions in each
five participants were from companies that provided in-house
phase of the threat hunting process. The pilot study data was
threat hunting services only, while eleven participants were
excluded from the final analysis.
from Managed Security Service Providers (MSSP). The re-
maining six were from companies that provided both in-house
3.2 Ethics and Participant Recruitment and external threat hunting services. While some participants
shared the same company affiliation, most worked in different
We obtained ethics clearance from our University of Bristol countries and states serving diverse clients around the world.
Ethics committee before beginning our study. Utilizing our Only P1 and P3 were based in the same office, while P4, P5,
personal, industrial, and academic connections, we recruited P9, P10, and P11 worked for the same company but across
diverse participants through social media, word-of-mouth, different offices in the US. As for their roles, participants
Slack, and academic and industry conferences. Our objective held a variety of positions, including analysts, consultants,
was to identify and recruit professionals whose daily work red team specialists, and security engineers. They also had
could be described as threat hunting or involved some aspect diverse educational backgrounds and held various security
of threat hunting and who had a minimum of 2.5 years of expe- certifications, such as the CISSP. Table 1 provides a summary
rience in the field. This was to ensure that our sample included of the demographics, a more detailed summary of our par-
the necessary expertise and familiarity with the problem space, ticipants’ demographic and professional backgrounds can be

USENIX Association 33rd USENIX Security Symposium 3315

 Table 1: Participants demographics details "-" : Prefer not to answer
ID Job Role Country Experience Education Company Type of Service Recruitment Method

P01 Senior Cybersecurity Analyst UK 10-15 years PhD Company 1 In-house Industry Connection
P02 Security Consultant Australia 15-20 years MSc Company 2 MSSP Industry Connection
P03 Threat Intelligence Analyst UK 5-10 years Bachelor Company 1 In-house Industry Connection
P04 Associate Director Threat Hunt US 10-15 years - Company 2 MSSP Industry Connection
P05 Threat Hunting Team Lead US - - Company 2 MSSP Industry Connection
P06 Digital Forensics Specialist UK 5-10 years Bachelor Company 3 MSSP Industry Connection
P07 SOC Analyst US 10-15 year Bachelor Company 4 In-house + MSSP Industry Connection
P08 Director for DFIR Singapore 10-15 years MSc Company 2 MSSP Industry Connection
P09 Consultant US 15-20 years Bachelor Company 2 MSSP Industry Connection
P10 Lead SOC Threat Hunter US 5-10 years High School Company 2 MSSP Industry Connection
P11 IT Security Engineer US 15-20 years Bachelor Company 2 MSSP Industry Connection
P12 SOC Analyst India 10-15 years Bachelor Company 4 In-house + MSSP Snowball
P13 Security Analyst L3 UAE 5-10 years - Company 5 In-house + MSSP Slack
P14 Program Lead Adv Sec Analytics US 15-20 years - Company 6 In-house Slack
P15 Threat analyst UK 5-10 years High school Company 7 MSSP Slack
P16 Cybersecurity Technical Specialist UK 10-15 year Bachelor Company 3 MSSP Snowball
P17 SOC Head UK 10-15 years MSc Company 8 MSSP Industry Connection
P18 Security Research Lead UK 15-20 years Bachelor Company 9 In-house + MSSP Snowball
P19 Cybersecurity Engineer Germany 15-20 years - Company 10 In-house Snowball
P20 Lead Cybersec Engineer US 10-15 years MSc Company 5 In-house + MSSP Slack
P21 Manager, Incident Handling US 10-15 years MSc Company 11 In-house Snowball
P22 Senior Incident Response Consultant Qatar 10-15 years MSc Company 9 In-house + MSSP Snowball

found in Table 2 in the Appendix 8.1. researchers independently coded the first two transcripts to
identify key themes, methods, processes, tools, and attitudes
related to threat hunting. Following this, they met and dis-
3.4 Interview Procedure cussed their findings to create a codebook. Discrepancies be-
At the beginning of each interview session, participants were tween the coders were resolved using the “arguing to consen-
reminded of the purpose of the study, their expected involve- sus” method [21]. The codebook was then shared with other
ment, and the withdrawal process. They were also asked to researchers for review and validation before finalizing. After
confirm their willingness to participate. After obtaining con- developing the codebook, two researchers proceeded to code
sent, we initiated the audio recording and began the interview. an additional three transcripts and calculated the inter-coder
Our first questions were about roles and responsibilities in reliability. The inter-coder reliability score using Cohen’s
their workplaces. We then proceeded to ask about their threat Kappa Coefficient was 0.81, indicating substantial agreement
hunting practices, guided by their responses. While we had in applying the codebook [24]. Then, the first author pro-
a script, we did not rigidly adhere to it in all cases, but we ceeded to code the rest of the scripts. High level codebook
ensured that all relevant questions were covered by the end attached in Table 4 in the Appendix 8.4.
of each session. The interviews concluded with exploring
potential areas for improvement in the threat hunting process.
Participants were then thanked for voluntary participation in
3.6 Limitations
the study, and no financial compensation was provided. On While we attempted to diversify and enhance our sample
average, each interview session took between 40 minutes to using snowball sampling, we acknowledge some common
1hr 15 minutes. Due to time differences with some partici- limitations associated with studies that have employed this
pants, some interviews took place in the early hours or late technique as a recruitment method. Firstly, snowballing can
evenings of our local time. Our complete interview protocol perpetuate power imbalances; some participants may have
can be found in the Appendix 8.3. participated in the study because they were recruited by in-
dividuals in higher positions, feeling obligated to participate.
3.5 Data Analysis To mitigate this, we emphasized to participants the voluntary
nature of their involvement and their right to withdraw at any
Once data collection was complete, we utilized a professional point, ensuring they felt no external pressure to participate.
transcription service that adhered to our university policy and Secondly, samples recruited through snowballing often lack
GDPR complaint to transcribe all our interview recordings. representativeness. For instance, our sample is biased towards
After transcribing all the audio files, we began the coding pro- participants from large companies that primarily offer man-
cess. We inductively coded [9,30] the scripts using the conven- aged security services. This bias may stem from participants
tional line-by-line method to identify key themes, methods, sharing the study among their peers or personal networks
processes, tools, and attitudes relating to threat hunting. Two limited to such companies. Moreover, some participants even

3316 33rd USENIX Security Symposium USENIX Association

came from the same large corporations. However, while they companies, the other big organizations in the United Nations
may have been affiliated with the same large companies, they family and so on.”
were situated in different countries, serving a diverse array of Regarding skills and qualifications, we found that threat
clients and dealing with various challenges and circumstances. hunting teams consisted of analysts with various skills and
This diversity instilled confidence in us that our sample has knowledge (captured in Table 1 and 2). However, many par-
provided a broad spectrum of perspectives. ticipants with teams explained that they either received or
Other limitations of our study include lack of generalizabil- provided training for their new team members. For instance,
ity. As prior studies [3, 4] have reported, SOCs and organi- one participant explained that their team comprised individ-
zations are unique, and other salient factors could influence uals with forensic and penetration testing backgrounds, but
the daily experiences of threat hunters that we may not have they had to provide training to cross-train the team members
captured in our study. Moreover, our results may be skewed in threat hunting. P05 said “I have a team of 8 people, most
towards companies that provide managed security services of them actually started on our forensic team, and I’ve kind
than those with their own in-house dedicated threat hunting of poached people from the forensic team... pentest team to
teams. We conducted recorded interviews, which might have form the threat hunt team. I had to cross-train the forensics
influenced participants’ responses. Some threat hunters de- guys and the pen-testers.”
clined to participate due to concerns about recording sessions, This finding is further supported by the skills and qualifi-
which could have led to under-reporting or giving answers cations of the participants in our sample. They have various
that they believe are socially or professionally acceptable. To skills and qualifications. Other teams started with one member,
mitigate this social desirability bias, we emphasized that we and later were joined by others. P10 explained:“Our company
were interested in their opinions on processes rather than spe- had one dedicated threat hunter, [name]. He’s also a science
cific sensitive issues. However, it remains essential for the instructor. He kind of worked on threat hunting but he didn’t
security community to explore alternative methods that pro- work in the SOC doing the day-to-day job, [he did] informal
tect participants’ identities or their organizations while still type of threat hunting stuff. Me and a few others started doing
obtaining valuable insights. threat hunting on our own and we proposed a threat hunting
program.” P10
4 Findings Take Away. Threat hunting is carried out by analysts with
diverse qualifications who may be part of dedicated teams or
4.1 Who performs threat hunting? have other organizational responsibilities. Since there are no
Our analysis suggests that threat hunting is performed by specific entry qualification requirements for threat hunting, or-
analysts with various skills and qualifications, who may ei- ganizations invest considerable time and resources in training
ther belong to dedicated teams or have other responsibilities their staff to excel in hunting for threats.
within their organizations. Participants engage in threat hunt-
ing both for their own organizations and as consultants for 4.2 How do they perform threat hunting?
external entities. From our sample, those who had dedicated
teams performed threat hunting for their organizations and Our analysis revealed that threat hunters employ various meth-
also provided external consultations:“I have 9 staff working ods when conducting threat hunting in the wild. These meth-
for me. We have staff in the US, India... It’s a dedicated ser- ods are often combined based on specific needs and available
vice, when a client signs up, we assign a dedicated analyst to resources. For easy explanation, we have categorized them
that particular client.” P04 into three broad groups: use-case hunting, signature or intel
Teams are often formed randomly, with their structure and hunting, and random hunting.
size depending on the organization’s size and requirements. In the first approach, use-case hunting (n=17), threat
Threat hunting teams in some organizations can consist of one hunters use predefined scenarios or patterns of suspicious
person or a team that leads the efforts and receives assistance activities to identify and investigate known threats and attack
from other teams or departments within the organization. For patterns. These use cases are based on the threat hunters’
instance, P7 explained that their threat hunting is part of a knowledge of threat actors, the systems they own, and the typ-
SOC, where they have a monitoring team, and others perform ical attack patterns. The second category, intel-based hunt-
threat investigations. “I work around the SOC, which also ing (n=7), involves leveraging technical threat intelligence,
has threat intel, threat hunting, [Incident] response, all of such as known indicators of compromise/attacks (IoC/IoA),
that. The organization is global in the sense that we have to guide the hunting process. This method relies on up-to-
office in possibly every country on this planet. [...] We run date threat intelligence to proactively detect potential threats.
our security operations centre from two geography’s and do The difference between these two methods is that in use-
cover 24 [countries] across seven services. There is a threat case based hunting, hunters rely on pre-defined scenarios or
intel function which works for big organizations, the other cases to hunt for threats, while intel-based hunting relies on

USENIX Association 33rd USENIX Security Symposium 3317

the threat intelligence they receive to investigate and iden- stated that they create use cases based on Tactics, Techniques,
tify threats. For example, under intel-based hunting, hunters and Procedures (TTPs), particularly leveraging the MITRE
may use malware signatures they have received to search for ATT&CK framework, historical incidents, threat intelligence,
threats. Under use-case based hunting, hunters formulate or their knowledge of the infrastructure in question, or the spe-
create scenarios to guide their efforts, and sometimes these cific requirements of their organization.“Using my experience
scenarios are informed by threat intel. For instance, a scenario of having worked in many environments and I look at how un-
may include the behavior of malware which was provided likely an attacker would target one of those individual systems.
as intelligence. The third category, random hunting (n=6), Then we build a threat mod el that goes from the outside in
encompasses methods where participants conduct hunts with- using info that is publicly available for the initial compromise.
out prior knowledge of specific indicators of compromise But then my experience comes in, how does a breach of [the]
or a predefined plan. This approach involves being alert to main frame look like, what data would they try to steal, what
potential threats during regular responsibilities or while con- would they do if they got access to a PLC.” P18
ducting other specific hunts, such as onboarding new clients Once the use case is finalized and the approach is outlined,
or responding to incidents initially considered benign. participants explained that they would determine or compile a
list of the necessary data for the hunt and identify the relevant
sources. This is to ensure that the data is readily available
when needed. “It’s really important that you identify the data
sources before you do an investigation, so you can onboard
them, normalize that data, so when you need to do any type
of hunting activity, it’s readily available.” P14
Participants who consulted for other organizations men-
tioned that the planning phase included assessing the client’s
infrastructure to understand normal behavior and whether
their existing resources and methods could be applied to the
client’s case. P04 said: “When a client signs up for our ser-
Figure 1: Threat Hunting Process vice, we assign a dedicated analyst specifically for that client.
This allows the analyst to become familiar with the client’s
Our analysis indicated that the choice of approach is influ- environment, understand what is considered normal in that
enced by several factors, including available resources (e.g., context, and identify abnormalities.”
malware lab), data availability (including external threat intel- Intel-based planning is primarily influenced by the intel-
ligence), the skills and experience of the hunter, and organiza- ligence or indicators of compromise (IoCs) hunters receive.
tional requirements. Despite their differences, these methods This intel may come as hash values, filenames, registry keys,
share a similar approach involving planning, data prepara- IP addresses, domain names, malware, and host or network
tion, threat identification, and remediating and reporting the activities associated with malicious activities. Participants
findings. Figure 1 shows the relationship between various explained that detailed and specific planning begins when
processes involved in threat hunting. they receive such intel. Consequently, they rely on getting
the latest threat intel. Participants received intel from vari-
4.2.1 Pre-hunt Plan ous sources, primarily external sources such as mainstream
vendors and open-source intelligence platforms. They shared
Our analysis indicated that most threat-hunting activities be- that they also actively participate in various security commu-
gin with a dedicated planning phase. During this stage, hunters nication groups on platforms such as Slack, LinkedIn, and
gather to formulate a plan that outlines the specific areas they Twitter, while others are public and private security exchange
intend to investigate. Participants emphasized that having a groups.“We start by looking at what’s available open source;
well-defined plan enables them to assume control over the ex- blogs and reports that other vendors have produced, maybe
ercise and effectively allocate resources. They also explained where they have only mentioned a particular piece of malware
that having a plan helps keep the entire team involved. “We or CnCs [Command&Control]. We then take those CnCs, an-
try to plan all that stuff out at the beginning. It’s planning, alyze them further, and try to build a bigger picture. We also
getting together in the beginning with everyone involved. And, get intelligence from other partners in the community, closed
then just let the system run, and then if something comes up intelligence exchanges, not open to the public, just between
later, you deal with it as a team." P19 security vendors or other companies in that kind of space.
Pre-hunt plan for use-case-based hunting involved defin- We have our tracking as well. We’ve paid for services like
ing the scope of the hunt, determining the type of threat to VirusTotal, an online malware repository service, and we pay
pursue, and identifying the required data sources based on the to put certain kinds of tracking in place on those sites, various
use cases and hypotheses they have developed. Participants services such as the known groups we track. We can say if it

3318 33rd USENIX Security Symposium USENIX Association

 satisfies this rule, it will alert us, and we can stay on top of would plan how to implement or run the IoCs in their systems.
things that way as well.” P16 This process involved collecting relevant data and preparing
Once intelligence is received, hunters review and try to the testing environments such as malware labs, to handle the
understand it; its origins, the systems that can be affected, received indicators. Hunters also used the intel to develop
and how the threats can be identified and mitigated. P03 new use cases or hypotheses for threat-hunting activities.
stated: “The idea is that once we have that, we would then run Those who engaged in random hunting used various data
that information through other data sources that we [have] logs for their hunts, often selecting sources based on what
purchased or have access to, and kind of enrich it.” was available or what they found interesting at the time.
For random hunting, we observed that there is minimal We found that threat hunters generally prioritize gathering
planning involved. These hunts are more ad hoc, happening as much data as possible. Participants argued that it could
spontaneously and sometimes driven by curiosity or a hunch. significantly improve the quality of the hunt. For example, P04
For instance, a threat hunter may impulsively investigate old stated, “We collect as much data as we can because the more
‘benign’ logs without a predefined plan or specific objectives. data you have, the better. Sometimes, you might search for an
ID, which might appear on three different devices, allowing
Take Away. Threat hunters use various methods in their you to trace how it entered the network. The more we can do
hunting activities, which they often combine based on their on the SIEM, the better, rather than having scattered devices.”
specific needs, skills, knowledge, and available resources. Most
activities start with a dedicated plan, where hunters define the Take Away. Threat hunters tailor their data collection to align
scope of the investigation, assess threat intelligence, generate with their hunting goals and collect as much data as possible
hypotheses or use cases, identify relevant data sources, and to maximize their effectiveness in detecting and responding to
allocate resources based on their chosen approach. security threats.

4.2.2 Data Collection and Preparation 4.2.3 Hunting and Validating

Once the pre-hunt plan is complete, participants indicated Our analysis indicates that threat hunters hunt and validate
they would collect and process the necessary data from vari- threats through an interactive and connected process. Threat
ous systems. Threat hunters collect data from various sources, hunters move between hunting and validating as needed, often
including firewalls, antivirus, network systems, Endpoint De- collecting more data/intel or creating new scenarios.
tection and Response tools (EDR), and proxy servers, and For use case hunting, hunters deploy the rules they have
these data are based on specific time frames or behaviors of built on the collected and filtered data. When specific condi-
the systems. Participants emphasized the importance of se- tions related to threats are met, alarms are triggered, prompt-
lecting the hunting approach before initiating data collection. ing the hunters to investigate further. P02 explained, “We
For example, P09 stated, “Before you even get to data collec- create use cases - if the rule is being triggered, we identify the
tion, you talk about the frame of where they are approaching specific behavior, whether that’s coming from an IP address
it from. [...] People will see something on Twitter and say, or a process.”
‘That’s a great idea; let me go and hunt for it,’ but they don’t For intel-based hunting, IoCs are fed into relevant systems
really bring a rigorous approach to it.” to trigger alerts and identify potential compromises. Hunters
After gathering all the required data, they curate and pre- compare their own data or logs against signatures or patterns
pared it for their hunts. The data preparation process involves of known indicators of compromise associated with specific
cleaning, filtering, consolidating, and transforming the data tactics, techniques, and procedures (TTPs). When a match
into a usable format. This is often within a single location is found, it suggests a potential compromise or the presence
or system such as a SIEM (Security Information and Event of a specific TTP. P22 elaborated, “if our threat intelligence
Management) solution, which also aids in the hunting process. profile tells us that we need to look for a specific binary name
Other steps included normalizing, parsing, and restructuring in the Windows system32 folder, then we’re going to search
the data. Filtering was typically performed to remove benign for that.”
activities from suspicious events, based on time frames, pat- On the other hand, random hunts tend to occur unplanned
terns, and thresholds defined during the planning phase. and are often initiated by hunches or when the hunter observes
For participants using use case hunting, data collection something that looks malicious. Participants described situa-
was aligned with the goal of the hunt. They would determine tions where, based on their knowledge of the normal state of
what the hunt needed and then search for the logs supporting the system, they occasionally discovered patterns of malicious
those detections. They would then build rules or triggers to behavior. P09 stated, “I could be looking at something and
detect threats, even those that may not be easily noticeable. think that looks weird. To me, that is an identification. An
Participants who utilized the intel-based approach stated analyst looking at data and saying, ’That does not quite look
that once they understood the campaigns or the intel, they right’ is absolutely identification.”

USENIX Association 33rd USENIX Security Symposium 3319

 After an investigation is triggered, threat hunters manu- that we wanna give a report, we want to tell a story, we want
ally search for threats or use tools to understand the alarms. to be able to say this is how it actually started. So and so
They employ tools such as Splunk, VirusTotal, or Reli- got a phishing email, they called the number on the phishing
aQuest/GreyMatter to validate whether they deal with threats email...” P05
or false alerts. Threat validation includes further investiga- The reporting process also includes lessons learned. Par-
tions (using additional hypotheses) and may involve inviting ticipants stated that this part of the report includes detailed
other team members to confirm the findings. Threat identifica- information about how the detection mechanisms missed the
tion is a process that requires knowledge and experience. P09 threat or specific steps to configure a system properly. These
emphasized, “I think it requires more knowledge and keeping lessons are valuable for future reference and can provide new
up-to-date with stuff. We would discuss it as a team, ‘Does intelligence on threats. P13 said, “As I said, in the lessons
this look suspicious? This doesn’t seem quite right to me.”’ learned part, we will be giving them more detailed informa-
In addition to team discussions, participants mentioned tion about how to enhance their detection analytics to fix
that they refer back to previous incidents or notes to confirm where they failed to alert.”
whether the behavior is normal or if the suspicious pattern Moreover, participants recognized the value of thorough
has been encountered before. P20 explained, “The first thing reporting as it allows them to demonstrate the value of their
I usually do is look back on my own notes and things and see hunt to the organization. By providing a detailed post-action
if I recognize them. If it’s a URL or a pattern, I look back on report, they can show how they addressed deficient posture in
past incidents and any notes I have to see if it’s familiar to me the organization, helping prevent potential damage or further
to jog my memory. If I can’t recall it, then I usually go to open threats. “For me, the hunt starts with that framework of a
sources first, and secondly, I’ll go to places like VirusTotal, plan and post-action report to help capture the value because
input the domains, and see what comes back.” then we can show the organization we have this posture im-
provement report that came out of our hunt, and we helped to
Take Away. Threat identification and validation are inter- address deficient posture in the organization before it caused
connected processes. Threat hunters employ various tools and damage or allowed damage to occur.” P09
manually search and validate indicators of compromise. In
most cases, validation is a team process. Take Away. Threat validation, remediation, and reporting
are interconnected and vital stages of threat hunting. They
are crucial for ensuring that threats are effectively addressed,
4.2.4 Remediation and Reporting lessons are learned, and necessary actions are taken to enhance
Once threats are validated, the remediation and reporting the organization’s security posture.
process is initiated. Participants described these processes as
interconnected and often addressed them together. Depending
4.3 Threat Hunting Challenges
on the case, remediation can either be carried out by the threat
hunting team or passed on to other teams in the SOC or the
client to decide. “If we do identify threats, and we prove our
hypothesis to be correct, we’ll inform the customer and move
to emergency response.” P22
They emphasized the critical importance of severity and
time in remediation and reporting. For severe issues, immedi-
ate action is taken, and relevant parties are contacted promptly
on how to proceed. In cases where they have permission or
authority to act on behalf of the client, they contain the situa-
tion. However, if authorization is needed, they would contact
relevant people immediately suggesting some remediation
Figure 2: Threat Hunting Challenges regarding Method, Data,
steps. “...depending on the severity, we contact the client; it
Organization and People
could be a phone call or an email. Then we check with them
and tell them some remediation steps they can take.” P02
Participants revealed various challenges they face as threat
Sometimes, the remediation process involves collaboration
hunters. We categorize and present these challenges in three
with other teams, requiring thorough communication. In such
groups as shown in Figure 2
cases, participants emphasized the need for detailed report-
ing to ensure everyone understands what took place, what
actions were taken, what was discovered, how the situation Method-related challenges: These are participants’ most
was contained, and any further recommendations to prevent common issues while attempting to identify, verify, and reme-
future occurrences. “For the breaches, the idea is pretty much diate threats.

3320 33rd USENIX Security Symposium USENIX Association

False alerts One common challenge reported by all our par- poses difficulties in application. “The biggest challenge we
ticipants is dealing with false alerts or false positives. They have is going through all the tactics and identifying what we
explained that many hunting activities are prone to trigger can look for in particular client’s environments because the
alerts that are non-malicious. However, despite being false, MITRE ATT&CK Matrix is very broad, there’s so much data
participants explained they cannot be ignored and require there that you have to go through and understand what data
thorough investigations, which often takes time and need sources are needed for a particular techniques to focus on.”
resources. Participants further indicated that several factors P08
maybe contribute to false alerts, such as vague detection rules, Furthermore, some participants highlighted the difficulty in
new devices joining the network, and internal users using determining what to hunt for when they are not even a victim
pirated or banned tools or accessing malicious sites. P08 men- or aware of what is happening outside their companies. They
tioned, “Yeah, there are many false positives, especially on find it challenging to identify what they should be monitoring
the PowerShell side. We built a usecase to block PowerShell in without any leads, leading them to rely on information from
the network, and it sometimes flagged it. When we ask about it, other sources. “If you are not a victim or an incident respon-
we received responses like, ’No, we’re not using PowerShell,’ der with access to active attacks, you have to rely on other
but it turned out there’s one admin who is very proficient with people to say, ‘Oh, we were just attacked by this. Here’s the
PowerShell and who used it. We thought, ’Oh, what’s happen- information,’ that’s different from a SOC where you’re the one
ing here?’ The analysts said, ’No one’s using PowerShell,’ but being attacked. You have the information. We wait on other
there was a senior guy who was actually using it.” people to present the information to us. That is a challenge.
Moreover, participants pointed out that verifying these Trying to find things of interest is always a challenge, but
anomalies can be particularly challenging, especially when fortunately, there’s a lot of security researchers out there who
users have flexible working conditions like working from are constantly blogging things, so we get by.” P16
home or during outside normal working hours. P19 further
explained that it is even be more complex when the company Evolving tactics and techniques. Participants (n=10) men-
operates globally; reaching an individual to verify alerts can tioned that one challenge in the threat identification process
be even more challenging: “We’re a global company and is from the ever-changing threat landscape. They explained
there are people working in different time zones. If people are that from time to time they encounter evolving threats and ac-
working on a Sunday, and they’ve turned off their phones be- tors as technologies change or advance, new threat actors and
cause they’re not technically supposed to be working, trying threats are discovered. Moreover, some participants explained
to catch up, we may not be able to reach them when they’ve that threat actors also adapt the way they operate, making
triggered the alerts. Sometimes that’s a big challenge.” threats increasingly challenging to understand. “The biggest
Participants also discussed false alerts as a challenge when challenge would be changing TTPs, when you have groups
automating processes. They explained that automation some- that change the way they operate. You can’t go in knowing
times results in false alerts or causes tools to miss out on that CONTI does this or APT28 does this ... you can’t re-
detecting threats. They also pointed out that due to the speed ally be used to that much because they could change tactics...
at which threats constantly evolve, automation may strug- Groups change their tactics all the time so that’s probably the
gle to keep up with newer threats, leading to countless false biggest thing is involving tactics.” P05
negatives. “False negatives, that’s a silent killer. Not under- Participants commonly described these tactics as a moving
standing that, it’s tough...There are dozens of cases that I can target, which some said it is overwhelming and hard to keep
think of where I looked at something and was like, hey, I don’t up with. They explained that the problem with these constant
see anything and I go back to the business owner and we look changes is that threat can sometimes evade their existing de-
at it together and then we’re like, hey yes, these things right tection methods; actors refusing to comply with predictable
here, that should never happen. The system should never do patterns-“not playing according to their wishes.” Evolving na-
that, even though to me they first looked benign.” P14 ture of threats demands continuous changes to approaches and
budget to threat hunting. “Big contenders of challenges, first
Building use cases - Complex TTP. Participants (n=5) of all, are adversaries are not rolling over and playing dead
also discussed building use cases or hypotheses for threat for us. They are specifically interested in evading detection.
hunting as one of their biggest challenges. They explained We are dealing with a moving target and that moving target
that TTPs are complex, making it difficult to formulate and has interest to succeed, the ability to evade our detections
focus on specific usecases for each client. They also stated which means they are developing evasions.” P09
that the complexity arises from each usecase requiring its
own unique data set, which limits the possibility of reusing Systems and Tools failing. We also found that tool perfor-
certain use cases. Some participants specifically emphasized mance poses many challenges. Participants (n=7) described
the challenges they face when using the MITRE ATT&CK various situations in which the tools or system they were using
Framework to build use cases, noting that its broad scope failed to deal with their request. For instance, a tool crashing

USENIX Association 33rd USENIX Security Symposium 3321

 because it is dealing with large amounts of data or queries. information is of little value for effective threat hunting. We
“Because we do correlation, some searches take more time and identified several factors contributing to data quality issues,
CPUs than others, so we need to optimise so that one search including the lack of knowledge among data collectors, lim-
does not crash all the system. We try to do that but the issue ited resources for long-term data storage, and excessive noise
is, in the lab we only have few rules and only running when in the data. Participants noted that data collection decisions,
we want them to run, whereas for a big production you have particularly by external companies, can be ill-informed. This
a thousand of rules all running in parallel together. So, when can result in unnecessary or incorrect event logging, which
we create a rule we have to make sure that it will fit in the big are common issues that lead to data not being useful for threat
production without crashing everything else” P02 hunting. Moreover, storage limitations, where companies fail
or do not have enough resources to retain logs for extended
Take Away. Building relevant use cases or hypotheses is periods was commonly mentioned. Regarding the lack of
a big challenge because attacks constantly evolve. Dealing
visibility due to missing data points, P18 stated: “Another
with false alerts, whether positive or negative, poses many
thing that becomes apparent is when you expect the data to
challenges, including time and team effort. While tools play a
be there, but it’s not. You should have had 30 days of logs, but
significant role in hunting, they sometimes fall short.
when you check, there are only 7. What happened there? I
think these are the basics for obtaining data, and some people
Data-related challenges: These are the most common would call this visibility. I need visibility into these things in
challenges threat hunters face concerning data for threat- order to do it.”
hunting purposes. Other participants argued that the low visibility of devices
or systems in the logs was sometimes caused by logs col-
Complex data. Participants (n=7) also highlighted that deal- lected in various places rather than in one location. Some
ing with complex data poses various challenges. While having participants said this challenge is sometimes caused by not
a lot of data available is considered a good thing, the task of having access to relevant systems or devices to collect data.
getting these data into a usable format presents many chal- Example of P17 explaining that they had to revisit a device
lenges. They explained that logs of various systems come in because it was missing from the logs:“The data in the logs
various formats or containing different bits of information. sometimes doesn’t give us enough information to go on and
Furthermore, technological advancements can also lead to hunt, what’s presented in front of the analysts is quite hard to
changes in logs, which is challenging to keep up with. As P07 use to make a decision. You end up having to spend extra time
explained: “it’s very challenging to keep up with technology actually going to the originated device to see what happened.
and available options, logs often change their format, the way So, we can’t pull back PCAPs [Packet capture ], things like
they appear. The way they parse. So, what that mean is that that, we’re solely based on what the logs present to us and
the detection that we burn is based on what the logs looked they’re defined by IS defenders so that can be quite tricky.”
like a year back. They’re not really what it is today.” Participants who cited noise in the data as a problem ex-
To make use of this data effectively, participants explained plained that it forces them to spend significant time filtering
that they must first get it into a standard and usable format. through irrelevant information.“You’re digging through large
This involves filtering, normalizing, and transforming it into data sets you’re always going to have limitations and time...
a format that is appropriate for analysis tools. But, in most You don’t get instant results for any query or analysis that
cases, these processes require time and effort. P01 said: “The you’re doing. Again the volume of data that you have to go
challenge is to get the data in a good shape. For instance, if through is so much data... it’s overwhelming sometimes how
you have email logs where you have one that is odd, for this much you have to dig through ... As you scale the data it just
message ID, you have different indicators, the subject, sender, gets harder and longer and more difficult.” P04
the recipients. Then you have another thing that says the file Regardless of the specific cause of low data quality or
attachments are this big. You have all these logs, and they all visibility issues, all participants emphasized that missing data
intermingled, so you need to do a lot of work on the data just or data gaps create blind spots, making it difficult to identify
to get one line of the email. That’s a challenge because you threats (from benign events) or determine the duration of an
need to be like a data scientist and programmer just to get the ongoing attack.
information you need before you even do your analysis as a
security person, so that’s definitely a challenge.” Data overload. Participants(8) also reported that the sheer
volume of data they have to deal with can sometimes pose
Incomplete and Low-Quality Data. Participants (n=11) challenges. They explained that going through all the data to
also highlighted the significant challenges they face when identify threats takes time and is complex. Some highlighted
dealing with incomplete or low-quality data; data lacking that it can be an overwhelming experience which may affect
critical points or information required for threat-hunting in- the hunter’s ability to spot obvious indicators of compromise.
vestigations. They said data full of noise or data missing some Another challenge mentioned concerning the abundance of

3322 33rd USENIX Security Symposium USENIX Association

 data was querying. Participants explained that having too that they need to provide training to every new hunter to en-
much data makes querying difficult and time-consuming. We sure they possess the right skills for the job, but this process
found that failing to promptly and effectively retrieve the nec- takes time and requires resources.“Our main challenge is
essary data slowed their efforts and made hunting challenging. that not a lot of IT people we encounter are well-versed in
“To make sense of the data, we have a lot of it and to get some- security and incident management, so we have to give them
thing interesting from it is a big challenge. You end up with step-by-step instructions on what to do, and it takes time, and
loads of results that are useless in the data set. It’s hard to even if you do, sometimes they don’t understand. ‘Oh, what
filter out new threats. It’s very challenging and it takes time is happening? Why do we need to do this?’ We have to let
to adjust the query to get the right data.” P15 them understand it, otherwise they are going to make mistakes”
P08
Limited data storage. Some (n=7) face challenges around In addition to skillsets, attracting and retaining skilled per-
data storage. They explained that they usually have limited sonnel emerged as another problem. Participants described
storage capacity, which means they sometimes do not have the difficulty in finding individuals with skillsets that strike
sufficient data to conduct thorough investigations. Participants the right balance between broad enough for various threats
also explained that the issue of data storage is challenging and specific enough for effective threat hunting. Furthermore,
because it is also hard to know how much data is needed retaining staff posed challenges as threat hunting can be de-
to be stored just in case it needs to be used for investiga- manding, making it crucial to maintain long-term commit-
tions. The participants who conducted external threat hunting ment from employees. P02 elaborated on the staffing chal-
stated that most of their clients usually do not keep much lenges, and said; “Yeah. I will start with the people. People is
data because they do not have the infrastructure to do so. finding people and retaining people especially with Level 1s
They explained that this leads to challenges when investigat- because they work on shifts, they work at night, then when you
ing threats or understanding how long the systems have been work online looking at the logs, again it’s a bit big. So, make
compromised.“Acquiring the data is the difficult part because sure they can see the end of the tunnel. We can keep them
they might not have storage for that stuff. It’s ridiculous. The interested and they can move on to the next level of analyst.”
security solutions that are out there are not made to store
data, they’re basically made to do things with it and then Communication. Participants (n=7) also reported that com-
remove it quickly. The problems I’ve seen, is that people don’t munication within teams, management and clients can some-
think about what happens when you need the forensics? What times be challenging. Some participants explained that some
happens when there’s compromise? You need 30 days of logs threat hunters have difficulty communicating threats or risks
or 60 days or 180” P22 they find during hunts. Poor communication or unclear report-
ing protocols (or channels) may lead to other team members
Take Away. Threat hunting relies extensively on the avail- underestimating the impact of the threats or the management
ability, usability, and quality of data. However, this crucial not understanding what is needed or how to respond. They
aspect presents a significant challenge. The difficulty arises emphasized the critical importance of communication, espe-
from (1) obtaining the required data for investigations due to cially since they work with various teams (e.g., legal), some of
poor logging (visibility) and storage practices within organiza- which may not possess the technical expertise to understand
tions, and (2) the complexity and overwhelming nature of the and assess cyber attacks. P03 highlighted the significance
data, making it challenging to analyze with the available tools of effective communication:“There’s no point having threat
and/or knowledge. intelligence if you can’t communicate it to someone in the
proper way, so someone who is very skilled at reports who
can create a lot of information. Communication is quite useful
People and Organizational-related Challenges These in- to the people that are working on the board and things like
clude issues around organizational culture and interpersonal that and who don’t understand cyber, presenting the threats
skills of individuals. Participants reported facing significant to them in a way that they can understand it, that’s useful.”
challenges in identifying and recruiting skilled staff, estab-
lishing effective communication channels, and managing con- Budget constraints and lack of resources. Participants
straints around budget and resources. (n=9) also emphasized the significant challenge posed by bud-
get constraints on security and threat-hunting efforts within
Skillsets and staffing. Participants (n=11) highlighted var- organizations. They stated that the lack of budget affected sev-
ious challenges related to their staffs’ necessary skillset for eral crucial aspects, including acquiring necessary resources,
effective hunting, including technical, communication, and availability of skilled workforce, dedicated teams, and time
analytical skills. They emphasized that threat hunting requires allocation. Some participants argued that this is due to the
more than just technical expertise; it demands a specialized management not understanding the importance of proactive
set of skills and critical thinking. As a result, they explained security response. Most participants shared that threat hunting

USENIX Association 33rd USENIX Security Symposium 3323

 and security are underfunded, making it difficult to acquire the Take Away. Threat hunting’s effectiveness is intricately tied
right equipment for storage or analysis, as it is often expensive. to the skills and experience of the threat hunters. However,
“Cost is one of the biggest issues because the technologies that finding and retaining qualified staff in this field presents a
companies need are expensive. I’ve been in threat hunting for significant challenge. Furthermore, threat hunting is frequently
almost 20 years one form or another as well as exploited all underfunded, exacerbating the challenges faced by security
kinds of stuff, we have to do the best job possible with a lack teams, like getting the right tools.
of information and a lack of technology at the client.” P05
Some participants also associated operational issues like
bad performance, time constraints, and lack of dedicated 4.4 Best Practices Strategies
threat-hunting teams with insufficient budgets. The shortage
To address the various challenges, participants shared several
of financial resources often forces threat hunters to perform
strategies they used and practices they perceived to be the
other duties unrelated to threat hunting because the company
most effective in improving threat hunting processes. We only
does not have enough security personnel. This affected their
discuss the most reported strategies.
threat identification performance, ability to learn and tune
their threat detecting rules, and response time to alerts and
investigations. “Another [challenge] is having time, being Strategy 1: Re-analysing, Re-tuning, and Collaborating.
able to dedicate – not just time, but continuous time, where Firstly, to address false alerts participants emphasized con-
you can continually tune the rules that we have, and actually ducting thorough analysis of the cases against historical data
go back and continually review the results of the rules that we to identify patterns or common characteristic of previous at-
have. So, I would say time is the biggest issue outside of data tacks. This method helped them distinguish genuine threats
collection. Time and dedicated man hours to threat hunting. from false positives more accurately. Secondly, some ex-
With respect to that, I guess having one person dedicated to plained they constantly fine-tune their approaches; refine de-
threat hunting would definitely be useful but I always say tection rules, use cases, and algorithms. Refining these ap-
there’s a saying, if you give me six hours to cut down a tree I proaches helps in reducing false positives. Participants also
will spend four hours sharpening my axe” P10 highlighted that relying only on rules for identifying threats
Furthermore, we found that budget constraints posed chal- might lead to false negatives, especially when dealing with
lenges in fostering a culture of innovation and continuous new and emerging threats (e.g., zero-days). To overcome this,
improvement among threat-hunting teams. Some participants they randomly conduct further analysis on logs that may have
were concerned that low or no budget could hinder analysts’ initially passed without triggering alerts. This active approach
exploration of new techniques and experimental approaches. helps them uncover potential threats that might have been
The lack of allocated time and flexibility for hunting duties af- missed by the existing rules or tools. Other participants em-
fected their ability to stay at the forefront of evolving threats phasized the importance of collaboration and shared learning
and identify potential security risks effectively.“The chal- to mitigate false negatives. They explained that when an IoC
lenge is that our organizations are telling the analysts don’t is missed, and later found, they come together to discuss and
be smart, don’t spend the time to learn new techniques, don’t understand the reasons behind the oversight. This collective
spend the time to experiment, don’t spend the time to try things effort helps them identify gaps in their detection capabili-
that might not work. That’s exactly what the organization is ties and implement necessary changes to avoid similar false
telling people when they take away the time allocation from negatives in the future.
hunting and flexibility. That’s the big pressure on us as to why
we’re not good at [threat] identification.” P09 Strategy 2: Automating Repetitive Tasks. Other partici-
Participants stated that budget constraints also their clients. pants reported that automating certain activities helps to scale
They said budget constraints affected what they could do or the search and improve its effectiveness. By streamlining
recommend for their clients. For instance, P04 explained that repetitive tasks, they can free some time and focus on other
their clients cannot collect all the necessary data they need strategic and analytical aspects of threat hunting. Moreover,
due to budget constraints. “I think the biggest challenge is they stated that automation also helps to ensure consistency
usually client budgets because you know it’s always kind of and accuracy around data collection and analysis. We also
[affect] the amount of data that goes to the SIEM [because] found that automation also help reduce the burden on threat
that’s how they get charged by EPS. The more data they send hunters; reduces the risk of errors that might result due to
in there, the higher their bill. You know budget constraints manual analysis of larger datasets.
seem to be the biggest barrier to collecting the data we really
need in order to do the kind of threat hunting that they would Strategy 3: Refining data collection strategies. To miti-
want us to do.” P04 gate issues around data collection, some participants reported
that they work on their data collection strategies, identify-
ing and collecting only info that is needed, and also using

3324 33rd USENIX Security Symposium USENIX Association

efficient methods of collecting and storing logs. Other partici- machine interaction in hunting. From our results, it is clear
pants explained that they encourage best practices around data that machines have a significant role to play in threat hunt-
collection and also by following best policies. For instance, ing, but alone, without humans, they are limited. For instance,
some argued for more centralization of data. They reasoned they may struggle to detect zero-day attacks or other new
that having all the data in one place improves the overall vis- and emerging attacks, which may require human involvement.
ibility of the data and the infrastructure being investigated. Our findings also suggest that threat hunting requires dynamic
While others stated that having centralised data was useful reasoning, such as intuition, creativity, and strategic thinking,
for use case generation; they explained that it made use case which cannot be fully replicated by tools alone. Moreover,
generation easier as one can clear visibility of what they have like hackers and testers [44], we found that experience plays
and what they do not have. a significant role in threat hunting. Some threats have been
discovered solely because of experience (Tacit knowledge).
Strategy 4: Being flexible and Open minded. Participants Also, some hunters have found threats due to their curiosity,
said one way of making threat hunting easier is not following personal creativity, and persistence. Future efforts could in-
a rigid and repetitive process. They explained that, as threats vestigate how these workers could collaborate to build morale,
constantly evolve, one needs to be creative and find new ways exchange ideas, and bring creativity to the field.
of hunting for threats. Being strict in approach or not adapting
the process could lead to missing new or emerging threats.
They explained that approaching threat hunting with an open In-House vs. Providing Services. Our sample represent
mind however helps to break routines and norms that develop various companies, with some conducting in-house hunting
within teams over time. With an open-minded approach, threat and others outsourcing their services. We observed signif-
hunting teams can promote an environment where fresh ideas icant similarities between both groups, particularly in how
and perspectives are valued which may eventually lead to new they apply each method. However, there were a few notable
novel techniques to threat hunting. differences (Table 3), for example, reporting. Participants ex-
pressed frustrations about reporting while providing services
Strategy 5: Keeping up with current threats. Some par- to other companies, highlighting unclear reporting lines than
ticipants emphasized that staying informed about the latest in-house which they believe affect the remediation of iden-
threats is crucial for enhancing threat hunting practices. They tified issues. Regarding challenges, though they face similar
recommended providing threat hunters with continuous learn- issues, the manifested differently. For example, both groups
ing opportunities to keep them updated on current threats faced challenges with data, but those offering services to out-
and how to identify and respond to them. Other participants side companies emphasized issues such as unusable data and
suggested that this could be achieved through training and missing data points. This discrepancy is perhaps unsurprising,
research efforts. While some participants had specific sug- as companies typically have more control over their in-house
gestions like having frequent training, others stressed on en- operations compared to external entities.
hancing the complexity of the training. In addition to training,
other participants suggested documenting and reporting new
threats as part of the learning and keeping up with new threats.
Lack of standardization. While we categorized threat hunt-
ing methods in our paper, in practice, they are borderless, and
Strategy 6: Asking for better budget allocation. Almost
not one size fits all. Hunters combine them for better out-
all the participants discussed budget in one way or another
put. While this flexibility may be beneficial in some cases, it
particularly how it can improve threat hunting activities. For
could also lead to undesirable practices and outcomes. For
example, acquiring tools and organizing and conducting train-
example, hunters may choose to prioritize certain activities
ings. Participants suggested that having adequate budget allo-
while neglecting others (e.g., skipping pre-planning), which
cation is critical to facilitate effective threat hunting activities
may be critical in the entire threat hunting process. There is
and enable teams to have access to necessary resources and
a need to standardize these methods to prevent the loss of
tools that can detect and mitigate threats.
critical processes and promoting best practices. Moreover, the
establishment of industry standards outlining how each threat
5 Discussion hunting approach should be executed may lead to greater con-
sistency, and minimization of errors. Furthermore, this could
Humans, Machines, and Collaborations. Our findings re- lead to the development of specific training materials to assist
veal that threat hunting is a multifaceted process that de- practitioners, such as detailed case studies that could be used
mands creativity, attentiveness, and the utilization of ap- worldwide. Future work should look into these methods and
propriate tools. These results align with existing litera- identify which ones are prevalent in the wild and the activities
ture [15, 36, 48, 50], emphasizing the significance of human- that should form the base of threat hunting framework.

USENIX Association 33rd USENIX Security Symposium 3325

Beyond Traditional SOC Boundaries. Beyond Traditional various security experts can perform threat-hunting and use
SOC Boundaries: Our findings indicate that threat hunting is different methods based on needs and resources. The chal-
expanding beyond traditional SOC boundaries. Some partici- lenges they encounter are not just technical but include orga-
pants reported that their threat hunting efforts began outside nizational challenges. Our results not only provide empirical
the SOC, while others mentioned that members of their threat evidence on threat hunters’ daily practices but also have the
hunting teams had additional responsibilities in their compa- potential to strengthen Cyber Defense Strategies and improve
nies and conducted threat hunting outside the SOC. Hunting decision-making in SOCs. Future studies could investigate
outside the SOC may have both positive and negative aspects. the unique challenges and opportunities for threat hunting in
In some cases, it may democratize threat hunting and engage specific areas, such as gaming.
everyone. However, it may also introduce additional privacy
and security risks or require extra measures to be put in place,
which might increase costs. Also, during the pandemic, there
7 Acknowledgement
were discussions about virtual SOCs (or metaverse SOCs), We thank all the security experts who took part in our inter-
which would enable practitioners to connect to SOCs remotely views, without their time and expertise, the publication of this
or integrate several SOCs virtually in real-time. As spatial paper would have not been possible. We would also like to
computing becomes more popular, virtual SOCs are feasi- thank the reviewers and shepherd for their valuable feedback
ble, and threat hunting outside physical SOCs might become to the paper. This work is supported in part by EPSRC CDT
common. We believe virtual threat hunting warrants further TIPS-at-Scale.
exploration to determine how it can be achieved securely.
Moreover, research should further investigate current prac-
tices of threat hunting outside SOC settings and how it can References
be made effective and secure.
[1] Enoch Agyepong, Yulia Cherdantseva, Philipp Reinecke,
and Pete Burnap. Towards a framework for measuring
Budget and Staffing. Similar to other works on security the performance of a security operations center analyst.
workers (e.g., [5, 16, 25]), recruiting and retaining skilled In Int. conference on cyber security and protection of
personnel is also a challenge in threat hunting. This is a mul- digital services, pages 1–8. IEEE, 2020.
tifaceted problem, involving low budgets, lack of motivation,
excessive responsibilities, and cognitive demands. We believe [2] Omer Akgul, Taha Eghtesad, Amit Elazari, Omprakash
there is no single solution to these issues, but management Gnawali, Jens Grossklags, Michelle L Mazurek, Daniel
should work on certain improvements. For example, removing Votipka, and Aron Laszka. Bug hunters’ perspectives on
additional non-hunting responsibilities, as they may indicate the challenges and benefits of the bug bounty ecosystem.
a lack of appreciation for proactive security measures. More- In 32nd USENIX Security Symposium., 2023.
over, this could maximize hunters’ time and effort. We also
acknowledge that hunters may take on other roles because [3] Olusola Akinrolabu, Ioannis Agrafiotis, and Arnau Erola.
they see no clear career progression in threat hunting. Conse- The challenge of detecting sophisticated attacks: In-
quently, there is a need for clear career paths in threat hunting, sights from soc analysts. In Proc. of the 13th Int. Conf on
so hunters can envision themselves in these roles for the long Availability, Reliability and Security, ARES 2018, NY,
term. Investing in threat hunting may also address some of NY, USA, 2018. Association for Computing Machinery.
these issues; for instance, providing training could ensure an [4] Bushra A. Alahmadi, Louise Axon, and Ivan Martinovic.
adequate number of personnel and boost team motivation, 99% false positives: A qualitative study of SOC analysts’
which is crucial in this field, as highlighted in previous stud- perspectives on security alarms. In 31st USENIX Secu-
ies [2, 44] on bug hunters. The research community should rity Symposium. USENIX Association, 2022.
also emphasize the importance of investing in proactive secu-
rity rather than relying solely on reactive measures, as this is [5] Noura Alomar, Primal Wijesekera, Edward Qiu, and
paramount to the success of threat-hunting initiatives. Serge Egelman. "You’ve got your nice list of bugs,
now what?" vulnerability discovery and management
processes in the wild. In Sixteenth Symposium on Us-
6 Conclusion able Privacy and Security (SOUPS), 2020.

In this work, we investigated threat hunters’ practices in the [6] Saed Alrabaee, Paria Shirani, Mourad Debbabi, and
wild, including who conducts the hunts, how they conduct Lingyu Wang. On the feasibility of malware authorship
them, the challenges they face, and the strategies they employ attribution. In Foundations and Practice of Security: 9th
to address these challenges and improve hunting processes. Int. Symposium, FPS 2016, Québec City, QC, Canada,
We found that threat-hunting activities are not standardized; pages 256–272. Springer, 2017.

3326 33rd USENIX Security Symposium USENIX Association

 [7] Abdulellah Alsaheel, Yuhong Nan, Shiqing Ma, Le Yu, //www.mandiant.com/resources/blog/evasive-
Gregory Walkup, Z Berkay Celik, Xiangyu Zhang, and attacker-leverages-solarwinds-supply-chain-
Dongyan Xu. Atlas: A sequence-based learning ap- compromises-with-sunburst-backdoor, 2020.
proach for attack investigation. In USENIX Security Accessed on May 2023.
Symposium, pages 3005–3022, 2021.
[18] Stefan Gast, Jonas Juffinger, Martin Schwarzl, Gururaj
[8] Louise Axon, Jassim Happa, Alastair Janse van Rens- Saileshwar, Andreas Kogler, Simone Franza, Markus
burg, Michael Goldsmith, and Sadie Creese. Sonifica- Köstl, and Daniel Gruss. Squip: Exploiting the sched-
tion to support the monitoring tasks of security opera- uler queue contention side channel. In 2023 IEEE Sym-
tions centres. IEEE Transactions on Dependable and posium on Security and Privacy (SP), pages 468–484.
Secure Computing, 18(3):1227–1244, 2021. IEEE Computer Society, 2022.
[9] Virginia Braun and Victoria Clarke. Using thematic anal- [19] Robert S Gutzwiller, Sunny Fugate, Benjamin D Sawyer,
ysis in psychology. Qualitative research in psychology, and PA Hancock. The human factors of cyber network
3(2):77–101, 2006. defense. In Proc. of the human factors and ergonomics
society annual meeting, number 1. SAGE publications
[10] Po-Chun Chen, Peng Liu, John Yen, and Tracy Mullen. Sage CA: Los Angeles, CA, 2015.
Experience-based cyber situation recognition using re-
laxable logic patterns. In IEEE Int. Multi-Disciplinary [20] Xinfeng Li1 Xiaoyu Ji, Chen Yan1 Chaohao Li, Yichen
Conference on Cognitive Methods in Situation Aware- Li, Zhenning Zhang, and Weyuan Xu. Learning nor-
ness and Decision Support, pages 243–250. IEEE, 2012. mality is enough: A software-based mitigation against
inaudible voice attacks. 2023.
[11] Peter Clay. A modern threat response framework. Net-
work Security, 2015(4):5–10, 2015. [21] Barbara Johnstone. Discourse analysis. John Wiley &
Sons, 2017.
[12] Stephanie de Smale, Rik van Dijk, Xander Bouwman,
Jeroen van der Ham, and Michel van Eeten. No one [22] Benjamin A Knott, Vincent F Mancuso, Kevin Bennett,
drinks from the firehose: How organizations filter and Victor Finomore, Michael McNeese, Jennifer A McK-
prioritize vulnerability information. In IEEE Symposium neely, and Maria Beecher. Human factors in cyber war-
on Security and Privacy (SP), 2023. fare: Alternative perspectives. In Proc. of the Human
Factors and Ergonomics Society Annual Meeting, vol-
[13] Constanze Dietrich, Katharina Krombholz, Kevin Bor- ume 57. SAGE Publications Sage CA: Los Angeles, CA,
golte, and Tobias Fiebig. Investigating system operators’ 2013.
perspective on security misconfigurations. In Proc. of
the 2018 ACM SIGSAC Conference on Computer and [23] Piergiorgio Ladisa, Henrik Plate, Matias Martinez,
Communications Security, pages 1272–1289, 2018. and Olivier Barais. Taxonomy of attacks on open-
source software supply chains. arXiv preprint
[14] Feng Dong, Liu Wang, Xu Nie, Fei Shao, Haoyu Wang, arXiv:2204.04008, 2022.
Ding Li, Xiapu Luo, and Xusheng Xiao. Distdet: A
cost-effective distributed cyber threat detection system. [24] J Richard Landis and Gary G Koch. The measurement
of observer agreement for categorical data. biometrics,
[15] Anita D’Amico, Laurin Buchanan, Drew Kirkpatrick, pages 159–174, 1977.
and Paul Walczak. Cyber operator perspectives on se-
curity visualization. In Advances in Human Factors [25] Chanel Macabante, Sherry Wei, and David Schuster. El-
in Cybersecurity: AHFE Int Conf on Human Factors ements of cyber-cognitive situation awareness in organi-
in Cybersecurity, Walt Disney World®, Florida, USA, zations. In Proc. of the Human Factors and Ergonomics
pages 69–81. Springer, 2016. Society Annual Meeting, volume 63, pages 1624–1628.
SAGE Publications Sage CA: Los Angeles, CA, 2019.
[16] Anita D’Amico and Kirsten Whitley. The real work of
computer network defense analysts: The analysis roles [26] Michele Marazzi, Flavien Solt, Patrick Jattke, Kubo
and processes that transform network data into secu- Takashi, and Kaveh Razavi. Rega: Scalable rowhammer
rity situation awareness. In VizSEC 2007: Proc. of the mitigation with refresh-generating activations. In 44rd
workshop on visualization for computer security, pages IEEE Symposium on Security and Privacy. IEEE, 2023.
19–37. Springer, 2008.
[27] Vasileios Mavroeidis and Audun Jøsang. Data-driven
[17] FireEye. Highly evasive attacker leverages so- threat hunting using sysmon. In Proceedings of the 2nd
larwinds supply chain to compromise multiple Int. conf on cryptography, security and privacy, pages
global victims with sunburst backdoor. https: 82–88, 2018.

USENIX Association 33rd USENIX Security Symposium 3327

[28] S Naveen, Rami Puzis, and Kumaresan Angappan. Deep S Raj Rajagopalan. Turning contradictions into innova-
learning for threat actor attribution from threat reports. tions or: How we learned to stop whining and improve
In 2020 4th Int. Conf on Computer, Communication and security operations. In Twelfth Symposium on Usable
Signal Processing (ICCCSP), pages 1–6. IEEE, 2020. Privacy and Security (SOUPS), pages 237–251, 2016.

[29] Lily Hay Newman. Russia’s fireeye hack is [39] Sathya Chandran Sundaramurthy, Michael Wesch, Xin-
a statement—but not a catastrophe. https: ming Ou, John McHugh, S Raj Rajagopalan, and Alexan-
//www.wired.com/story/russia-fireeye-hack- dru G Bardas. Humans are dynamic - our tools should
statement-not-catastrophe/, 2020. Accessed on be too. IEEE Internet Computing, 21(3):40–46, 2017.
April 2023.
[40] Joe Tidy. Solarwinds: Why the sunburst hack is so se-
[30] Lorelli S Nowell, Jill M Norris, Deborah E White, and rious. https://www.bbc.co.uk/news/technology-
Nancy J Moules. Thematic analysis: Striving to meet 55321643, 2020. Accessed on April 2023.
the trustworthiness criteria. Int. journal of qualitative
[41] Swaathi Vetrivel, Veerle Van Harten, Carlos H Gañán,
methods, 16(1):1609406917733847, 2017.
Michel Van Eeten, and Simon Parkin. Examining con-
[31] Antonio Pecchia, Domenico Cotroneo, Rajeshwari sumer reviews to understand security and privacy issues
Ganesan, and Santonu Sarkar. Filtering security alerts in the market of smart home devices. In 32nd USENIX
for the analysis of a production saas cloud. In IEEE 7th Security Symposium, 2023.
Int Conference on Utility and Cloud Computing. IEEE,
[42] Manfred Vielberth, Fabian Böhm, Ines Fichtinger, and
2014.
Günther Pernul. Security Operations Center: A System-
[32] Akalanka Perera, Shanith Rathnayaka, N. D. Perera, atic Study and Open Challenges. IEEE Access, 2020.
W.W. Madushanka, and Amila Nuwan Senarathne. The [43] Daniel Votipka, Seth Rabin, Kristopher Micinski, Jef-
next gen security operation center. In 2021 6th Int. Conf frey S Foster, and Michelle L Mazurek. An observa-
for Convergence in Technology (I2CT), pages 1–9, 2021. tional investigation of reverse {Engineers’} processes.
[33] Rami Puzis, Polina Zilberman, and Yuval Elovici. In 29th USENIX Security Symposium, 2020.
Athafi: Agile threat hunting and forensic investigation. [44] Daniel Votipka, Rock Stevens, Elissa Redmiles, Jeremy
arXiv preprint arXiv:2003.03663, 2020. Hu, and Michelle Mazurek. Hackers vs. testers: A com-
[34] Sagar Samtani, Ryan Chinn, Hsinchun Chen, and Jay F parison of software vulnerability discovery processes.
Nunamaker Jr. Exploring emerging hacker assets and In IEEE Symposium on Security and Privacy. IEEE,
key hackers for proactive cyber threat intelligence. Jour- 2018.
nal of Management Information Systems, 34(4):1023– [45] Maurice Weber, Xiaojun Xu, Bojan Karlaš, Ce Zhang,
1053, 2017. and Bo Li. Rab: Provable robustness against backdoor
attacks. arXiv preprint arXiv:2003.08904, 2020.
[35] Jordan Shropshire and Christopher Kadlec. I’m leaving
the it field: The impact of stress, job insecurity, and [46] Rodrigo Werlinger, Kasia Muldner, Kirstie Hawkey,
burnout on it professionals. Int. Journal of Information and Konstantin Beznosov. Preparation, detection, and
and Communication Technology Research, 2(1), 2012. analysis: the diagnostic work of it security incident re-
sponse. Information Management & Computer Security,
[36] Xiaokui Shu, Frederico Araujo, Douglas L Schales,
18(1):26–42, 2010.
Marc Ph Stoecklin, Jiyong Jang, Heqing Huang, and
Josyula R Rao. Threat intelligence computing. In Proc. [47] Chong Xiang, Alexander Valtchanov, Saeed Mahloujifar,
of the 2018 ACM SIGSAC conference on computer and and Prateek Mittal. Objectseeker: Certifiably robust
communications security, pages 1883–1898, 2018. object detection against patch hiding attacks via patch-
agnostic masking. 2022.
[37] Sathya Chandran Sundaramurthy, Alexandru G Bar-
das, Jacob Case, Xinming Ou, Michael Wesch, John [48] Heng Yin, Dawn Song, Manuel Egele, Christopher
McHugh, and S Raj Rajagopalan. A human capi- Kruegel, and Engin Kirda. Panorama: capturing system-
tal model for mitigating security analyst burnout. In wide information flow for malware detection and anal-
Eleventh Symposium On Usable Privacy and Security ysis. In Proc. of the 14th ACM conf on Computer and
(SOUPS), pages 347–359, 2015. communications security, pages 116–127, 2007.

[38] Sathya Chandran Sundaramurthy, John McHugh, Xin- [49] Kim Zetter. The untold story of the boldest supply-chain
ming Ou, Michael Wesch, Alexandru G Bardas, and hack ever. https://www.wired.com/story/the-

3328 33rd USENIX Security Symposium USENIX Association

 untold-story-of-solarwinds-the-boldest- Aim: To identify and understand the data collection process, various sources of logs
and address the challenges linked with the collected data.
supply-chain-hack-ever/, 2024. Accessed on May
2024. Q1. How do you do data collection?
Follow up: Can you tell us about the types of information you collect?
[50] Chen Zhong, John Yen, Peng Liu, and Robert F Er- Follow up: How often do you collect logs for analysis?
Q2. How do you get your logging information ? (event logs/ Cyber Threat Intelligence)
bacher. Learning from experts’ experience: toward Follow up: What are the security devices or methods you use to collect logs? (Source of
automated cyber security data triage. IEEE Systems the logs: Where does the data come from?)
Q3. After collecting all your logs, what do you do next?
Journal, 13(1):603–614, 2018. Follow up: Do you use any parser or tools for the analysis?
Follow up: How much data do you keep and for how long?
Q4. How do you do log aggregation?
Q5. What kind of challenges do you normally face during data collection?
8 Appendix Q6. How do you think the challenges can be mitigated?
Follow-up: Any suggestion for improvement of the current approach?

8.1 Participants demographics details showing Threat Identification

their skills and certifications Aim: Gain insight into the practical aspects of threat hunting and identify challenges
when extracting threats from extensive event logs. Additionally, explore how analysts
cope with uncovering advance threats while encountering the challenges in the data
collection process.
Table 2: Participants’ skills and certifications
ID Job Role Certification(s)
Q1. What is your process of hunting the threats/IoCs out of the logs?
Follow-up: What procedure you follow for threat hunting? Is this process automated or
P01 Senior Cybersecurity Analyst - manual, or reported?
P02 Security Consultant CISSP, CISM, CCSP, GICSP,Fortinet NSE7 Firewall, Follow-up: What sort of tools and techniques do you use?
NSE 7 OT, NSE7 SD WAN. Fortinet NSE5 FortiAna-
lyzer, NSE5 FortiManager, Fortinet NSE4
Follow-up: Do you use any specific tool or technique to identify thyreats?
P03 Threat Intelligence Analyst - Q2.How do you prioritise threat and threat validation?
P04 Associate Director Threat Hunt GSIF, GCIH, GCFE, GREM, GMON, GSEC, GPEN Q3. How is the severity of incidents defined?
P05 Threat Hunting Team Lead - Follow-up: Who defines the severity of incidents?
P06 Digital Forensics Specialist -
P07 SOC Analyst -
Q4. What kind of challenges do you face during hunting process?
P08 Director for DFIR CISA, CISM, CISSP, GSE, GIAC x12 Q5. Threat hunting sometimes includes false alarms, do you also face such false
P09 Consultant CISSP, GSEC, GCIA, GCIH, GCFA, GPEN, GMOB, positive alerts?
GPYC, GASF, GXPN, GREM Follow-up: How do you deal with false positive?
P10 Lead SOC Threat Hunter SANS (GDAT, GREM), CCNA,Security+, CEH
P11 IT Security Engineer MTA Sec Fundamentals, Comptia Security+, Comptia
Follow-up: What do you do in case false negative?
CySA+,GIAC, GCIH Q6. Based on your experience, What can be done to improve the process?
P12 SOC Analyst -
P13
P14
Security Analyst L3 -
Program Lead Adv Sec Analytics
-
GPYC (Python Coder), AWS Security
Analysis of Threats:
P15 Threat analyst High school -
P16 Cybersecurity Technical Specialist - Aim: Now that you have cultivated a good amount of evidence indicating anomalies,
P17 SOC Head - the next step would be, analysis of threats or anomalies. We want to understand the
P18 Security Research Lead - challenges in the threat analysis part.
P19 Cybersecurity Engineer -
P20 Lead Cybersecurity Engineer – SOC and GCIH Note: The person identifying threats also analyzing the threats?
Blue Team
P21 Manager, Incident Handling - Q1. Once IoCs are identified, how do you analyse the threats?
P22 Senior Incident Response Consultant -
Prompt: Is this automated or manually?
Follow-up: What challenges do you face while analysing threats?
Follow-up: Do you have labs or simulation environments to test out the particular
executable behavior?
Follow-up: What challenges do you face when testing or simulating these behaviours?
Q2. How do you resolve the identified incidents?
8.2 In-house Vs Outsourcing services Prompt: Once the incident source has been identified, what are actions taken to resolve
it?
8.3 Interview Protocol Follow-up: Is this process based on severity measurements?

Q1. Please tell us about job role/position/level Suggestion/Scope of improvement:

Q2. Please tell us about your work responsibilities within the company?
Prompt: What do you have to do in your day-to-day job role? Q1. In general, as a person who works in SOCs or deal with identifying threats, what
Follow-up: Are you working in a team or individually? kind of challenges in terms of people, process and technology?
Q3. Could you please tell me about your approach of threat hunting? Prompt: How do Follow-up: the processes involved in threat hunting?
you perform threat hunting? Follow-up: the technologies used for threat hunting?
Q2.What would you like to change in this whole threat hunting process?
Study Intro: This study is divided into three parts: Q3.What tools and technologies do you think can benefit threat hunting process?
1) Data collection 2) Threat Identification 3) Threat Analysis

Data\Log Collection 8.4 Codebook

USENIX Association 33rd USENIX Security Symposium 3329

 Table 3: In-house Vs Outsourcing services
Threat Hunting Process In-house Outsourcing services

Have deeper knowledge of the network, systems, and data being collected. Cannot only rely on the client’s in-house team, they must learn the systems themselves from
scratch
Have defined communication line so they know who to communicate with during planning about data and Communication is usually challenging because sometimes it is not clear who oversees what.
devices.
Pre-hunt Plan Flexibility to adapt and refine planning Services are for limited time, and they must follow standardize process.
Priority issues (analysts may have other work responsibilities) No priority issues, as this is a dedicated team
Other local teams and management usually understand scope of the threat hunting Keeping everyone on the same page as sometime client have different expectations.
Data Collection and Prepara- Have direct access to data/logs makes the process easy Limited data access and availability because sometimes the logs are not even collected or have
tion been collected but deleted.
They are usually part of the team that decides which logs needs to be collected and retention period. Lots of time can be spent on trying to understand logs. Data sometimes is missing
Hunting and Validating Misunderstanding threats may lead to false negative alerts Limited access to data and lack of data source results in false negative
Due to direct access to internal team can promptly respond to threats and start remediation actions Must wait and be advised on how to respond to threats.
Remediation and Can communicate directly with other teams during remediation Communication can be challenging which may delay the threat response process
Reporting Reporting channels are clear Reporting can be cumbersome

Table 4: Codebook
Theme Code Subcode Description
Use case-bases Hunting Participants describing a process of threat hunting that involves building cases or scenarios on
possible attack patterns, tactics, techniques, and behaviors that adversaries might employ.
Threat Hunting Intel-based Hunting Participants explaining they leveraged threat intelligence feeds, open-source intelligence, or vendor-
Method based intelligence for threat hunting.
Random-based Hunting Participants describing a method of hunting that is random, possible based on suspicion or knowl-
edge from past events.
Threat Hunting Pre-hunt Plan Participants describing a process in which they carefully plan for the hunting process. This includes
Practices defining the objectives, scope, choosing network or system logs to investigate, and determining
the tools and techniques and who will carry out the search.
Threat Hunting Data Collection and Prepara- Participants describing an approach for collecting data from various sources such as network, tools,
Process tion and other security devices deployed to collect data within the organizations.
Hunting and Validating Participants describing the hunting process and the validation process they follow to identify and
verify threats.
Remediation and Reporting Participants explaining the process they follow when they have discovered and confirmed a threat
or indicator of compromise including how they report it and to whom.
Method- False Alerts Participants explaining sometimes they receive potential threat alerts that turn out to be false or
related Chal- benign.
lenges
Building Usecases - Complex Participants explaining that building use cases requires deep understanding of potential attacks
TTPs tactic and technique, also highlighting that this is not easy.
Evolving tactics and tech- Participants explaining that because threat actors continually adapt and innovate their approaches
niques to bypass security measures and avoid detection, then threat hunting is more challenging.
System & Tools failing Participants explaining that systems and tools sometimes fail to provide the services required.
Complex Data Participants explaining that data being extracted can come in diverse formats, sometimes even
challenging to understand.
Threat Hunting Data-related Incomplete & Low-Quality Participants explaining that data is sometimes incomplete or of low quality leading to inaccurate
Challenges Challenges Data assessments or provide insufficient insights on potential threats
Data Overload Participants explaining that identifying threats from large volume of data is complicated and
challenging.
Limited Data Storage Participants explaining that their challenge is having limited storage which affects their threat
hunting efforts.
Organization & Skillsets & Staffing Participants discussing challenges around finding people with the right skills to perform threat
People related hunting.
Challenges
Communication Participants explaining that communication is one of their challenges in threat hunting.
Budget Constrains & Lack of Participants explaining the constraints that they have in threat hunting due to lack of budget and
Resources having the right resources.
Re-analysing, Re-tuning, and Participants describing how they address false alerts.
Collaborating
Automating Repetitive Tasks Participants explaining how they use automation to address some of the challenges they face.
Threat Hunting Best Practices Refining data collection strate- Participants explaining that they continuously refine their data collection strategies to have useful
Strategies Strategies gies data for threat hunting.
Being flexible and Open Participants explaining that they approach threat hunting with an open mind to ease the tasks.
minded
Keeping up with current threats Participants describing the importance of continuous learning to keep up with threats.
Asking for better budget alloca- Participants explaining how better budget would solve majority of their challenges.
tion

3330 33rd USENIX Security Symposium USENIX Association