Splunk 20230910-Wa0000
Splunk 20230910-Wa0000
COM
mail us: [email protected]
SPLUNK
SIEM ADMINISTRATOR AND
SECURITY
At its core, SIEM is a data aggregator, search, and reporting system. SIEM gathers immense amounts
of data from your entire networked environment, consolidates and makes that data human accessible.
With the data categorized and laid out at your fingertips, you can research data security breaches
with as much detail as needed.
There are various SIEM tools are available in the Market like -
Arcsight
Splunk
Qradar
McAfee etc.
Depend on the pros and cons of any SIEM tool and how it fits to our enterprise network we select the
tool
Like Arcsight is very good in correlation and alerting while Splunk is very good in Analytics and
visualization while Qradar is good from simplicity perspective
Modern SOC :
Security Monitoring:
Splunk History
Splunk Inc. is an American public multinational corporation based in San Francisco, California
Michael Baum, Rob Das and Erik Swan co-founded Splunk Inc in 2003
What is Splunk?
Splunk is an advanced, scalable, and effective technology that indexes and searches log files stored in
a system. It analyses the machine-generated data to provide operational intelligence. The main
advantage of using Splunk is that it does not need any database to store its data, as it extensively
makes use of its indexes to store the data
Splunk is not only an SIEM tool but it can work more as an analytical tool and can receive the data
from any data source
To use Splunk as an SIEM tool we have to deploy Splunk Enterprise Security app over the Splunk
Splunk Products
Splunk is available in three different versions.
Splunk Enterprise
Splunk Light
Splunk Cloud
Splunk Enterprise
Splunk Enterprise edition is used by large IT business. It helps you to gather and
analyse the data from applications, websites, network devices, security devices and
many more
Splunk Cloud
Splunk Cloud is a hosted platform. It has the same features as the enterprise
version. It can be availed from Splunk or using AWS cloud platform.
Splunk Light
Splunk Light is a free version. It allows search, report and alter your log data. It has
limited functionalities and feature compared to other versions.
Splunk Architecture
Splunk Forwarder
Splunk Forwarder is the component which you have to use for collecting the logs. Suppose,
you want to collect logs from a remote machine, then you can accomplish that by using
Splunk’s remote forwarders which are independent of the main Splunk instance.
In fact, you can install several such forwarders in multiple machines, which will forward the
log data to a Splunk Indexer for processing and storage. What if you want to do real-time
analysis of the data? Splunk forwarders can be used for that purpose too. You can configure
the forwarders to send data to Splunk indexers in real-time. You can install them in multiple
systems and collect the data simultaneously from different machines in real time.
Compared to other traditional monitoring tools, Splunk Forwarder consumes very less cpu
~1-2%. You can scale them up to tens of thousands of remote systems easily, and collect
terabytes of data with minimal impact on performance.
Universal Forwarder – You can opt for a universal forwarder if you want to forward the
raw data collected at the source. It is a simple component which performs minimal
processing on the incoming data streams before forwarding them to an indexer.
Data transfer is a major problem with almost every tool in the market. Since there is minimal
processing on the data before it is forwarded, lot of unnecessary data is also forwarded to
the indexer resulting in performance overheads.
Why go through the trouble of transferring all the data to the Indexers and then filter out
only the relevant data? Wouldn’t it be better to only send the relevant data to the Indexer
and save on bandwidth, time and money? This can be solved by using Heavy forwarders
which I have explained below.
Heavy Forwarder – You can use a Heavy forwarder and eliminate half your problems, because
one level of data processing happens antnt the source itself before forwarding data to the indexer.
Heavy Forwarder typically does parsing and indexing at the source and also intelligently routes the
data to the Indexer saving on bandwidth and storage space. So when a heavy forwarder parses the
data, the indexer only needs to handle the indexing segment.
Splunk Indexer: Indexer is the Splunk component which you will have to use for
indexing and storing the data coming from the forwarder. Splunk instance transforms the incoming
data into events and stores it in indexes for performing search operations efficiently
If you are receiving the data from a Universal forwarder, then the indexer will first parse the
data and then index it. Parsing of data is done to eliminate the unwanted data. But, if you are
receiving the data from a Heavy forwarder, the indexer will only index the data.
As the Splunk instance indexes your data, it creates a number of files. These files contain one
of the below:
• Raw data in compressed form
• Indexes that point to raw data (index files, also referred to as tsidx files), plus some
metadata files
Look at the above image to understand the end to end working of Splunk. The images shows
a few remote Forwarders that send the data to the Indexers. Based on the data present in
the Indexer, you can use the Search Head to perform functions like searching, analysing,
visualizing and creating knowledge objects for Operational Intelligence.
The Management Console Host acts as a centralized configuration manager responsible for
distributing configurations, app updates and content updates to the Deployment Clients. The
Deployment Clients are Forwarders, Indexers and Search Heads.
Indexing volume. How much data are planning to index on a daily basis? To handle
increased indexing loads, you might need multiple indexers.
Number and type of searches. How frequently will you be running searches, either
scheduled or ad hoc? What type of searches will you be running? Large numbers of
searches, or frequent process-intensive searches, can tax both search head and indexer
resources.
Number of concurrent users. How many users will be viewing dashboards or running
searches concurrently? To handle increased numbers of users, you might need to add
search heads, usually through a search head cluster.
Data fidelity requirements. If you must ensure that the system never loses data,
an indexer cluster is a necessity.
Availability requirements. What requirements do you have for data availability? If you
must always have access to the full set of data, you might need to deploy both an indexer
cluster and a search head cluster.
# of forwarders Median < 10; Median in the Median in the 10's; Median in the
maximum 100 10's; maximum maximum in the 10's; maximum in
in the 100's low 1000's the 1000's
# of users Median < 10 Median in the Median in the 10's; Median in the
10's maximum in the 10's; maximum
low 100's 500+
Path to discovery
To determine your deployment topology, you must identify the components and their relationships.
Discovery involves these steps:
Determine which machines contain instances of your deployment. Although it is possible for a
single machine to host multiple instances, such a configuration is unusual except in test
environments. In production environments, each Splunk Enterprise instance usually resides on
its own machine.
2. Identify your components.
For each instance, identify the components that it hosts. Components define the roles that the
instances play in the deployment. A single instance can host multiple components.
3. Identify the relationships between components.
All components run on Splunk Enterprise instances, except for the universal forwarder. The universal
forwarder is a lightweight version of Splunk Enterprise with its own executable.
Splunk Enterprise instances usually reside on dedicated machines, as a best practice. However, you
might discover an instance running on a machine that is also performing some entirely different
function.
Universal forwarder instances usually reside on machines that host other applications, such as web
servers. The forwarders ingest data produced by those applications.
A single machine can host multiple instances, although the best practice is for each instance to reside
on its own machine.
The absence of Splunk Web, the Splunk graphical user interface, is not a reliable indicator that the
machine does not host a Splunk Enterprise instance. On most deployments, only a subset of Splunk
Enterprise instances, such as search heads and some management components, have a running web
interface.
You can identify machines hosting Splunk Enterprise and universal forwarder instances by looking for the
presence of Splunk subdirectories on the machines' file systems.
Splunk documentation refers to the base directory for the Splunk file system as $SPLUNK_HOME .
Instances typically reside under these locations on a file system:
Caution: This table shows default or typical locations for $SPLUNK_HOME . However, the installation process
permits the user to install to any location and to change the name of the base directory.
Operating system Locations for Splunk Enterprise $SPLUNK_HOME Locations for universal forwarder
$SPLUNK_HOME
C:\Splunk C:\SplunkUniversalForwarder
C:\SPL
Linux /opt/splunk /opt/splunkforwarder
Solaris
AIX
HP-UX
FreeBSD
Mac OS X /Applications/splunk /Applications/splunkforwarder
Therefore, if you cannot immediately identify $SPLUNK_HOME , look for a directory that contains a set of
Splunk subdirectories. These subdirectories include bin, etc, include, lib, openssl, share , and var.
You can also verify that the machine hosts $SPLUNK_HOME by looking for a bin subdirectory that contains
the splunk, splunkd , and btool executables, among others. The parent of that bin subdirectory
is $SPLUNK_HOME .
Once you identify a machine with an installed instance, confirm that the instance is currently running. Use
a system tool such as ps or Task Manager to look for the splunkd process.
Monitoring console
Deployment server
License master
Indexer cluster mas+ter
Search head cluster deployed
X: Bold X in a box that intersects the computing platform and Splunk software type you want means that
Splunk software is available for that platform and type.
A: The software for this platform is available for download from splunk.com, but there is no official
support for the platform.
D: Splunk supports this platform and architecture but might remove support in a future release.
Version History
Splunk Enterprise / Splunk Analytics for Hadoop / Splunk Light*
**********Installation Of Splunk*********
When you set up and begin using your Splunk Enterprise installation or upgrade, perform some additional
steps to ensure that Splunk Enterprise and your data are secure. Taking the proper steps to secure Splunk
Enterprise reduces its attack surface and mitigates the risk and impact of most vulnerabilities.
This section highlights some of the ways that you can secure Splunk Enterprise before, during, and after
installation.
Before you install Splunk Enterprise, make your operating system secure. Harden all Splunk Enterprise
server operating systems.
The user you choose depends on what you want Splunk Enterprise to monitor
The user that Splunk Enterprise runs as determines what Splunk Enterprise can monitor. The Local System
user has access to all data on the local machine by default, but nothing else. A user other than Local
System has access to whatever data you want, but you must give the user that access before you install
Splunk Enterprise.
To do any of the following actions with Splunk Enterprise, you must install it as a domain user:
Be a member of the Active Directory domain or forest that you want to monitor (when using AD)
Be a member of the local Administrators group on the server on which you install Splunk Enterprise
Be assigned specific user security rights
If you are not sure which user Splunk Enterprise should run as, then see Considerations for deciding how
to monitor remote Windows data in the Getting Data In manual for information on how to configure the
Splunk Enterprise user with the access it needs.
Note: The Splunk Enterprise indexing subsystem requires high disk throughput. Any software with a
device driver that intermediates between Splunk Enterprise and the operating system can restrict
processing power available to Splunk Enterprise, causing slowness and even an unresponsive system. This
includes anti-virus software.
You must configure such software to avoid on-access scanning of Splunk Enterprise installation directories
and processes before you start a Splunk installation.
3. To continue the installation, check the "Check this box to accept the License Agreement"
checkbox. This activates the "Customize Installation" and "Next" buttons.
4. (Optional) If you want to view the license agreement, click View License Agreement.
Installation Options
The Windows installer gives you two choices: Install with the default installation settings, or configure all
settings prior to installing.
When you choose to install with the default settings, the installer does the following:
Installs Splunk Enterprise in \Program Files\Splunk on the drive that booted your Windows machine.
Installs Splunk Enterprise with the default management and Web network ports.
Configures Splunk Enterprise to run as the Local System user.
Prompts you to create a Splunk administrator password. You must do this before installation can
continue.
Creates a Start Menu shortcut for the software.
If you want to change any of these default installation settings, click Customize Options and proceed with
the instructions in "Customize Options" in this topic.
Otherwise, click Next. You will be prompted for a password for the Splunk admin user. After you supply a
password, installation begins and you can continue with the "Complete the install" instructions later in this
topic.
By default, the installer puts Splunk Enterprise into \Program Files\Splunk on the system drive. This
documentation set refers to the Splunk Enterprise installation directory
as $SPLUNK_HOME or %SPLUNK_HOME% .
Splunk Enterprise installs and runs two Windows services, splunkd and splunkweb . The splunkd service
handles all Splunk Enterprise operations, and the splunkweb service installs to run only in legacy mode.
These services install and run as the user you specify on the "Choose the user Splunk Enterprise should run
as" panel. You can choose to run Splunk Enterprise as the Local System user, or another user.
When the installer asks you the user that you want to install Splunk Enterprise as, you must specify the
user name in domain\username format. The user must be a valid user in your security context, and must be
an active member of an Active Directory domain. Splunk Enterprise must run under either the Local
System account or a valid user account with a valid password and local administrator privileges. Failure to
include the domain name with the user will cause the installation to fail.
1. Click Change… to specify a different location to install Splunk Enterprise, or click Next to accept
the default value. The installer displays the "Choose the user Splunk Enterprise should run as"
panel.
4. Enter the Windows credentials that Splunk Enterprise uses to run on the machine and
click Next.
These credentials are different from the Splunk administrator credentials that you create in the
next step.
5. Create credentials for the Splunk administrator user by entering a username and password that
meets the minimum eligibility requirements as shown in the panel and click Next.
You must perform this action as the installation cannot proceed without your completing it. If
you do not enter a username, the installer creates the admin user during the installation
process.
If you specified the wrong user during the installation procedure, you will see two pop-up error windows
explaining this. If this occurs, Splunk Enterprise installs itself as the Local System user by default. Splunk
Enterprise does not start automatically in this situation. You can proceed through the final panel of the
installation, but uncheck the "Launch browser with Splunk" checkbox to prevent your browser from
launching. Then, use these instructions to switch to the correct user before starting Splunk.
1. (Optional) Check the boxes to Launch browser with Splunk and Create Start Menu Shortcut.
2. Click Finish. The installation completes, Splunk Enterprise starts and launches in a supported
browser if you checked the appropriate box
Now to access the Splunk from browser type https://localhost:8000 if you are access the Splunk from any
other system enter the IP address and port number on the browser, let say if Splunk IP is 192.168.0.24
then you can access the Splunk by typing https://192.168.024:8000