Splunk Fundamentals 2

Instructor on Demand
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
1 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Course Prerequisites
• To be successful in this course, you should have Note
In order to receive credit for this
completed: course, you must complete all lab
exercises.
– Splunk Fundamentals Part 1

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
2 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Course Guidelines
• Hands-on lab exercises reinforce information presented in the
lecture modules
• To receive a completion for the course, you must complete the lab
exercises
• The lab exercises must be completed sequentially
– Laterlab exercises often depend on steps completed in previous lab
exercises

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
3 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Course Goals
• Use transforming commands and visualizations
• Filter and format the results of a search
• Correlate events into transactions
• Create and manage Knowledge Objects
• Create & manage extracted fields, field aliases, calculated fields
• Create tags and event types
• Create and use macros and workflow objects
• Create and manage data models
• Use the Splunk Common Information Model (CIM)
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
4 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Course Outline
Module 1: Course Introduction
Module 2: Beyond Search Fundamentals
Module 3: Commands for Visualizations
Module 4: Advanced Visualizations
Module 5: Filtering and Formatting Data
Module 6: Correlating Events
Module 7: Introduction to Knowledge Objects
Module 8: Creating and Managing Fields
Module 9: Creating Field Aliases and Calculated Fields
Module 10: Creating Tags and Event Types
Module 11: Creating and Using Macros
Module 12: Creating and Using Workflow Actions
Module 13: Creating Data Models
Module 14: Using the Common Information Model (CIM) Add-On
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
5 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Course Scenario
• Use cases in this course are based on Buttercup Games, a
gaming company
• Searches and reports are based on:
– Business analytics from the web access logs and lookups
– Internal operations information from mail and internal network data
– Security operations information from internal network and badge
reader data

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
6 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Buttercup Games, Inc.
• Multinational company with HQ in
San Francisco and offices in
Boston and London
• Sells product mainly through its
worldwide chain of third party
stores, but also sells through its
online store

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
7 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Your Role at Buttercup Games
• You are a Splunk power user with a great understanding of all your
company’s data
• Your responsibilities are to provide information to users throughout
the company and to create and manage Splunk knowledge objects
for your stakeholders
• You implement best practices for naming conventions of all
knowledge objects
• You gather data and statistics, and report on Security, IT
Operations, Operational Intelligence, etc.

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
8 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
 Buttercup Games Network
Index Description Sourcetype Host
web Online transactions access_combined www1
www2
www3
security Badge reader data history_access badgesv1
AD/DNS data winauthentication_security adldapsv1
Web login data linux_secure www1
www2
www3
sales Retail sales data vendor_sales vendorUS1
BI data sales_entries ecommsv1
network Firewall data cisco_firewall cisco_router1
Email data cisco_esa
Web appliance data cisco_wsa_squid
games Game logs SimCubeBeta sim_cube_server

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
9 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
 Module 2:
Beyond Search
Fundamentals

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
10 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Module Objectives
• Review basic search commands
• Use case correctly in searches
• Describe Splunk’s search process

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
11 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Basic Search Review
• Keywords
For example, search for a single word (e.g., error) or group of words (e.g., error password)
• Booleans
NOT, OR, AND; AND is implied; MUST be uppercase; can use ( )’s to force precedence
sourcetype=vendor_sales OR (sourcetype=access_combined action=purchase)
• Phrases
"web error" (different than web AND error)
• Field searches
status=404, user=admin
• Wildcard (*)
– status=40* matches 40, 40a, 404, etc.
– Starting keywords with a wildcard is very inefficient, e.g. *dmin
• Comparisons
=, !=, <, <=, >=, > status>399, user!=admin
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
12 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Basic Search Review (cont.)
• table: returns table containing only specified fields in result set
• rename: renames a field in results
• fields: includes or excludes specified fields
• dedup: removes duplicates from results
• sort: sorts results by specified field
• lookup: adds field values from external source (e.g., csv files)

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
13 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
 Case Sensitivity – Sensitive
Case sensitive Examples
AND, OR, NOT (Boolean operators)
Boolean operators (uppercase)
and, or, not (literal keywords)
productId vs. productid
Field names
eval cs_username = "Total Access"
Field values from lookup product_name="Tulip Bouquet" vs.
(default, but configurable) product_name="tulip bouquet"
Regular expressions \d\d\d vs. \D\D\D
eval action=if(action=="view",...)
eval and where commands where action="Purchase"
stats count(eval(action="view") as…
Tags tag=DMZ vs. tag=dmz

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
14 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Case Sensitivity – Insensitive
Case insensitive Examples
Command names STATS, stats, sTaTs

AS used by stats, rename, ...;

Command clauses BY used by stats, chart, top, ...;
WITH used by replace

Search terms failed, FAILED, Failed

Statistical functions avg, AVG, Avg used by stats, chart, …

host=www1, host=WWW1
Field values
(unless coming from a lookup)

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
15 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
How Splunk Searches – Buckets
• As events come in, Splunk places them into an index's hot bucket
(only writable bucket)
• As buckets age, they roll from the hot to warm to cold
• Each bucket has its own raw data, metadata, and index files
• Metadata files track source, sourcetype, and host
• Admins can add more

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
16 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
How Splunk Searches – Searching
Hot: Now to -3h index raw events
• When you search, Splunk uses the
Hot: -3 to -6h index raw events
time range to choose which buckets
Hot: -5 to -8h index raw events
to search and then uses the bucket
Warm: -9 to -12h index raw events
indexes to find qualifying events
Warm: -12 to -15h index raw events
• When you search for Warm: -14 to -17h index raw events
index=web password fail* ..... index raw events
during the last 24 hours: Warm: -42 to -45h index raw events
– Splunk identifies the buckets for the Warm: -45 to -48h index raw events
last 24 hours Cold: -48 to -51h index raw events
 And searches the indexes of those Cold: -51 to -54h index raw events
buckets for the search terms Cold: -54 to -57h... index raw events
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
17 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
General Search Practices
• As events are stored by time, time is the most efficient filter
• After time, most powerful keywords are host, source, sourcetype
• To make searches more efficient, include as many terms as
possible
– e.g., searching for sourcetype=x failure is better than failure
• Use the fields command to extract (discover) only fields you need
• Example: Search last 365 days, scans 566,720 events (in secs):
index=web sourcetype=access_combined 15.16
index=web sourcetype=access_combined | fields clientip bytes referrer 4.49

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
18 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
General Search Practices – Wildcards
• Splunk only searches for whole words, but wildcards allowed
• Only trailing wildcards make efficient use of index
– Wildcards at beginning of string scan all events within time frame
– Wildcards in middle of string may return inconsistent results
– So use fail* (not *fail or *fail* or f*il)

• Wildcards tested after all other terms

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
19 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
General Search Practices
• Inclusion is generally better than exclusion
for "access denied" is faster than
– Searching
NOT "access granted"
• Filter as early in your search as possible
– Removing duplicates then sorting is faster than sorting then removing
duplicates
• Use the appropriate search mode
– Fast- performance over completeness
– Smart [default]
– Verbose - completeness over performance
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
20 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Transforming Search Commands
• A transforming command:
– Massages raw data into a data table
– 'Transforms' specified cell values for each event into numerical values
that you can use for statistical purposes
– Is required to 'transform' search results into visualizations

• Transforming commands include:

– top
– rare
– chart
– timechart
– stats
– geostats
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
21 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Reviewing Search Mode – Fast Mode
index=web sourcetype=access_combined
• Emphasizes performance, returning
only essential and required data
• For non-transforming searches:
✓ Events – fields sidebar displays only
those fields required for the search
✓ Patterns
✗ Statistics or visualizations
• Contents of interesting fields sidebar
are lost

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
22 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Reviewing Search Mode – Fast Mode (cont.)
index=web sourcetype=access_combined
• For transforming searches: | stats count by action
✗ Events
✗ Patterns
✓ Statistics or visualizations

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
23 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Reviewing Search Mode – Smart Mode (Default)
• Designed to give you the best results for your search
• Combination of Fast and Verbose modes
• For non-transforming searches [Verbose]:
✓ Events – fields sidebar displays all fields
✓ Patterns
✗ Statistics or visualizations
• For transforming searches:
✗ Events
✗ Patterns
✓ Statistics or visualizations
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
24 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Reviewing Search Mode – Verbose Mode
• Emphasizes completeness by returning all possible
field and event data
• For non-transforming searches:
✓ Events – fields sidebar displays all fields
✓ Patterns
✗ Statistics or visualizations
• For transforming searches:
✓ Events
✓ Patterns
✓ Statistics or visualizations
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
25 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Search Performance – Modes
• Use the most appropriate search mode: Note
These searches were run in our
index=web sourcetype=access_combined lab environment, not a production
environment - your results may
| chart count by product_name vary.

• Time range: last 365 days

Mode Returned Results Events Scanned Time

Fast 14 566,731 1.82
Smart 14 566,731 1.91
Verbose 14 566,731 15.21

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
26 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
 Search Performance – Types of Searches
Search Description Indexer throughput
A large percentage of the data matches the search
Up to 50K matching EPS
Dense Use Cases: computing stats, reporting (Events per second)
CPU bound
index=web sourcetype=access_combined | timechart count

A small percentage of data matches the search

Up to 5K matching EPS
Sparse Use Cases: troubleshooting, error analysis
CPU bound
index=web sourcetype=access_combined status=404 | timechart count

Returns a small number of results from each index bucket matching the search

I/O intensive as the indexer looks through all of an index's buckets Up to 2 seconds per index
Super Sparse bucket
With a lot of data, with a lot of buckets, it can take a long time to finish I/O bound
index=network sourcetype=cisco_wsa_squid action=denied src_ip=10.2.3.11

The indexer checks all buckets to find results, but bloom filters eliminate those buckets
that don’t include search results Up to 10-50 index
Rare buckets/second
Use Cases: user behavior tracking
I/O bound
index=web sourcetype=access_combined sessionID=1234
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
27 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Search Job Inspector
• Tool allows you to examine:
– Overall stats of search (e.g., records processed and returned,
processing time)
– How search was processed
– Where Splunk spent its time
• Use to troubleshoot search’s performance and understand impact of
knowledge objects on processing (e.g., event types, tags, lookups)
• Any existing (i.e., not expired) search job can be inspected

Note
For more information, see:
docs.splunk.com/Documentation/Splunk/latest/Sear
ch/ViewsearchjobpropertieswiththeJobInspector
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
28 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Search Job Inspector – 3 Components

• Header
• Execution costs
• Search job properties
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
29 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Search Job Inspector – Header

Top of Search job inspector provides basic information, including

time to run and # of events scanned

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
30 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Search Job Inspector – Execution Costs
• Provides details on cost to
retrieve results, such as:
– command.search.index
Time to search the index
for the location to read in
rawdata files
– command.search.filter
Time to filter out events
that do not match
– command.search.rawdata
Time to read events from
the rawdata files
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
31 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Search Job Inspector – Search Job Properties
Example:

• Produces scanCount of 127,201 events

• Returns resultCount of 2,144 in 3.01 seconds
• To calculate performance:
– Do not use resultCount/time 2,144 / 3.01 = 712 EPS*
– Rather, calculate scanCount/time 127,201/ 3.01 = 40,892 EPS

* EPS= events per second

Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
32 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
 Module 3:
Commands for Visualizations

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
33 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Module Objectives
• Explore data structure requirements
• Explore visualization types
• Create and format charts
• Create and format timecharts
• Explain when to use each type of reporting command

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
34 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
 Visualization Types
• When a search returns statistical
values, results can be viewed with a
wide variety of visualization types
– Statisticstable
– Charts: Line, column, pie, etc
– Single value, gauges
– Maps
– Many more

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
35 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Viewing Results as a Visualization
• Not all searches can be
visually represented
• A data series is a
sequence of related data
points that are plotted in a
visualization
• Data series can generate
any statistical or
visualization results

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
36 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Data Structure Requirements – Single Series
• Most visualizations require search results
structured as tables, with at least two
columns, a single series
– Leftmost column provides x-axis values
– Subsequent columns provide numeric y-axis
values for each series in the chart

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
37 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Data Structure Requirements – Multi-Series
To get multi-series tables, you need to set up the underlying search with reporting
search commands like chart or timechart

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
38 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Data Structure Requirements – Time Series
• Time series displays
statistical trends over time
• Can be single-series or multi-
series

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
39 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Viewing Results as a Chart
• There are seven chart types:
– Line
– Area
– Column
– Bar
– Bubble
– Scatter
– Pie

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
40 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Charts – Line
index=web sourcetype=access_combined action=*
| timechart count

index=web sourcetype=access_combined action=*

| timechart count by action

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
41 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Charts – Area
index=web sourcetype=access_combined action=*
| timechart count

index=web sourcetype=access_combined action=*

| timechart count by action

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
42 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Charts – Column
index=web sourcetype=access_combined action=*
| chart count over action

index=web sourcetype=access_combined action=*

| chart count over action by host

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
43 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Charts – Column (Formatted as Stacked)
index=web sourcetype=access_combined action=*
| chart count over action

index=web sourcetype=access_combined action=*

| chart count over action by host

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
44 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Charts – Bar
index=web sourcetype=access_combined action=*
| chart count over action

index=web sourcetype=access_combined action=*

| chart count over action by host

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
45 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Charts – Pie
index=web sourcetype=access_combined action=*
| chart count over action

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
46 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Charts – Scatter
• Scatter chart shows trends in the index=sales sourcetype=vendor_sales
VendorID >=7000 AND VendorID <=8999
relationships between discrete | stats sum(price) as "Total Sales",
values(price) as "Price",
data values count by VendorCountry, product_name

• Generally, it shows discrete values that do not occur at regular intervals or

belong to a series

Note
Although three values are
returned from this search, a
scatter chart can only show two
dimensions.

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
47 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
 Charts – Bubble
• Bubble chart provides a visual way to index=sales sourcetype=vendor_sales
VendorID >=7000 AND VendorID <=8999
view a three dimensional series | stats sum(price) as "Total Sales",
values(price) as "Price",
• Each bubble plots against two count by VendorCountry, product_name

dimensions on the X and Y axes

• The size of the bubble represents the value for the third
dimension

Note
Compare this with the two-
dimensional scatter chart from the
same search, shown on the
previous slide.

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
48 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Visualizations – x and y Axes
• For line, area, and column charts, index=security sourcetype=linux_secure
| chart count over vendor_action by app
the x axis is horizontal

• For bar chart, the

x axis is vertical

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
49 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
chart Command
• chart command can display any series of data that you want to plot
• You decide which field to plot on the x-axis
– The function defines the value of the y-axis, therefore it should be numeric
– The first field after the over clause is the x-axis
– Using the over and by clauses divides the data into sub-groupings
 The values from the by clause display in the legend
chart avg(bytes) over host
– The host values display over the x-axis
chart avg(bytes) over host by product_name
– The host field is the x-axis and the series is further split by product_name
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
50 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
chart Command – over field
count function tallies the number of Scenario
Display a count of vendor actions over the
events for each value in the result set last 60 minutes.

index=security sourcetype=linux_secure
| chart count over vendor_action

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
51 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
chart Command – over field by field
• You can use the by clause with the over clause Scenario
Display a count of vendor actions by user
to split results (over vendor_action by user) over the last 60 minutes.

• Alternatively, you can just use two by clauses

(by vendor_action, user) index=security sourcetype=linux_secure
| chart count over vendor_action by user
• You can only split chart results over TWO
dimensions (unlike stats results)

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
52 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
 Stack Mode

Stack Mode OFF

Stack Mode ON

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
53 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Including NULL and OTHER Values
• chart and timechart commands automatically Scenario
Display a count of unsuccessful web
filter results to include the ten highest values transactions by host for each item
– Surplus values are grouped into OTHER over the last 7 days.

• In this example, the results are skewed by NULL index=web sourcetype=access_combined

and OTHER status>399
| chart count over host by itemId
– These values are shown by default

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
54 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
 Omitting NULL and OTHER Values
• To remove empty (NULL) and
OTHER field values from the display,
use these options:
– useother=f
– usenull=f
index=web sourcetype=access_combined status>399
| chart count over host by itemId
Note useother=f usenull=f
To remove null values, add
itemId=* to the base search.

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
55 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Limiting the Number of Values
• To adjust the number of plotted series, use the limit argument
• For unlimited values, use limit=0 Scenario
Display sales per host for the top 5 best
selling products over the last 7 days.
Note
When splitting on two dimensions, index=web sourcetype=access_combined
the limit option applies to the action=purchase status=200
second split (so in this case,
| chart sum(price) over host
product_name).
by product_name limit=5 useother=f

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
56 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
timechart Command – Overview
• timechart command performs statistical aggregations against
time
• Plots and trends data over time
• _time is always the x-axis
• You can optionally split data using the by clause for one other field
- Each distinct value of the split by field is a separate series in the chart
• Timecharts are best represented as line or area charts

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
57 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
timechart Command – Example
Scenario index=network sourcetype=cisco_wsa_squid usage=Violation
How many usage violations have occurred | timechart count
during the last 7 days?

Note
Functions and arguments used with stats and
chart can also be used with timechart.
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
58 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
timechart Command – Multiple Values
• Splitting by the usage field, each line represents a unique field value
– Unlike stats, only ONE field can be specified after by
• y-axis represents the count for each field value
Note
Scenario index=network sourcetype=cisco_wsa_squid Using timechart, you can split by
What is the overall usage trend | timechart count by usage a maximum of one field because
for the last 24 hours? _time is the implied first by field.

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
59 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
 timechart Command – Multi-series: No
When the multi-series mode is set to No, all fields share the
y-axis index=network sourcetype=cisco_wsa_squid
| timechart count by usage

Scenario
What is the overall usage trend
for the last 24 hours?

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
60 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
timechart Command – Multi-series: Yes
• Setting multi-series mode to Yes causes
the y-axis to split for each field value
• y-axis is divided into sections, each
spanning the same max and min count

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
61 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
timechart Command – Adjusting the Sampling Interval
• The timechart command index=security sourcetype=linux_secure
vendor_action=*
"buckets" the values of the | timechart span=15m count by vendor_action
_time field
– Thisprovides dynamic sampling
intervals, based upon the time
range of the search
• Example defaults:
– Last 60 minutes uses span=1m
– Last 24 hours uses span=30m

• Adjust the interval using the

span argument, e.g. span=15m
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
62 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
timechart Command – Statistical Functions
As with the stats and chart commands, Scenario
How much Web activity of each type took
you can apply statistical functions to the place during the last 24 hours?

timechart command
index=web sourcetype=access_combined action=*
| timechart span=1h count by action

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
63 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Trellis Layout
• Display multiple charts index=web sourcetype=access_combined action=*
| timechart count by action
based on one result set
• Allows visual
comparison between
different categories
• Data only fetched once

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
64 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Visualization Formatting
index=web sourcetype=access_combined
| chart count by host

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
65 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
 Formatting – Chart Overlay
Scenario
A
Graphically compare the total price of online sales to sales made
in physical stores for the last 7 days, on an hour-by-hour basis.

(index=web sourcetype=access_combined action=purchase status<400) OR

(index=sales sourcetype=vendor_sales)
| timechart span=1h sum(price) by sourcetype
| rename access_combined as webSales, vendor_sales as retailSales

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
66 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
 Transforming Command Summary
Feature stats chart timechart
Multi-level breakdown Many 2 1
[by clause]

Limit # series shown NA limit=n limit=n

Default=10 Default=10

Filter other series NA useother=f useother=f

Filter null values NA usenull=f usenull=f
Set time value on x axis NA NA span
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
67 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Transforming Command Summary (cont.)
To count the frequency of a field(s), use top/rare
index=security sourcetype=linux_secure
| top src_ip, user, vendor_action, app

index=security sourcetype=linux_secure
| rare src_ip, user, vendor_action, app

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
68 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Transforming Command Summary (cont.)
Use stats to calculate statistics for two or more by fields (non time-
based) index=security sourcetype=linux_secure
| stats count by src_ip, user, vendor_action, app

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
69 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Transforming Command Summary (cont.)
• To calculate statistics with an arbitrary field
as the x-axis (not _time), use chart
– When you use a by field, the output is a index=security sourcetype=linux_secure
table | chart count over src_ip
by vendor_action
– Each column represents a distinct value
of the split-by field

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
70 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Transforming Command Summary (cont.)
• Use timechart to calculate
statistics with _time as the
x-axis
• If a by field is used, the output
is a table
• Each column represents a distinct value of
the split-by field
... | timechart span=1h count by itemId limit=3 useother=f

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
71 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
 Module 4:
Advanced Visualizations

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
72 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Module Objectives
• Create a trendline
• Create maps
– iplocation
– geostats
– geom

• Create and format single values

• Use the addtotals command

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
73 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
trendline Command
• Allows you to overlay a computed moving average on a chart
• trendline computes the moving averages of a field
trendline <trendtype><period>(field) [AS newfield]

• trendtype:
– sma - simple moving average
– ema - exponential moving average
– wma - weighted moving average

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
74 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
trendline Command (cont.)
• Must define the period over which to compute the trend
• period must be an integer between 2 and 10000
– For example, sma2(sales) is valid Scenario
sma(sales) would fail as it is
– But Display total sales and sales
trends over the past 24 hours.
missing an integer, the defining period

Note
Autocomplete displays functions in purple. Here
however, since it does not recognize sma as a
function, it is shown in black.
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
75 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
trendline Command – Example
Scenario index=web sourcetype=access_combined action=purchase status=200
Display total sales and sales | timechart span=2h sum(price) as sales
trends over the past 24 hours. | trendline sma2(sales) as trend

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
76 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Viewing Results as a Map
There are two map types

Cluster Map Choropleth Map

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
77 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
iplocation Command
Scenario index=security sourcetype=linux_secure (fail* OR invalid)
Discover longitude and latitude data for | iplocation src_ip
src_ip for the last 60 minutes.

• Use iplocation to look up and add location information

to an event
– This information includes city, country, region, latitude and
longitude
• Not all of the information is available for all ip address
ranges
• Automatically defines the default lat and lon fields
required by geostats

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
78 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
geostats Command
• Use geostats to compute statistical functions and render a
cluster map
geostats [latfield=string] [longfield=string] [stats-agg-
term]* [by-clause]
• Data must include latitude and longitude values
• Define the latfield and longfield only if they differ from the
default lat and lon fields
• To control the column count:
– On a global level, use the globallimit argument
– On a local level, depending on where your focus is (i.e., where
you’ve zoomed in), use the locallimit argument
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
79 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
geostats Command – Example
Scenario
Map the users of failed actions on the network
worldwide during the last 24 hours.

index=security sourcetype=linux_secure
(fail* OR invalid)
| iplocation src_ip
| geostats globallimit=5 count by user

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
80 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Choropleth Map
• Uses shading to show relative metrics, such as sales, network
intruders, etc. for predefined geographic regions
• To define regional boundaries, you must have either a:
– KML (Keyhole Markup Language) file
– KMZ (compressed Keyhole Markup Language) file

• Splunk ships with:

– geo_us_states, United States
– geo_countries, countries of the world

…| geom [featureCollection] [featureIdField=string]

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
81 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
geom Command
Scenario
Display the previous week’s retail sales in
EMEA.

index=sales sourcetype=vendor_sales
VendorID > 4999 AND VendorID < 6000
| stats count as Sales by VendorCountry
| geom geo_countries featureIdField=VendorCountry

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
82 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Viewing Results as a Single Value
Single value visualizations
provide various formatting index=security sourcetype=linux_secure vendor_action=failed
| stats count
options
Marker Filler Radial

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
83 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
 Single Value Visualizations: Formatting
Set color using UI or with the
gauge command

sourcetype=access_combined action=purchase
| stats sum(price) as count
| gauge count 0 5000 10000 15000

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
84 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
 Single Value Visualizations: Formatting (cont.)
index=security sourcetype=linux_secure
(fail* OR invalid)
| stats count(vendor_action)

index=security sourcetype=linux_secure
(fail* OR invalid)
| chart count by src_ip
| sort -count

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
85 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Single Value Visualizations: Formatting (cont.)
index=security sourcetype=linux_secure
(fail* OR invalid)
| stats count

To resize the font,

resize the pane

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
86 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Single Value Visualizations: Formatting (cont.)
Optionally, specify number format index=security sourcetype=linux_secure
(fail* OR invalid)
information for the single value on the | stats count
Number Format tab

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
87 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Single Value Visualizations: timechart
index=security sourcetype=linux_secure
• With the timechart fail* OR invalid
command, you can add a | timechart span=15m count(vendor_action)

sparkline and a trend

• A sparkline is an inline chart
– It is designed to display time-
based trends associated with
the primary key
• The trend shows the direction
in which values are moving
– It appears to the right of the
single value

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
88 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Adding Totals Using Format Options
• Automatically total every column using the Scenario
For the last 60 minutes, display the total number of
Format options events, with the total and average size (in bytes) by
web server. Also, calculate the total bytes.
• When using this approach, you:
– Cannot indicate which column to total; all index=web sourcetype=access_combined
| stats sum(bytes) as Bytes,
columns are always totaled avg(bytes) as avgBytes,
count as totalEvents by host
– Cannot add labels

3
1
2

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
89 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Adding Totals Using Format Options (cont.)
• Using the Summary tab, you can add a % row to the end of the statistics table
• All columns are used to compute the percentage – i.e., all percentages from all
columns combined will equal approximately 100%
index=sales sourcetype=vendor_sales
Scenario | chart count over product_name by VendorCountry
Display the retail products sold by country with totals
by product and by country during the last 4 hours.

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
90 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Adding Totals Using addtotals Command
• Alternatively, use the addtotals command to:
– Compute the sum of all or selected numeric fields for each column
and place the total in the last row
– Compute the sum of all or selected numeric fields for each row and
place the total in the last column

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
91 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
addtotals Command: Syntax
addtotals [row=bool] [fieldname=field]
[col=bool][labelfield=field] [label=string] field-list

Row Options Column Options

row=true/false A column is created that contains col=true/false A row is created that contains
(Default= true) numeric totals for each row. (Default= false) numeric totals for each column.
fieldname=field Defines a string used to create a label=string Defines a string used to name the
(Default=Total) field name for the totals column. (Default=Total) totals row.
labelfield= Defines where the label string is
fieldname placed. (Generally, you should make
this the first column.)

General Options
field-list=one or more numeric fields. Defines the numeric fields to be totaled.
(Default: all numeric fields)
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
92 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
addtotals Command – Example 1
• row=t (default) counts the index=sales sourcetype=vendor_sales
| chart count over product_name by VendorCountry
fields in each row under a | addtotals
fieldname="Total Per Product" A
column named "Total Per col=t B
Product” label="Total Per Country" labelfield=product_name C

A
C B

C B

• col=t counts the fields in Scenario

Calculate the total retail products sold by country
each row in a row named totaled by product and by country during the last 60

"Total Per Country"

minutes.

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
93 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
addtotals Command – Example 2
Scenario index=web sourcetype=access_combined
For the last 60 minutes, display the total | stats sum(bytes) as Bytes,
number of events with the total and average avg(bytes) as avgBytes,
size (in bytes) by web server, and then total count as totalEvents by host
the bytes. | addtotals row=f A col=t B label=totalBytes C
labelfield=host D Bytes E
A Do not total rows
B Total columns D
B

C Add the label totalBytes

D Place the label under the
host column
C E
E Only total the Bytes column

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
94 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
 Module 5:
Filtering and
Formatting Data

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
95 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Module Objectives
• Use the eval command to:
– Perform calculations
– Convert values
– Round values
– Format values
– Use conditional statements

• Use the search and where commands to filter calculated results

• Use fillnull command

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
96 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
eval Command – Overview
• eval allows you to calculate and manipulate field values in your
report
eval fieldname1 = expression1 [, fieldname2 =
expression2...]
• Supports a variety of functions
• Results of eval written to either new or existing field you specify
- If the destination field exists, the values of the field are replaced by the
results of eval
- Indexed data is not modified, and no new data is written into the index
- Field values are treated in a case-sensitive manner

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
97 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
eval Command
• The eval command allows you to:
– Calculate expressions
– Place the results in a field
– Use that field in searches or other expressions

• Type Operators
Arithmetic + - * / %
Concatenation +.
Note
Boolean AND OR NOT XOR For more info on common eval
functions, see
Comparison < > <= >= != = == LIKE http://docs.splunk.com/Documenta
tion/Splunk/latest/SearchReferenc
e/CommonEvalFunctions
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
98 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
eval Command – Convert Values
• This example report displays Scenario
What types of websites used the most bandwidth
the sum of bytes used for in bytes during the previous month?

each usage category

index=network sourcetype=cisco_wsa_squid
• It is difficult to determine how | stats sum(sc_bytes) as Bytes by usage

much bandwidth is being used

by looking at bytes
• First, use eval to convert the
bytes value into megabytes

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
99 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
eval Command – Convert Values (cont.)
• Results of eval must be set to Scenario
What types of websites used the most bandwidth
a new or existing field in megabytes during the previous month?

• In this example: index=network sourcetype=cisco_wsa_squid

| stats sum(sc_bytes) as Bytes A by usage
A Calculate the number of bytes | eval bandwidth B = Bytes/(1024*1024) C
for each usage type
B Create a new field named A B
bandwidth C

C Convert the values of the

Bytes field into MB by dividing
Bytes field values by
(1024*1024)
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
100 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
eval Command – Round Values
• The results of Bandwidth are hard to Scenario
What types of websites used the most bandwidth in
read with so many decimal points megabytes, rounded to 2 decimal places, during the
previous month? Sort by bandwidth.
• round(field/number, decimals)
index=network sourcetype=cisco_wsa_squid
function sets the value of a field to | stats sum(sc_bytes) as Bytes by usage
| eval bandwidth = round(Bytes/(1024*1024), 2) A
the number of decimals you specify | sort -bandwidth
| rename bandwidth as "Bandwidth (MB)"
• In this example:
– Divide the value of the Bytes field by
(1024*1024) A

- Round the result to two decimal points

• If the number of decimals is
unspecified, the result is a whole
number Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
101 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Removing Fields
• The "Bandwidth (MB)" field has Scenario
What types of websites used the most bandwidth in
the data in the desired format megabytes, rounded to 2 decimal places, during the
previous month? Sort by bandwidth.

• The Bytes field is no longer index=network sourcetype=cisco_wsa_squid

needed | stats sum(sc_bytes) as Bytes by usage
| eval bandwidth = round(Bytes/(1024*1024), 2)
– The Bytes field can be removed | sort -bandwidth
| rename bandwidth as "Bandwidth (MB)"
| fields - Bytes A

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
102 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
eval Command – Calculating Values
Scenario
You can perform mathematical
Calculate total online sales for last week; include price, sales price,
functions against fields with numeric and discount percentage. Sort by descending discount value.

field values
index=web sourcetype=access_combined product_name=*
A In this example, stats calculates action=purchase
A | stats sum(price) as tp, sum(sale_price) as tsp by product_name
the total list price and total sale B | eval Discount = round(((tp - tsp)/ tp)*100)
price by product_name | sort -Discount
| eval Discount = Discount."%"
B eval calculates the discount | rename tp as "Total List Price", tsp as "Total Sale Price",
product_name as Product
percentage and formats the
discount field

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
103 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
eval Command – Calculating Values (cont.)
C sort lists the highest discounted Scenario
Calculate total online sales for last week; include price, sales price,
items first and discount percentage. Sort by descending discount value.
D eval converts Discount to a string
and concatenates the % character index=web sourcetype=access_combined product_name=*
action=purchase
E rename provides user friendly | stats sum(price) as tp, sum(sale_price) as tsp by product_name
| eval Discount = round(((tp - tsp)/ tp)*100)
headings C | sort -Discount
D | eval Discount = Discount."%"
| rename tp as "Total List Price", tsp as "Total Sale Price",
E
product_name as Product

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
104 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
eval Command – tostring Function
• tostring converts a numeric field Scenario
How much potential online sales revenue was lost during the
value to a string previous week due to 503 server errors?

tostring(field,"option")
index=web sourcetype=access_combined
• Options: action=purchase status=503
| stats count(price) as NumberOfLostSales, A
– "commas": applies commas avg(price) as AverageLostSales,
 If the number includes decimals, it sum(price) as TotalLostRevenue
| eval AverageLostSales =
rounds to two decimal places "$" + tostring(AverageLostSales, "commas"), B
TotalLostRevenue =
– "duration": formats the number as "$" + tostring(TotalLostRevenue, "commas") C
"hh:mm:ss"
– "hex": formats the number
in hexadecimal A B C

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
105 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
tostring Function – duration Option
This example shows "duration" Scenario
option of tostring function Identify the five longest client sessions over the last 4 hours in
HH:MM:SS format.

A stats calculates sessionTime for

index=web sourcetype=access_combined
each session (JSESSIONID) | stats range(_time) as sessionTime by JSESSIONID A
| sort 5 -sessionTime B
– Use the range function to return the | eval duration = tostring(sessionTime,"duration") C
difference between the max and min
values of _time A A

B sort 5 displays the top 5 most

frequent values C
B
C The duration option formats the
time as "hh:mm:ss"

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
106 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Formatting and Sorting Values
• eval with added characters converts numeric field values to strings
• To order numerically, first sort, then use eval
index=web sourcetype=access_combined price=* index=web sourcetype=access_combined price=*
| stats values(price) as price by product_name | stats values(price) as price by product_name
| eval price = "$".price | sort -price
| sort -price | eval price = "$".price

Alpha Numeric

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
107 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
eval Commands with Multiple Expressions
• Multiple expressions can be combined into one eval command
• Each subsequent expression references the results of previous
expressions
• Expressions must be separated by commas

eval fieldname1 = expression1,

fieldname2 = expression2,
fieldname3 = expression3...

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
108 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
eval Commands with Multiple Expressions – Example
A Based on the values of list_price Scenario
and current_sale_price, calculate Calculate a new sale price that is 5% less than the current
discount percentage for online sales data over the last hour.
the current_discount percentage
B Calculate the new_discount value by index=web sourcetype=access_combined price=*
| stats values(price) as list_price, values(sale_price)
subtracting 5 from current_discount as current_sale_price by product_name
| eval current_discount = round((list_price – A
C Calculate the new_sale_price by current_sale_price)/list_price*100,2),
new_discount = (current_discount - 5), B
applying the new_discount percentage new_sale_price = list_price - (list_price *
(new_discount/100)) C

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
109 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
eval Command – if Function Syntax
if(X,Y,Z)
• The if function takes three arguments
• The first argument, X, is a Boolean expression
- If it evaluates to TRUE, the result evaluates to the second argument, Y
- If it evaluates to FALSE, the result evaluates to the third argument, Z

• Non-numeric values must be enclosed in "double quotes"

• Field values are treated in a case-sensitive manner

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
110 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
eval Command – if Function Example
index=sales sourcetype=vendor_sales
Scenario | eval SalesTerritory =
if((VendorID >= 7000 AND VendorID < 8000), "Asia", "Rest of the World")
Display retail sales for the previous week, broken
X Y Z
down by Asia and the Rest of the World.
| stats sum(price) as TotalRevenue by SalesTerritory
| eval TotalRevenue = "$" + tostring(TotalRevenue, "commas")

• Create a new field, SalesTerritory

• Evaluate VendorID
– If >= 7000 AND < 8000 is TRUE, set
result to "Asia"
 Remember, arguments must be
enclosed in quotes
– If it evaluates to FALSE, set result to
"Rest of the World"
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
111 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
eval Command – case Function
case(X1,Y1,X2,Y2...)
Note
The like operator is discussed later in
this module.
• The first argument, X1, is a Boolean expression
• If it evaluates to TRUE, the result evaluates to Y1
• If it evaluates to FALSE, the next Boolean expression, X2, is
evaluated, etc.
• If you want an “otherwise” clause, just test for a condition you
know is true at the end (e.g., 0=0)
index=web sourcetype=access_combined
X1 Y1 X2 Y2
| eval rating = case(productId LIKE "WC%", "Teen", productId LIKE "FS%", Mature",

0=0, "Unrated")
X3 Y3
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
112 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
eval function
• To count the number of events Scenario
Count the number of events that occurred yesterday where
that contain a specific field the vendor action was Accepted, Failed, or session opened.

value, use the count and eval

functions index=security sourcetype=linux_secure vendor_action=*
– Used within a transforming | stats
count(eval(vendor_action="Accepted")) as Accepted, A
command, such as stats count(eval(vendor_action="Failed")) as Failed, B
count(eval(vendor_action="session opened")) as
– Requires an as clause SessionOpened C

– Double quotes are required for

character field values
– Field values are case-sensitive A B C

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
113 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Filtering Results – search and where
• The search and where commands both filter results
– search
 May be easier if you’re familiar with basic search syntax
 Treats field values in a case-insensitive manner
 Allows searching on keyword
 Can be used at any point in the search pipeline
– where
 Can compare values from two different fields
 Functions are available, such as isnotnull()
 Treats field values in a case-sensitive manner
 Can’t appear before first pipe in search pipeline

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
114 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
search Command
• To filter results, use search at any Scenario
Report which products during the last 24 hours
point in the search pipeline have sold more than $500 online.

• Behaves exactly like search strings index=web sourcetype=access_combined

action=purchase status=200
before the first pipe | stats sum(price) as sales by product_name
| search sales>500 A
– Uses the "*" wildcard | sort –sales
| eval sales="$"+sales
– Treats field values in a case- | rename sales as "Popular Products",
product_name as "Product Name"
insensitive manner

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
115 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
where Command
where eval-expression
• Uses same expression syntax as eval command
• Uses boolean expressions to filter search results and only keeps
results that are True
• Double quoted strings are interpreted as field values
– Treats field values in a case-sensitive manner

• Unquoted or single-quoted strings are treated as fields

Note
To view all of the functions for where, see:
http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Where?r=searchtip

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
116 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
where Command – Example
• Filters search results using eval Scenario
Report which days over the previous week
expressions have seen more remove actions than change
quantity actions.

• Used to compare two different

index=web sourcetype=access_combined
fields | timechart count(eval(action="changequantity"))
as changes, count(eval(action="remove")) as removals
| where removals > changes
A

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
117 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
where Command With like Operator
• Can do wildcard searches with Scenario

where command
Report the number of events over the past 24
hours by IP address for a specific range of
addresses.

• Use (_) for one character and

(%) for multiple characters
index=security sourcetype=linux_secure
• Must use the like operator | stats count by src_ip
| where src_ip like "10_.%"

with wildcards

Note
As you've seen, the like operator can
also be used with the case function.

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
118 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
fillnull Command
• Use fillnull to replace null values in fields
• Use value=string to specify a string you want displayed instead
Example: fillnull value=NULL
• If no value= clause, default replacement value is 0
• Optionally, restrict which fields fillnull apply to by listing them at
end of command
Example: fillnull VALUE=“N/A” discount refund

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
119 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
 fillnull Command – Examples
Scenario
Evaluate vendor sales by country for the last
hour.

index=sales sourcetype= vendor_sales

| chart sum(price) over product_name by VendorCountry
| fillnull

index=sales sourcetype= vendor_sales

| chart sum(price) over product_name by VendorCountry
| fillnull value="No Value"

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
120 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
 Module 6:
Correlating Events

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
121 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Module Objectives
• Identify transactions
• Group events using fields
• Group events using fields and time
• Search with transactions
• Report on transactions
• Determine when to use transaction vs. stats

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
122 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
What is a Transaction?
• A transaction is any group of related events that span time
• Events can come from multiple applications or hosts
- Events related to a single purchase from an online store can span across
an application server, database, and e-commerce engine
- One email message can create multiple events as it travels through
various queues
– Each event in the network traffic logs represents a single user generating
a single http request
– Visiting a single website normally generates multiple http requests
 HTML, JavaScript, CSS files
 Flash, images, etc.
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
123 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
transaction Command
• transaction field-list
– field-list can be one field name or a list of field names
– Events are grouped into transactions based on the values of these fields
– If multiple fields are specified and a relationship exists between those
fields, events with related field values are grouped into a single
transaction

• Common constraints:
maxspan maxpause startswith endswith

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
124 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Events That Have the Same JSESSIONID
• The log shows a number of Scenario
events that share the same Display customer transactions in the online
store during the last 60 minutes.
JSESSIONID value
(SD0SL10FF3ADFF4950) index=web sourcetype=access_combined

• However, it is difficult to:

• View the events as a group
• Gain insight as to what is
happening with these
events
• Know if there are other
events scattered in the
results set
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
125 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
transaction Command – Example 1
• The transaction command creates a Scenario
single event from a group of events Group together Buttercup Games online store
events based on the JSESSIONID value for
– The events must share the same value the last 15 minutes.

in a specified field
index=web sourcetype=access_combined
• Transactions can cross multiple tiers | transaction JSESSIONID

such as web servers or

application servers
• For example, you can
easily view the events
for JSESSIONID
SD0SL10FF3ADFF4950

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
126 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
transaction Command – Example 2
• Use the search command at any point in Scenario
Display transactions that included a 404 error
the search pipeline to filter results during the last 60 minutes.

• Behaves exactly like search strings index=web sourcetype=access_combined

| transaction JSESSIONID A
before the first pipe | search status=404
| highlight JSESSIONID, 404 B
– search uses the "*" wildcard
and treats field values in a
case-insensitive manner A

– status=404 finds the errors

A
– highlight highlights the
terms you specify A

A
B

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
127 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
 transaction Command – Example 3
Scenario
For failed network logins, display different index=security sourcetype=linux_secure failed
users from the same IP during the last 60 | transaction src_ip
minutes.

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
128 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
transaction Command – Specific Fields
The transaction command produces additional fields, such as:
– duration – the difference between the timestamps for the first and
last event in the transaction
– eventcount – the number of events in the transaction

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
129 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
transaction Command – maxspan/maxpause
• You can also define a max overall Scenario
Display customer actions on the website
time span and max gap between during the last 4 hours.

events
index=web sourcetype=access_combined
- maxspan=10m | transaction clientip maxspan=10m maxpause=1m

▸Maximum total time between the

| eval duration = tostring(duration,"duration")
| sort -duration
| table clientip duration action
earliest and latest events | rename clientip as "Client IP",
▸If not specified, default is -1 (or no limit) action as "Client Actions"

- maxpause=1m
▸Maximum total time between events
▸If not specified, default is -1 (or no limit)
Note
Assumptions: Transactions spanning more than 10 minutes with
the same client IP are considered unrelated. Also, there can be
no more than one minute between any two related events.
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
130 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
transaction Command – startswith/endswith
• To form transactions based on Scenario
Determine the length of time spent to complete a purchase
terms, field values, or by customers in the online store over the last 24 hours.

evaluations, use startswith

and endswith options index=web sourcetype=access_combined
| transaction clientip JSESSIONID
startswith=eval(action="addtocart")
• In this example: endswith=eval(action="purchase")
| table clientip, JSESSIONID, duration, eventcount
– The first event in the
transaction includes
addtocart
– The last event includes
purchase

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
131 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Investigating with Transactions
• Transactions can be useful when Scenario
a single event does not provide Find emails that were rejected during the last
24 hours.
enough information
• This example searches email index=network sourcetype=cisco_esa REJECT

logs for the term “REJECT”

• Events that include the
term do not provide
much information
about the rejection

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
132 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Investigating with Transactions (cont.)
• By creating a transaction, you Scenario
Find emails that were rejected in the last 24
can then search and see hours.
additional events related to the
index=network sourcetype=cisco_esa
rejection, such as: | transaction mid dcid icid
| search REJECT
- IP address of sender
- Reverse DNS lookup results
- Action taken by the mail
system following the rejection
• mid – Message ID
• dcid – Delivery Connection ID
• icid – Incoming Connection
ID
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
133 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Reporting on Transactions
• You can use statistics and reporting Scenario

commands with transactions Create a chart to show the number of purchase

transactions based on their duration.

• In this example, a transaction is defined

by events that share clientip and fit index=web sourcetype=access_combined
status=200 action=purchase
within a 10 minute span | transaction clientip maxspan=10m
| chart count BY duration span=log2
• count() function counts number of
transactions and separates the count by
the duration of each

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
134 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
transaction vs. stats
• When you have a choice, use stats—it’s faster and more efficient,
especially in large Splunk environments
• Only use transaction when you:
– Need to see events correlated together
– Must define event grouping based on start/end values or segment on time
• Use stats when you:
– Want to see the results of a calculation
– Can group events based on a field value (e.g., by src_ip)

• By default, there’s a limit of 1,000 events per transaction

– No such limit applies to stats
– Admins can change limit by configuring max_events_per_bucket in
limits.conf
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
135 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
transaction vs. stats: Example 1
Scenario
index=web Find online purchase transactions over the • Searches produce
sourcetype=access_combined past year.
earliest=-1y@y latest=@y
same result
A
| transaction JSESSIONID
| table JSESSIONID, • A took 23.381
action, product_name
| sort JSESSIONID seconds
• B took 2.077
index=web
seconds
sourcetype=access_combined
earliest=-1y@y latest=@y • stats faster than
B
| stats values(action)
as "action",
transaction
values(product_name)
as "product_name”
by JSESSIONID
| sort JSESSIONID

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
136 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
 transaction vs. stats: Example 2
A B

index=security Note index=security

sourcetype=linux_secure failed 1. transaction has a limit of 1,000 sourcetype=linux_secure failed
| transaction src_ip 2. Count of transactions vs. count of IPs | stats count as eventcount
| table src_ip, eventcount by src_ip
| sort - eventcount | sort - eventcount

• A took 6.163
seconds
• B took 4.643
seconds

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
137 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
 Module 7:
Introduction to
Knowledge Objects
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
138 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Module Objectives
• Identify the categories of knowledge objects
• Define the role of a knowledge manager
• Identify naming conventions
• Review permissions
• Manage knowledge objects
• Describe the Splunk Common Information Model (CIM)

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
139 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
What are Knowledge Objects?
• Knowledge objects are tools you use to discover and analyze
various aspects of your data

– Data interpretation – Fields and field

extractions
– Data classification – Event types
– Data enrichment – Lookups and workflow
actions
– Normalization – Tags and field aliases
– Datasets – Data models

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
140 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
What are Knowledge Objects? (cont.)
• Shareable
– Can be shared between users
• Reusable
– Persistent
objects that can be used by multiple people or apps, such
as macros and reports
• Searchable
– Since the objects are persistent, they can be used in a search

Note
For more information go to:
http://docs.splunk.com/Documenta
tion/Splunk/7.0.0/Knowledge/What
isSplunkknowledge
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
141 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
What is a Knowledge Manager?
• Oversees knowledge object creation and usage for a group or
deployment
• Normalizes event data
• Creates data models for Pivot users

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
142 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Defining Naming Conventions
• This course uses simple names for lab exercises, but using a
naming convention in your production environment is
recommended. For example:
–Group: Corresponds to the working group(s) of the user saving the
object (examples: SEG. NEG. OPS. NOC)
–Object Type: Indicates the type of object (alert, report, summary-
index-populating) (examples: Alert, Report, Summary)
–Description: A meaningful description of the context and intent of the
search, limited to one or two words if possible; Note
ensures the search name is unique For more information go to:
http://docs.splunk.com/Documenta
• So, for example: SEG_Alert_WinEventlogFailures tion/Splunk/7.0.0/Knowledge/Deve
lopnamingconventionsforknowledg
eobjecttitles
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
143 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Reviewing Permissions
Description Create Read Edit (write)
Private Only the person who User Person who created it Person who created it
created the object can Power
use it and edit it Admin Admin Admin

This app only User* User*

Object persists in the Power
Power* Power*
context of a specific app Admin
Admin Admin
All apps User* User*
Object persists globally
Admin Power* Power*
across all apps
Admin Admin

* Permission to read and/or write if creator gives permission to that role

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
144 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Reviewing Permissions (cont.)
• When an object is created, Display
For is set to Owner by default
• When object’s permissions are set to
App or All apps, all roles are given
read permission
– Write permission is reserved for
admin and the object creator unless
the creator edits permissions
• Only the admin role can promote an
object to All apps
– Other roles have All Apps button
grayed out

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
145 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Managing Knowledge Objects
• Knowledge objects are
centrally managed from
Settings > Knowledge
• Your role and permissions
determine your ability to
modify an object’s settings

Note
By default, objects for all owners
are listed.

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
146 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Using the Splunk Common Information Model (CIM)

• Methodology for normalizing data

• Easily correlate data from different sources and source types
• Leverage to create various objects discussed in this course—field
extractions, field aliases, event types, tags
• More details discussed in Module 13

Unix
Search CIM Network Web

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
147 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
 Module 8:
Creating and Managing
Fields

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
148 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Module Objectives
• Review the Field Extractor (FX) methods
– Regex
– Delimiter

• Identify the different options to get to the Field Extractor

– Settings
– Fieldssidebar
– Event actions

• Review the process of extracting fields manually using regular

expressions
• Use the Field Extraction Manager to modify extracted fields
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
149 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Field Auto-Extraction
• Splunk automatically discovers many fields based on source type
and key/value pairs found in the data
• Prior to search time, some fields are already stored with the event
in the index
– Meta fields, such as host, source, and sourcetype
– Internal fields such as _time and _raw

• At search time, field discovery discovers fields directly related to

the search’s results
• Splunk may also extract other fields from raw event data that aren’t
directly related to the search
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
150 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Performing Field Extractions
• In addition to the many fields Splunk auto-extracts, you can also
extract your own fields with the Field Extractor (FX)
• Use FX to extract fields that are static and that you use often in
searches
- Graphical UI
- Extract fields from events using regex or delimiter
- Extracted fields persist as knowledge objects
- Can be shared and re-used in multiple searches
Note
• Access FX via Settings, Fields Sidebar, For more information, see:
http://docs.splunk.com/Documenta
or Event Actions menu tion/Splunk/7.0.0/Knowledge/Extra
ctfieldsinteractivelywithIFX
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
151 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Field Extraction Methods
• Regex
– Use this option when your event contains unstructured data like a
system log file
– FX attempts to extract fields using a Regular Expression that matches
similar events
• Delimiter
- Use this option when your event contains structured data like a .csv
file
- The data doesn’t have headers and the fields must be separated by
delimiters (spaces, commas, pipes, tabs, or other characters)

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
152 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Field Extraction Workflows – RegEx

Settings

Fields Sidebar

Event Actions
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
153 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Regex Field Extractions from Settings
Settings > Fields > Field extractions > Open Field Extractor

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
154 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Regex Field Extractions from Settings (cont.)
1. Select the Data Type
– sourcetype
– source

2. Select the Source Type

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
155 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Regex Field Extractions from Settings – Select
Sample
3. Select a sample event by clicking on it
4

4. Click Next >

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
156 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Regex Field Extractions from Settings – Select Method
5. Select Regular Expression
6
6. Click Next >

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
157 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Regex Field Extractions from Settings – Select Values
7. Select the value(s) you want to extract. In this example, two fields
are being extracted
8. Provide a field name
9. Click Add Extraction

Note 8

Require option – only events with

the highlighted string are included
in the extraction. 9

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
158 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Regex Field Extractions from Settings –
Preview
11
10. Preview the sample
events
11. Click Next >

10

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
159 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Regex Field Extractions from Settings – Validate
12.Validate the proper field values are extracted
13
13.Click Next >

12

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
160 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Regex Field Extractions from Settings – Save
14.Review the name for the newly extracted fields and set
permissions 15

15.Click Finish

14

Note
An extractions name is provided
by default. However, this name
can be changed.
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
161 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Using the Extracted Fields

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
162 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Editing Regex for Field Extractions
1. From Select Method, click Regular Expression
2

2. Click Next >

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
163 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Editing Regex for Field Extractions – Select Field
Note
3. Select the field to extract For more information about Splunk
Regular Expressions, see:

4. Provide a Field Name http://docs.splunk.com/Documentation/

Splunk/7.0.0/Knowledge/AboutSplunkre
gularexpressions

5. Click Add Extraction

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
164 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Editing Regex for Field Extractions – Show Regex
6. Click Show Regular Expression >
7. Click Edit the Regular Expression

7
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
165 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Editing Regex for Field Extractions – Modify RegEx
8. Update the regular expression Warning
After you edit the regular
expression, you cannot go back to
9. Click Save the Field Extractor UI.

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
166 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Editing Regex for Field Extractions - Save
10.Review the Extractions Name and set permissions
11.Click >Finish 11

10

Note
For more information about directly
editing these objects go to:
http://docs.splunk.com/Documentation/
Splunk/7.0.0/Admin/Propsconf

http://docs.splunk.com/Documentation/
Splunk/7.0.0/Admin/Transformsconf

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
167 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Delimited Field Extractions
• Use delimited field extractions when the event log does not have a
header and fields are separated by spaces, commas, or
characters
• In this example, the fields are separated by commas

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
168 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Field Extraction Workflows – Delimiters

Settings

Fields Sidebar

Event Actions
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
169 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Delimited Field Extractions from Settings
Settings > Fields > Field extractions > Open Field Extractor

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
170 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Delimited Field Extractions (Settings) – Select Sample
1. Select the Data Type
– sourcetype
– source
2. Select the Source Type

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
171 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Delimited Field Extractions (Settings) – Select Event
3. Select a sample event 4

4. Click Next >

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
172 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Delimited Field Extractions (Settings) – Select Method
5. Select Delimiters 6

6. Click Next >

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
173 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Delimited Field Extractions (Settings) - Select Delimiter
7. Select the
Delimiter used in
your event
7

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
174 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Delimited Field Extractions (Settings) – Rename Field
8. Click the icon
next to the default
field name
9. Enter a new field 8
11

name 9

10.Click Rename 10

Field
11.Repeat these
steps for all fields

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
175 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Delimited Field Extractions (Settings) – Rename Field (cont.)
12. After all the fields are renamed, click Next >
12

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
176 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Delimited Field Extractions (Settings) – Save
13. Name your 14

extraction
14.Click Finish>
13

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
177 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Using a Delimited Field Extraction

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
178 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
 Module 9:
Creating Field Aliases
and Calculated Fields

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
179 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Module Objectives
• Create and use field aliases
• Create calculated fields

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
180 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Field Aliases
• A way to normalize data over any default field (host, source or
sourcetype)
• Multiple aliases can be applied to one field
• Applied after field extractions, before lookups
• Can apply field aliases to lookups

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
181 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Field Alias Example
• Several source types contain some type of a username field
• To make data correlation and searching easier, normalize the username field

sourcetype=cisco_firewall

sourcetype=cisco_wsa_squid

sourcetype=winauthentication_security
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
182 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Creating a Field Alias
Settings > Fields > Field Aliases >
New Field Alias
1. Select the app associated with
the field alias
2. Enter a Name for the field alias 1

3. Apply the field alias to a default 2

field: 3
• Host
4
• Source
• Sourcetype existing field name new field alias

4. Enter the name for the existing

field and the new alias

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
183 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Creating a Field Alias (cont.)
In this example, one field alias
used for new ‘user’ fields in
multiple source types

New field alias required for

each sourcetype

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
184 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Testing the Field Alias
After the field alias is created, perform a search using the new field alias

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
185 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Field Alias and Original Fields
• When you create a field alias,
the original field is not affected
• Both fields appear in the All
Fields and Interesting Fields
lists, if they appear in at least
20% of events

field alias
original field

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
186 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Field Aliases and Lookups
After you have defined your field aliases, you can reference them in
a lookup table

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
187 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
 What is a Calculated Field?
• Shortcut for performing repetitive, long, or complex transformations using
the eval command
• Must be based on an extracted field

Note
Output fields from a lookup table or
fields/columns generated from within a
search string are not supported.

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
188 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Creating a Calculated Field
Settings > Fields > Calculated
Fields > New Calculated Field
1. Select the app that will use the
calculated field
1

2. Select host, source, or 2

sourcetype to apply to the 3

calculated field and specify 4

the related name

3. Name the calculated field
4. Define the eval expression
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
189 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Using a Calculated Field
After you have created a calculated field, you can use it in a search
like any other extracted field

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
190 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
 Module 10:
Working with Tags and
Event Types

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
191 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Module Objectives
• Create and use tags
• Describe event types and their uses
• Create an event type

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
192 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Describing Tags
• Tags are like nicknames that you create for related field/value pairs
• Tags make your data more understandable and less ambiguous
• You can create one or more tags for any field/value combination
• Tags are case sensitive

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
193 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
 Creating Tags
To create a tag:
1. Click on the arrow for event
details
2. Under Actions, click the down
arrow
3. Select Edit Tags
4. Name the tags, separated by 2
3
commas

1
4

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
194 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Viewing Tags
• When tagged field/value pairs are
selected, the tags appear: A

– In the results as tags A B

– In parentheses next to the

associated field/value pairs B

B
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
195 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Using Tags
To use tags in a search, use the syntax: tag=<tag name>

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
196 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Searching for Tags
• To search for a tag associated with a value:
– tag=<tagname> Note
Tag names are case sensitive.

• To search for a tag associated with a value on a specific field:

– tag::<field>=<tagname>

• To search for a tag using a partial field value:

– Use (*) wildcard

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
197 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Managing Tags – List by Field Value Pair
• Settings > Tags > List by field value pair
– Edit permissions
– Disable all tags for pair – disables the tag in searches and prevents
it from being listed under List by Tag Name and All unique tag
objects

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
198 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Adding/Changing the Tag Name
Click List by field value pair to add another tag or change the
name of the tag

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
199 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Adding/Changing the Field Value Pair
Click List by tag name to add or edit the field value pair for the tag

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
200 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Describing Event Types
• A method of categorizing events based on a search
• A useful method for institutional knowledge capturing and sharing
• Can be tagged to group similar types of events

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
201 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Creating an Event Type from the Search Page
1. Run a search and verify that all results meet your event type criteria
2. From the Save As menu, select Event Type
3. Provide a Name for your event type (name should not contain spaces)

Note
Must be a basic search (cannot
contain pipes or subsearches).
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
202 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Using the Event Type Builder
1. From the event details, select Event Actions > Build
Event Type

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
203 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Using the Event Type Builder (cont.)
2. Refine the criteria for your
event type such as:
• Search string
• Field values
• Tags
3. Verify your selections and click
Save

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
204 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Using Event Types
• To verify the event type, search for eventtype=web_error
• ‘eventtype’ displays in the Fields sidebar and can be added as a selected
field
• Splunk evaluates the events
and applies the appropriate
event types at search time

• Using the Fields sidebar, you

can easily view the individual
event types, the number of
events, and percentage

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
205 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Tagging Event Types
You can tag event types two ways:
1. Settings > Event Types
2. Event details > Actions

Priority determines which

event type color displays for
an event
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
206 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Event Types vs. Saved Reports
• Event Types
– Categorize events based on a search string
– Tag event types to organize data into categories
– The eventtype field can be included in a search string
– Does not include a time range

• Saved Reports
– Search criteria will not change
– Includes a time range and formatting of the results
– Can be shared with Splunk users and added to dashboards

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
207 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
 Module 11:
Creating and Using
Macros

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
208 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Module Objectives
• Describe macros
• Manage macros
• Create a basic macro
• Use a basic macro
• Define arguments / variables for a macro
• Add and use arguments with a macro

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
209 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Macros Overview
• Useful when you frequently run searches or reports with similar
search syntax
• The time range is selected at search time
• Macros can be a full search string or a portion of a search that can
be reused in multiple places
• Allows you to define one or more arguments within search segment
– Pass parameter values to macro at execution time
– Macro uses values to resolve search string

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
210 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Creating a Basic Macro
Settings > Advanced search > Search Macros
1. Click New Search Macro
2. Select the destination app
3. Enter a name
4. Type the search string
5. Save

Note
Test your search string before saving your macro.
You can check the contents of your macro with
keyboard shortcuts (Command-Shift-E on Mac OS
or Control-Shift-E on Linux or Windows) from the
Search bar in the Search page.

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
211 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Using a Basic Macro
`US_sales`
• Type the macro name into the
search bar
sourcetype=vendor_sales
• Surround the macro name with the VendorCountry="United States"
| stats sum(price) as USD by product_name
backtick (or grave accent) | eval USD = "$" + tostring('USD’,"commas")

character
- `macroname` != ‘macroname’
- Do not confuse with single-quote
character (‘)
• Pipe to more commands, or
precede with search string
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
212 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Adding Arguments
• Include the number of
arguments in parentheses after
the macro name
- monthly_sales(3)
• Within the search definition, use
$arg$
- currency=$currency$
- symbol=$symbol$
- rate=$rate$
• In the Arguments field, enter
the name of the argument(s)
• Provide one or more variables
of the macro at search time
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
213 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
 Using Arguments
• When using a macro with arguments,
include the argument(s) in parentheses
following the macro name
• Be sure to pass in the arguments in the
same order as you defined them
sourcetype=vendor_sales VendorCountry=Germany
OR VendorCountry=France OR VendorCountry=Italy
| `monthly_sales(euro,€,0.79)`

sourcetype=vendor_sales VendorCountry=Germany OR
VendorCountry=France OR VendorCountry=Italy |
stats sum(price) as USD by product_name
| eval euro = "€" + tostring(USD*0.79, "commas")
| eval USD = "$" + tostring(USD, "commas")

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
214 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
 Validating Macros
Note
You can validate argument values in your macro Don’t create macros with leading
pipes—someone may put a pipe in
• Validation Expression: you can enter an expression front of the macro when using it in
the actual search string.
for each argument
– Argument must be enclosed
in dollar signs
• Validation Error Message:
message that appears
when you run the macro

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
215 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
 Module 12:
Creating and Using
Workflow Actions

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
216 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Module Objectives
• Create a GET workflow action
• Create a POST workflow action
• Create a Search workflow action

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
217 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
What are Workflow Actions?
• Execute workflow actions from an event in your search results to
interact with external resources or run another search
– GET - retrieve information from an external resource
– POST - send field values to an external resource
– Search - use field values to perform a secondary search

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
218 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Creating a GET Workflow Action
Settings > Fields > Workflow
actions > New Workflow Action
1. Select the app
1

2. Name the workflow action with 2

no spaces or special characters
3. Define the label, which will 3

appear in the Event Action menu

4. Determine if your workflow
action applies to a field or event
type 4

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
219 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Creating a GET Workflow Action (cont.)
5. From the Show action in drop-
down list, select Event menu
5
6. From Action type dropdown list, 6
select link
7
7. Enter the URI of where the user
will be directed 8

8. Specify if the link should open 9

in a New window or Current

window 10

9. Select the Link method of get

10. Save
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
220 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Testing the GET Workflow Action

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
221 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Creating a POST Workflow Action
Settings > Fields >
Workflow actions > New 1
Workflow Action 2

Complete steps 1 – 6 as 3
described in the previous
example, Creating a GET
Workflow Action
4

6
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
222 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Creating a POST Workflow Action (cont.)
7. Enter the URI of where
the user will be directed
7
8. Open the link in a New
window or Current 8

window 9

9. Select the Link method 10

of post
10. Provide post argument
11
parameters
11.Save
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
223 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Creating a Search Workflow Action
Settings > Fields > Workflow
actions> New
1
Complete steps 1 – 5 as described
2
in the previous example, Creating a
GET Workflow Action 3

6. From the Action type dropdown

list, select search 4

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
224 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Creating a Search Workflow Action (cont.)
7. Enter the Search string
8. Select the app if it is different
from the current app 7

9. Enter the view name where 8

the search will execute
9
10.Indicate if the search should
run in a New window or the 10
Current window
11.Enter the time range for the 11

search or choose to use the

same time range as the
search
12.Save 12

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
225 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Testing the Search Workflow Action

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
226 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
 Module 13:
Creating Data Models

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
227 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Objectives
• Describe the relationship between data models and Pivot
• Identify data model datasets
• Identify dataset fields
• Create a data model
• Use a data model in Pivot

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
228 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Reviewing Pivot
• Used for creating reports and dashboards (discussed in Splunk
Fundamentals 1)
• As a knowledge manager, you’re responsible for building the data model
that provides the datasets for Pivot

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
229 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Overview of Data Models
• Hierarchically
structured datasets
that generate
searches and drive
Pivot
• Pivot reports are
created based on
datasets
• Each event, search or
transaction is saved
as a separate dataset
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
230 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Data Model Dataset Types
A data model can
consist of 3 types of
datasets
Events
• Events
• Searches
• Transactions
Searches

Transactions

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
231 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Data Model Events
• Event datasets contain constraints and fields
• Constraints are essentially the search broken down into a
hierarchy
• Fields are properties base search

associated with the events

fields

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
232 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Event Object Hierarchy and Constraints
Each constraint inherits the
base search – all access_combined events parent search string

2
successful requests

3 successful requests for purchases

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
233 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Dataset Fields
• Select the fields you want
to include in the dataset
• Like constraints, fields are
inherited from parent
objects

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
234 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Creating a Data Model
Settings > Data Models

ID is automatically populated from

Title, but can be overridden

Choose app context

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
235 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Adding a Root Event
Constraints are essentially search terms – add child events
(discussed later in this section) to further "narrow" your search

Click Preview to view the events that

the constraint returns

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
236 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Adding a Root Event (cont.)
• In this example, the root event of this data model represents all
web requests
• The Inherited attributes are default fields
• Use Add Field > Auto-Extracted to add more fields

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
237 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Adding Fields
• Auto-Extracted – can be
default fields or manually
extracted fields
• Eval Expression – a new field
based on an expression that
you define
• Lookup – leverage an existing
lookup table
• Regular Expression – extract
a new field based on regex
• Geo IP – add geographical
fields such as
latitude/longitude, country, etc.
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
238 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Adding Fields – Auto-Extracted
Fields that already exist for the constraint can be added as
attributes to the data model

View a field's
example values

Give the field a friendly name for use in Pivot

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
239 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Field Types
• String: Field values are recognized as alpha-numeric
• Number: Field values are recognized as numeric
• Boolean: Field values are recognized as true/false
or 1/0
• IPV4: Field values are recognized as IP addresses
– Thisis an important field type, as at least one IPV4 attribute type must
be present in the data model in order to add a Geo IP attribute

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
240 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Field Flags
• Optional: This field doesn't have to appear in every
event
• Required: Only events that contain this field are
returned in Pivot
• Hidden: This field is not displayed to Pivot users when they select
the dataset in Pivot
– Usefor fields that are only being used to define another field, such as
an eval expression
• Hidden & Required: Only events that contain this field are returned,
and the fields are hidden from use in Pivot
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
241 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Adding Fields – Eval Expressions
• You can define a new field using an eval expression
– In this example, you create a field named Error Reason that evaluates the
value of the status field

Click Preview to verify your eval

expression returns events

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
242 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Adding Fields – Lookups
• Leverage an existing lookup definition to add fields to your event
object
• Configure the lookup attribute in the same way as an automatic
lookup

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
243 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Adding Fields – Lookups (cont.)
• Use Preview to test your lookup settings
• Use the Events and Values tab to verify your results

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
244 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Adding Fields – Regular Expression
You can define a new field using a regular expression

Click Preview to view the events that

Match
or Don’t Match the regular expression

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
245 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Adding Fields - GeoIP
• Map visualizations require latitude/longitude fields
• To use Geo IP Lookup, at least one IP field must be configured as
an IPv4 type
• While the map function isn't available in Pivot, the data model can
be called using the
| pivot command and <map>
element in a dashboard population search
– Select the field that contains
the mapping to lat/lon
– Identify the lat/lon and geo
fields in the data
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
246 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Adding Child Datasets
When you create a new child dataset, you give it one or more
additional constraints

All events that have a status less than 400

(successful http request)

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
247 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Adding Child Datasets (cont.)
• Child datasets inherit
all fields from the
parent events
• You can add more
fields to child datasets

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
248 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Testing the Data Model
• Click Pivot to access the Select a Dataset window
• Choose an object from the selected data model to begin building the report

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
249 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Using the Data Model in Pivot
The New Pivot window automatically
populates with a count of events for the
selected dataset

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
250 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Pivot – Using Fields
• The fields associated
with each dataset are
available as splits for
rows or columns
• In this example, the
Pivot report will show
a count of failed
request actions by
status

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
251 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Pivot – Using Fields (cont.)
• Fields can also be
used to filter events in
the Pivot interface
• In this example, the
Pivot report is filtered
to only return results
where status=503

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
252 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Underlying Search

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
253 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Underlying Search (cont.)
Data model name Object name

| pivot Buttercup_Games_Site_Activity failed_request

count(failed_request) AS "Count of Failed requests"
Split row field (or attribute)

SPLITROW action AS action TOP 100

count(failed_request)
Split column field (or attribute) and filter field/value pair

SPLITCOL status
FILTER status = 503
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
254 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Data Model Search Datasets
• Arbitrary searches that
include transforming
commands to define the
dataset that they
represent
• Search datasets can also
have fields, which are
added via the Add Field
button

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
255 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Data Model Transaction Datasets
• Enable the creation of
datasets that represent
transactions
• Use fields that have
already been added to the
model using event or
search datasets

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
256 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Adding a Transaction
• You can add a transaction to the data model
• The transaction dataset below would equate to the search:
sourcetype=access_* | transaction clientip maxpause=10s

Select a dataset from the data

model to base the transaction on

Optionally select max pause and

max span

Select a group by field

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
257 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Adding a Transaction (cont.)
• You can then add an eval
expression or any other field
to your transaction to further
define the results
• This example shows dividing
the duration field value by 60
to convert the duration field to
minutes

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
258 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Search and Transaction Dataset Considerations
• There must be at least one event or search dataset before adding
a transaction dataset
• Search and transaction datasets cannot benefit from persistent
data model acceleration
• As you learn to create data models, consider the types of reports
your users will run
– Can the same report be achieved with event datasets?
– Will users need raw events or transactional data?

Note
Data model acceleration is
discussed later in this module.

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
259 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Set Permissions
• When a data model is created, the
owner can determine access based
on the following permissions:
– Who can see the data models
 Owner, App, or All Apps
– Which users can perform which
actions (Read/Write)
 Everyone
 Power
 User
 Admin-defined roles, if applicable
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
260 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Download and Upload Data Models
• Use the Splunk Web interface to download or upload data models:
– Back up important data models
– Collaborate with other Splunk users to create/modify/test data models
– Move data models from a test environment to production instance

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
261 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Downloading a Data Model

Note
An HTML 5 supported browser
must be used to download data
models.
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
262 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Uploading a Data Model

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
263 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Data Model Acceleration
• Uses automatically created summaries to speed completion
times for pivots
• Takes the form of inverted time-series index files (tsidx) that
have been optimized for speed
• Discussed in more detail in Advanced Searching and
Reporting

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
264 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Accelerating a Data Model
Note
• With persistent data model acceleration, all fields Only root events can be
in the model become "indexed" fields accelerated. If there are multiple
root events, only the first root
event is accelerated.
• You must have administrative permissions or the
accelerate_datamodel capability to accelerate a data model
• Private data models cannot be accelerated
• Accelerated data models cannot be edited

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
265 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
 Module 14:
Using the Common Information
Model (CIM) Add-On

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
266 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Objectives
• Describe the Splunk Common Information Model
• List the knowledge objects included with the Splunk CIM Add-On
• Use the CIM Add-On to normalize data

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
267 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
What is the Common Information Model (CIM)?
• The Splunk Common Information Model provides a methodology to
normalize data
• Leverage the CIM when creating field extractions, field aliases,
event types, and tags to ensure:
– Multiple apps can co-exist on a single Splunk deployment
– Object permissions can be set to global for the use of multiple
apps
– Easier and more efficient correlation of data from different
sources and source types

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
268 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
How the Splunk CIM Works
General users Network team Server admins Web admins

Search Unix Web

CIM Network

Search Head Search

Time apps

TA Checkpoint

Logs and Data

Indexer Index
TA1 TA Checkpoint TA2 TA3 Time apps

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
269 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Normalized Field Names – Email Data
Field name Data type Description Possible values

delivered, blocked,
action string Action taken by the reporting device. quarantined,
deleted, unknown

The amount of time for the completion of the messaging

duration number Email
event, in seconds.

The system that sent the message. May be aliased from

src string
more specific fields such as src_host, src_ip, or src_name.

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
270 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Normalized Field Names – Network Traffic
Field name Data type Description Possible values

allowed, blocked,
action string The action taken by the network device.
dropped, unknown

Total count of bytes handled by this device/interface (bytes_in +

bytes number
bytes_out).
bytes_in number How many bytes this device/interface received.

bytes_out number How many bytes this device/interface transmitted.

The source of the network traffic (the client requesting the

src string connection). May be aliased from more specific fields such as
src_host, src_ip, or src_name.

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
271 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
 Normalized Field Names – Web Data
Field name Data type Description Possible values
action string The action taken by the server or proxy.

duration number The time taken by the proxy event, in milliseconds.

GET, PUT, POST,

http_method string The HTTP method used in the request.
DELETE, etc.
The source of the network traffic (the client requesting the
src string
connection).

The HTTP response code indicating the status of the

status string 404, 302, 500, and so on.
proxy request.

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
272 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Splunk CIM Add-on
• Set of 22 pre-configured data models Splunk CIM Add-On Data Models

– Fields
Alerts Java Virtual Machines (JVM)
and event category tags Application State Malware

– Least common denominator Authentication Network Resolution (DNS)

of a domain of interest Certificates Network Sessions

Change Analysis Network Traffic

• Leverage the CIM so that knowledge objects CIM Validation (S.o.S) Performance

in multiple apps can co-exist on a single Databases

Email
Splunk Audit Logs
Ticket Management
Splunk deployment Interprocess Messaging Updates
Intrusion Detection Vulnerabilities
• Available on splunkbase: Inventory Web
– https://splunkbase.splunk.com/app/1621/
• Use the CIM Reference Tables
Note
– https://docs.splunk.com/Documentation/CIM/4.9.0
The data models included in the CIM add-on are
/User/Howtousethesereferencetables configured with data model acceleration turned
off.
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
273 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Using the CIM Add-on
1. Examine your data
– Go to Settings >
Data Models
– Identify a data model
relevant to your
dataset

Best Practice
Keep the CIM Reference Tables in
Splunk Docs page open in a
separate tab.

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
274 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Using the CIM Add-on (cont.)
2. Create event types & tags
– Identify the CIM datasets relevant
to your events
– Observe which tags are required
for that dataset or any parent
datasets
– Apply those tags to your events
using event types

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
275 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Using the CIM Add-on (cont.)
3. Create field aliases
– Determine whether any
existing fields in your data
have different names than the Field name
in CIM object

names expected by the data

models
– Define field aliases to capture
the field with a different name
Field name
in your original data and map in your data

it to the field name that the

CIM expects

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
276 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Using the CIM Add-on (cont.)
4. Add missing fields
– Create field extractions
– Write lookups to add fields and
normalize field values
5. Validate against data model
– Use the datamodel command
– Use Pivot in Splunk Web

Note
For more information, see the Common
Information Model Add-on Manual:
http://docs.splunk.com/Documentation/CIM/latest/
User/Overview
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
277 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
datamodel Command
• Search against a specified data
model object
• Return a description of all or a
specified data model and its objects
• Is a generating command and
should be the first command in the
pipeline

Important
The object name and search keyword aren't valid unless
preceded by the data model name. The command search
cannot be substituted with a search string or name.
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
278 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
 datamodel Command – Example
| datamodel Web Web search | fields Web*
A B C D E

Dataset name
prepended to field
A• Command names in your data
B• Data model name
C• Data model dataset name
D• Command
E• Find field names with Web prefix
Note
When using the datamodel command, the data
model name and dataset name are case-
sensitive.
Generated for ([email protected]) (C) Splunk Inc, not for distribution
Splunk Fundamentals 2
279 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
from Command
• Retrieves data from a data model or named dataset
• Must be the first command in a search
• Different than just using datamodel
– datamodelreturns all fields prepended with data model name
– from datamodel returns specified fields only

| from datamodel:"internal_server.splunkdaccess"

• from can also retrieve data from saved searches, reports, or

lookup files
| from savedsearch:mysecurityquery

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
280 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018
Additional CIM Resources
• Understand and use the CIM Add-on
http://docs.splunk.com/Documentation/Splunk/latest/Knowledge/UnderstandandusetheCommonInformationModel

• Overview of the Splunk CIM

http://docs.splunk.com/Documentation/CIM/latest/User/Overview

• Use the CIM to normalize data at search time

http://docs.splunk.com/Documentation/CIM/latest/User/UsetheCIMtonormalizedataatsearchtime

Generated for ([email protected]) (C) Splunk Inc, not for distribution

Splunk Fundamentals 2
281 Copyright © 2018 Splunk, Inc. All rights reserved | 14 June 2018