5DV Advanced_Dashboards_and_Visualisations.pdf

Advanced Dashboards & Visualizations
Listen to your data™ 1
Copyright © 2015 Splunk, Inc. All rights reserved. | 6 November 2015
Generated for Marco Tavares (marco.tavares@foxtel.com.au) (C) Splunk Inc, not for distribution

Document Usage Guidelines
•Should be used only by enrolled students
•Not meant to be a self-paced document, an instructor is needed
•Do not distribute
6 November 2015

Course Prerequisites
Classes • Splunk Tutorial (eLearning)
• Using Splunk (ILT or eLearning)
• Searching and Reporting with Splunk (ILT)
• Creating Splunk Knowledge Objects (ILT)
Skills • Working knowledge of HTML
• Some XMLand JavaScript experience (recommended)
In order to receive credit for this course,
you must complete all lab exercises.
Important

Course Goals
Upon completion of this course you will be able to:
•Use best practices for planning and creating views
•Define data structure requirements for visualizations
•Create efficient, well-formed searches that generate charts
•Edit simple xml to enable:
– dynamic drilldowns
– global searches
– tokens
•Use JavaScript and CSS to create interactive visualizations
•Troubleshoot views

Course Outline
Module 1: Introduction to Views
Module 2:Adding Content
Module 3: Using Simple XML
Module 4: Creating Forms
Module 5: Customizing Dashboards

Course Scenario
•As in the other Splunk courses, the use
cases in this course are based on
Buttercup Games, a gaming company
•The views are based on business analytics
from web access logs and lookups
Data Host
Sourcetype

Online transactions
& web server
www1 access_combined
linux_secure

www2
www3
Retail sales data vendorUS1 vendor_sales


Callouts
Scenarios
•Many of the examples in this course
relate to a specific scenario
•For each example, a question is
posed from a colleague or manager
at Buttercup Games
Notes & Tips
•References for more information on
a topic and tips for best practices
How can we use an view to limit
types of searches?
Scenario ?
Functions and arguments used with
stats and chart can also be used
with timechart.
Note

Module 1:
Introduction to Views

Module Objectives
Upon successful completion of this module, you will be able to:
•Define what is a view
•Identify best practices for creating views
•Define the common information model
•Normalize data to CIM

What is a view?
•Every page in Splunk Web is a view
– Dashboards
– Forms
•Each view is a web page built from:
– XMLfile that defines the content
– HTMLfile that defines the layout
– CSS and JavaScript files that define
the appearance and interactions

Dashboards
•Most common type of view
•Collection of data visualizations
that tell a meaningful story
•Display results as event list or in
graphical form: charts, graphs,
tables, etc.
•Typically, limited user input
•Default interactive features
– Mouse over values, core drill-down

Forms
•An interface that allows
users to enter values for
one or more search terms
from a variety of inputs
•Shield users from the details
of the underlying search

Dashboards & Forms
•Similarities
– Layout of rows, panels,
and panel visualizations
•Differences
– Top-level element:
<dashboard> and <form>
– Forms have user inputs:
time range picker, drop-down lists,
radio buttons, text box, etc.

Stakeholders
•Depending on the complexities of your
dashboard, your stakeholders may include:
– SplunkAdministrator
– JavaScript developer
– Security expert
– Business user
– UX designer
•Questions to ask:
– How will users access your dashboards?
– Will the dashboards use JavaScript?
– Should the views be deployed with their own app?

Best Practice
• Plan: Identify key metrics, time frames,
visualizations; then, wireframe the view
• Add content: Create basic searches,
views, and visualizations
• Add interactivity: Add buttons, inputs,
tokens, and dynamic drilldowns
• Get stakeholder sign-off
• Refine: Optimize searches, add tokens, use
post-processes, create macros and data models
• Customize: Add custom features available
with CSS and JavaScript

Plan
•An iterative process between you and
the stakeholders
– What critical metrics do users want?
– What is the time span for the data?
– What is the timeframe for refreshing data?
– What visualizations will be required?
– What should the layout look like?
•Wireframing is the process of designing
a view through prototyping
Use sketches then build
Or add a static version as an interim step

Managing Views
Views are scoped to your app context and permissions can be applied to it
1
2
4
3

Troubleshooting Views
•Examine the panel's search
– Check for syntax errors
– Run it manually in the relevant app context
•View all previous searches with their stats
– Run | history
•Examine the view's source
– Appending "?showsource=1"
– Use "&showsource=1" if other parameters
have already been appended
– Expand macros and event types

Troubleshooting Views – Job Inspector
Examine impact of
knowledge object
processing, such as
event types, tags,
lookups and so on
Click inspect to open
the job inspector
For more information on the Job Inspector,
see the Knowledge Manager Manual.
Note
1
2

Troubleshooting Views – Job Inspector (cont.)
Debug messages
You won't see these messages
until the search has completed.
Note

Common Information Model
•Methodology to normalize data
•Match a common standard, using the same field names and event tags
for equivalent events from different sources or vendors
Normalized
Field Names
Data Source 2
Data Source 1

Splunk CIM Add-on
•Set of 22 pre-configured data models
– Fields and event category tags
– Least common denominator
of a domain of interest
•Leverage the CIM so that knowledge
objects in multiple apps can co-exist
on a single Splunk deployment
The data models included in the CIM add-on are
configured with data model acceleration turned off.
Note
Splunk CIM Add-On Data Models
Alerts Java Virtual Machines (JVM)
Application State Malware
Authentication Network Resolution (DNS)
Certificates Network Sessions
Change Analysis Network Traffic
CIM Validation (S.o.S) Performance
Databases Splunk Audit Logs
Email Ticket Management
Interprocess Messaging Updates
Intrusion Detection Vulnerabilities
Inventory Web

Get Data In
1
Using the CIM
Examine Your Data Create Event
Types & Tags
Create Field Aliases
2 3 4 6
Validate Against Model
✓
✓
✓
Best Practice
Add Missing
Fields & Tags
5

Using the CIM (cont.)
•Examine Your Data
– Go to Settings > Data Models
– Identify a data model relevant
to your dataset
Object Hierarchy
Data Types
Tags and Other Constraints
Inherited
Fields
Extracted
Fields
2
Keep the CIM Reference Tables in Splunk
Docs page open in a separate tab.
Best Practice Calculated
Fields

•Create Event Types & Tags
– Identify the CIM objects relevant
to your events
– Observe which tags are required
for that object or any parent objects
– Apply those tags to your events
using event types
3

•Create FieldAliases
– Determine whether any existing
fields in your data have different
names than the names expected
by the data models
– Define field aliases to capture the
differently named field in your original
data and map it to the field name that
the CIM expects
4
Field name
in CIM object
Field name
in your data

•Add Missing Fields
– Create field extractions
– Write lookups to add fields
and normalize field values
•ValidateAgainst Data Model
– Using datamodel command
OR
– Using Pivot in Splunk Web
5
6
For more information, see the
Common Information Model Add-on Manual.
Note

datamodel
Command
•Search against a specified data
model object
•Return a description of all or a specified
data model and its objects
•Is a generating command and should
be the first command in the pipeline

The object name and search keyword aren't valid unless
preceded by the data model name. The keyword search
cannot be substituted with a search string or name.
Important

datamodel
Command – Example
A
|
datamodel
Web
Web
search
|
fields
Web*

A B C
command
data model name
data model object name
keyword
find field names with Web prefix
B
C
D
D
When using the datamodel command, the data
model name and object name are case-sensitive
Note
Object name
prepended to field
names in your data
E
E

Lab 1 – Get to Know Your Data
Time: 20 – 25 minutes
Tasks:
– Log into Splunk Web
– Change the account name and time zone
– Examine the use case and wireframe
– Make your data CIM-compliant

Module 2:
Adding Content

Module Objectives
•Define data structure requirements
•Identify the primary transforming commands
•Split values into multiple series
•Chart multiple values on the same timeline

Create Basic Searches
•Basic, hard coded searches
– Add tokens, macros,
data models later
•Use naming conventions
– At least the same prefix
–Group, search type, view type,
platform, category, time interval,
description, and project
Command Description
bucket Puts continuous numerical values into discrete sets.
chart Returns results in a tabular output for charting.
dedup Removes subsequent results that match a specified criteria.
eval Calculates an expression and puts the value into a field.
fields Adds or removes fields from search results.
lookup Explicitly invokes field value lookups.
multikv Extracts field-values from table-formatted events.
rangemap Sets RANGE field to the name of the ranges that match.
rex
Specify a Perl regular expression named groups to extract
fields while you search.
spath Extracts key-value pairs from XML or JSON formats.
stats Provides statistics, grouped optionally by fields.
timechart Create a time series chart and corresponding table of statistics.
transaction Groups search results into transactions.
where Performs arbitrary filtering on your data.

Data Structure Requirements
•Search not generating any statistical values, or the visualization you want
not available?
•Modify the search to get the visualization you want
Data series: a sequence of related
data points that are plotted in a chart.
Note

Data Structure Requirements – Single Series
•Search results structured as tables
with at least two columns
– First column provides x-axis values
– Second column provides y-axis
values for the chart
sourcetype=vendor_sales

|
chart
avg(price)
over
Vendor


Data Structure Requirements – Multi-Series
•Search results structured as tables
with three or more columns
– First column provides x-axis values
– Subsequent columns provide y-axis
values for each series in the chart
– Underlying search must use
transforming commands like
stats, chart, or timechart
VendorID<4000

|
chart
count
over
VendorCountry

by
product_name
limit=5
useother=f


Data Structure Requirements – Time Series
•Time series display statistical trends over time
– Any search using the timechart command
– First column provides _time values
– Subsequent columns provide y-axis values for each series in the chart
• Underlying search must use transforming commands like stats, chart, or timechart
sourcetype=access_combined
action=purchase
status=200

|
timechart
count(action)
by
host


Data Structure Requirement Summary
Built-in Visualization Series Dimensions Table Columns
Column, line,
and area charts
single, multiple 2 Two column minimum:
first = x-axis, second = y-axis, additional = y-axis
Bar charts single, multiple 2 Two column minimum:
first = y-axis, second = x-axis, additional = x-axis
Pie charts single 1 Two column:
first = slice label, second = label value, additional = ignored
Bubble charts single, multiple 4 Three column:
first = series label, second = x-axis, third = y-axis (numerical for best results),
additional = ignored
Scatter charts single, multiple 3 Two or three columns:
single series (two column): first = x-axis, second = y-axis,
multiple series (three column): first = series names, second = x-axis, third: y-axis
Single value – 1 One column
Gauges single 1 Two column:
first = range value, second = x-axis, additional = ignored
Map single 2 Two column:
first = slice label, second = label value, third = longitude, fourth = latitude

Transforming Commands
•Required to transform search results into visualizations
•The primary transforming commands are:
– chart: displays any series of data that you want to plot
– timechart: displays trend over time; display _time on the x-axis
– stats, eventstats, geostats, geom, and streamstats: display summary statistics
– top: displays the most common values of a field
– rare: displays the least common values of a field
– associate, correlate, and diff: display associations, correlations,
and differences between fields in your data

Statistical Functions
•These five transforming commands
work with statistical functions:
– chart, timechart, stats, eventstats,
streamstats
•Available statistical functions:
– count, distinct count
– mean, median, mode
– min, max, range, percentiles
– standard deviation, variance
– sum
– first occurrence, last occurrence
action=purchase

|
timechart
span=1h

min(price)
as
min,

max(price)
as
max,

mean(price)
as
mean,

median(price)
as
median,

stdev(price)
as
"standard
dev",

range(price)
as
range


Characteristic Visualization
Temporal Data Stacked Line and Area Charts
Y-Axis is Numerical Line, Area, Column Charts
X-Axis is Numerical Bar Chart
Constituent Totals Stacked Bar, Column Charts
Percentage Pie Chart
Discrete Events Scatterplot
Discrete Events with X, Y, Z axis Bubble Chart
Lots of measures at once,
High-level
Single Value
Gauge
Compare Locations Map
Splunk Visualizations
Charts
Single Value
Maps

Which Visualization Should I Use?
Do I have at
least 3 or 4
dimensions
of data?
Yes: use a scatterplot or bubble chart
No, two dimensions: use a line or
area chart
No, one dimension: use a bar chart,
column chart, line chart or area chart
Do I need
to show
something
changing
in time?
Yes: use the timechart command
to plot a line, column, area, or a
stacked area chart

xyseries Command
•xyseries
<x-‐field>
<y-‐name-‐field>
<y-‐data-‐field>

•<x-‐field> is the field to use as the x-axis
•<y-‐name-‐field> is the field that contains
the values to be used as labels for the
data series
•<y-‐data-‐field> is the field(s) that
contains the data to be charted

xyseries Command – Example
Evaluate online sales for the previous month.
Scenario ?
A B
C
action=purchase

|
stats
count
by
product_name
categoryId

|
where
count
>
20

|
xyseries

product_name

categoryId

count

|
fillnull
value=-‐

A B C

xyseries vs. stats

•Generally, instead of xyseries, you would use chart
a
over
b
by
c

chart
a
over
b
by
c
is equivalent to:
stats
a
by
b,c
|
xyseries
b
c
a

•However, if you need to do some processing after the chart command,
use stats followed by xyseries


chart vs. xyseries – Examples

|
chart
count(price)
as
count
over

product_name
by
categoryId

|
fillnull
value=0


|
chart
count(price)
as
count
over

product_name
by
categoryId

|
where
count
>
100


|
stats
count(price)
as
count
by

product_name,
categoryId

|
where
count
>
100

|
xyseries
product_name,
categoryId,
count

|
fillnull
value=0

A
B
C
Display the number of retail sales the previous week
that exceeded 100, by product name and category.
Scenario ? A
B
C

Improving Performance
•Refine Searches
•Create Reports
•Schedule Reports
•Accelerate Reports
•Use Summary Indexes
•Accelerate Data Models
•Use tstats Command
•Use Tokens

Schedule Reports
•Schedule a new report when you save
a search or pivot
•Schedule an existing report:
– Navigate to the Reports page, and locate the report
– In theActions column, click Edit > Edit Schedule
– Select a Schedule type
▸Basic: choose from a range of preset options
▸Cron: set up a schedule using standard cron
– Select a Schedule Window (optional)
Note:Thisisforwhentherearemanyconcurrently
scheduledreports.
1
2
3
4

Refine Searches
•Use the most efficient
command for the use case
– tstats
– stats vs. transaction
•Make the base search
as specific as possible
...
|
transaction
trade_id
|
chart
count
by
duration

...
|
stats
range(_time)
as
duration
by
trade_id

|
chart
count
by
duration


Refine Searches (cont.)
•Restrict your search to the specific host, index, source, source type,
or Splunk server
•Limit your search to a specific time window
•Limit the quantity of data retrieved
– For example, use the head command: sourcetype=access_* | head 1000.
•Avoid using NOT expressions when possible
– Instead of using (NOThost=d NOThost=e) or (host!=d OR host!=e),
use (host=a OR host=b OR host=c)

Search Acceleration
Report
Acceleration
• Accelerates individual reports
• Uses automatically-created summaries to speed completion times for qualified reports
• Easier to create than summary indexes and backfill automatically
• Depending on the defined time span, periodically ages out data
• Can correct gaps and overlaps from the UI 'rebuild' feature
• Cannot create a data cube and report on smaller subsets
Summary
Indexing
• Accelerates reports that don't qualify for report acceleration
• Uses manually created summary indexes that exist separate from main indexes
• Useful for searches that don't qualify for report acceleration
• Can persist after events have been frozen by controlling retention period or index size
• Backfill is a manual (scripted) process
Data Model
Acceleration
• Accelerates all of the fields defined in a data model
• Uses automatically-created summaries to speed completion times for pivots
• Takes the form of time-series index files

Report Acceleration
• You
– Your role has the schedule search capability
– You have write permissions for the report you want to accelerate
• The report
– Was not created via Pivot
– The underlying search qualifies for acceleration:
▸usesatransformingcommand(suchaschart,timechart,stats,andtop)
▸onlystreamingcommandsbeforethetransformingcommand
▸basesearchdoesnotuseeventsampling
• Search Mode
– If the underlying search uses verbose mode, it is automagically changed to smart or fast
– You cannot change search mode of an accelerated report to verbose

When Splunk Does NOTAccelerate Reports
•Splunk generally won't generate a summary if:
– There are fewer than 100K events in the summary range –
It’s faster executing the search without a summary
– Summary size is projected to be too large –
It’s faster executing the search because the main index is smaller
•If a summary is defined and not created for the above reasons, Splunk
continues to check periodically, then automatically creates a summary
after it meets the requirements

Summary Indexing
•Efficiently report on large volumes of data
– When a search is run results are saved to a summary index
– Then you can run searches against this smaller, and thus faster, summary index
– Allows the cost of a computationally expensive report to be spread over time
•Amortize costs of reports, over different but overlapping time range
– Summary data generated on aTuesday can be used for a report of the previous
7 days done on the Wednesday,Thursday, or the following Monday.
Summary indexing volume is not counted against your
license, even if you have multiple summary indexes.
Note

Accelerate Data Models
•Use to speed up data models that represent extremely large datasets
•Speeds up reporting for the entire set of attributes (fields)
– Report acceleration and summary indexing speed up
individual searches on a report by report basis
•Set of .tsidx file summaries is created and scheduled searches
are run every five minutes to keep current
•Affects only event object hierarchies
•Most efficient if the root event objects include the index(es) in their
initial constraint search

tstats Command
•Perform statistical queries on
indexed fields in tsidx files
– Normal index data
– tscollect data
– Accelerated Data Models
•Query the tsidx files of a specific
accelerated data model in Search
•Verify a data model is capturing
the data you expect

tstats Command – Example
•Agenerating processor, so it must be the first command in a search pipeline
•Use prestats to pipe results to chart or timechart for creating visualizations
...|
tstats
prestats=t
count
by
_time
span=1d

|
timechart
span=1d
count

Gives a timechart of all the data in your
default indexes with a day granularity.
Scenario ?

Lab 2 – Create a Prototype
Time: 25 – 30 minutes
Questions:
– Answer a set of questions concerning types
of searches.
Tasks:
– Create a dashboard with basic searches
– Schedule and accelerate reports
Challenge Task:
– Create a data model

Module 3:
Using Simple XML

Module Objectives
•Define the simple XMLsyntax
•Name three types of dashboard panels
•Identify six simple XMLpanel objects
•Create a dashboard with panels that use post process searches

Simple XML
•Create and modify dashboards without
writing any simple XMLcode
– Dashboard editor
– Form editor
– Visualization editor
•Can also code simple XML
– Custom charts, gauges, and rangemaps
– Dynamic drilldown behaviors
– Tokenization
– Post-process searches
…and much more
Visualization
Editor
Dashboard & Form Editor

Simple XML (cont.)
•There are many ways to customize
views by editing the simple XML
– Layout
– Tokenization
– CSS styles
– Custom scripts

Simple XML Syntax
Label
Root Element
<dashboard>

<label>Your
Dashboard
Title</label>

...

</dashboard>

Chart Title
Panel Title

Simple XML Syntax (cont.)
inline search
chart

CDATATags
•Special characters in XML files
– Some characters have special
meaning in an XMLfile and cannot
be used literally.
– Wrap the text with special
characters within CDATAtags.
– Or escape special characters
using HTMLentities.
Character
HTML
En/ty

' '
< <
> >
& &
<link>
<![CDATA[
/app/search/form_for_drilldown?
form.sourcetype=$row.sourcetype$&earliest=
$earliest$&latest=$latest$
]]>
</link>
The search in this dynamic
drilldown has an ampersand
CDATA Tags
<![CDATA[
"Text
within
tags"
]]>


Panels
Each panel has six possible visualization elements:
Panel Description Create or Edit Using
1. Chart Displays search results as a chart Visualization editor or XML editor
2. Event List Displays search results as individual events Visualization editor or XML editor
3. HTML Displays inline HTML XML editor only
4. Map Displays search results as map Visualization editor or XML editor
5. Single value Displays a single value visualization
and various styles of gauges
Visualization editor or XML editor
6. Table Displays search results as a table Visualization editor or XML editor
In Splunk 6 and prior, panel type referred to
a visualization. In with Splunk 6.1 and later
a panel is a container for a visualization.
Note

Panel Categories
•Inline: displays content of a search string or post-process
•Report: displays contents of a report
•Prebuilt: displays contents of a preexisting panel
Inline Panel Report Panel
Prebuilt Panel

Creating Prebuilt Panels
•To create a prebuilt panel:
– Select Edit > Edit Panels
– In the panel's options menu, select
Convert to Prebuilt Panel
– Enter a Panel ID
– Select Private or Shared inApp permissions
– Click Save
1
2
4
3
5
Note
Repeat these steps on a converted prebuilt panel
to go back to an inline panel or panel from report.

Adding Prebuilt Panels
•To add a prebuilt panel:
– Click +Add Panel
– Click Add Prebuilt Panel
– Select a panel
– Click Add to Dashboard
4
5
3
2
1

Cloning Panels
•You can also add a panel
from another dashboard
by cloning the panel:
– Click +Add panel
– Click Clone from Dashboard
– Select a dashboard
– Select a panel
– Click Add to Dashboard
1
2
3
4
5
6
Note
You can also clone a panel from the Prebuilt
Panels page.

Managing Panels
•To manage all prebuilt panels:
– Select Settings > User interface
– Click Prebuilt panels

Grouping Panels
•Defined in simple XML
– Single value panels group horizontally
– All other panels group vertically

Chart Customization
•Using the Panel Editor
– change chart axis labels
– define color ranges for gauges
– and much more
Default Chart
Options

Chart Customization (cont.)
•Edit the panel XMLdirectly
to customize the appearance
and behavior of your charts
– axis label text styles
– reverse chart axes
– chart colors
– chart height
– and much more
Default Chart Colors
Custom Chart Colors

Chart Customization – Example
• Changing Chart Colors
1. Add charting.fieldcolors option
2. Add hexidecimal for each data series
2
1

Panel Customization – Example
• Changing Chart & Panel Colors
– Set background color: charting.backgroundColor
– Set foreground color: charting.foregroundColor
– Set font color: charting.fontColor
– Set series colors: charting.seriesColor
4
3
2
1

Single Value Customization
Example: Using a timechart command to
get a total number of purchases over time.
4
3
2
1

Single Value Customization (cont.)
Example: Using a stats command
to get a total number of purchases.
4
3
2
1

Choropleth Map
•Uses shading to show relative metrics
•Color Modes
– Categorical mode: Identify regions that share the same value
– Divergent mode: Identify regions with high or low values.
– Sequential mode: Identify regions with high values.
Categorical Divergent Sequential

Choropleth Map Components
•Data with location information
– latitude and longitude coordinates OR
– location names that match names in a lookup
•KMZ (Keyhole Markup Language) Lookup Table
– Defines region boundaries
– Built-in KMZ Lookups:
ê geo_us_states, United States
ê geo_countries, countries of the world
•Lookup definition
– Matches location coordinates in data to the KMZ file
– Built-in lookups definitions for the U.S. and world countries
KML files are not currently supported. Convert
a KML file into KMZ, by compressing the file
and replacing the '.zip' extension with '.kmz'
Note

Choropleth Map Search
•Data with locations Transforming search geom Command
VendorID
<
4000

|
stats
count
as
Sales
by
VendorStateProvince

|
geom
geo_us_states
featureIdField=VendorStateProvince

status!=200

|
dedup
clientip

|
iplocation
prefix=cip_
clientip

|
stats
count
by
cip_Country

|
geom
geo_countries
featureIdField=cip_Country

Example 2
Example 1
1
2
3
1
2
3
Display web server errors worldwide.
Scenario ?
Display vendor sales by U.S. State.
Scenario ?
1 2 3

Global & Post Process Searches
•Specify the global search using
<search id="MyBaseSearch">
– Typically, the global search is a
transforming search
•Specify post process using
<search base="MyBaseSearch">
•Can use multiple global
searches on a dashboard
Passing a large number of search results from
a global search can cause a server time out.
Warning

• The global search must gather appropriate statistics for the downline processing
• The results are further processed
Global & Post Process Searches (cont.)
Base search ID
Base search reference
Post-process
Base search
Note
See also, Post-process searches in the
Dashboards & Visualizations manual

• Base search requires all fields identified in the data cube to have a value
• If necessary, remove null values in the post-process
Global & Post Process Searches – Null Values

Lab 3 – Improve Performance
Time: 30 - 35 minutes
Tasks:
– Create prebuilt panels
– Create a dashboard
– Create a global search that uses:
▸tstats command
▸accelerated data model
– Add panels driven by post-process searches
– Add a choropleth map
– Customize chart and single value colors

Module 4:
Creating Forms

Module Objectives
•Name two types of forms
•Identify seven types of form inputs
•Describe how tokens work
•Use tokens to create:
– cascading menus
– link switchers

Dynamic Form
Forms
•Present a simplified search interface
•Hides an underlying search string
•Can be generated as PDFs
•Form types
– Simple: contain one or more input boxes
– Dynamic: inputs are dynamically
populated by searches
Simple Form

Simple XML Syntax
•Form views begin and end with the <form></form> tags
•The <fieldset> tag defines a form input
Root element
Search to perform with
variable ($token$)

Tokens
•Placeholder for dynamic values
•Use in search strings, form inputs,
panel titles, HTMLpanels, links,
and chart options
•When the value changes, all
related items are updated
sourcetype="vendor_sales"
product_name="$p_name$"

|
stats
count
by
VendorCity,
Vendor
|
sort
-‐count

Access the value of a token
using $...$ delimiters.

Tokens – Example
Define in the form input.
Use in <title>
and <query>


Token Filters
Token filters ensure that you correctly capture the value of a token
Use Case
Token
Description
Wrap in quotes
$token_name|s$

Adds quotation marks around the value referenced
by the token and escapes those within the value.
HTML format
$token_name|h$

Ensures the token value is valid for HTML formatting.
Default for <HTML> element token values.
URL format
$token_name|u$

Ensures the token value is valid to use as a URL.
Default for <link> element token values.
If you include static text that contains the $ character,
use $$ to escape the token delimiter value.
Note

Token Filters – Example
index=main
sourcetype="access_combined"
|
timechart
count
by
sourcetype
Here the value of sourcetype_tok
is access_combined in quotes.
<search>

<query>

index=main
sourcetype=$sourcetype_tok|s$
|
timechart
count
by
sourcetype

</query>

</search>

Use the |s filter to place quotation
marks around the value returned.

Tokens Summary
Use Case Defined Used Within Description Element
1 Search User defined Search string
Insert a term within a search string that uses a
value defined elsewhere.
<query>

2 Inputs User defined Form input
Capture user input to modify the data displayed
in a panel.
<input>

3 Multiple time pickers User defined Form panel
Indicate which time picker to use for each panel,
on forms with multiple time pickers.
<input
type=time>

4 Conditional display User defined Dashboard panel Specify conditions for the display of panels and
their contents based on the value of the token.
Can be used with <drilldown> or <input>
<condition>

5 Dynamic drilldown Predefined Dashboard panel Capture the value clicked for use in the drilldown. <drilldown>

6 Pan and zoom Predefined Dashboard panel Select a time range within a chart to zoom into. <selection>


Form Inputs
•Seven types
– Text box
– Dropdown list
– Radio button
– Checkbox
– Multi-selection
– Time range picker
– Submit
•Add to a form or panel
– Drag inputs onto panels
•Free form input for multi-select
and dropdown

Form Inputs – Text
Search that returns results to the
table – variable part of search (the
token) is surrounded by $’s
Part of the search
the user enters

Form Inputs – Multi-select
Search that returns
results to table – the
variable (token) is
surrounded by $’s
Part of the search
the user selects
Search that generates
multiple select options

Form Inputs – Checkbox
Search that generates
checkbox selections
Part of the search
the user selects
Search that returns
results to table – the
variable (token) is
surrounded by $’s

Tokens – Cascading Inputs
Use the selection of one form
input to reduce or set the
values of another form input

Tokens – Link Switcher
•Toggle dashboard content
– Time range
– Visualization
– Search Time picker

Core Drilldown
•Allows for user interaction
with view objects
•Can dispatch a search on
chart or table elements
– Clicking an object redirects
you to search view
– Ctrl-clicking an object opens
search in new window

Ultra Drilldown
•Provides drilldown options
without hidden modifier keys
•Click on an event to see a
contextual pop-up menu
– Add to search
– Exclude from search
– Create a search

Dynamic Drilldown
•Pass a value to another panel, form,
dashboard or external site from:
– Chart
– Map
– Table row or field
– Multiple table fields
•Use hidden fields in tables
– Referenced only in drilldown links

Dynamic Drilldown – Chart
Dashboard: use $click.value$ to pass a value from a chart click
token field destination value
target view path

Dynamic Drilldown – Table Row
Dashboard: use $row.fieldname$ to pass a value from row click
token field destination value
target view path

Dynamic Drilldown – Table Field
Dashboard: use the field=name option to pass a value from a field click
field=name option
Same syntax as table row

Dynamic Drilldown – Destination Form
Add the token field to the search string to receive the value in the form view
replacement token

Dynamic Drilldown – Multiple Table Fields
Dashboard: use field=name with multiple link tags to make multiple fields clickable
Same syntax as:
- table row
- table field

Dynamic Drilldown – Multiple Table Fields (cont.)
Destination Form: Make sure you indicate default values for all token fields

Dynamic Drilldowns – dest_value
Indicates the value to capture from a table or chart
dest_value Description Notes
click.name
click.name2
Passes the name of the field
that is clicked on.
• click.name: The value in a table column of the field that is clicked.
• click.name2: The value in a table row of the field that is clicked.
• Use with tables.
click.value
click.value2
Passes the value of the field
that is clicked on.
• click.value: The value in a table column of the field that is clicked.
• click.value2: The value in a table row of the field that is clicked.
• Use with all charts, except bar charts. For bar charts, these values are reversed.
• Multivalue fields in a table use click.value2.
earliest
latest
Passes the earliest and latest
times of a search.
• Use as parameters to URL for the target view.
• For example, add: &earliest=$earliest$&latest=$latest$ to the drilldown target view
URL. In this example, use CDATA to escape the '&’.
form.token
Passes the token accepted as
input by the target form.
• Use as a parameter to URL for the target form.
row.fieldname
Passes the value of the field
named, for the entire row.
• Specifies the field from the selected row or column from which to capture the value.
• Use with tables.

Event Handlers
•Event handlers "listen" for specific events to occur, then perform
an event action:
– Set or unset tokens
– Execute an eval function
– Link to a page
•Three types of event handlers:
– Form inputs
– Searches
– Visualizations

Event Action Syntax
•Set or unset tokens
<set
token="myToken">$text$</set>

<unset
token="myToken">$text$</unset>

•Execute an eval function
<eval
token="myToken">tostring(round('field'))</eval>

•Link to a page
<link>/app/myApp/my_form?form.token=$text$</link>


Form Input Event Handler
•Use the <change> element to
capture the label or value from
an input and perform event actions
with them
•Applies to these inputs:
– checkbox
– dropdown
– link
– multiselect
– radio
– text

Form Input Event Handler – Example

Search Event Handler
•Use predefined tokens to access
search results or properties and
perform event actions with them
•Predefined tokens include:
– results.field: access the
value of the named field
– job.property: access the value
of the named search job property
See the Knowledge Manager Manual for
a complete list of search job properties.
Note
Search Event Description
<cancelled> Execute actions when a search is cancelled.
<done> Execute actions based on done search events.
<error> Execute actions when there is an error in the search.
<finalized> Execute an action when a search finalizes and data is
available
<progress> • Execute an action on search progress events.
• Similar to the preview event handler.
• Access job properties and field results. The progress
event has only job properties information available.
<preview> • Execute an action when search preview data is available.
• Similar to the progress event handler.
• Access job properties and field results. The preview
event has only preview information available.

Search Event Handler – Example

Visualization Event Handler
•Use the drilldown or selection elements to perform event actions
– Available for: chart, event, map, single, or table
•<selection>

– Sets the time window for the pan and zoom feature of charts
– Use tokens to set other values, such as the numerical values of a chart's x-axis
•<drilldown>

– Enable event actions for drilldown behaviors
Selection element applies only to area,
column, or line charts.
Note

Visualization Event Handler – Example
•Pan & Zoom
Example: Use <selection> in chart 1
to pick a time range for chart 2

condition Element
•Use with event handlers to specify the scope of event actions
•Define complex conditional matching, token filtering and formatting
•Use tokens based on search metadata, results, and job information
Attributes
Name Type Default Description
label text *
Specifies the input <label> element to which the condition
applies. '*' applies the condition to all input <label> elements.
match eval expression —
An eval expression that defines the conditions needed for
actions to be executed.
value text *
Specifies the input <value> element to which the condition
applies. '*' applies the condition to all input <value> elements.

condition Element – Input
•Specify actions based on input choices
The <condition> element is not
available for multiselect inputs.
Note
Example: Use conditional inputs to
select preset time ranges for a search.
<condition
label="[text]>">

..
actions
...

</condition>


condition Element – Search
•Specify the scope of actions based
on an eval expression
<condition
match="[eval
expression]>">

..
actions
...

</condition>

Example: If zero
results are returned,
show the html panel.
If the results = 0,
set a token called foob.
If there is a token called foob,
show the html panel.

condition Element – Drilldown
•Limits the scope of drilldown actions to clicks on specific fields and adds logic
<condition
field="[text]>">

..
actions
...

</condition>

Example: In-page
drilldown to hidden panel.

Lab 4 – Add Interactivity
Tasks:
– Create a form
– Add cascading multiselect inputs
– Add chart panels
– Add a visualization event handler
– Add a dynamic drilldown
– Create a drilldown destination form

Module 5:
Customizing Dashboards

Module Objectives
•Use simple XMLextensions
•Identify types of search managers
•Link search managers to views
•Explain how autodiscovery works
•Add a D3 visualization to a dashboard

Navigation
•Once you've built reports and views,
you can create navigation
– Go to Settings > User interface
> Navigation menus
– Select the app context for your view
– Edit the default file to specify the
order and menus for your views
nav menu
bar color
Default Navigation Bar
1
2
3

Navigation (cont.)
Sub-menu with HTML pages
renamed for this menu
Views with file names
that include "marketing"
Searches with names
that include "Top"
All views not listed
in default.xml
Default view
Nav bar color

Simple XML Extensions
•CSS and JavaScript extensions
•Customize the look and behavior of an app
by adding or modifying CSS & JS files in:
$APP_HOME/appserver/static

and referencing them in the view:

<dashboard
stylesheet="my_style.css"

script="my_script.js">

•Customize dashboards and panels
– For all, use: dashboard.css or dashboard.js
– For individual: my_style.css or my_script.js


Simple XML Extensions (cont.)
•Six extension points are provided:
– Drilldown: customize the drilldown event
for tables, charts, and other elements
– Visualization: integrate a custom
visualization as a panel
– Layout: make simple layout changes
– Stylesheet: use a custom CSS
– Table Cell Renderer: custom styles and
behaviors within table cells
– Token Setting: set custom tokens for a
dashboard page With simple XML extensions, you can still use the
visual editor and PDF generation is available.
Note

Extension Points – Drilldown
table_drilldown_url_field.js

Extension Points – Visualization
parallelcoords.js

Extension Points – Layout
custom_layout_overlay_single.css

Extension Points – Stylesheet
custom_decorations.css

Extension Points – Table Cell Renderer
table_icons_inline.js table_decorations.css

Extension Points – Tokens
Use JavaScript to set tokens in panel titles, html panels, and for drilldown.
set_app_token.js

Linking Searches & Views
•Search ID for search
•Link the search to the
view with the search ID
•Reference the path
and JavaScript file
Search
1
3
1
2
3
2
View

Linking Reports & Views
•Search ID for a report
(saved search)
•Link the report to the
view with the search ID
•Reference the path
and JavaScript file
1
3
1
2
2
Report
3
View
There is an assumed path that includes:
/appserver/static/
For example, "app/sales/appserver/static/
Important

Linking Global Searches & Views
•Search ID for a
global search
•Post process
•Link the post process
to the view with the
search ID
1
2
3
2
1
3
Post process
View

Troubleshooting
•Verify the manager id matches the search manager id
•Verify the path of data-‐require
– app/<app_name>/path/to/<js_file_without_extension>

•Verify all XMLattributes are properly escaped
– Escape all special characters in the search query: ' < > "
•Verify any tokens with values that contain the $ character are escaped by
using two dollar signs: $$token$$

•Verify the value of data-options is a valid JavaScript object
– Comma and quote placements
•Look for any JavaScript errors in the browser

SimpleSplunkView.js
•Extend this base class to create custom views
•Several methods that you can override as needed:
– initialize: the constructor
– formatData: formats results data from Splunk and returns a handle to the data
– formatResults: same as formatData, except you can format the full set of data
from the results model
– createView: configures the custom visualization and returns a handle to it
– updateView: puts Splunk data into the view
– clearView: resets rendering
– render: creates the initial view and draws it on the screen. On subsequent calls,
runs a full update cycle by calling formatResults, formatData, then updateView


Example – tagcloud.js
•Load dependencies
– Underscore and jquery libraries
– SimpleSplunkView base class
– any optional SplunkJS libraries
and stylesheets
•Declare a new class (inheriting
from SimpleSplunkView base
class) and its options
•Define any events
– Here, drilldown when clicking a tag
•Re-render the view when the
settings change
1
1
2
3
2
3
4
4

Example – tagcloud.js (cont.)
•When the search runs,
update the view
– Clear the current view
– Extract and convert the
magnitude field value
– Find the maximum and minimum
of the magnitude values
– Calculate relative size of
each tag and render
5
5

autodiscover.js
•Re-use custom components,
especially visualizations, in
multiple dashboards and apps
•It looks for splunk-‐view
or class elements in your
embedded JavaScript
•The splunkjs/ready script
must be loaded to perform
auto-discovery
autodiscover.js
splunkjs/ready! script should not be loaded before
dashboard renders. Ensure this by loading it within a
splunkjs/mvc/simplexml/ready! loader script callback.
Note

autodiscover.js (cont.)
•Two methods to enable auto-discover:
– Use a dedicated JavaScript file in the
appserver/static directory and load
the code from the dashboard using the
custom script attribute
– Include the code in dashboard.js
ê This loads automatically for every simple XML
dashboard within the app
ê With this method, auto-discovery works for all
dashboards without specifically enabling it
Dedicated JS file referenced in XML

HTML Conversion
•To work in HTML& JavaScript, you can easily convert simple XMLviews
– Then, add CSS, change the layout, edit the JavaScript to add interactivity, etc.

HTML Conversion (cont.)
•Layout is converted into Splunk styles
•Elements and forms are converted to
SplunkJS Stack equivalents
•Definition of each element and form
is converted into JavaScript
– Properties
– Auto-generated ID
•All searches are extracted and
represented in SplunkJS

Lab 5 – Add Advanced Visualizations
Tasks:
– Create a dashboard
– Add simple xml extensions
– Add an HTMLpanel
– Add a search id
– Add search manager and visualization
Challenge Tasks:
– Add a D3 visualization
– Customize multiselect inputs

Summary

Wrap-up
You should now be able to:
•Generate, format, and customize charts
•Create dashboards and forms that use:
– post process searches
– tokens
•Use JavaScript and CSS to create customize visualizations

Wrap-up
You should now be able to:
•Use best practices for planning and creating views
•Define data structure requirements for visualizations
•Create efficient, well-formed searches that generate charts
•Edit simple xml to enable:
– global searches
– tokens
•Use JavaScript and CSS to create interactive visualizations
•Troubleshoot views

Next Steps
Apps
•Splunk Dashboard Examples
•Common Information Model
Courses
•Advanced Search & Reporting
•Building SplunkApps
•Splunk 6Administration
License
•Developer Trial License

Support Programs
• Community
– SplunkAnswers:answers.splunk.com
– SplunkDev:dev.splunk.com
– SplunkDocs:docs.splunk.com
– Wiki:wiki.splunk.com
– IRC Channel: #splunk on the EFNet IRC server
• Global Support
Supportforcriticalissues,adedicatedresourcetomanageyouraccount–24x7x365.
– Email:support@splunk.com
– Web:http://www.splunk.com/index.php/submit_issue
• Enterprise Support
– Access customer support by phone and manage your cases online 24 x 7
(depending on support contract).

• Complete the survey to be in this month's drawing for a $100 Splunk Store voucher
– Checkyourinboxforalinktothesurvey,oraccessthelinkontheMyProfilepage
Thank You

5DV Advanced_Dashboards_and_Visualisations.pdf

More Related Content

Similar to 5DV Advanced_Dashboards_and_Visualisations.pdf (20)

Recently uploaded (20)

5DV Advanced_Dashboards_and_Visualisations.pdf