SlideShare a Scribd company logo
IRM Summit 2014
Customer Intelligence:
Using the ELK stack (Elasticsearch,
Logstash and Kibana) to analyse
ForgeRock OpenAM audit data
warren.strange@forgerock.com
IRM Summit 2014
Make pretty pictures
for the boss
3IRM Summit 2014
Coincidence?
OpenDJ, OpenIDM,
OpenAM produce copious
amounts of audit data
Analysis of that data is left as an
exercise for the reader
Many great SIEM tools
Desire for an Open Source
solution for data analysis
What is the ELK stack?
Elasticsearch: “No SQL” database
Logstash: Log collection and
transformation
Kibana: Data visualizer for
Elasticsearch
Yes, but what does ELK do?
Collect, analyse and visualize data
Any kind of data
Github (8 Million repos), Soundcloud (30M users), The Guardian (40M documents)
Answer questions:
● Where are my users coming from?
● What is the traffic in North America vs.
Europe?
● Why do I see an employee logging in from
Canada?
Elasticsearch
● NoSQL, REST/json, document oriented,
schemaless, “Big data” full text search engine
● Apache 2.0 license
● Sweet spot is rapid full text search / ad hoc queries
● Not a replacement for an RDBMS
● Not transactional, not ACID, etc.
● Built on Apache Lucene project
Logstash
● Swiss army knife of log collection,
transformation and forwarding
● JRuby based
● Large footprint :-(
● lumberjack
● go based collector that feeds into logstash
● Very lightweight, small footprint
Kibana
Logstash flow
Input source
files, database,
syslog, etc.
Filters
grep, regex,
geoIP, ...
Output
elasticsearch, file,
db, syslog
“Plugin” based architecture. Add new
plugins for input, output and filters
Logstash example
Input source
file: amAccess.*
type: amAccess
Filters
Map IP
address to
GEO location
Output
elasticsearch:9100
Read from
OpenAM access
logs
Add Geo
Location data
Write the result
to Elasticsearch
Geek Alert!!
input {
file {
type => amAccess
path => "/logs/am/log/amAuthentication.*"
}
}
Input section
Wildcards can be
used
Data is tagged with a type.
Use this to classify & search
by type
filter {
if [type] == "amAccess" {
csv {
columns => [time,Data,LoginID,ContextID, IPAddr, LogLevel,
Domain, LoggedBy, MessageID, ModuleName, NameID,
HostName]
separator => " "
}
date {
match => ["time", "yyyy-MM-dd HH:mm:ss"]
}
geoip {
database => "/usr/share/GeoIP/GeoIP.dat"
source => ["IPAddr"]
}
}
}
Filter apply to type
Parse the data
as csv
Normalize the date to
a common format
Enrich the record
with GEO location
output {
stdout {
codec => rubydebug
}
elasticsearch { host => localhost }
}
Output
Send the data to
Elasticsearch and the
stdout
Demo Time
As seen on youtube!
http://youtu.be/tvrYuSLuGik
27 49 views!
ELK Demo Environment
OpenAM OpenDJ OpenIDM
logstash
elasticsearch:9100
Apache:80/443 +
Policy Agent
Log Files
Kibana
Marketing Genius?
Where to hold the next ForgeRock Summit: Europe, USA, or Canada?
Asks you to find out pronto:
● What country are customers visiting the ForgeRock website from?
● How are they authenticating (forgerock account, or federated?)
The next IRM summit
location:
We have beer!
Bring your toque!
Next Steps
Delivery Models
Cloud or Appliance?
Interested in collaborating?
Share logstash config, kibana reports,
etc.
Puppet/Chef/Ansible/Docker installers?
Ad

More Related Content

What's hot (20)

Elastic Stack Introduction
Elastic Stack IntroductionElastic Stack Introduction
Elastic Stack Introduction
Vikram Shinde
 
Elk - An introduction
Elk - An introductionElk - An introduction
Elk - An introduction
Hossein Shemshadi
 
Log analysis with the elk stack
Log analysis with the elk stackLog analysis with the elk stack
Log analysis with the elk stack
Vikrant Chauhan
 
Elastic stack Presentation
Elastic stack PresentationElastic stack Presentation
Elastic stack Presentation
Amr Alaa Yassen
 
Introduction to Kibana
Introduction to KibanaIntroduction to Kibana
Introduction to Kibana
Vineet .
 
Log analytics with ELK stack
Log analytics with ELK stackLog analytics with ELK stack
Log analytics with ELK stack
AWS User Group Bengaluru
 
ELK introduction
ELK introductionELK introduction
ELK introduction
Waldemar Neto
 
Centralized log-management-with-elastic-stack
Centralized log-management-with-elastic-stackCentralized log-management-with-elastic-stack
Centralized log-management-with-elastic-stack
Rich Lee
 
Introducing ELK
Introducing ELKIntroducing ELK
Introducing ELK
AllBits BVBA (freelancer)
 
Log analysis using elk
Log analysis using elkLog analysis using elk
Log analysis using elk
Rushika Shah
 
The Elastic ELK Stack
The Elastic ELK StackThe Elastic ELK Stack
The Elastic ELK Stack
enterprisesearchmeetup
 
Elastic Stack ELK, Beats, and Cloud
Elastic Stack ELK, Beats, and CloudElastic Stack ELK, Beats, and Cloud
Elastic Stack ELK, Beats, and Cloud
Joe Ryan
 
Elk stack
Elk stackElk stack
Elk stack
Jilles van Gurp
 
Fleet and elastic agent
Fleet and elastic agentFleet and elastic agent
Fleet and elastic agent
Ismaeel Enjreny
 
Redis + Apache Spark = Swiss Army Knife Meets Kitchen Sink
Redis + Apache Spark = Swiss Army Knife Meets Kitchen SinkRedis + Apache Spark = Swiss Army Knife Meets Kitchen Sink
Redis + Apache Spark = Swiss Army Knife Meets Kitchen Sink
Databricks
 
HTTP Analytics for 6M requests per second using ClickHouse, by Alexander Boc...
HTTP Analytics for 6M requests per second using ClickHouse, by  Alexander Boc...HTTP Analytics for 6M requests per second using ClickHouse, by  Alexander Boc...
HTTP Analytics for 6M requests per second using ClickHouse, by Alexander Boc...
Altinity Ltd
 
Introduction to elasticsearch
Introduction to elasticsearchIntroduction to elasticsearch
Introduction to elasticsearch
hypto
 
LDAP
LDAPLDAP
LDAP
Chandanapriya Sathavalli
 
Kibana Tutorial | Kibana Dashboard Tutorial | Kibana Elasticsearch | ELK Stac...
Kibana Tutorial | Kibana Dashboard Tutorial | Kibana Elasticsearch | ELK Stac...Kibana Tutorial | Kibana Dashboard Tutorial | Kibana Elasticsearch | ELK Stac...
Kibana Tutorial | Kibana Dashboard Tutorial | Kibana Elasticsearch | ELK Stac...
Edureka!
 
Processing Large Datasets for ADAS Applications using Apache Spark
Processing Large Datasets for ADAS Applications using Apache SparkProcessing Large Datasets for ADAS Applications using Apache Spark
Processing Large Datasets for ADAS Applications using Apache Spark
Databricks
 
Elastic Stack Introduction
Elastic Stack IntroductionElastic Stack Introduction
Elastic Stack Introduction
Vikram Shinde
 
Log analysis with the elk stack
Log analysis with the elk stackLog analysis with the elk stack
Log analysis with the elk stack
Vikrant Chauhan
 
Elastic stack Presentation
Elastic stack PresentationElastic stack Presentation
Elastic stack Presentation
Amr Alaa Yassen
 
Introduction to Kibana
Introduction to KibanaIntroduction to Kibana
Introduction to Kibana
Vineet .
 
Centralized log-management-with-elastic-stack
Centralized log-management-with-elastic-stackCentralized log-management-with-elastic-stack
Centralized log-management-with-elastic-stack
Rich Lee
 
Log analysis using elk
Log analysis using elkLog analysis using elk
Log analysis using elk
Rushika Shah
 
Elastic Stack ELK, Beats, and Cloud
Elastic Stack ELK, Beats, and CloudElastic Stack ELK, Beats, and Cloud
Elastic Stack ELK, Beats, and Cloud
Joe Ryan
 
Redis + Apache Spark = Swiss Army Knife Meets Kitchen Sink
Redis + Apache Spark = Swiss Army Knife Meets Kitchen SinkRedis + Apache Spark = Swiss Army Knife Meets Kitchen Sink
Redis + Apache Spark = Swiss Army Knife Meets Kitchen Sink
Databricks
 
HTTP Analytics for 6M requests per second using ClickHouse, by Alexander Boc...
HTTP Analytics for 6M requests per second using ClickHouse, by  Alexander Boc...HTTP Analytics for 6M requests per second using ClickHouse, by  Alexander Boc...
HTTP Analytics for 6M requests per second using ClickHouse, by Alexander Boc...
Altinity Ltd
 
Introduction to elasticsearch
Introduction to elasticsearchIntroduction to elasticsearch
Introduction to elasticsearch
hypto
 
Kibana Tutorial | Kibana Dashboard Tutorial | Kibana Elasticsearch | ELK Stac...
Kibana Tutorial | Kibana Dashboard Tutorial | Kibana Elasticsearch | ELK Stac...Kibana Tutorial | Kibana Dashboard Tutorial | Kibana Elasticsearch | ELK Stac...
Kibana Tutorial | Kibana Dashboard Tutorial | Kibana Elasticsearch | ELK Stac...
Edureka!
 
Processing Large Datasets for ADAS Applications using Apache Spark
Processing Large Datasets for ADAS Applications using Apache SparkProcessing Large Datasets for ADAS Applications using Apache Spark
Processing Large Datasets for ADAS Applications using Apache Spark
Databricks
 

Similar to Customer Intelligence: Using the ELK Stack to Analyze ForgeRock OpenAM Audit Data (20)

Logstash-Elasticsearch-Kibana
Logstash-Elasticsearch-KibanaLogstash-Elasticsearch-Kibana
Logstash-Elasticsearch-Kibana
dknx01
 
Sumo Logic QuickStart Webinar - Jan 2016
Sumo Logic QuickStart Webinar - Jan 2016Sumo Logic QuickStart Webinar - Jan 2016
Sumo Logic QuickStart Webinar - Jan 2016
Sumo Logic
 
Sumo Logic "How to" Webinar: Advanced Analytics
Sumo Logic "How to" Webinar: Advanced AnalyticsSumo Logic "How to" Webinar: Advanced Analytics
Sumo Logic "How to" Webinar: Advanced Analytics
Sumo Logic
 
Elk with Openstack
Elk with OpenstackElk with Openstack
Elk with Openstack
Arun prasath
 
Elk presentation 2#3
Elk presentation 2#3Elk presentation 2#3
Elk presentation 2#3
uzzal basak
 
How ElasticSearch lives in my DevOps life
How ElasticSearch lives in my DevOps lifeHow ElasticSearch lives in my DevOps life
How ElasticSearch lives in my DevOps life
琛琳 饶
 
Strata Presentation: One Billion Objects in 2GB: Big Data Analytics on Small ...
Strata Presentation: One Billion Objects in 2GB: Big Data Analytics on Small ...Strata Presentation: One Billion Objects in 2GB: Big Data Analytics on Small ...
Strata Presentation: One Billion Objects in 2GB: Big Data Analytics on Small ...
randyguck
 
Fluentd and Docker - running fluentd within a docker container
Fluentd and Docker - running fluentd within a docker containerFluentd and Docker - running fluentd within a docker container
Fluentd and Docker - running fluentd within a docker container
Treasure Data, Inc.
 
Fluentd and Docker - running fluentd within a docker container
Fluentd and Docker - running fluentd within a docker containerFluentd and Docker - running fluentd within a docker container
Fluentd and Docker - running fluentd within a docker container
Treasure Data, Inc.
 
Azure Stream Analytics : Analyse Data in Motion
Azure Stream Analytics  : Analyse Data in MotionAzure Stream Analytics  : Analyse Data in Motion
Azure Stream Analytics : Analyse Data in Motion
Ruhani Arora
 
What is going on? Application Diagnostics on Azure - Copenhagen .NET User Group
What is going on? Application Diagnostics on Azure - Copenhagen .NET User GroupWhat is going on? Application Diagnostics on Azure - Copenhagen .NET User Group
What is going on? Application Diagnostics on Azure - Copenhagen .NET User Group
Maarten Balliauw
 
Elk scilifelab
Elk scilifelabElk scilifelab
Elk scilifelab
Guillermo Carrasco Hernández
 
«Что такое serverless-архитектура и как с ней жить?» Николай Марков, Aligned ...
«Что такое serverless-архитектура и как с ней жить?» Николай Марков, Aligned ...«Что такое serverless-архитектура и как с ней жить?» Николай Марков, Aligned ...
«Что такое serverless-архитектура и как с ней жить?» Николай Марков, Aligned ...
it-people
 
Visualizing Big Data in Realtime
Visualizing Big Data in RealtimeVisualizing Big Data in Realtime
Visualizing Big Data in Realtime
DataWorks Summit
 
Data saturday malta - ADX Azure Data Explorer overview
Data saturday malta - ADX Azure Data Explorer overviewData saturday malta - ADX Azure Data Explorer overview
Data saturday malta - ADX Azure Data Explorer overview
Riccardo Zamana
 
Using Sumo Logic - Apr 2018
Using Sumo Logic - Apr 2018Using Sumo Logic - Apr 2018
Using Sumo Logic - Apr 2018
Sumo Logic
 
ELK-Stack-Grid-KA-School.pptx
ELK-Stack-Grid-KA-School.pptxELK-Stack-Grid-KA-School.pptx
ELK-Stack-Grid-KA-School.pptx
abenyeung1
 
Berlin Buzz Words - Apache Drill by Ted Dunning & Michael Hausenblas
Berlin Buzz Words - Apache Drill by Ted Dunning & Michael HausenblasBerlin Buzz Words - Apache Drill by Ted Dunning & Michael Hausenblas
Berlin Buzz Words - Apache Drill by Ted Dunning & Michael Hausenblas
MapR Technologies
 
Elasticsearch an overview
Elasticsearch   an overviewElasticsearch   an overview
Elasticsearch an overview
Amit Juneja
 
Sherlock: an anomaly detection service on top of Druid
Sherlock: an anomaly detection service on top of Druid Sherlock: an anomaly detection service on top of Druid
Sherlock: an anomaly detection service on top of Druid
DataWorks Summit
 
Logstash-Elasticsearch-Kibana
Logstash-Elasticsearch-KibanaLogstash-Elasticsearch-Kibana
Logstash-Elasticsearch-Kibana
dknx01
 
Sumo Logic QuickStart Webinar - Jan 2016
Sumo Logic QuickStart Webinar - Jan 2016Sumo Logic QuickStart Webinar - Jan 2016
Sumo Logic QuickStart Webinar - Jan 2016
Sumo Logic
 
Sumo Logic "How to" Webinar: Advanced Analytics
Sumo Logic "How to" Webinar: Advanced AnalyticsSumo Logic "How to" Webinar: Advanced Analytics
Sumo Logic "How to" Webinar: Advanced Analytics
Sumo Logic
 
Elk with Openstack
Elk with OpenstackElk with Openstack
Elk with Openstack
Arun prasath
 
Elk presentation 2#3
Elk presentation 2#3Elk presentation 2#3
Elk presentation 2#3
uzzal basak
 
How ElasticSearch lives in my DevOps life
How ElasticSearch lives in my DevOps lifeHow ElasticSearch lives in my DevOps life
How ElasticSearch lives in my DevOps life
琛琳 饶
 
Strata Presentation: One Billion Objects in 2GB: Big Data Analytics on Small ...
Strata Presentation: One Billion Objects in 2GB: Big Data Analytics on Small ...Strata Presentation: One Billion Objects in 2GB: Big Data Analytics on Small ...
Strata Presentation: One Billion Objects in 2GB: Big Data Analytics on Small ...
randyguck
 
Fluentd and Docker - running fluentd within a docker container
Fluentd and Docker - running fluentd within a docker containerFluentd and Docker - running fluentd within a docker container
Fluentd and Docker - running fluentd within a docker container
Treasure Data, Inc.
 
Fluentd and Docker - running fluentd within a docker container
Fluentd and Docker - running fluentd within a docker containerFluentd and Docker - running fluentd within a docker container
Fluentd and Docker - running fluentd within a docker container
Treasure Data, Inc.
 
Azure Stream Analytics : Analyse Data in Motion
Azure Stream Analytics  : Analyse Data in MotionAzure Stream Analytics  : Analyse Data in Motion
Azure Stream Analytics : Analyse Data in Motion
Ruhani Arora
 
What is going on? Application Diagnostics on Azure - Copenhagen .NET User Group
What is going on? Application Diagnostics on Azure - Copenhagen .NET User GroupWhat is going on? Application Diagnostics on Azure - Copenhagen .NET User Group
What is going on? Application Diagnostics on Azure - Copenhagen .NET User Group
Maarten Balliauw
 
«Что такое serverless-архитектура и как с ней жить?» Николай Марков, Aligned ...
«Что такое serverless-архитектура и как с ней жить?» Николай Марков, Aligned ...«Что такое serverless-архитектура и как с ней жить?» Николай Марков, Aligned ...
«Что такое serverless-архитектура и как с ней жить?» Николай Марков, Aligned ...
it-people
 
Visualizing Big Data in Realtime
Visualizing Big Data in RealtimeVisualizing Big Data in Realtime
Visualizing Big Data in Realtime
DataWorks Summit
 
Data saturday malta - ADX Azure Data Explorer overview
Data saturday malta - ADX Azure Data Explorer overviewData saturday malta - ADX Azure Data Explorer overview
Data saturday malta - ADX Azure Data Explorer overview
Riccardo Zamana
 
Using Sumo Logic - Apr 2018
Using Sumo Logic - Apr 2018Using Sumo Logic - Apr 2018
Using Sumo Logic - Apr 2018
Sumo Logic
 
ELK-Stack-Grid-KA-School.pptx
ELK-Stack-Grid-KA-School.pptxELK-Stack-Grid-KA-School.pptx
ELK-Stack-Grid-KA-School.pptx
abenyeung1
 
Berlin Buzz Words - Apache Drill by Ted Dunning & Michael Hausenblas
Berlin Buzz Words - Apache Drill by Ted Dunning & Michael HausenblasBerlin Buzz Words - Apache Drill by Ted Dunning & Michael Hausenblas
Berlin Buzz Words - Apache Drill by Ted Dunning & Michael Hausenblas
MapR Technologies
 
Elasticsearch an overview
Elasticsearch   an overviewElasticsearch   an overview
Elasticsearch an overview
Amit Juneja
 
Sherlock: an anomaly detection service on top of Druid
Sherlock: an anomaly detection service on top of Druid Sherlock: an anomaly detection service on top of Druid
Sherlock: an anomaly detection service on top of Druid
DataWorks Summit
 
Ad

More from ForgeRock (20)

Digital Identities in the Internet of Things - Securely Manage Devices at Scale
Digital Identities in the Internet of Things - Securely Manage Devices at ScaleDigital Identities in the Internet of Things - Securely Manage Devices at Scale
Digital Identities in the Internet of Things - Securely Manage Devices at Scale
ForgeRock
 
Get the Exact Identity Solution You Need - In the Cloud - AWS and Beyond
Get the Exact Identity Solution You Need - In the Cloud - AWS and BeyondGet the Exact Identity Solution You Need - In the Cloud - AWS and Beyond
Get the Exact Identity Solution You Need - In the Cloud - AWS and Beyond
ForgeRock
 
Identity Live Sydney: Identity Management - A Strategic Opportunity
Identity Live Sydney: Identity Management  - A Strategic OpportunityIdentity Live Sydney: Identity Management  - A Strategic Opportunity
Identity Live Sydney: Identity Management - A Strategic Opportunity
ForgeRock
 
Identity Live Singapore: Transform Your Cybersecurity Capability
Identity Live Singapore: Transform Your Cybersecurity CapabilityIdentity Live Singapore: Transform Your Cybersecurity Capability
Identity Live Singapore: Transform Your Cybersecurity Capability
ForgeRock
 
Identity Live Singapore 2018 Keynote Presentation
Identity Live Singapore 2018 Keynote PresentationIdentity Live Singapore 2018 Keynote Presentation
Identity Live Singapore 2018 Keynote Presentation
ForgeRock
 
Identity Live Sydney 2018 Keynote Presentation
Identity Live Sydney 2018 Keynote PresentationIdentity Live Sydney 2018 Keynote Presentation
Identity Live Sydney 2018 Keynote Presentation
ForgeRock
 
Identity Live Singapore: Just Ask 'Em
Identity Live Singapore: Just Ask 'EmIdentity Live Singapore: Just Ask 'Em
Identity Live Singapore: Just Ask 'Em
ForgeRock
 
Identity Live Singapore: Building Trust & Privacy in a Connected Society
Identity Live Singapore: Building Trust & Privacy in a Connected SocietyIdentity Live Singapore: Building Trust & Privacy in a Connected Society
Identity Live Singapore: Building Trust & Privacy in a Connected Society
ForgeRock
 
Identity Live Sydney: Intelligent Authentication
Identity Live Sydney: Intelligent Authentication Identity Live Sydney: Intelligent Authentication
Identity Live Sydney: Intelligent Authentication
ForgeRock
 
Identity Live Sydney: Building Trust and Privacy in a Connected Society
Identity Live  Sydney:  Building Trust and Privacy in a Connected SocietyIdentity Live  Sydney:  Building Trust and Privacy in a Connected Society
Identity Live Sydney: Building Trust and Privacy in a Connected Society
ForgeRock
 
Get the Exact Identity Solution you Need in the Cloud - Deep Dive
Get the Exact Identity Solution you Need in the Cloud - Deep DiveGet the Exact Identity Solution you Need in the Cloud - Deep Dive
Get the Exact Identity Solution you Need in the Cloud - Deep Dive
ForgeRock
 
Get the Exact Identity Solution You Need - In the Cloud - Overview
Get the Exact Identity Solution You Need - In the Cloud - OverviewGet the Exact Identity Solution You Need - In the Cloud - Overview
Get the Exact Identity Solution You Need - In the Cloud - Overview
ForgeRock
 
ForgeRock and Trusona - Simplifying the Multi-factor User Experience
ForgeRock and Trusona - Simplifying the Multi-factor User ExperienceForgeRock and Trusona - Simplifying the Multi-factor User Experience
ForgeRock and Trusona - Simplifying the Multi-factor User Experience
ForgeRock
 
Opening Keynote (Identity Live Berlin 2018)
Opening Keynote (Identity Live Berlin 2018)Opening Keynote (Identity Live Berlin 2018)
Opening Keynote (Identity Live Berlin 2018)
ForgeRock
 
Steinberg - Customer identity as the cornerstone of our approach to digitaliz...
Steinberg - Customer identity as the cornerstone of our approach to digitaliz...Steinberg - Customer identity as the cornerstone of our approach to digitaliz...
Steinberg - Customer identity as the cornerstone of our approach to digitaliz...
ForgeRock
 
BMW Group - Identity Enables the Next 100 Years.. (Identity Live Berlin 2018)
BMW Group - Identity Enables the Next 100 Years..  (Identity Live Berlin 2018)BMW Group - Identity Enables the Next 100 Years..  (Identity Live Berlin 2018)
BMW Group - Identity Enables the Next 100 Years.. (Identity Live Berlin 2018)
ForgeRock
 
Trust is Everything - The Future of Identity and the ForgeRock Platform (Iden...
Trust is Everything - The Future of Identity and the ForgeRock Platform (Iden...Trust is Everything - The Future of Identity and the ForgeRock Platform (Iden...
Trust is Everything - The Future of Identity and the ForgeRock Platform (Iden...
ForgeRock
 
Silo Busters- The Value of User and Data Centricity beyond IoT Devices (Ident...
Silo Busters- The Value of User and Data Centricity beyond IoT Devices (Ident...Silo Busters- The Value of User and Data Centricity beyond IoT Devices (Ident...
Silo Busters- The Value of User and Data Centricity beyond IoT Devices (Ident...
ForgeRock
 
Shift from GDPR readiness to sustained compliance to improve your business an...
Shift from GDPR readiness to sustained compliance to improve your business an...Shift from GDPR readiness to sustained compliance to improve your business an...
Shift from GDPR readiness to sustained compliance to improve your business an...
ForgeRock
 
Intelligent Authentication (Identity Live Berlin 2018)
Intelligent Authentication  (Identity Live Berlin 2018)Intelligent Authentication  (Identity Live Berlin 2018)
Intelligent Authentication (Identity Live Berlin 2018)
ForgeRock
 
Digital Identities in the Internet of Things - Securely Manage Devices at Scale
Digital Identities in the Internet of Things - Securely Manage Devices at ScaleDigital Identities in the Internet of Things - Securely Manage Devices at Scale
Digital Identities in the Internet of Things - Securely Manage Devices at Scale
ForgeRock
 
Get the Exact Identity Solution You Need - In the Cloud - AWS and Beyond
Get the Exact Identity Solution You Need - In the Cloud - AWS and BeyondGet the Exact Identity Solution You Need - In the Cloud - AWS and Beyond
Get the Exact Identity Solution You Need - In the Cloud - AWS and Beyond
ForgeRock
 
Identity Live Sydney: Identity Management - A Strategic Opportunity
Identity Live Sydney: Identity Management  - A Strategic OpportunityIdentity Live Sydney: Identity Management  - A Strategic Opportunity
Identity Live Sydney: Identity Management - A Strategic Opportunity
ForgeRock
 
Identity Live Singapore: Transform Your Cybersecurity Capability
Identity Live Singapore: Transform Your Cybersecurity CapabilityIdentity Live Singapore: Transform Your Cybersecurity Capability
Identity Live Singapore: Transform Your Cybersecurity Capability
ForgeRock
 
Identity Live Singapore 2018 Keynote Presentation
Identity Live Singapore 2018 Keynote PresentationIdentity Live Singapore 2018 Keynote Presentation
Identity Live Singapore 2018 Keynote Presentation
ForgeRock
 
Identity Live Sydney 2018 Keynote Presentation
Identity Live Sydney 2018 Keynote PresentationIdentity Live Sydney 2018 Keynote Presentation
Identity Live Sydney 2018 Keynote Presentation
ForgeRock
 
Identity Live Singapore: Just Ask 'Em
Identity Live Singapore: Just Ask 'EmIdentity Live Singapore: Just Ask 'Em
Identity Live Singapore: Just Ask 'Em
ForgeRock
 
Identity Live Singapore: Building Trust & Privacy in a Connected Society
Identity Live Singapore: Building Trust & Privacy in a Connected SocietyIdentity Live Singapore: Building Trust & Privacy in a Connected Society
Identity Live Singapore: Building Trust & Privacy in a Connected Society
ForgeRock
 
Identity Live Sydney: Intelligent Authentication
Identity Live Sydney: Intelligent Authentication Identity Live Sydney: Intelligent Authentication
Identity Live Sydney: Intelligent Authentication
ForgeRock
 
Identity Live Sydney: Building Trust and Privacy in a Connected Society
Identity Live  Sydney:  Building Trust and Privacy in a Connected SocietyIdentity Live  Sydney:  Building Trust and Privacy in a Connected Society
Identity Live Sydney: Building Trust and Privacy in a Connected Society
ForgeRock
 
Get the Exact Identity Solution you Need in the Cloud - Deep Dive
Get the Exact Identity Solution you Need in the Cloud - Deep DiveGet the Exact Identity Solution you Need in the Cloud - Deep Dive
Get the Exact Identity Solution you Need in the Cloud - Deep Dive
ForgeRock
 
Get the Exact Identity Solution You Need - In the Cloud - Overview
Get the Exact Identity Solution You Need - In the Cloud - OverviewGet the Exact Identity Solution You Need - In the Cloud - Overview
Get the Exact Identity Solution You Need - In the Cloud - Overview
ForgeRock
 
ForgeRock and Trusona - Simplifying the Multi-factor User Experience
ForgeRock and Trusona - Simplifying the Multi-factor User ExperienceForgeRock and Trusona - Simplifying the Multi-factor User Experience
ForgeRock and Trusona - Simplifying the Multi-factor User Experience
ForgeRock
 
Opening Keynote (Identity Live Berlin 2018)
Opening Keynote (Identity Live Berlin 2018)Opening Keynote (Identity Live Berlin 2018)
Opening Keynote (Identity Live Berlin 2018)
ForgeRock
 
Steinberg - Customer identity as the cornerstone of our approach to digitaliz...
Steinberg - Customer identity as the cornerstone of our approach to digitaliz...Steinberg - Customer identity as the cornerstone of our approach to digitaliz...
Steinberg - Customer identity as the cornerstone of our approach to digitaliz...
ForgeRock
 
BMW Group - Identity Enables the Next 100 Years.. (Identity Live Berlin 2018)
BMW Group - Identity Enables the Next 100 Years..  (Identity Live Berlin 2018)BMW Group - Identity Enables the Next 100 Years..  (Identity Live Berlin 2018)
BMW Group - Identity Enables the Next 100 Years.. (Identity Live Berlin 2018)
ForgeRock
 
Trust is Everything - The Future of Identity and the ForgeRock Platform (Iden...
Trust is Everything - The Future of Identity and the ForgeRock Platform (Iden...Trust is Everything - The Future of Identity and the ForgeRock Platform (Iden...
Trust is Everything - The Future of Identity and the ForgeRock Platform (Iden...
ForgeRock
 
Silo Busters- The Value of User and Data Centricity beyond IoT Devices (Ident...
Silo Busters- The Value of User and Data Centricity beyond IoT Devices (Ident...Silo Busters- The Value of User and Data Centricity beyond IoT Devices (Ident...
Silo Busters- The Value of User and Data Centricity beyond IoT Devices (Ident...
ForgeRock
 
Shift from GDPR readiness to sustained compliance to improve your business an...
Shift from GDPR readiness to sustained compliance to improve your business an...Shift from GDPR readiness to sustained compliance to improve your business an...
Shift from GDPR readiness to sustained compliance to improve your business an...
ForgeRock
 
Intelligent Authentication (Identity Live Berlin 2018)
Intelligent Authentication  (Identity Live Berlin 2018)Intelligent Authentication  (Identity Live Berlin 2018)
Intelligent Authentication (Identity Live Berlin 2018)
ForgeRock
 
Ad

Recently uploaded (20)

Exceptional Behaviors: How Frequently Are They Tested? (AST 2025)
Exceptional Behaviors: How Frequently Are They Tested? (AST 2025)Exceptional Behaviors: How Frequently Are They Tested? (AST 2025)
Exceptional Behaviors: How Frequently Are They Tested? (AST 2025)
Andre Hora
 
How to Optimize Your AWS Environment for Improved Cloud Performance
How to Optimize Your AWS Environment for Improved Cloud PerformanceHow to Optimize Your AWS Environment for Improved Cloud Performance
How to Optimize Your AWS Environment for Improved Cloud Performance
ThousandEyes
 
How to Batch Export Lotus Notes NSF Emails to Outlook PST Easily?
How to Batch Export Lotus Notes NSF Emails to Outlook PST Easily?How to Batch Export Lotus Notes NSF Emails to Outlook PST Easily?
How to Batch Export Lotus Notes NSF Emails to Outlook PST Easily?
steaveroggers
 
Expand your AI adoption with AgentExchange
Expand your AI adoption with AgentExchangeExpand your AI adoption with AgentExchange
Expand your AI adoption with AgentExchange
Fexle Services Pvt. Ltd.
 
TestMigrationsInPy: A Dataset of Test Migrations from Unittest to Pytest (MSR...
TestMigrationsInPy: A Dataset of Test Migrations from Unittest to Pytest (MSR...TestMigrationsInPy: A Dataset of Test Migrations from Unittest to Pytest (MSR...
TestMigrationsInPy: A Dataset of Test Migrations from Unittest to Pytest (MSR...
Andre Hora
 
The Significance of Hardware in Information Systems.pdf
The Significance of Hardware in Information Systems.pdfThe Significance of Hardware in Information Systems.pdf
The Significance of Hardware in Information Systems.pdf
drewplanas10
 
Adobe Illustrator Crack FREE Download 2025 Latest Version
Adobe Illustrator Crack FREE Download 2025 Latest VersionAdobe Illustrator Crack FREE Download 2025 Latest Version
Adobe Illustrator Crack FREE Download 2025 Latest Version
kashifyounis067
 
Who Watches the Watchmen (SciFiDevCon 2025)
Who Watches the Watchmen (SciFiDevCon 2025)Who Watches the Watchmen (SciFiDevCon 2025)
Who Watches the Watchmen (SciFiDevCon 2025)
Allon Mureinik
 
Interactive odoo dashboards for sales, CRM , Inventory, Invoice, Purchase, Pr...
Interactive odoo dashboards for sales, CRM , Inventory, Invoice, Purchase, Pr...Interactive odoo dashboards for sales, CRM , Inventory, Invoice, Purchase, Pr...
Interactive odoo dashboards for sales, CRM , Inventory, Invoice, Purchase, Pr...
AxisTechnolabs
 
Microsoft AI Nonprofit Use Cases and Live Demo_2025.04.30.pdf
Microsoft AI Nonprofit Use Cases and Live Demo_2025.04.30.pdfMicrosoft AI Nonprofit Use Cases and Live Demo_2025.04.30.pdf
Microsoft AI Nonprofit Use Cases and Live Demo_2025.04.30.pdf
TechSoup
 
Requirements in Engineering AI- Enabled Systems: Open Problems and Safe AI Sy...
Requirements in Engineering AI- Enabled Systems: Open Problems and Safe AI Sy...Requirements in Engineering AI- Enabled Systems: Open Problems and Safe AI Sy...
Requirements in Engineering AI- Enabled Systems: Open Problems and Safe AI Sy...
Lionel Briand
 
WinRAR Crack for Windows (100% Working 2025)
WinRAR Crack for Windows (100% Working 2025)WinRAR Crack for Windows (100% Working 2025)
WinRAR Crack for Windows (100% Working 2025)
sh607827
 
Meet the Agents: How AI Is Learning to Think, Plan, and Collaborate
Meet the Agents: How AI Is Learning to Think, Plan, and CollaborateMeet the Agents: How AI Is Learning to Think, Plan, and Collaborate
Meet the Agents: How AI Is Learning to Think, Plan, and Collaborate
Maxim Salnikov
 
PDF Reader Pro Crack Latest Version FREE Download 2025
PDF Reader Pro Crack Latest Version FREE Download 2025PDF Reader Pro Crack Latest Version FREE Download 2025
PDF Reader Pro Crack Latest Version FREE Download 2025
mu394968
 
Adobe Master Collection CC Crack Advance Version 2025
Adobe Master Collection CC Crack Advance Version 2025Adobe Master Collection CC Crack Advance Version 2025
Adobe Master Collection CC Crack Advance Version 2025
kashifyounis067
 
Kubernetes_101_Zero_to_Platform_Engineer.pptx
Kubernetes_101_Zero_to_Platform_Engineer.pptxKubernetes_101_Zero_to_Platform_Engineer.pptx
Kubernetes_101_Zero_to_Platform_Engineer.pptx
CloudScouts
 
Designing AI-Powered APIs on Azure: Best Practices& Considerations
Designing AI-Powered APIs on Azure: Best Practices& ConsiderationsDesigning AI-Powered APIs on Azure: Best Practices& Considerations
Designing AI-Powered APIs on Azure: Best Practices& Considerations
Dinusha Kumarasiri
 
Explaining GitHub Actions Failures with Large Language Models Challenges, In...
Explaining GitHub Actions Failures with Large Language Models Challenges, In...Explaining GitHub Actions Failures with Large Language Models Challenges, In...
Explaining GitHub Actions Failures with Large Language Models Challenges, In...
ssuserb14185
 
F-Secure Freedome VPN 2025 Crack Plus Activation New Version
F-Secure Freedome VPN 2025 Crack Plus Activation  New VersionF-Secure Freedome VPN 2025 Crack Plus Activation  New Version
F-Secure Freedome VPN 2025 Crack Plus Activation New Version
saimabibi60507
 
Mastering Fluent Bit: Ultimate Guide to Integrating Telemetry Pipelines with ...
Mastering Fluent Bit: Ultimate Guide to Integrating Telemetry Pipelines with ...Mastering Fluent Bit: Ultimate Guide to Integrating Telemetry Pipelines with ...
Mastering Fluent Bit: Ultimate Guide to Integrating Telemetry Pipelines with ...
Eric D. Schabell
 
Exceptional Behaviors: How Frequently Are They Tested? (AST 2025)
Exceptional Behaviors: How Frequently Are They Tested? (AST 2025)Exceptional Behaviors: How Frequently Are They Tested? (AST 2025)
Exceptional Behaviors: How Frequently Are They Tested? (AST 2025)
Andre Hora
 
How to Optimize Your AWS Environment for Improved Cloud Performance
How to Optimize Your AWS Environment for Improved Cloud PerformanceHow to Optimize Your AWS Environment for Improved Cloud Performance
How to Optimize Your AWS Environment for Improved Cloud Performance
ThousandEyes
 
How to Batch Export Lotus Notes NSF Emails to Outlook PST Easily?
How to Batch Export Lotus Notes NSF Emails to Outlook PST Easily?How to Batch Export Lotus Notes NSF Emails to Outlook PST Easily?
How to Batch Export Lotus Notes NSF Emails to Outlook PST Easily?
steaveroggers
 
Expand your AI adoption with AgentExchange
Expand your AI adoption with AgentExchangeExpand your AI adoption with AgentExchange
Expand your AI adoption with AgentExchange
Fexle Services Pvt. Ltd.
 
TestMigrationsInPy: A Dataset of Test Migrations from Unittest to Pytest (MSR...
TestMigrationsInPy: A Dataset of Test Migrations from Unittest to Pytest (MSR...TestMigrationsInPy: A Dataset of Test Migrations from Unittest to Pytest (MSR...
TestMigrationsInPy: A Dataset of Test Migrations from Unittest to Pytest (MSR...
Andre Hora
 
The Significance of Hardware in Information Systems.pdf
The Significance of Hardware in Information Systems.pdfThe Significance of Hardware in Information Systems.pdf
The Significance of Hardware in Information Systems.pdf
drewplanas10
 
Adobe Illustrator Crack FREE Download 2025 Latest Version
Adobe Illustrator Crack FREE Download 2025 Latest VersionAdobe Illustrator Crack FREE Download 2025 Latest Version
Adobe Illustrator Crack FREE Download 2025 Latest Version
kashifyounis067
 
Who Watches the Watchmen (SciFiDevCon 2025)
Who Watches the Watchmen (SciFiDevCon 2025)Who Watches the Watchmen (SciFiDevCon 2025)
Who Watches the Watchmen (SciFiDevCon 2025)
Allon Mureinik
 
Interactive odoo dashboards for sales, CRM , Inventory, Invoice, Purchase, Pr...
Interactive odoo dashboards for sales, CRM , Inventory, Invoice, Purchase, Pr...Interactive odoo dashboards for sales, CRM , Inventory, Invoice, Purchase, Pr...
Interactive odoo dashboards for sales, CRM , Inventory, Invoice, Purchase, Pr...
AxisTechnolabs
 
Microsoft AI Nonprofit Use Cases and Live Demo_2025.04.30.pdf
Microsoft AI Nonprofit Use Cases and Live Demo_2025.04.30.pdfMicrosoft AI Nonprofit Use Cases and Live Demo_2025.04.30.pdf
Microsoft AI Nonprofit Use Cases and Live Demo_2025.04.30.pdf
TechSoup
 
Requirements in Engineering AI- Enabled Systems: Open Problems and Safe AI Sy...
Requirements in Engineering AI- Enabled Systems: Open Problems and Safe AI Sy...Requirements in Engineering AI- Enabled Systems: Open Problems and Safe AI Sy...
Requirements in Engineering AI- Enabled Systems: Open Problems and Safe AI Sy...
Lionel Briand
 
WinRAR Crack for Windows (100% Working 2025)
WinRAR Crack for Windows (100% Working 2025)WinRAR Crack for Windows (100% Working 2025)
WinRAR Crack for Windows (100% Working 2025)
sh607827
 
Meet the Agents: How AI Is Learning to Think, Plan, and Collaborate
Meet the Agents: How AI Is Learning to Think, Plan, and CollaborateMeet the Agents: How AI Is Learning to Think, Plan, and Collaborate
Meet the Agents: How AI Is Learning to Think, Plan, and Collaborate
Maxim Salnikov
 
PDF Reader Pro Crack Latest Version FREE Download 2025
PDF Reader Pro Crack Latest Version FREE Download 2025PDF Reader Pro Crack Latest Version FREE Download 2025
PDF Reader Pro Crack Latest Version FREE Download 2025
mu394968
 
Adobe Master Collection CC Crack Advance Version 2025
Adobe Master Collection CC Crack Advance Version 2025Adobe Master Collection CC Crack Advance Version 2025
Adobe Master Collection CC Crack Advance Version 2025
kashifyounis067
 
Kubernetes_101_Zero_to_Platform_Engineer.pptx
Kubernetes_101_Zero_to_Platform_Engineer.pptxKubernetes_101_Zero_to_Platform_Engineer.pptx
Kubernetes_101_Zero_to_Platform_Engineer.pptx
CloudScouts
 
Designing AI-Powered APIs on Azure: Best Practices& Considerations
Designing AI-Powered APIs on Azure: Best Practices& ConsiderationsDesigning AI-Powered APIs on Azure: Best Practices& Considerations
Designing AI-Powered APIs on Azure: Best Practices& Considerations
Dinusha Kumarasiri
 
Explaining GitHub Actions Failures with Large Language Models Challenges, In...
Explaining GitHub Actions Failures with Large Language Models Challenges, In...Explaining GitHub Actions Failures with Large Language Models Challenges, In...
Explaining GitHub Actions Failures with Large Language Models Challenges, In...
ssuserb14185
 
F-Secure Freedome VPN 2025 Crack Plus Activation New Version
F-Secure Freedome VPN 2025 Crack Plus Activation  New VersionF-Secure Freedome VPN 2025 Crack Plus Activation  New Version
F-Secure Freedome VPN 2025 Crack Plus Activation New Version
saimabibi60507
 
Mastering Fluent Bit: Ultimate Guide to Integrating Telemetry Pipelines with ...
Mastering Fluent Bit: Ultimate Guide to Integrating Telemetry Pipelines with ...Mastering Fluent Bit: Ultimate Guide to Integrating Telemetry Pipelines with ...
Mastering Fluent Bit: Ultimate Guide to Integrating Telemetry Pipelines with ...
Eric D. Schabell
 

Customer Intelligence: Using the ELK Stack to Analyze ForgeRock OpenAM Audit Data

  • 1. IRM Summit 2014 Customer Intelligence: Using the ELK stack (Elasticsearch, Logstash and Kibana) to analyse ForgeRock OpenAM audit data [email protected]
  • 2. IRM Summit 2014 Make pretty pictures for the boss
  • 4. OpenDJ, OpenIDM, OpenAM produce copious amounts of audit data Analysis of that data is left as an exercise for the reader Many great SIEM tools Desire for an Open Source solution for data analysis
  • 5. What is the ELK stack? Elasticsearch: “No SQL” database Logstash: Log collection and transformation Kibana: Data visualizer for Elasticsearch
  • 6. Yes, but what does ELK do? Collect, analyse and visualize data Any kind of data Github (8 Million repos), Soundcloud (30M users), The Guardian (40M documents) Answer questions: ● Where are my users coming from? ● What is the traffic in North America vs. Europe? ● Why do I see an employee logging in from Canada?
  • 7. Elasticsearch ● NoSQL, REST/json, document oriented, schemaless, “Big data” full text search engine ● Apache 2.0 license ● Sweet spot is rapid full text search / ad hoc queries ● Not a replacement for an RDBMS ● Not transactional, not ACID, etc. ● Built on Apache Lucene project
  • 8. Logstash ● Swiss army knife of log collection, transformation and forwarding ● JRuby based ● Large footprint :-( ● lumberjack ● go based collector that feeds into logstash ● Very lightweight, small footprint
  • 10. Logstash flow Input source files, database, syslog, etc. Filters grep, regex, geoIP, ... Output elasticsearch, file, db, syslog “Plugin” based architecture. Add new plugins for input, output and filters
  • 11. Logstash example Input source file: amAccess.* type: amAccess Filters Map IP address to GEO location Output elasticsearch:9100 Read from OpenAM access logs Add Geo Location data Write the result to Elasticsearch
  • 13. input { file { type => amAccess path => "/logs/am/log/amAuthentication.*" } } Input section Wildcards can be used Data is tagged with a type. Use this to classify & search by type
  • 14. filter { if [type] == "amAccess" { csv { columns => [time,Data,LoginID,ContextID, IPAddr, LogLevel, Domain, LoggedBy, MessageID, ModuleName, NameID, HostName] separator => " " } date { match => ["time", "yyyy-MM-dd HH:mm:ss"] } geoip { database => "/usr/share/GeoIP/GeoIP.dat" source => ["IPAddr"] } } } Filter apply to type Parse the data as csv Normalize the date to a common format Enrich the record with GEO location
  • 15. output { stdout { codec => rubydebug } elasticsearch { host => localhost } } Output Send the data to Elasticsearch and the stdout
  • 16. Demo Time As seen on youtube! http://youtu.be/tvrYuSLuGik 27 49 views!
  • 17. ELK Demo Environment OpenAM OpenDJ OpenIDM logstash elasticsearch:9100 Apache:80/443 + Policy Agent Log Files Kibana
  • 18. Marketing Genius? Where to hold the next ForgeRock Summit: Europe, USA, or Canada? Asks you to find out pronto: ● What country are customers visiting the ForgeRock website from? ● How are they authenticating (forgerock account, or federated?)
  • 19. The next IRM summit location: We have beer! Bring your toque!
  • 20. Next Steps Delivery Models Cloud or Appliance? Interested in collaborating? Share logstash config, kibana reports, etc. Puppet/Chef/Ansible/Docker installers?