SlideShare a Scribd company logo
Christian Schneider | @cschneider4711
Security DevOps


Wie Sie in agilen Projekten trotzdem sicher bleiben
`whoami`
• Software Developer, Whitehat Hacker & Trainer
• als Freelancer im Projektgeschäft seit 1997
• Fokus auf JavaEE Architekturen und Web Security
• Speaker auf Konferenzen (JAX,WebTechCon, OWASP
AppSecEU, DevOps Conference, …) + HackPra
• @cschneider4711
www.

mail@
Christian-Schneider.net
Security – Status Quo
Static Application Security Testing (SAST)
Dynamic Application Security Testing (DAST)
Fun Part
The Ratio Problem
Dev Ops
Sec
100 10 1: :
The Frequency Problem
mehr Rollouts als Pentests
Security DevOps ?
Integrate Security Tools into Build-Jobs / CI-Chain
on every
commit or nightly
Reaching the different levels
of "Security DevOps" integration…
Security DevOps Maturity Model
(SDOMM)
implicit master
four axes each with four
belts as small steps
Four different axes
Dynamic Depth
Intensity
Static DepthConsolidation
4
3
2
1
1
2
3
4
4 3 2 1 1 2 3 4
Four different axes
Dynamic Depth
Intensity
Static DepthConsolidation
4
3
2
1
1
2
3
4
4 3 2 1 1 2 3 4
Axis of "Dynamic Depth"
How deep are dynamic scans executed
within a Security DevOps CI chain?

i.e. "where" are dynamic 

security tests applied?
Axis "Dynamic Depth": Level 1
Scanning of public attack surface (pre-auth):
• Spidering of UI layer
• No requirement to authenticate scanner with target
• Easy integration of scanner(s) in nightly build as post-step
• "Throw tool at it (in CI-chain) and see what it generates…"
Tool example: ZAP
"OWASP ZAP" features:
• passive scanning (Proxy / Spider)
• active scanning (Proxy / Spider)
• manual Payload Delivery (Intercepting Proxy)
• Spider (classic & AJAX)
• Fuzzing
• Plugins
• …
Request/Response History (Proxy)
Active Scanning…
Findings
ZAP in SecDevOps?
"OWASP ZAP" features relevant for Security DevOps integration:
• …
• Headless Operation Mode / Daemon
• REST-API (with several language bindings as pre-built clients)
• Scriptable
• CLI
ZAP + Jenkins = SecDevOps?
"OWASP ZAP" (spider & scanner) + Jenkins plugin "ZAProxy"
• Allows us to "Spider & Scan" as step in build job via Jenkins plugin
• Point plugin config to URL of integration system to test
• Plugin saves HTML-report in project’s job for inspection
• Best as separate Jenkins job to run during nightly build (duration)
ZAProxy Jenkins Plugin: ZAP startup
ZAProxy Jenkins Plugin: ZAP scan
UI- and Service-Tests in CI
Web Browser
Build-Scripts
UI-Tests
(Selenium, etc.)
Service-Tests
(REST, SOAP)
Application Server
CI Jobs
… enhanced with OWASP ZAP
Web Browser
Build-Scripts
UI-Tests
(Selenium, etc.)
Service-Tests
(REST, SOAP)
Application Server
OWASP ZAP
CI Jobs
passive & active
scanning
place report (html,xml) 

into workspace
HTML report of ZAP scan
Tool example:Arachni
"Arachni Scanner" features relevant for Security DevOps integration:
• Passive scanning (Proxy / Spider)
• Active scanning (Proxy / Spider)
• Uses internally a headless browser-cluster (for apps with lots of JS)
• Automation?
• CLI + RPC API
• Web-UI (helpful when permanently running as server)
Arachni + Jenkins = SecDevOps?
"Arachni Scanner" + Jenkins CLI step in build
• Start in build job as CLI step and point to URL of system under test
• Generate HTML report and place into workspace for inspection
• Better execute within nightly build job (due to duration)
HTML report of Arachni scan
(Summary) Bildquelle: Arachni Scanner
HTML report of Arachni scan 

(Details) Bildquelle: Arachni Scanner
Tool example: BDD-Security
BDD-based framework for functional and technical security tests:
• Technical security tests (i.e. check against XSS, SQL-Injection, XXE, etc.)
• mainly uses ZAP as scanning engine
• Functional security tests (i.e. check UserA doesn’t see data from UserB)
• Tightly integrates with Selenium based app navigation workflows
• Uses JBehave for G/W/T stories & reporting
• Can run within CI (Jenkins, etc.) due to JBehave 

or as JUnit tests
BDD-Security Story: Scan for XSS
HTML report of BDD-Security
Other dynamic scanners exist also…
Some have different focus
• Nikto
• w3af
• IronWASP
• Nessus
• Gauntlt (BDD-based Security tool orchestration)
• Mittn (BDD-based)
• Minion (integration framework)
• OWTF (integration framework)
• good commercial ones
• etc.
Axis "Dynamic Depth": Level 2
Scanning of authenticated parts (= "post-auth") via UI layer
• Properly maintaining sessions
• Logout-detection & automatic re-login
• Different users / roles
• Spider & scan post-auth

Handling of hardening measures of application under test
• CSRF-Tokens, CAPTCHAs, etc.
Guide ZAP into Post-Auth
Many ways to give authentication infos to ZAP
• Use ZAP manually to configure "Context": RegExps for Logged-In/Out
Indicators, Users etc. + save as "ZAP Session-File" (could be in SCM)
• use that "Session-File" from SCM as starting point of scan 

(loaded via Jenkins plugin)
• Also Scripts in JavaScript or Zest can be registered in ZAP 

context to programmatically give authentication to ZAP
• Alternatively set these auth values via ZAP’s REST-API 

during each build before scan starts (from Jenkins/Maven/…)
"Forced User Mode" to stick to certain users during scans
Login config example within ZAP
ZAProxy Jenkins Plugin: ZAP session use
Guide Arachni into Post-Auth
Give authentication infos to Arachni (Auth, Logged-In Indicators, Users)
• Use Arachni "autologin" plugin to specify via command line
• Login URL, formfield names, credentials, logged-in indicator, excludes
• Alternatively write custom ruby script for "login_script" plugin
• Individual custom login logic possible
• Logged-In indicators (RegExp) to know when to re-login


Login config example within Arachni
./arachni 



--plugin=autologin:

url=http://10.33.48.80/marathon/secured/profile.page,

parameters='j_username=john&j_password=john',

check='Logout' 



--scope-exclude-pattern=logout.page 



http://10.33.48.80/marathon
Or individual ruby script if more custom login logic required…
Guide BDD-Security into Post-Auth
Use Selenium to navigate through the login process
• Based on excellent integration of BDD-Security with Selenium
• Separate app navigation code (Selenium) from Security testing code
• Use (along with test user credentials) within BDD stories
• Perform further spidering & active scanning (through ZAP) post-auth
BDD-Security based stories integrate with custom code to deliver 

test CAPTCHA value to scanner
• while also checking that mismatches are handled with denial)
What about CSRF-Tokens?
ZAP can handle CSRF-Tokens
• Can be individualised via Config
BDD-Security uses (among others) ZAP for scanning
Arachni can also handle CSRF-Tokens during spidering & scanning
Apply targeted delta scanning:
• Developer teams maintain "scope delta files" in SCM
• like "what parts of the application have changed during this sprint" (= URL
patterns)
• DAST Scanner uses it to scan only this during onCommit or nightly CI run
• Feed this delta information as argument to scope definitions

Arachni: --scope-include-pattern / --scope-exclude-pattern

ZAP: Include/Excludes in Context (via Session-File or REST-API)

BDD-Security: maintain as "table file" of RegExps as scope
If scanning takes too long?
Train spiders about the structure of application to reduce scan redundancy
• ZAP: use RegExp "Exclude from scanner" rules to prevent redundant
scans in data-driven applications (i.e. scan only one instance of it)
• Arachni: --scope-redundant-path-pattern / --scope-auto-redundant
Don’t forget to scan complete application at least on nightly or weekly run
• Only apply full scan, if simple smoke-tests of system 

under test are green
If scanning takes too long?
Axis "Dynamic Depth": Level 3
Separate scanning of different application layers / backends
• Scan internal WebServices (e.g. SOAP / REST) = directly scan backends
• Detect and scan parameter positions within XML, JSON, …
• Scan from "within" the different application’s layers
• IAST with distributed agents & instrumentation aims into that direction
• At least one simple step in that direction:
• Use the proxy also between your backend service calls
Backend scans with ZAP
How to achieve this with ZAP?
• ZAP operates as proxy server: place it between backend calls
• ZAP can inject payloads in observed XML tags/attributes & JSON fields
• Capture service call traffic in integration test during CI while either
A. executing service tests that directly access the service endpoint, or
B. frontend UI tests execute service backend calls indirectly
• Automatically scan as new requests are seen: "ATTACK Mode"
Also keep an eye on an alpha-level SOAP-Scanner (by WSDL) ZAP addon
Backend scans with Arachni
How to achieve this with Arachni?
• Arachni can also operate as proxy server: place it between backend calls
• Use passive proxy plugin to "train" Arachni of the XML / JSON requests
• New addition in v1.1 to extract XML / JSON input vectors from that
• Use that collected input vector data to feed the active scan for the
observed requests
Axis "Dynamic Depth": Level 4
Targeted scanning of individual forms / wizards (UI) and service layers
• More individualised workflow coverage (not just simple spidering)
• Business-logic compliant usage patterns & inputs
• "fill shopping cart followed by checkout process"
• "access backendWebServices in special order to test workflow"
• etc.
• Custom coded security tests tailored to the application
ZAP with special workflows (1/2)
Re-use existing UI tests (Selenium, …) and/or WebService tests
• Proxy Selenium-generated traffic through ZAP (in integration test phase) 

in ATTACK-Mode
More individual: Re-use & enhance existing "UI test code" at the proper 

workflow steps with points of where to attack by calling ZAP’s REST-API

(simple to use REST-API clients are pre-shipped with ZAP)
• Type of scans at these positions can be individualised (policy config, etc.)
• See https://github.com/continuumsecurity/zap-webdriver 

for a working example with ZAP
• Start ZAP as Daemon from Jenkins with Zapper Jenkins plugin
ZAP with special workflows (2/2)
Alternatively "train" ZAP about the workflow by recording Zest scripts
• Keep an eye on "Sequence Scanning" alpha-level ZAP extension
• Still alpha-level (as of May 2015), but interesting approach
Use Selenium to further drive BDD-Security initiated checks:
• Selenium-based test code navigates application workflows
• This code is integrated with BDD (via Java interfaces), so that:
• BDD-Security stories can use that code to navigate 

and generate traffic
• This generated traffic will be scanned by ZAP via BDD
BDD with special workflows
BDD-Security: Selenium integration
If no Selenium testcases exist?
Simply give developer teams access to ZAP to pre-seed the scanner (at least):
• Developer teams use browser to navigate app workflows while proxying
• Thereby seed the ZAP session(s) with navigation nodes/workflows
• Save the ZAP session(s) and check-in into SCM (Git, SVN, …)
• Point the Jenkins ZAP plugin to the saved ZAP session(s) as starting point
• Devs can add to this list of URLs for ZAP with each new UI
BTW: ZAP is also available as Docker image…
Security-UnitTests for APIs
Sometimes it’s useful to have programmed unit-tests directly calling a business API
• Test directly at the API-level (i.e. Java code level)
• Utilise fuzzing values from popular sources like "fuzzdb" 

and/or generate on-the-fly with "radamsa"
• Check the API method’s response 

(like regular unit test, but seeking for signs of successful attack tests)
• Leverage the benefit of direct API-level testing (i.e. testing "from within")
• Place "probes" into the application to watch out for successful attack tests
• Mock objects (Mockito, etc.) or weave aspects (AOP) 

collecting data (i.e. method calls and arguments, etc.)
• Even logs (of error messages) are good candidates to check
Axis of "Static Depth"
How deep is static code analysis performed 

within a Security DevOps CI chain?

i.e. "where" are static 

security tests applied?
Axis "Static Depth": Level 1
Assurance that no third-party code with known vulnerabilities is used
• Check application’s server-side dependencies
• For Java applications: JAR files, etc.
• Check application’s client-side dependencies
• JavaScript frameworks, etc.

Useful even for projects not currently under development
• New vulnerabilities could become public: Build regularly to check
Tool: OWASP Dependency Check
Scans all dependencies (even transitive ones) against CVE list
• Available as Maven plugin and Ant task
• CLI version also available
• Jenkins plugin for nice reporting (and build breaking thresholds)
• Not false-positive free
• regularly needs some time to triage findings
Reporting in Jenkins
Security DevOps: Wie Sie in agilen Projekten trotzdem sicher bleiben // JAX 2015
Findings (CVE-Details)
For JavaScript libs: retire.js
Scans application’s JavaScript files against list of known vulnerable ones
• Available also as Maven Plugin…
• … and as CLI (point it to your folder containing .js files)
Axis "Static Depth": Level 2
Scan important parts of source code for vulnerability patterns
• At least important parts of applications’ codebases are scanned
• like in-house reused code & custom developed frameworks
• for big multi-project Maven projects: 

at least frontend and backend projects
• or code of changed modules within a sprint
Java: FindSecurityBugs
Plugin for FindBugs with enhanced checks for security issues in Java code
• Runs within FindBugs, so that it can execute in Maven and/or Sonar
• Generates XML-file with potential findings
• Jenkins plugin for representing results (also possible to display in Sonar)
• Integrates with Jenkins "Static Code Analysis" as part of FindBugs
• Delta reporting of new (and fixed) findings & trends
FindBugs in Jenkins: Trends, etc.
in Sonar: Vulnerability & Code-Pointer
JavaScript: ScanJS
Scans JavaScript code for vulnerabilities
• Most importantly DOM-based XSS (Cross-Site Scripting) issues
• Flags also "Usage of sensitive API" issues
• Executable as CLI from within Jenkins build steps
• Optional Web-UI (running as local server)
ScanJS Findings as HTML: DOM-XSS
Ruby on Rails apps: Brakeman
Scans RoR code for vulnerabilities
• CLI based
• Nicely integrates with Jenkins
• Rendering scan results and trend analysis
• Allows to compare scan results (delta detection)
• "What has been fixed and what was newly introduced?"
Other static scanners exist also…
Some have different focus
• PrimeJS (like ScanJS for JavaScript regarding DOM-XSS etc.)
• PMD (for Java code regarding API safety issues)
• good commercial ones
• etc.
Axis "Static Depth": Level 3
Scan the complete applications’ source code for vulnerability patterns
• Just like "Level 2", but now for the complete codebase (no excludes)
• i.e. all projects of a multi-project Maven project
• best via parent or corporate pom.xml
For FindSecurityBugs this means:
• pre-compile even all JSPs (if you have any) to scan them
Axis "Static Depth": Level 4
Scan source code of (important) third-party dependencies used in application
• If the component is open-source:
• include it in regular scanning activities
• If the component is closed-source:
• consider using code scanners that operate on binaries
• FindBugs (hence its FindSecurityBugs plugin) 

can also scan binaries, since it operates on Bytecode
Axis of "Intensity"
How intense are the majority of the executed
attacks within a Security DevOps CI chain?

i.e. "what" is being 

checked for?
Axis "Intensity": Level 1
Dynamic checks: Only passive scanning
• Simply proxy testcase generated traffic through passive operating scanners
• Quick-win for using existing testcase code (i.e. Service- or UI-tests)
• Can execute within existing tests that execute on every commit
• No requirement for a separate nightly CI job (i.e. doesn’t slow down build)


Static checks: Just scan the code along with other code metrics
• Use the default rules of the scanners and integrate them into CI
Axis "Intensity": Level 2
Use lightweight active scanning options for dynamic checks
• ZAP: active scanning of observed URLs (proxied or spidered)
• perform an "active scan": For Jenkins ZAProxy Plugin provide a ZAP policy
with desired active scan intensity (or via ZAP’s REST-API during build for
other setups)
• Arachni: enable "active scan" during spidering
• BDD-Security stories can call ZAP to actively attack the application
Better to execute in separate (nightly) build job due it’s to duration
Think of resetting databases (+ more environment) prior or after test execution
Axis "Intensity": Level 3
Use heavyweight scanning options on important parts of application
Dynamic checks: create "riskier" custom attack profiles
ZAP:
• set "Threshold" to "Low" / "Strength" to "High" (or selectively "Insane")
• enable more scanning rules via Policy Files (could be maintained & used via SCM)
• use ZAP’s "Advanced SQLInjection Scanner" extension (or directly call sqlmap)
Arachni: Enable the more aggressive "audit flags" like --audit-forms, …
Static checks: FindSecurityBugs: set "Threshold" to "Low" / "Effort" to "Max"
Axis "Intensity": Level 4
Use customised rule sets for dynamic checks
• ZAP (which is very extensible via custom scripts, recored and customised Zest
or plugins):
• Custom coded scan scripts as active & passive scan rules for company-
individual checks
• Security regression tests (in Zest) against previous found vulns (also logic
flaws)
• Custom input vector scripts: e.g. define certain injection points in custom
encoded params
• Proxy scripts (for example for generic request/response modifications)
Scripting possibilities in ZAP
Axis "Intensity": Level 4
Use customised rule sets for static checks
• FindSecurityBugs: Custom coded rule files (in Java) to check for 

wrong usages of server-side inhouse frameworks (or alike)
• ScanJS: Custom rule files (in JSON) to check for wrong usages 

of client-side inhouse frameworks (or alike)
Axis of "Consolidation"
How complete is the process of handling
findings within a Security DevOps CI chain?

i.e. "how" are the 

results used?
Axis "Consolidation": Level 1
Generate human-readable (HTML) reports from tools and link them in Jenkins
• All relevant mentioned static and dynamic scanners generate HTML reports
• Collect and publish them in Jenkins build: via Jenkins "HTML Publisher Plugin"
Use simple criteria to "break the build" on heavy findings (ok, at least "unstable")
• Dependency-Check, BDD-Security (with the JBehave-stories),

FindSecurityBugs (via Sonar when rated as blocker),

Arachni (via Gauntlt execution with BDD-like stories), etc.

all have capabilities to automatically flag the build
• For others: at least do a simple log parse from Jenkins 

"Log Parser Plugin" to flag the build as instable and/or broken
OWASP Dependency Check: 

Thresholds to break build
Reports & Sessions
Simple reporting with ZAP in CI:
• Export HTML report & link from build (via Jenkins Plugin "ZAProxy" or "HTML
Publisher")
• Save the ZAP session with build for later manual inspection (by security team)
For Arachni scans in CI:
• Generate & link readable HTML report from that (via arachni-report tool in build)
• Optionally also save AFR file with build
BDD-Security directly generates reports in JBehave formats (HTML, XML, …) 

or via JUnit from within build job
Jenkins HTML Publisher Plugin:
Configuration of HTML reports to link
Jenkins HTML
Publisher Plugin:
Result in build
FindSecurityBugs: IDE support & Collaboration
FindBugs has excellent Jenkins integration
Additionally consider placing the FindBugs generated XML report file within Jenkins
• Developers can load the XML report file into FindBugs IDE plugins 

(Eclipse, IntelliJ, NetBeans, …)
• Developers & Security Analysts can collaborate on bugs via the IDE plugins
• See https://code.google.com/p/findbugs/wiki/FindBugsCloudTutorial 

for more details
• Consider installing it as your private on-premise 

"FindBugs"-Server to keep results only on your servers…
FindSecurityBugs Eclipse Perspective
with Collaboration View + Remediation
Axis "Consolidation": Level 2
Custom logic to make build unstable and/or broken depending on
• Type of vulnerability (CWE or WASC or …)
• Confirmation level (firm)
• Severity rating (high risk)
Provide useful remediation info to developers
Respect suppression mechanisms to rule out false positives
Flagging builds from reports
How (from within a CI job)?
• Most scanners also emit XML reports that can be parsed
• mostly a simple XPath count should be fine
• Alternatively fetch the results by accessing the scanner’s APIs
Providing useful infos to Devs
Lookup findings by CWE-ID (provided by most scanners) in vulndb on GitHub
• https://github.com/vulndb/data (currently 65+ vulns described)
• Useful if the messages from scanners are too generic or don’t help much
• Maps vulnerability to useful remediation messages
• Possibility to extend that repository locally with company-individual texts
Suppression of false positives
BDD-Security
• Use false positive tables in story files
FindSecurityBugs
• Use exclusion lists in config (XML filter files)
• Use annotation @SuppressWarnings on false positive code lines
Dependency-Check
• XML file of suppressions (checked-in along with the project)
retire.js
• "ignore file" parameters on CLI (cloud be loaded in CI from SCM)
Axis "Consolidation": Level 3
Consolidation goals:
• Consolidate & de-duplicate findings from different 

scanner reports (with better false positive handling)
• Push consolidated findings into established bug-tracker 

(known to devs)
• Delta analysis & trends over consolidated data sets
ThreadFix as result consolidator
Use a local ThreadFix server, which imports native scanner outputs
• does the heavy lifting of consolidation & de-duplication
• pushes findings toward bug-tracker and IDE (via plugins)
• process can be customised using it’s own REST-API
• ThreadFix integrates with ZAP to import scans from ZAP
Report generated by ThreadFix
Bildquelle: ThreadFix
Trend Chart generated by ThreadFix
Bildquelle: ThreadFix
Trends about vulnerable dependencies
OWASP Dependency-Track provides trend insight into findings about use of
vulnerable components (from OWASP Dependency-Check):
• across multiple applications and versions
• across time
Trend Chart generated by 

OWASP Dependency Track Bildquelle: OWASP
Axis "Consolidation": Level 4
Measure the concrete code coverage of your security testing activities
• Find untested "white spots"
• Derive where static checks and code reviews should 

focus more to compensate
How?
• Use "OWASP Code Pulse", which instruments your Java app via agent
• collects coverage data during dynamic security testing scans
• generates reports ("code treemaps") of coverage
Code Treemap of dynamic scan coverage
Bildquelle: OWASP Code Pulse
Without CI build chain?
Now that we’ve seen several ways to introduce security
checks into the build chain

(four axes with four levels each)

…
What if we don’t have a full-blown CI?
Or simply cannot extend it that way 

(i.e. build job would run too long)?
Use recurring scans instead!
As an alternative to a build-chain based scan recurring scans can be used:
• Don’t block the build, but get nightly or weekly results delivered
• No need to integrate into full build chain
• Still leverage the other benefits
• Dev-maintained delta files defining scopes
• Consolidation levels of results
• Basically all four levels for each axis apply the same
• = Quick-win for DAST + SAST scans independent of build-chain
Recurring scans:Arachni
Remote Arachni (non-blocking in build chain):
• Instead of a scan step in Jenkins build, the RPC-API of a

remote Arachni server can be used to run/schedule a scan
• Collect findings separately after scan 

or link to WebUI of Arachni Server to present them
Recurring scans: ZAP
Remote OWASP ZAP (non-blocking in build chain):
• Instead of a scan step in Jenkins build, the REST-API of a remote 

ZAP can be used to run/schedule a scan (cron, etc.)
• Collect findings separately after scan
• Or use Seccubus to have an integrated framework 

for scan scheduling with ZAP
• also has comparison of different scans to report deltas
• ZAP authenticated scans scheduled via Seccubus:

-o "--auth-form http://example.com/login --auth-user
myuser --auth-pass mypass --auth-verify-url http://
example.com/profile -X /logout" --hosts @HOSTS
Thank you very much!
Links
OWASP ZAP https://www.owasp.org/index.php/OWASP_Zed_Attack_Proxy_Project
ZAP Selenium Demo https://github.com/continuumsecurity/zap-webdriver
ZAP Jenkins Plugin https://wiki.jenkins-ci.org/display/JENKINS/ZAProxy+Plugin
BDD-Security http://www.continuumsecurity.net/bdd-intro.html
Arachni http://www.arachni-scanner.com
OWASP Dependency Check https://www.owasp.org/index.php/OWASP_Dependency_Check
OWASP Dependency Track https://www.owasp.org/index.php/OWASP_Dependency_Track_Project
FindSecurityBugs http://h3xstream.github.io/find-sec-bugs/
FindSecurityBugs-Cloud https://code.google.com/p/findbugs/wiki/FindBugsCloudTutorial
retire.js http://bekk.github.io/retire.js/
ScanJS https://github.com/mozilla/scanjs
Jenkins Log Parser Plugin https://wiki.jenkins-ci.org/display/JENKINS/Log+Parser+Plugin
ThreadFix http://www.threadfix.org
OWASP Code Pulse https://www.owasp.org/index.php/OWASP_Code_Pulse_Project


Interested in more web security stuff?
Visit my Blog: www.Christian-Schneider.net
@cschneider4711
Bildquelle: dreamstime.com

More Related Content

PDF
Security DevOps - Wie Sie in agilen Projekten trotzdem sicher bleiben // DevO...
Christian Schneider
 
PDF
Security DevOps - Free pentesters' time to focus on high-hanging fruits // Ha...
Christian Schneider
 
PPT
Automating security test using Selenium and OWASP ZAP - Practical DevSecOps
Mohammed A. Imran
 
PPTX
Syntribos API Security Test Automation
Matthew Valdes
 
PDF
An Attacker's View of Serverless and GraphQL Apps - Abhay Bhargav - AppSec Ca...
Abhay Bhargav
 
ODP
Automating OWASP ZAP - DevCSecCon talk
Simon Bennetts
 
PDF
N Different Strategies to Automate OWASP ZAP - OWASP APPSec BUCHAREST - Oct 1...
gmaran23
 
PDF
Merging Security with DevOps - An AppSec Perspective
Abhay Bhargav
 
Security DevOps - Wie Sie in agilen Projekten trotzdem sicher bleiben // DevO...
Christian Schneider
 
Security DevOps - Free pentesters' time to focus on high-hanging fruits // Ha...
Christian Schneider
 
Automating security test using Selenium and OWASP ZAP - Practical DevSecOps
Mohammed A. Imran
 
Syntribos API Security Test Automation
Matthew Valdes
 
An Attacker's View of Serverless and GraphQL Apps - Abhay Bhargav - AppSec Ca...
Abhay Bhargav
 
Automating OWASP ZAP - DevCSecCon talk
Simon Bennetts
 
N Different Strategies to Automate OWASP ZAP - OWASP APPSec BUCHAREST - Oct 1...
gmaran23
 
Merging Security with DevOps - An AppSec Perspective
Abhay Bhargav
 

What's hot (20)

PPTX
Security Testing with Zap
Soluto
 
PDF
Zed Attack Proxy (ZAP)
JAINAM KAPADIYA
 
PPTX
Security Testing - Zap It
Manjyot Singh
 
PPTX
Zed attack proxy [ What is ZAP(Zed Attack Proxy)? ]
raj upadhyay
 
PPTX
The OWASP Zed Attack Proxy
Aditya Gupta
 
PDF
Tw noche geek quito webappsec
Thoughtworks
 
ODP
OWASP 2013 APPSEC USA Talk - OWASP ZAP
Simon Bennetts
 
ODP
OWASP Zed Attack Proxy Demonstration - OWASP Bangalore Nov 22 2014
gmaran23
 
PDF
Owasp zap
ColdFusionConference
 
PDF
N Different Strategies to Automate OWASP ZAP - Cybersecurity WithTheBest - Oc...
gmaran23
 
PPTX
Learn to pen-test with OWASP ZAP
Paul Ionescu
 
PPTX
Integrating Splunk into your Spring Applications
Damien Dallimore
 
ODP
2014 ZAP Workshop 2: Contexts and Fuzzing
Simon Bennetts
 
PPTX
OWASP Zed Attack Proxy
Fadi Abdulwahab
 
ODP
BlackHat 2014 OWASP ZAP Turbo Talk
Simon Bennetts
 
PPTX
InSpec Workshop DevSecCon 2017
Mandi Walls
 
ODP
BSides Manchester 2014 ZAP Advanced Features
Simon Bennetts
 
PDF
"Design First" APIs with Swagger
scolestock
 
PDF
Node.js vs Play Framework
Yevgeniy Brikman
 
PPTX
DevSecCon Tel Aviv 2018 - Integrated Security Testing by Morgan Roman
DevSecCon
 
Security Testing with Zap
Soluto
 
Zed Attack Proxy (ZAP)
JAINAM KAPADIYA
 
Security Testing - Zap It
Manjyot Singh
 
Zed attack proxy [ What is ZAP(Zed Attack Proxy)? ]
raj upadhyay
 
The OWASP Zed Attack Proxy
Aditya Gupta
 
Tw noche geek quito webappsec
Thoughtworks
 
OWASP 2013 APPSEC USA Talk - OWASP ZAP
Simon Bennetts
 
OWASP Zed Attack Proxy Demonstration - OWASP Bangalore Nov 22 2014
gmaran23
 
N Different Strategies to Automate OWASP ZAP - Cybersecurity WithTheBest - Oc...
gmaran23
 
Learn to pen-test with OWASP ZAP
Paul Ionescu
 
Integrating Splunk into your Spring Applications
Damien Dallimore
 
2014 ZAP Workshop 2: Contexts and Fuzzing
Simon Bennetts
 
OWASP Zed Attack Proxy
Fadi Abdulwahab
 
BlackHat 2014 OWASP ZAP Turbo Talk
Simon Bennetts
 
InSpec Workshop DevSecCon 2017
Mandi Walls
 
BSides Manchester 2014 ZAP Advanced Features
Simon Bennetts
 
"Design First" APIs with Swagger
scolestock
 
Node.js vs Play Framework
Yevgeniy Brikman
 
DevSecCon Tel Aviv 2018 - Integrated Security Testing by Morgan Roman
DevSecCon
 
Ad

Viewers also liked (20)

PPTX
Pwning with XSS: from alert() to reverse shell: Defcon Banglore 2013
Ajin Abraham
 
PDF
2014 11-06-sonarqube-asfws-141110031042-conversion-gate01
Cyber Security Alliance
 
PPTX
Owasp A9 USING KNOWN VULNERABLE COMPONENTS IT 6873 presentation
Derrick Hunter
 
PPTX
Managing Security in External Software Dependencies
thariyarox
 
PDF
27 jan 2012[1]
Biblioteca Escolar Aeob
 
PPTX
Continuous Security - TCCC
Wendy Istvanick
 
PPTX
Dependency check
David Karlsen
 
PDF
Hiding in Plain Sight: The Danger of Known Vulnerabilities
Imperva
 
PPTX
Live 2014 Survey Results: Open Source Development and Application Security Su...
Sonatype
 
PDF
Managing third party libraries
n|u - The Open Security Community
 
PDF
Abusing, Exploiting and Pwning with Firefox Add-ons: OWASP Appsec 2013 Presen...
Ajin Abraham
 
PDF
[Poland] SecOps live cooking with OWASP appsec tools
OWASP EEE
 
PPTX
News Bytes - December 2015
n|u - The Open Security Community
 
PDF
DevOpsCon 2016 - Continuous Security Testing - Stephan Kaps
Stephan Kaps
 
PPTX
Simplify Dev with Complicated Security Tools
Kevin Fealey
 
PDF
Continuous Integration with Maven for Android apps
Hugo Josefson
 
PDF
Les principales failles de sécurité des applications web actuelles
Bee_Ware
 
PDF
Owasp Project を使ってみた
Akitsugu Ito
 
PDF
Secure Software Development with 3rd Party Dependencies
thariyarox
 
PDF
Devops/Sysops security
Logicaltrust pl
 
Pwning with XSS: from alert() to reverse shell: Defcon Banglore 2013
Ajin Abraham
 
2014 11-06-sonarqube-asfws-141110031042-conversion-gate01
Cyber Security Alliance
 
Owasp A9 USING KNOWN VULNERABLE COMPONENTS IT 6873 presentation
Derrick Hunter
 
Managing Security in External Software Dependencies
thariyarox
 
27 jan 2012[1]
Biblioteca Escolar Aeob
 
Continuous Security - TCCC
Wendy Istvanick
 
Dependency check
David Karlsen
 
Hiding in Plain Sight: The Danger of Known Vulnerabilities
Imperva
 
Live 2014 Survey Results: Open Source Development and Application Security Su...
Sonatype
 
Managing third party libraries
n|u - The Open Security Community
 
Abusing, Exploiting and Pwning with Firefox Add-ons: OWASP Appsec 2013 Presen...
Ajin Abraham
 
[Poland] SecOps live cooking with OWASP appsec tools
OWASP EEE
 
News Bytes - December 2015
n|u - The Open Security Community
 
DevOpsCon 2016 - Continuous Security Testing - Stephan Kaps
Stephan Kaps
 
Simplify Dev with Complicated Security Tools
Kevin Fealey
 
Continuous Integration with Maven for Android apps
Hugo Josefson
 
Les principales failles de sécurité des applications web actuelles
Bee_Ware
 
Owasp Project を使ってみた
Akitsugu Ito
 
Secure Software Development with 3rd Party Dependencies
thariyarox
 
Devops/Sysops security
Logicaltrust pl
 
Ad

Similar to Security DevOps: Wie Sie in agilen Projekten trotzdem sicher bleiben // JAX 2015 (20)

PDF
Security DevOps - Staying secure in agile projects // OWASP AppSecEU 2015 - A...
Christian Schneider
 
PPTX
OWASP ZAP Workshop for QA Testers
Javan Rasokat
 
ODP
Simon Bennetts - Automating ZAP
DevSecCon
 
ODP
2017 Codemotion OWASP ZAP in CI/CD
Simon Bennetts
 
PDF
Security Testing with OWASP ZAP in CI/CD - Simon Bennetts - Codemotion Amster...
Codemotion
 
ODP
AllDayDevOps ZAP automation in CI
Simon Bennetts
 
PDF
Automated Security Testing
seleniumconf
 
PPTX
OWASP ZAP API Automation
Thivya Lakshmi
 
PDF
DAST in CI/CD pipelines using Selenium & OWASP ZAP
srini0x00
 
PDF
2021 ZAP Automation in CI/CD
Simon Bennetts
 
PDF
Security Automation using ZAP
Vaibhav Gupta
 
ODP
2014 ZAP Workshop 1: Getting Started
Simon Bennetts
 
ODP
JavaOne 2014 Security Testing for Developers using OWASP ZAP
Simon Bennetts
 
PPT
Zap attack proxy
Artem Vasilenko
 
PPTX
OWSAP Zap Tool Execution - API Security Scan
Palani Kumar
 
ODP
OWASP 2012 AppSec Dublin ZAP Intro
Simon Bennetts
 
PPTX
Owasp zap
penetration Tester
 
ODP
OWASP 2013 EU Tour Amsterdam ZAP Intro
Simon Bennetts
 
PDF
we45 DEFCON Workshop - Building AppSec Automation with Python
Abhay Bhargav
 
ODP
OWASP 2013 AppSec EU Hamburg - ZAP Innovations
Simon Bennetts
 
Security DevOps - Staying secure in agile projects // OWASP AppSecEU 2015 - A...
Christian Schneider
 
OWASP ZAP Workshop for QA Testers
Javan Rasokat
 
Simon Bennetts - Automating ZAP
DevSecCon
 
2017 Codemotion OWASP ZAP in CI/CD
Simon Bennetts
 
Security Testing with OWASP ZAP in CI/CD - Simon Bennetts - Codemotion Amster...
Codemotion
 
AllDayDevOps ZAP automation in CI
Simon Bennetts
 
Automated Security Testing
seleniumconf
 
OWASP ZAP API Automation
Thivya Lakshmi
 
DAST in CI/CD pipelines using Selenium & OWASP ZAP
srini0x00
 
2021 ZAP Automation in CI/CD
Simon Bennetts
 
Security Automation using ZAP
Vaibhav Gupta
 
2014 ZAP Workshop 1: Getting Started
Simon Bennetts
 
JavaOne 2014 Security Testing for Developers using OWASP ZAP
Simon Bennetts
 
Zap attack proxy
Artem Vasilenko
 
OWSAP Zap Tool Execution - API Security Scan
Palani Kumar
 
OWASP 2012 AppSec Dublin ZAP Intro
Simon Bennetts
 
OWASP 2013 EU Tour Amsterdam ZAP Intro
Simon Bennetts
 
we45 DEFCON Workshop - Building AppSec Automation with Python
Abhay Bhargav
 
OWASP 2013 AppSec EU Hamburg - ZAP Innovations
Simon Bennetts
 

Recently uploaded (20)

PDF
Microsoft Teams Essentials; The pricing and the versions_PDF.pdf
Q-Advise
 
PPTX
AI-Ready Handoff: Auto-Summaries & Draft Emails from MQL to Slack in One Flow
bbedford2
 
PDF
Key Features to Look for in Arizona App Development Services
Net-Craft.com
 
PPTX
Role Of Python In Programing Language.pptx
jaykoshti048
 
PPTX
ConcordeApp: Engineering Global Impact & Unlocking Billions in Event ROI with AI
chastechaste14
 
PPTX
Odoo Integration Services by Candidroot Solutions
CandidRoot Solutions Private Limited
 
PDF
Micromaid: A simple Mermaid-like chart generator for Pharo
ESUG
 
PDF
On Software Engineers' Productivity - Beyond Misleading Metrics
Romén Rodríguez-Gil
 
PDF
Protecting the Digital World Cyber Securit
dnthakkar16
 
PDF
Multi-factor Authentication (MFA) requirement for Microsoft 365 Admin Center_...
Q-Advise
 
DOCX
The Five Best AI Cover Tools in 2025.docx
aivoicelabofficial
 
PPTX
Presentation of Computer CLASS 2 .pptx
darshilchaudhary558
 
PDF
QAware_Mario-Leander_Reimer_Architecting and Building a K8s-based AI Platform...
QAware GmbH
 
PDF
ShowUs: Pharo Stream Deck (ESUG 2025, Gdansk)
ESUG
 
PDF
Exploring AI Agents in Process Industries
amoreira6
 
PPTX
EU POPs Limits & Digital Product Passports Compliance Strategy 2025.pptx
Certivo Inc
 
PDF
A REACT POMODORO TIMER WEB APPLICATION.pdf
Michael624841
 
DOCX
The Future of Smart Factories Why Embedded Analytics Leads the Way
Varsha Nayak
 
PPTX
oapresentation.pptx
mehatdhavalrajubhai
 
PDF
Build Multi-agent using Agent Development Kit
FadyIbrahim23
 
Microsoft Teams Essentials; The pricing and the versions_PDF.pdf
Q-Advise
 
AI-Ready Handoff: Auto-Summaries & Draft Emails from MQL to Slack in One Flow
bbedford2
 
Key Features to Look for in Arizona App Development Services
Net-Craft.com
 
Role Of Python In Programing Language.pptx
jaykoshti048
 
ConcordeApp: Engineering Global Impact & Unlocking Billions in Event ROI with AI
chastechaste14
 
Odoo Integration Services by Candidroot Solutions
CandidRoot Solutions Private Limited
 
Micromaid: A simple Mermaid-like chart generator for Pharo
ESUG
 
On Software Engineers' Productivity - Beyond Misleading Metrics
Romén Rodríguez-Gil
 
Protecting the Digital World Cyber Securit
dnthakkar16
 
Multi-factor Authentication (MFA) requirement for Microsoft 365 Admin Center_...
Q-Advise
 
The Five Best AI Cover Tools in 2025.docx
aivoicelabofficial
 
Presentation of Computer CLASS 2 .pptx
darshilchaudhary558
 
QAware_Mario-Leander_Reimer_Architecting and Building a K8s-based AI Platform...
QAware GmbH
 
ShowUs: Pharo Stream Deck (ESUG 2025, Gdansk)
ESUG
 
Exploring AI Agents in Process Industries
amoreira6
 
EU POPs Limits & Digital Product Passports Compliance Strategy 2025.pptx
Certivo Inc
 
A REACT POMODORO TIMER WEB APPLICATION.pdf
Michael624841
 
The Future of Smart Factories Why Embedded Analytics Leads the Way
Varsha Nayak
 
oapresentation.pptx
mehatdhavalrajubhai
 
Build Multi-agent using Agent Development Kit
FadyIbrahim23
 

Security DevOps: Wie Sie in agilen Projekten trotzdem sicher bleiben // JAX 2015

  • 1. Christian Schneider | @cschneider4711 Security DevOps 
 Wie Sie in agilen Projekten trotzdem sicher bleiben
  • 2. `whoami` • Software Developer, Whitehat Hacker & Trainer • als Freelancer im Projektgeschäft seit 1997 • Fokus auf JavaEE Architekturen und Web Security • Speaker auf Konferenzen (JAX,WebTechCon, OWASP AppSecEU, DevOps Conference, …) + HackPra • @cschneider4711 www.
 mail@ Christian-Schneider.net
  • 5. Dynamic Application Security Testing (DAST) Fun Part
  • 6. The Ratio Problem Dev Ops Sec 100 10 1: :
  • 7. The Frequency Problem mehr Rollouts als Pentests
  • 9. Integrate Security Tools into Build-Jobs / CI-Chain on every commit or nightly
  • 10. Reaching the different levels of "Security DevOps" integration…
  • 11. Security DevOps Maturity Model (SDOMM) implicit master four axes each with four belts as small steps
  • 12. Four different axes Dynamic Depth Intensity Static DepthConsolidation 4 3 2 1 1 2 3 4 4 3 2 1 1 2 3 4
  • 13. Four different axes Dynamic Depth Intensity Static DepthConsolidation 4 3 2 1 1 2 3 4 4 3 2 1 1 2 3 4
  • 14. Axis of "Dynamic Depth" How deep are dynamic scans executed within a Security DevOps CI chain?
 i.e. "where" are dynamic 
 security tests applied?
  • 15. Axis "Dynamic Depth": Level 1 Scanning of public attack surface (pre-auth): • Spidering of UI layer • No requirement to authenticate scanner with target • Easy integration of scanner(s) in nightly build as post-step • "Throw tool at it (in CI-chain) and see what it generates…"
  • 16. Tool example: ZAP "OWASP ZAP" features: • passive scanning (Proxy / Spider) • active scanning (Proxy / Spider) • manual Payload Delivery (Intercepting Proxy) • Spider (classic & AJAX) • Fuzzing • Plugins • …
  • 20. ZAP in SecDevOps? "OWASP ZAP" features relevant for Security DevOps integration: • … • Headless Operation Mode / Daemon • REST-API (with several language bindings as pre-built clients) • Scriptable • CLI
  • 21. ZAP + Jenkins = SecDevOps? "OWASP ZAP" (spider & scanner) + Jenkins plugin "ZAProxy" • Allows us to "Spider & Scan" as step in build job via Jenkins plugin • Point plugin config to URL of integration system to test • Plugin saves HTML-report in project’s job for inspection • Best as separate Jenkins job to run during nightly build (duration)
  • 22. ZAProxy Jenkins Plugin: ZAP startup
  • 24. UI- and Service-Tests in CI Web Browser Build-Scripts UI-Tests (Selenium, etc.) Service-Tests (REST, SOAP) Application Server CI Jobs
  • 25. … enhanced with OWASP ZAP Web Browser Build-Scripts UI-Tests (Selenium, etc.) Service-Tests (REST, SOAP) Application Server OWASP ZAP CI Jobs passive & active scanning place report (html,xml) 
 into workspace
  • 26. HTML report of ZAP scan
  • 27. Tool example:Arachni "Arachni Scanner" features relevant for Security DevOps integration: • Passive scanning (Proxy / Spider) • Active scanning (Proxy / Spider) • Uses internally a headless browser-cluster (for apps with lots of JS) • Automation? • CLI + RPC API • Web-UI (helpful when permanently running as server)
  • 28. Arachni + Jenkins = SecDevOps? "Arachni Scanner" + Jenkins CLI step in build • Start in build job as CLI step and point to URL of system under test • Generate HTML report and place into workspace for inspection • Better execute within nightly build job (due to duration)
  • 29. HTML report of Arachni scan (Summary) Bildquelle: Arachni Scanner
  • 30. HTML report of Arachni scan 
 (Details) Bildquelle: Arachni Scanner
  • 31. Tool example: BDD-Security BDD-based framework for functional and technical security tests: • Technical security tests (i.e. check against XSS, SQL-Injection, XXE, etc.) • mainly uses ZAP as scanning engine • Functional security tests (i.e. check UserA doesn’t see data from UserB) • Tightly integrates with Selenium based app navigation workflows • Uses JBehave for G/W/T stories & reporting • Can run within CI (Jenkins, etc.) due to JBehave 
 or as JUnit tests
  • 33. HTML report of BDD-Security
  • 34. Other dynamic scanners exist also… Some have different focus • Nikto • w3af • IronWASP • Nessus • Gauntlt (BDD-based Security tool orchestration) • Mittn (BDD-based) • Minion (integration framework) • OWTF (integration framework) • good commercial ones • etc.
  • 35. Axis "Dynamic Depth": Level 2 Scanning of authenticated parts (= "post-auth") via UI layer • Properly maintaining sessions • Logout-detection & automatic re-login • Different users / roles • Spider & scan post-auth
 Handling of hardening measures of application under test • CSRF-Tokens, CAPTCHAs, etc.
  • 36. Guide ZAP into Post-Auth Many ways to give authentication infos to ZAP • Use ZAP manually to configure "Context": RegExps for Logged-In/Out Indicators, Users etc. + save as "ZAP Session-File" (could be in SCM) • use that "Session-File" from SCM as starting point of scan 
 (loaded via Jenkins plugin) • Also Scripts in JavaScript or Zest can be registered in ZAP 
 context to programmatically give authentication to ZAP • Alternatively set these auth values via ZAP’s REST-API 
 during each build before scan starts (from Jenkins/Maven/…) "Forced User Mode" to stick to certain users during scans
  • 37. Login config example within ZAP
  • 38. ZAProxy Jenkins Plugin: ZAP session use
  • 39. Guide Arachni into Post-Auth Give authentication infos to Arachni (Auth, Logged-In Indicators, Users) • Use Arachni "autologin" plugin to specify via command line • Login URL, formfield names, credentials, logged-in indicator, excludes • Alternatively write custom ruby script for "login_script" plugin • Individual custom login logic possible • Logged-In indicators (RegExp) to know when to re-login 

  • 40. Login config example within Arachni ./arachni 
 
 --plugin=autologin:
 url=http://10.33.48.80/marathon/secured/profile.page,
 parameters='j_username=john&j_password=john',
 check='Logout' 
 
 --scope-exclude-pattern=logout.page 
 
 http://10.33.48.80/marathon Or individual ruby script if more custom login logic required…
  • 41. Guide BDD-Security into Post-Auth Use Selenium to navigate through the login process • Based on excellent integration of BDD-Security with Selenium • Separate app navigation code (Selenium) from Security testing code • Use (along with test user credentials) within BDD stories • Perform further spidering & active scanning (through ZAP) post-auth BDD-Security based stories integrate with custom code to deliver 
 test CAPTCHA value to scanner • while also checking that mismatches are handled with denial)
  • 42. What about CSRF-Tokens? ZAP can handle CSRF-Tokens • Can be individualised via Config BDD-Security uses (among others) ZAP for scanning Arachni can also handle CSRF-Tokens during spidering & scanning
  • 43. Apply targeted delta scanning: • Developer teams maintain "scope delta files" in SCM • like "what parts of the application have changed during this sprint" (= URL patterns) • DAST Scanner uses it to scan only this during onCommit or nightly CI run • Feed this delta information as argument to scope definitions
 Arachni: --scope-include-pattern / --scope-exclude-pattern
 ZAP: Include/Excludes in Context (via Session-File or REST-API)
 BDD-Security: maintain as "table file" of RegExps as scope If scanning takes too long?
  • 44. Train spiders about the structure of application to reduce scan redundancy • ZAP: use RegExp "Exclude from scanner" rules to prevent redundant scans in data-driven applications (i.e. scan only one instance of it) • Arachni: --scope-redundant-path-pattern / --scope-auto-redundant Don’t forget to scan complete application at least on nightly or weekly run • Only apply full scan, if simple smoke-tests of system 
 under test are green If scanning takes too long?
  • 45. Axis "Dynamic Depth": Level 3 Separate scanning of different application layers / backends • Scan internal WebServices (e.g. SOAP / REST) = directly scan backends • Detect and scan parameter positions within XML, JSON, … • Scan from "within" the different application’s layers • IAST with distributed agents & instrumentation aims into that direction • At least one simple step in that direction: • Use the proxy also between your backend service calls
  • 46. Backend scans with ZAP How to achieve this with ZAP? • ZAP operates as proxy server: place it between backend calls • ZAP can inject payloads in observed XML tags/attributes & JSON fields • Capture service call traffic in integration test during CI while either A. executing service tests that directly access the service endpoint, or B. frontend UI tests execute service backend calls indirectly • Automatically scan as new requests are seen: "ATTACK Mode" Also keep an eye on an alpha-level SOAP-Scanner (by WSDL) ZAP addon
  • 47. Backend scans with Arachni How to achieve this with Arachni? • Arachni can also operate as proxy server: place it between backend calls • Use passive proxy plugin to "train" Arachni of the XML / JSON requests • New addition in v1.1 to extract XML / JSON input vectors from that • Use that collected input vector data to feed the active scan for the observed requests
  • 48. Axis "Dynamic Depth": Level 4 Targeted scanning of individual forms / wizards (UI) and service layers • More individualised workflow coverage (not just simple spidering) • Business-logic compliant usage patterns & inputs • "fill shopping cart followed by checkout process" • "access backendWebServices in special order to test workflow" • etc. • Custom coded security tests tailored to the application
  • 49. ZAP with special workflows (1/2) Re-use existing UI tests (Selenium, …) and/or WebService tests • Proxy Selenium-generated traffic through ZAP (in integration test phase) 
 in ATTACK-Mode More individual: Re-use & enhance existing "UI test code" at the proper 
 workflow steps with points of where to attack by calling ZAP’s REST-API
 (simple to use REST-API clients are pre-shipped with ZAP) • Type of scans at these positions can be individualised (policy config, etc.) • See https://github.com/continuumsecurity/zap-webdriver 
 for a working example with ZAP • Start ZAP as Daemon from Jenkins with Zapper Jenkins plugin
  • 50. ZAP with special workflows (2/2) Alternatively "train" ZAP about the workflow by recording Zest scripts • Keep an eye on "Sequence Scanning" alpha-level ZAP extension • Still alpha-level (as of May 2015), but interesting approach
  • 51. Use Selenium to further drive BDD-Security initiated checks: • Selenium-based test code navigates application workflows • This code is integrated with BDD (via Java interfaces), so that: • BDD-Security stories can use that code to navigate 
 and generate traffic • This generated traffic will be scanned by ZAP via BDD BDD with special workflows
  • 53. If no Selenium testcases exist? Simply give developer teams access to ZAP to pre-seed the scanner (at least): • Developer teams use browser to navigate app workflows while proxying • Thereby seed the ZAP session(s) with navigation nodes/workflows • Save the ZAP session(s) and check-in into SCM (Git, SVN, …) • Point the Jenkins ZAP plugin to the saved ZAP session(s) as starting point • Devs can add to this list of URLs for ZAP with each new UI BTW: ZAP is also available as Docker image…
  • 54. Security-UnitTests for APIs Sometimes it’s useful to have programmed unit-tests directly calling a business API • Test directly at the API-level (i.e. Java code level) • Utilise fuzzing values from popular sources like "fuzzdb" 
 and/or generate on-the-fly with "radamsa" • Check the API method’s response 
 (like regular unit test, but seeking for signs of successful attack tests) • Leverage the benefit of direct API-level testing (i.e. testing "from within") • Place "probes" into the application to watch out for successful attack tests • Mock objects (Mockito, etc.) or weave aspects (AOP) 
 collecting data (i.e. method calls and arguments, etc.) • Even logs (of error messages) are good candidates to check
  • 55. Axis of "Static Depth" How deep is static code analysis performed 
 within a Security DevOps CI chain?
 i.e. "where" are static 
 security tests applied?
  • 56. Axis "Static Depth": Level 1 Assurance that no third-party code with known vulnerabilities is used • Check application’s server-side dependencies • For Java applications: JAR files, etc. • Check application’s client-side dependencies • JavaScript frameworks, etc.
 Useful even for projects not currently under development • New vulnerabilities could become public: Build regularly to check
  • 57. Tool: OWASP Dependency Check Scans all dependencies (even transitive ones) against CVE list • Available as Maven plugin and Ant task • CLI version also available • Jenkins plugin for nice reporting (and build breaking thresholds) • Not false-positive free • regularly needs some time to triage findings
  • 61. For JavaScript libs: retire.js Scans application’s JavaScript files against list of known vulnerable ones • Available also as Maven Plugin… • … and as CLI (point it to your folder containing .js files)
  • 62. Axis "Static Depth": Level 2 Scan important parts of source code for vulnerability patterns • At least important parts of applications’ codebases are scanned • like in-house reused code & custom developed frameworks • for big multi-project Maven projects: 
 at least frontend and backend projects • or code of changed modules within a sprint
  • 63. Java: FindSecurityBugs Plugin for FindBugs with enhanced checks for security issues in Java code • Runs within FindBugs, so that it can execute in Maven and/or Sonar • Generates XML-file with potential findings • Jenkins plugin for representing results (also possible to display in Sonar) • Integrates with Jenkins "Static Code Analysis" as part of FindBugs • Delta reporting of new (and fixed) findings & trends
  • 64. FindBugs in Jenkins: Trends, etc.
  • 65. in Sonar: Vulnerability & Code-Pointer
  • 66. JavaScript: ScanJS Scans JavaScript code for vulnerabilities • Most importantly DOM-based XSS (Cross-Site Scripting) issues • Flags also "Usage of sensitive API" issues • Executable as CLI from within Jenkins build steps • Optional Web-UI (running as local server)
  • 67. ScanJS Findings as HTML: DOM-XSS
  • 68. Ruby on Rails apps: Brakeman Scans RoR code for vulnerabilities • CLI based • Nicely integrates with Jenkins • Rendering scan results and trend analysis • Allows to compare scan results (delta detection) • "What has been fixed and what was newly introduced?"
  • 69. Other static scanners exist also… Some have different focus • PrimeJS (like ScanJS for JavaScript regarding DOM-XSS etc.) • PMD (for Java code regarding API safety issues) • good commercial ones • etc.
  • 70. Axis "Static Depth": Level 3 Scan the complete applications’ source code for vulnerability patterns • Just like "Level 2", but now for the complete codebase (no excludes) • i.e. all projects of a multi-project Maven project • best via parent or corporate pom.xml For FindSecurityBugs this means: • pre-compile even all JSPs (if you have any) to scan them
  • 71. Axis "Static Depth": Level 4 Scan source code of (important) third-party dependencies used in application • If the component is open-source: • include it in regular scanning activities • If the component is closed-source: • consider using code scanners that operate on binaries • FindBugs (hence its FindSecurityBugs plugin) 
 can also scan binaries, since it operates on Bytecode
  • 72. Axis of "Intensity" How intense are the majority of the executed attacks within a Security DevOps CI chain?
 i.e. "what" is being 
 checked for?
  • 73. Axis "Intensity": Level 1 Dynamic checks: Only passive scanning • Simply proxy testcase generated traffic through passive operating scanners • Quick-win for using existing testcase code (i.e. Service- or UI-tests) • Can execute within existing tests that execute on every commit • No requirement for a separate nightly CI job (i.e. doesn’t slow down build) 
 Static checks: Just scan the code along with other code metrics • Use the default rules of the scanners and integrate them into CI
  • 74. Axis "Intensity": Level 2 Use lightweight active scanning options for dynamic checks • ZAP: active scanning of observed URLs (proxied or spidered) • perform an "active scan": For Jenkins ZAProxy Plugin provide a ZAP policy with desired active scan intensity (or via ZAP’s REST-API during build for other setups) • Arachni: enable "active scan" during spidering • BDD-Security stories can call ZAP to actively attack the application Better to execute in separate (nightly) build job due it’s to duration Think of resetting databases (+ more environment) prior or after test execution
  • 75. Axis "Intensity": Level 3 Use heavyweight scanning options on important parts of application Dynamic checks: create "riskier" custom attack profiles ZAP: • set "Threshold" to "Low" / "Strength" to "High" (or selectively "Insane") • enable more scanning rules via Policy Files (could be maintained & used via SCM) • use ZAP’s "Advanced SQLInjection Scanner" extension (or directly call sqlmap) Arachni: Enable the more aggressive "audit flags" like --audit-forms, … Static checks: FindSecurityBugs: set "Threshold" to "Low" / "Effort" to "Max"
  • 76. Axis "Intensity": Level 4 Use customised rule sets for dynamic checks • ZAP (which is very extensible via custom scripts, recored and customised Zest or plugins): • Custom coded scan scripts as active & passive scan rules for company- individual checks • Security regression tests (in Zest) against previous found vulns (also logic flaws) • Custom input vector scripts: e.g. define certain injection points in custom encoded params • Proxy scripts (for example for generic request/response modifications)
  • 78. Axis "Intensity": Level 4 Use customised rule sets for static checks • FindSecurityBugs: Custom coded rule files (in Java) to check for 
 wrong usages of server-side inhouse frameworks (or alike) • ScanJS: Custom rule files (in JSON) to check for wrong usages 
 of client-side inhouse frameworks (or alike)
  • 79. Axis of "Consolidation" How complete is the process of handling findings within a Security DevOps CI chain?
 i.e. "how" are the 
 results used?
  • 80. Axis "Consolidation": Level 1 Generate human-readable (HTML) reports from tools and link them in Jenkins • All relevant mentioned static and dynamic scanners generate HTML reports • Collect and publish them in Jenkins build: via Jenkins "HTML Publisher Plugin" Use simple criteria to "break the build" on heavy findings (ok, at least "unstable") • Dependency-Check, BDD-Security (with the JBehave-stories),
 FindSecurityBugs (via Sonar when rated as blocker),
 Arachni (via Gauntlt execution with BDD-like stories), etc.
 all have capabilities to automatically flag the build • For others: at least do a simple log parse from Jenkins 
 "Log Parser Plugin" to flag the build as instable and/or broken
  • 81. OWASP Dependency Check: 
 Thresholds to break build
  • 82. Reports & Sessions Simple reporting with ZAP in CI: • Export HTML report & link from build (via Jenkins Plugin "ZAProxy" or "HTML Publisher") • Save the ZAP session with build for later manual inspection (by security team) For Arachni scans in CI: • Generate & link readable HTML report from that (via arachni-report tool in build) • Optionally also save AFR file with build BDD-Security directly generates reports in JBehave formats (HTML, XML, …) 
 or via JUnit from within build job
  • 83. Jenkins HTML Publisher Plugin: Configuration of HTML reports to link
  • 85. FindSecurityBugs: IDE support & Collaboration FindBugs has excellent Jenkins integration Additionally consider placing the FindBugs generated XML report file within Jenkins • Developers can load the XML report file into FindBugs IDE plugins 
 (Eclipse, IntelliJ, NetBeans, …) • Developers & Security Analysts can collaborate on bugs via the IDE plugins • See https://code.google.com/p/findbugs/wiki/FindBugsCloudTutorial 
 for more details • Consider installing it as your private on-premise 
 "FindBugs"-Server to keep results only on your servers…
  • 86. FindSecurityBugs Eclipse Perspective with Collaboration View + Remediation
  • 87. Axis "Consolidation": Level 2 Custom logic to make build unstable and/or broken depending on • Type of vulnerability (CWE or WASC or …) • Confirmation level (firm) • Severity rating (high risk) Provide useful remediation info to developers Respect suppression mechanisms to rule out false positives
  • 88. Flagging builds from reports How (from within a CI job)? • Most scanners also emit XML reports that can be parsed • mostly a simple XPath count should be fine • Alternatively fetch the results by accessing the scanner’s APIs
  • 89. Providing useful infos to Devs Lookup findings by CWE-ID (provided by most scanners) in vulndb on GitHub • https://github.com/vulndb/data (currently 65+ vulns described) • Useful if the messages from scanners are too generic or don’t help much • Maps vulnerability to useful remediation messages • Possibility to extend that repository locally with company-individual texts
  • 90. Suppression of false positives BDD-Security • Use false positive tables in story files FindSecurityBugs • Use exclusion lists in config (XML filter files) • Use annotation @SuppressWarnings on false positive code lines Dependency-Check • XML file of suppressions (checked-in along with the project) retire.js • "ignore file" parameters on CLI (cloud be loaded in CI from SCM)
  • 91. Axis "Consolidation": Level 3 Consolidation goals: • Consolidate & de-duplicate findings from different 
 scanner reports (with better false positive handling) • Push consolidated findings into established bug-tracker 
 (known to devs) • Delta analysis & trends over consolidated data sets
  • 92. ThreadFix as result consolidator Use a local ThreadFix server, which imports native scanner outputs • does the heavy lifting of consolidation & de-duplication • pushes findings toward bug-tracker and IDE (via plugins) • process can be customised using it’s own REST-API • ThreadFix integrates with ZAP to import scans from ZAP
  • 93. Report generated by ThreadFix Bildquelle: ThreadFix
  • 94. Trend Chart generated by ThreadFix Bildquelle: ThreadFix
  • 95. Trends about vulnerable dependencies OWASP Dependency-Track provides trend insight into findings about use of vulnerable components (from OWASP Dependency-Check): • across multiple applications and versions • across time
  • 96. Trend Chart generated by 
 OWASP Dependency Track Bildquelle: OWASP
  • 97. Axis "Consolidation": Level 4 Measure the concrete code coverage of your security testing activities • Find untested "white spots" • Derive where static checks and code reviews should 
 focus more to compensate How? • Use "OWASP Code Pulse", which instruments your Java app via agent • collects coverage data during dynamic security testing scans • generates reports ("code treemaps") of coverage
  • 98. Code Treemap of dynamic scan coverage Bildquelle: OWASP Code Pulse
  • 99. Without CI build chain? Now that we’ve seen several ways to introduce security checks into the build chain
 (four axes with four levels each)
 … What if we don’t have a full-blown CI? Or simply cannot extend it that way 
 (i.e. build job would run too long)?
  • 100. Use recurring scans instead! As an alternative to a build-chain based scan recurring scans can be used: • Don’t block the build, but get nightly or weekly results delivered • No need to integrate into full build chain • Still leverage the other benefits • Dev-maintained delta files defining scopes • Consolidation levels of results • Basically all four levels for each axis apply the same • = Quick-win for DAST + SAST scans independent of build-chain
  • 101. Recurring scans:Arachni Remote Arachni (non-blocking in build chain): • Instead of a scan step in Jenkins build, the RPC-API of a
 remote Arachni server can be used to run/schedule a scan • Collect findings separately after scan 
 or link to WebUI of Arachni Server to present them
  • 102. Recurring scans: ZAP Remote OWASP ZAP (non-blocking in build chain): • Instead of a scan step in Jenkins build, the REST-API of a remote 
 ZAP can be used to run/schedule a scan (cron, etc.) • Collect findings separately after scan • Or use Seccubus to have an integrated framework 
 for scan scheduling with ZAP • also has comparison of different scans to report deltas • ZAP authenticated scans scheduled via Seccubus:
 -o "--auth-form http://example.com/login --auth-user myuser --auth-pass mypass --auth-verify-url http:// example.com/profile -X /logout" --hosts @HOSTS
  • 103. Thank you very much!
  • 104. Links OWASP ZAP https://www.owasp.org/index.php/OWASP_Zed_Attack_Proxy_Project ZAP Selenium Demo https://github.com/continuumsecurity/zap-webdriver ZAP Jenkins Plugin https://wiki.jenkins-ci.org/display/JENKINS/ZAProxy+Plugin BDD-Security http://www.continuumsecurity.net/bdd-intro.html Arachni http://www.arachni-scanner.com OWASP Dependency Check https://www.owasp.org/index.php/OWASP_Dependency_Check OWASP Dependency Track https://www.owasp.org/index.php/OWASP_Dependency_Track_Project FindSecurityBugs http://h3xstream.github.io/find-sec-bugs/ FindSecurityBugs-Cloud https://code.google.com/p/findbugs/wiki/FindBugsCloudTutorial retire.js http://bekk.github.io/retire.js/ ScanJS https://github.com/mozilla/scanjs Jenkins Log Parser Plugin https://wiki.jenkins-ci.org/display/JENKINS/Log+Parser+Plugin ThreadFix http://www.threadfix.org OWASP Code Pulse https://www.owasp.org/index.php/OWASP_Code_Pulse_Project 
 Interested in more web security stuff? Visit my Blog: www.Christian-Schneider.net @cschneider4711 Bildquelle: dreamstime.com