Curiosity and fourTheorem present: From Coverage Guesswork to Targeted Test Generation

© Curiosity Software Ireland Ltd. 2023
Housekeeping
You have all joined the Zoom webinar as an attendee. You can listen but cannot speak live.
Please connect to audio via phone or VOIP or phone, using the “Audio Settings” menu.
If you have questions as we go along, please hit Q&A to enter them using instant messenger.
If you have issues with audio or video, please use “Chat” and message “Hosts and panelists”.
We’ll have a live Q&A at the end.
This session will be recorded and the recording shared publicly.

From coverage guesswork
to targeted test generation
January 18th 2023
James Walker
Chief Technology Officer, Curiosity
James.Walker@Curiosity.Software
@CuriositySoft
Peter Elger
Founder and CEO, fourTheorem
Peter.Elger@fourTheorem.com
@pelger

Requirements
Why fourTheorem, why Curiosity, why
today?
UNDERSTANDING OBSERVABILITY
Test Results
Tests
Code
Coverage

From coverage guesswork to targeted
test generation
1. fourTheorem: Looking “under the hood” of complex systems
2. The under-used applications of deep monitoring to testing
3. Demo: Boost coverage and get the right test results
4. Q&A and next steps

fourTheorem
Looking under the hood of complex systems

About fourTheorem
• Our mission: Enable our clients to realize the full benefits of modern
cloud.
• By providing world class products, services, training and consulting
expertise to reliably and repeatably help our customers transform their
businesses and technology to modern cloud.
• AWS Advanced Consulting Partner
• Expertise in Serverless, cloud migration, application modernization

About fourTheorem
• Open-source creators
• Checkout Middy framework
• Community contributors
• Checkout AWS Bites podcast
• Book Authors
• Regular conference speakers
• AWS Serverless hero program

Example Work
• Reinsurance on premise risk analysis platform. Legacy monolith.
Execution times constraining business growth
• The Goal. Transform the risk analysis platform to run on AWS. Utilize
commodity, serverless infrastructure. Deliver horizontal scale.
• The Outcome
• Execution time cut by 90%
• Code base reduced by 70%
• Multiple parallel risk forecasting now a reality
• High Performance Computing using Serverless at scale!

Fission - by fourTheorem
• Fission is technology that has been developed by fourTheorem in
collaboration with Dublin City University.
• The research was partly funded under the Disruptive Technologies
Innovation Fund (DTIF)
• The original goal was NOT testing related, more oriented towards
the decomposition of software monoliths (as will be described).
• However, there are interesting applications in the testing domain.

Fission by fourTheorem
• Decision support for application
modernization
• Analyzes large applications (circa 500K
LOC and upwards)
• Determines optimal service decomposition
• Drastically shortens time to value for
migrations

© Curiosity Software Ireland Ltd. 2023 11
© FSSA Consortium 2021
CSE
1970 1980 1990 2000 2010
Waterfall
V-Model
RIPP
(Dupont)
RAD
[---- Other Agile Methods ---->
XP
RUP Lean
DevOps
Multiple Years
<1 day
Microservices
Large Increments
Continuous
Many Months
Spiral
Scrum
© Paul Clarke, 2021.
Established/Classical software
architectures constrain our
ability to:
- Ship frequently
- Ship small valuable new
features in isolation
- Take full economic
advantage of serverless
computing
Historical Perspective - Unit of Scale

Industry Background
Physical
On-Prem
Virtual
On-Prem
Virtual
On-Cloud
Containers
On-Cloud
Serverless
Traditional
Slow to Change
Monolithic
High TCO
High
Maintenance
Cloud 2.0
Highly Agile
Distributed
Lower TCO
Lower
Maintenance
Commoditization
The rise of the Serverless computing is a
natural result of commoditization.
Serverless/Cloud 2.0 will become the de-facto
standard for enterprise compute.

Why Fission?
• To realize the benefits of the next wave of cloud technology
organizations need to migrate.
• Direct lift and shift is often sub-optimal, resulting in high TCO.
• Decayed, monolithic, poorly maintained and poorly understood
code is often the cause.
• E.g. circa 1.5M LOC, over 20 years old, current maintenance
programmers understand perhaps 10% of the code

Monolith to (micro) Services
• Fission analyzes an application to establish the Domain Boundaries
as it runs
• Allows architects to play out decomposition scenarios
• To establish the optimal extraction patterns
• Automatically extracts code from the application into services
• Reducing analysis time from months to days
• Reducing the risk associated with modernization

Fission
• Collection agents gather
static and dynamic data
sets.
• Analysis pipeline
determines optimal domain
structure
• Results data used to extract
services
• No code leaves the
building!

Static Data Collection
• Static Collection is concerned with
the application source code.
• Static Collection works by breaking
code into symbols which form an
Abstract Syntax Tree (AST)

Dynamic Data Collection
• Dynamic Collection is concerned with the
running application.
• Dynamic collection works by placing an
agent on a running system. The agent
instruments the code to collect stack
traces as the program runs.
• Dynamic collection records what actually
happens on a running system by tracing
execution paths through the codebase.

Analytical Pipeline
• Applies multiple, composable, statistical machine learning algorithms
to the Dynamic and Static data sets.
• This includes both structural and semantic type analysis of the data sets.
• The pipeline is tuned by a human in the loop to play out different
scenarios and zone in on the optimal decomposition.
• Essentially the system identifies the logical domain structure within
the application and figures out the fracture lines between them.

Console - Recommended Services
• Services categorized
by extraction
coefficient
• Measures coupling
and ‘overlappedness’
of services

Console - Service Coupling
• View of the coupling between
services
• Coupling means the method
calls from one service to
another

Console - Overlap
• View of the
‘overlapped-ness’ of
services
• Overlapping means
methods that are
contained in both
services

Console - Service Details
• URLS, Database
access, methods that
constitute a service

Fission by fourTheorem
• So, how does this help with software testing?

The under-used
applications of this
approach to testing
Boosting coverage, adding traceability and enriching test assertions

Did your tests get the right result, for
the right reason?
Typical UI
testing
What happened
beneath the page?
</>
UI test
Now you’ve
partially tested
the service layer
Request
Response
+ API
testing
And how do you know that bugs haven’t masked bugs?
Is your system a black box or can you see its inner workings?
How are you targeting and maintaining tests?
But did the UI
interactions and API
calls update databases
correctly?
 Error message  Loaded screen

Test Modeller
Synchronise data from across
the SDLC to generate targeted
tests in-sprint
Find / Make / Allocate Test Data
Run Results
Tests
New
Iteration
 Seamless collaboration
 Better RQMTs  Better code
 Rigorous testing in-sprint
 Quality with every release
Test Automation
Test Automation
Test Cases
User stories
BDD/Gherkin
Scans and
Recordings
Requirements
Diagrams
Living Documentation
Cross-functional teams
BAs, POs,
testers, Dev,
SDETs, DBAs.
Find / Make / Allocate Test Data

Test Modeller with fourTheorem
</>
…for your
frameworks
…And systems
Generate tests…
Call
Highly granular run results
Target coverage
Boost traceability  Accelerate testing/dev
Stories
</>
Tests Code

+ DB comparisons during
your UI and API tests
Auto-generate UI & API tests
Database Comparison Engine
Baseline Data Data After Testing
</>
Your Automation
Frameworks
 Test across all
system tiers
 Boost coverage
 Granular run
results

Demo

Test Modeller x fourTheorem
Know what’s going on
under the covers
Focus where it will have
the greatest impact
Continuously generate
targeted tests and stories
Link test cases to code modules
Track tests at the data & code path levels
Domain and class analysis over time
Quality governance
Discovery
Pay off technical debt
Generate stories, tests and scripts
Integrated Test Data Automation
Risk and coverage-based test creation

Questions and Discussion

Carry on the conversation?
• Book a demo and discussion with a Curiosity expert!
• Connect fourTheorem on LinkedIn and connect to speak with Peter.
• Try Test Modeller. Visit Testmodeller.io for a free trial and online tutorials.
• Follow Curiosity Software on LinkedIn and @CuriositySoft on Twitter for
resources and news.
• Follow fourTheorem on LinkedIn and @fourTheorem on Twitter.

Thank you
James.Walker@Curiosity.Software
Peter.Elger@fourTheorem.com

Curiosity and fourTheorem present: From Coverage Guesswork to Targeted Test Generation

Recommended

More Related Content

Similar to Curiosity and fourTheorem present: From Coverage Guesswork to Targeted Test Generation (20)

More from Curiosity Software Ireland (11)

Recently uploaded (20)

Curiosity and fourTheorem present: From Coverage Guesswork to Targeted Test Generation