SAP Cloud ALM for Implementation

Test Management
Reiner Markheiser, SAP
February 01, 2024

Public
Disclaimer

The information in this presentation is confidential and proprietary to SAP and may not be disclosed without the permission of SAP.
Except for your obligation to protect confidential information, this presentation is not subject to your license agreement or any other service
or subscription agreement with SAP. SAP has no obligation to pursue any course of business outlined in this presentation or any related
document, or to develop or release any functionality mentioned therein.
This presentation, or any related document and SAP's strategy and possible future developments, products and or platforms directions and
functionality are all subject to change and may be changed by SAP at any time for any reason without notice. The information in this
presentation is not a commitment, promise or legal obligation to deliver any material, code or functionality. This presentation is provided
without a warranty of any kind, either express or implied, including but not limited to, the implied warranties of merchantability, fitness for a
particular purpose, or non-infringement. This presentation is for informational purposes and may not be incorporated into a contract. SAP
assumes no responsibility for errors or omissions in this presentation, except if such damages were caused by SAP’s intentional or gross
negligence.
All forward-looking statements are subject to various risks and uncertainties that could cause actual results to differ materially from
expectations. Readers are cautioned not to place undue reliance on these forward-looking statements, which speak only as of their dates,
and they should not be relied upon in making purchasing decisions.

Public 2
Agenda

Introduction SAP Cloud ALM for Implementation

Overview of Test Management

Functions & Features of Test Management

Analytics for Test Management

Test Automation

Questions

Public 3
One Suite from Strategy to Execution: Transformation Suite Approach

Built-In Support

Strategy Capabilities Processes Solutions Execution

powered by AI

One Methodology One process One integrated Toolchain

Public 4
ONE Integrated Tool-Chain
Integration with additional tools

Solution pre check

Document & Content Custom Code Mitigation Data Transformation
with SAP Readiness
Management with Service Partners with Service Partners
Check
with MS 365
Core Capabilities Solution Configuration
OKR &
Strategy Execution with SAP Central
Change Requirements Solution Business
Management Process
Management & Management Documentation Configuration
with Tangible Growth Owner
Collaboration Approval
Process
Project & Task
Governance
Management
Built-In Support
Test Automation
Process Analysis with TTA for
(“smart mining”) Test SAP and SAP
Advanced S/4HANA Cloud
Planning
Reporting
with SAP Analytics
Cloud Test Execution

Defect
Experience Data based Management
Employee Process Improvement
Engagement
Management
with Qualtrics Change & Feature
Transport
Management
Management
Collaborative Process
with SAP ABAP
Improvement Collaborative
Value Accelerator Monitoring & Deployment Release TMS and SAP
Process
Risk & Compliance Consumption Alerting Orchestration Management Cloud TMS
Definition &
Management (Reference Business and Solution
Architectures, Best Practice Processes) Design
with GRC

Operation Automation,
Process Automation with SAP Ticket integration Integration Automation
Build Process Automation with SAP Cloud Integration
Transformation Automation Service
Initiative Management
Public 5
This is the current state of planning and may be changed by SAP at any time.
We manage Your Solution

Out-of-the-Box Open

Cloud Native Integrated

SAP Standard
Traceable
Content

SAP Cloud ALM provides the harmonized implementation experience for the intelligent enterprise,
covering all relevant SAP solutions for all customers

Public 6
We manage Your Solution with SAP Cloud ALM

Manage all project activities Ensure Quality

§ Based on the SAP Activate methodology with § Manage all test from user acceptance to
template roadmaps for agile implementation regression
projects § Manual testing with test step based approach
§ Extended by your own tasks and user stories § Integration of automatic execution engines
§ Integrated planning and reporting

Design the to be solution Manage Successful

Deployment & Go-Live
§ Start with Best Practice Content
to run your Fit-2-Standard § Plan your releases and
workshops orchestrate consistent
§ Create own processes with deployment to production
integrated authoring capabilities § Enable change and provide
§ Collect requirements as an initial traceability and audit trails
backlog

Public 7
Some number for SAP Cloud ALM for implementation

~2000 ~2k
active created
parallel features
projets per per month
month

+11k +5k
Daily Users in managed
defects per
110 countries month

+2k +5k
created Test runs
requirements per per month
month

+82k ~2k
Scoped +4k created
Solution Custom diagrams
Processes Solution per month
Processes
Public 8
 Business Key User Test Managers
Project Leads Change Manager
Process Expert Test Experts
Functional Consultants All Project Members Testers Release Manager

Functionalities Detailed Reporting and Analytics

Integration of Change
Fit-2-Standard Process Project Test Test Automation Deployment
Workshops Hierarchy Planning Landscape Planning provider
Documentation
Planning
S/4HANA
BPMN DMS SAP Standard Test Automation Traceability Deployment
Modelling Integration Templates Configuration * Test Cases Tool Automation
SAP Status & Task Test Defect Change Feature
Requirements
Best Practice Approval Distribution Preparation Management Reporting Delivery
Refinement
Requirements Manual Audit Trails
Open APIs SAP Activate Open APIs Open APIs Open APIs
collection Testing

Tricentis Test Automation for SAP

Core Capabilities Project

Process Project & Task
Documentation Execution &
Management Management
Tracking

Phase
Design Build Test Deploy
Technology

Public 9
High flexibility provided by SAP Cloud ALM

Free choice of used capabilities depending on

§ Functional needs
– Process Management
– Requirements Management
– User Stories Processes Requirements

– Task Management
– Test Management
– Change & Deployment Management User Stories Project & Tasks

– Analytics
§ Implementation use case; e.g. S/4HANA Cloud,
Test Orchestration Change enablement
S/4HANA Private Cloud, Success Factors
§ Integration of third party solutions

Public 10
Maximum Value by following E2E implementation process

Benefits of using the core tool chain in

SAP Cloud ALM Embedded Analytics

Accelerated and content driven implementation

• Best Practice processes Processes Requirements

• Best Practice Test cases

User Stories Project & Tasks
• SAP Activate roadmap / task templates

Tightly integrated to ensure customer success

Test Orchestration Change enablement

• Minimum setup and no integration effort

• Transparency and traceability out-of-the-box

Public 11
Maximum Value by following E2E implementation process

§ Utilize entity relations as basis for an integrated

process Embedded Analytics
§ Manage entity dependencies and implementation
process governance via traceability views
– Solution Process Traceability
Processes Requirements
– Requirement Traceability
– User Story Traceability
– Feature Traceability
User Stories Project & Tasks
§ Requirement driven implementation to support
customer specific innovation
Test Orchestration Change enablement

Public 12
 SAP Cloud ALM Entities
Project
Discover Prepare Explore Realize Deploy Run

Task Template Sprint 1 Sprint 2 Sprint 3

Cross Project
Entities Team 1 Project Tasks Project Tasks Project Tasks

Roles Sub-Tasks Sub-Tasks

Scope 1
Customer
Processes
Process

Tags Process Requirement

User Story User Story Test Case Defect

Workstreams

User Story Test Case

Test Plan

Sub-Tasks Feature
Systems

Landscape
Document
assignments

Deployment Release 1 Release 2

Plan
Public 13
Agenda

Introduction SAP Cloud ALM for Implementation

Overview of Test Management

Functions & Features of Test Management

Analytics for Test Management

Test Automation

Questions

Public 14
Testing concepts in SAP Cloud ALM

Lean Agile Flexible Test Levels Process-Oriented

Simple concepts Test as soon as possible Test with a purpose Derive the structure from the
process flows
No complexity Test as fast as possible Relation to
overhead Process, Derive the content from the
Requirement, process assets
User Story (S/4 HANA)
to achieve traceability
Add test actions

Rich text support

Advanced Test Orchestration using Test Plans

Public 15
SAP Cloud ALM Test Management
Objectives and Capabilities

Easy to consume solution.

Open to support all products in the future: SAP Cloud ALM

• Cloud and On-Premise
• SAP and non-SAP Test Scope
§ Management
Test Orchestration
Test Orchestration § Execution
• Structure Test Cases via Scopes and Processes § Monitoring
• Manual Functional Testing § Signoff Manual Testing

• Automated Functional Testing § Reporting/Auditing

Test Automation
• Multiple Test Cycles via Test Plans Framework
• Traceability

Test Automation:
§ Integration of Cloud-based Test Automation providers:
Test automation tool for SAP Third-party Cloud Test
§ Test automation tool for SAP S/4 HANA Cloud S/4HANA Cloud Automation Providers
§ Tricentis Test Automation for SAP Cloud ALM

Public 16
SAP Cloud ALM Test Management

SAP Cloud ALM

Accelerated Implementation and Innovation based on SAP Best Practice Content

Test Test
Preparation Execution
Test
Planning

Manual Tests

Requirements User Stories

Automated Tests

S/4HANA Test Automation Tool

Efficient Regression Testing Test Automation

via integrated Test Automation

Public 17
Orchestration via Traceability

Process, Requirements and User Stories within SAP Cloud ALM can be associated to test cases to
ensure their test coverage before deployment to production

Process Integration
Test

Requirement Acceptance
Test
This allows us to
achieve traceability User Functional
§ from demand Stories Test

§ through realization
§ through testing
§ until deployment
Public 18
Agenda

Introduction SAP Cloud ALM for Implementation

Overview of Test Management

Functions & Features of Test Management

Analytics for Test Management

Test Automation

Questions

Public 19
SAP Cloud ALM – Overview Page

Overview Page can be used as a central entry for all test related information and activities
§ Cards of interest and filters can be managed via views e.g. to get a test centric view
§ All test entities and activities are managed with in the context of a project and scope
§ Option to use Tags for more granular view

Public 20
Manual Testing
Test Preparation

§ Creation of manual test cases is

done based on business activities
from the selected process flow

§ Relevant business activities

should be scoped so they are
included in the test case

§ The overall structure of the test

case is created through the
numerical order of these activities

Alternative approach using Best Practice

Test Assets for S4/HANA see blog post:
Import Test Scripts from the SAP Best
Practice content for SAP S/4 HANA

Public 21
Manual Testing
Test Preparation

Use Content tab to enhance each

process activity with test actions
§ Title
§ Instructions
§ Expected Results

Additional tabs to assign

§ Requirements
§ User Story
for traceability

Simple copy of test case to reuse existing

test case content or just parts of it

Public 22
Manual Testing
Test Preparation

General Information tab shows useful

information like:
§ Scope (inherited from Solution Process)
§ Solution Process, Flow and Diagram
§ On top you can create and assign Tags to further
categorize and classify your test cases

When done, you can set Test Case to

Prepared and save, so it appears in the Test
Execution Overview

Public 23
Test Planning

Benefits
§ Supports multiple test cycles or roll-
outs to multiple organizations re-using
same or similar set of test cases
§ Test Plan Status to manage
preparation, execution and completion
of test phases or test cycles
§ Tester assignment per test case
§ Dedicated execution context (status
and defects) per test plan
§ Start and end date can be set for test
plan
§ Embedded draft handling to keep
unsaved data
§ Efficient reporting and details per test
plan

Public 24
 Test Preparation Test Plan Test Execution

Test Case 1 ! Defect 1

! Defect 3
none
Test Case 2
! Defect 2

Test Case 1 Test Case 4

Test Case 2

Test Case 3

Test Case 4

Public 25
 Test Preparation Test Plan Test Execution

Test Case 1 ! Defect 1

! Defect 3
none
Test Case 2
! Defect 2

Test Case 1 Test Case 4

Test Plan A Test Case 1

Test Case 2

Test Case 2

Test Case 3 Test Case 3

Test Case 4

Public 26
 Test Preparation Test Plan Test Execution

Test Case 1 ! Defect 1

! Defect 3
none
Test Case 2
! Defect 2

Test Case 1 Test Case 4

Test Plan A Test Case 1 Tester A

Test Case 2

Test Case 2 Tester B

Test Case 3 Tester C

Test Case 3

Test Case 4

Public 27
 Test Preparation Test Plan Test Execution

Test Case 1 ! Defect 1

! Defect 3
none
Test Case 2
! Defect 2

Test Case 1 Test Case 4

Test Plan A Test Case 1 Tester A Test Case 1

! Defect 4
Test Case 2

Test Case 2 Tester B Test Case 2

Test Case 3 Tester C

Test Case 3

Test Case 4

Public 28
 Test Preparation Test Plan Test Execution

Test Case 1 ! Defect 1

! Defect 3
none
Test Case 2
! Defect 2

Test Case 1 Test Case 4

Test Plan A Test Case 1 Tester A Test Case 1

! Defect 4
Test Case 2

Test Case 2 Tester B Test Case 2

Test Case 3 Tester C

Test Case 3

Test Plan B Test Case 2 Tester X

Test Case 4

Test Case 3 Tester Y

Test Case 4 Tester Z

Public 29
Test Planning and Tester assignment

Benefits

• Easy assignment of Test

Cases by search and filter
capabilities

Public 30
Test Planning and Tester assignment

Benefits

• Easy assignment of Test

Cases by search and filter
capabilities

• Assign Tester to Test Case

Public 31
Test Execution with Test Plans and Tester assignments

Benefits

• Navigation from Overview Page,

Analytics and Traceability views

• Search and various Filters can be

used to find the relevant test cases

• Allows users to set a Tester filter to

view their own test cases or those
of multiple users on a team.

• Test plans with status ‘In testing’

available for execution in your
current test phase or test cycle.

• Test Case are grouped by Test

Plans

• Lean testing without the need to

create test plans will be still
available via filter value (None)

• Test Manager and Auditors can use

this list view for reporting purpose

Public 32
Manual Testing
Test Execution

Test Execution Detail view is used by tester to

work on Test Actions
§ Read instructions, expected results and set
status for each Test Action

Test run could be withdrawn

when needed to rate current run
as obsolete

Public 33
Manual Testing
Test Execution

Document Test Results / Evidence via Comments including screenshots

Public 34
Manual Testing
Create Defect

In case of failed test actions you can create defects to follow-up resolution

Public 35
Manual Testing
Defect Management

Defect detail view

• Description (rich text)
• Related Test Cases
• References (e.g. URL to app or
documentation)
• Additional information with status,
useful attributes, responsible team,
role, assignee and tags for follow-up
and reporting.

Public 36
Manual Testing
Test Execution: Download Test Execution Results

In the test execution, you can download the

test execution result to excel including most
important data like
§ who tested when
§ number of total, passed, failed and untested
actions
§ number of defects

Public 37
Agenda

Introduction SAP Cloud ALM for Implementation

Overview of Test Management

Functions & Features of Test Management

Analytics for Test Management

Test Automation

Questions

Public 38
Test Management Analytics
Overview Page Defects Reporting

Test Execution Analysis Traceability Reports

Public 39
Test Management Analytics
Overview Page

Overview Page can be used as a central entry for all test related information and activities
§ Cards of interest and filters can be managed via variants e.g. to have a test centric view
§ All test entities and activities are managed with in the context of a project and scope
§ Option to use Tags and Test Plans for more granular view (hidden by default)

Public 40
Test Management Analytics
Test Execution Analysis

Monitor the progress of Test Execution with different chart types

• Execution by Test Status, Filter by Test Plan

• Burn-up of Executed Test Actions versa Burn-Down of Remaining Test Actions

Public 41
Manual Test Action Analysis Report

Executed Manual Test Analysis can be found

as a tab in test Execution analysis app.

The report provides visibility on the amount of

actions remaining (work to be done for testers)
before a test run could be finished

It offers next level drill down to see that a test

run is in progress as just the status is not
enough. Progress can be viewed in terms of
actions executed v/s remaining

Manual test runs can be weighed by the

number of actions to be done comprises
actions grouped into activities

Public 42
Test Management Analytics
Defects Reporting

Monitor the progress of Defect Resolution with different chart types

Navigation to
Defect details

Public 43
Test Management Analytics
Traceability views for Processes, Requirements and User Stories

Check Readiness
for Test Execution

Processes Requirements User Stories Testing Deployment

Public 44
Test Management Analytics
Traceability views Requirements
Which Requirements are ready for Test Execution

Selection Criteria Action to be performed

§ Requirement Status is ‘In Realization’ § Set Requirement Status ‘In Testing’
§ Feature Status is ‘In Testing’ § Start test execution
§ All User Stories are set to ‘Done’
§ All Test Cases are set to ‘Prepared’
Public 45
Test Management Analytics
Traceability views for Processes, Requirements and User Stories

Check Readiness
for Deployment

Processes Requirements User Stories Testing Deployment

Public 46
Test Management Analytics
Traceability views Requirements
Which Requirements are ready for deployment to production ?

Selection Criteria Action to be performed

§ Requirement Status is ‘In Testing’ § Navigate to feature of requirement to set status to
§ Feature Status is ‘In Testing’ “Ready for Deployment”
§ All user stories are set to ‘Done’ § Deploy transports to production
§ All test cases are set to ‘Passed’
§ Defects are either ‘Closed’ or ‘No defects assigned’
Public 47
Traceability Enhancements : Indirect Assignments

Traceability apps now offer next

level of transparency by including
‘Indirect Assignments’

Indirect Assignments are indirectly

related objects that are not
assigned to the parent object, but
contribute in assessment of overall
readiness of the main object.

Requirements Traceability: By
switching on the Test case in
indirect assignment button, test
cases that are not assigned to the
parent requirement but assigned to
a child user story are also counted
and included under Test Execution Test case that was not
and Test Preparation columns. assigned to requirement

Related user story to

Public
which it is assigned 48
Traceability Enhancements : Indirect Assignments

Requirements Traceability:
Defects are always indirectly
related to requirements as the
relationship exists only with test
runs.

When the Indirect Assignment –

Defect is switched on, the defects
column is shown in the matrix.

The defects and its related test

case is shown in the pop over

Defect as indirect assignment

Test exe to which it is assigned

Public 49
Traceability Enhancements : Indirect Assignments

User Story Traceability: Defects

are always indirectly related to
user stories as the relationship
exists only with test runs.

When the Indirect Assignment –

Defect is switched on, the defects
column is shown in the matrix.

The defects and its related test

case is shown in the pop over

Public 50
Agenda

Introduction SAP Cloud ALM for Implementation

Overview of Test Management

Functions & Features of Test Management

Analytics for Test Management

Test Automation

Questions

Public 51
Test Automation

Do What You Do Best Simple Integration Evolutive

SAP Cloud ALM Only synchronize necessary Based on versioned REST

orchestrates info APIs

Automation provider does Loose coupling Ready for future evolutions

automation and requirements

Public 52
 Test Automation
Automation Provider Cloud
Demand Build Test Deploy
Test Automation
Test Orchestration
Test Cases and Data Variants
ALM Entities:
• Processes • Manual Test Cases
• Requirements • Automated Test Cases API
• User Stories and Variant
• Defects • Test Results

Authoring Environment
Flow Test Data

Analytics & Traceability

Landscape / Systems under Test

On Premise Hybrid / Private Cloud Public Cloud
Execution Environment

Public 53
Test Preparation for Automated Tests
3

1 2

4
5

1. With the Create feature you can create new Automated Test Cases in the integrated Test Automation Tool in the right Cloud ALM context
(Scope / Solution Process)
2. When you add new Solution Processes to a Scope, related Automated Test Cases are automatically synchronized to Test Preparation. Sync button
manually triggers synchronization for scoped Solution Processes
3. With the Import feature you can import any Automated Test Case that is available in the integrated Test Automation Tool
4. Display and navigation to Data Variant in Test Automation Tool
5. Display and navigation to endpoint for Test Automation Provider in Landscape Management
Public 54
Test Execution for Automated Tests

2 1 3

1. With Execute you can start the execution of Automated Test Cases in the integrated Test Automation Tool
2. The Sync button trigger an update of status and progress information
3. Open details for Executed Test Runs
4. Navigation to Test Execution Log in Test Automation Tool

Public 55
Integration of ALM Solutions from SAP and Test Automation by Tricentis

Tricentis Tosca Tricentis Test Automation

SAP Enterprise Continuous Testing by Tricentis SAP Test Automation by Tricentis*

*Planned Product or Offering

Tricentis Test Automation for SAP integrated with Tricentis Test Automation for SAP integrated with
SAP Solution Manager SAP Cloud ALM

Test Management in SAP Solution Manager Test Orchestration in SAP Cloud ALM

Additional license required

Included in SAP Enterprise Support and SAP Cloud Subscription

Public 56
 SAP S/4HANA Cloud Public – Test Automation Tool

Robust Recording & Execution Error Analysis driven by ML to

engine hosted on BTP integrated provide recommendations and
with Hyperscalers follow ups steps
Integrated with
SAP Cloud ALM

Industry first approach No-code approach to UI

to regression testing and API Test
with Post Upgrade S/4HANA Cloud
Automation
Tests Test Automation Tool

Accelerated implementation with Centralized Test Data repository

pre-delivered SAP Best Practice for scalable and repeated test
Content executions

Public
Driven by Customer Voice 57
Agenda

Introduction SAP Cloud ALM for Implementation

Overview of Test Management

Functions & Features of Test Management

Analytics for Test Management

Test Automation

Questions

Public 58
Outlook* SA
P Lab
sP
rev
iew

Roadmap explorer (Roadmap)

• Direct Upload of Test Test Sequences in

Cases
• • API for Test Cases
Test Plans

• Improved Test Case / • Document-based Test

Process integration Cases

• Variant Management in
Test Execution

• Variant Management in
Test Preparation

Public *This is the current state of planning and may be changed by SAP at any time. 59
Additional Information SAP Cloud ALM

Find out more Learn what’s new

(Homepage) (SAP Help Portal)

See what’s planned Experience it

(Roadmap) (Demo Center)

Request it Get help

(Order form) (In-app help)

Give feedback Contact us

(Qualtrics) (E-mail)

Public 60
Join our SAP Cloud ALM Community!

Follow the tag SAP Cloud ALM on SAP Community

Engage with the best and brightest of the software industry

Learn from blog posts and questions and answers

Public 61
Thank you.
Contact information:

Reiner Markheiser
[email protected]

© 2024 SAP SE or an SAP affiliate company. All rights reserved. See Legal Notice on www.sap.com/legal-notice for use terms, disclaimers, disclosures, or restrictions related to this material.