Bridging the Gap Between Relational Data and Application-

Level Business Objects with Core Data Services (CDS)

Stefan Bäuerle, Alexander Böhm SAP SE
September 2017

PUBLIC
Agenda

• Hybrid Transactional/Analytic Processing (HTAP)

• Challenges
• Motivation for CDS

• CDS as a possible solution

• Core CDS concepts
• Types
• Entities
• Associations
• Annotations
• Results from S/4HANA

© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 2

 S/4HANA
Application

HTAP Challenges SQL!(?)

HANA
DBMS
HTAP in S/4HANA

• S/4 builds on the classical business suite data model

• Normalized (usually 3NF or higher)
• Lots of small tables with configuration data, translated texts, etc.
• Lots of legacy to deal with:
• German column names difficult for international customer base (MANDT, BUKRS, …)
• Heavy use of NVARCHAR for non-textual data (DATE, TIMESTAMP, BOOLEAN, numbers, …)
• Abuse of fields (e.g. number encoded as NVARCHAR[1], but need 20 values -> “X”)

• Database views are at the core of the data model

• Translate technical, legacy data to modern representation
• Meaningful column names
• Get rid of “hacks”, proper types
• Enable analytics by creating meaningful business objects (e.g. invoice) from database tables
• Business objects span multiple database tables
• Business objects often need to be combined to derive insights

© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 4

A simple view from DBMS perspective

CREATE VIEW "SAPQM7"."MBVMBEW" AS

SELECT "B"."MANDT", "B"."MATNR", "B"."BWKEY", "B"."BWTAR", "B"."LVORM",
CAST( CASE WHEN ( "B"."BWTAR" = N'' AND NOT ( "B"."BWTTY" = N'' ) )
THEN "MOTHER"."LBKUM" ELSE "B"."LBKUM" END AS DECIMAL( 000013, 000003) ) AS "LBKUM",
CAST( CASE WHEN ( "B"."BWTAR" = N'' AND NOT ( "B"."BWTTY" = N'' ) )
THEN "MOTHER"."SALK3" ELSE "B"."SALK3" END AS DECIMAL( 000013, 000002) ) AS "SALK3",
Type"B"."PEINH",
"B"."VPRSV", "B"."VERPR", "B"."STPRS", adjustment "B"."BKLAS",
CAST( "B"."SALKV" AS DECIMAL( 000013, 000002) ) AS "SALKV",
Field
--73 attributes selection
left out from input tables based on dependent fields
CAST( CASE WHEN ( "B"."BWTAR" = N'‘ AND NOT ( "B"."BWTTY" = N'' ) )
THEN "MOTHER"."VKSAL" ELSE "B"."VKSAL" END AS DECIMAL( 000013, 000002) ) AS "VKSAL",
"B"."HKMAT", --20 attributes left out
"MBEW"."DUMMY_VAL_INCL_EEW_PS", "MBEW"."OIPPINV"
FROM ( "MBEW" "MBEW" LEFT OUTER MANY TO ONE JOIN "MBVMBEWBASE" "B" ON ( "MBEW"."KALNR" =
"B"."KALNR" AND "MBEW"."MANDT" = "B"."MANDT" AND "MBEW"."MANDT" = "B"."MANDT" ) )
LEFT OUTER MANY TO ONE JOIN "MBVMBEWMOTHSEGM" "MOTHER" ON ( "MOTHER"."MATNR" = "B"."MATNR" AND
"MOTHER"."BWKEY" = "B"."BWKEY" AND "MOTHER"."MANDT" = "B"."MANDT" AND "MBEW"."MANDT" =
"MOTHER"."MANDT" ) WITH READ ONLY

© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 5

Reverse-engineering MBVMBEW

MBVMBEW

MBEW ⟕ MBVMBEWBASE ⟕ MBVMBEWMOTHSEGM

MANDT BWKEY, MATNR
KALNR MANDT

MBEW ⟕ V_ML_ACDOC_EX_UL ⟕ CKMLCR MBEW ⟕ V_ML_ACDOC_EX_UL

KALNR BWKEY, MATNR
KALNR
MANDT MANDT
BDATJ

ACDOCA_M_EXTRACT ⨝ FINSV_CURTP_ML
BUKRS, RLDNR, MANDT Π
FMLT_CURTP_ML

© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 6

Adding complex analytics

• MBVMBEW, NSDM_V_MARC and friends are simple elements of the S/4 stack

• Analytical processing creates stacks of these basic views

• Business: New reporting opportunities across verticals
• DBMS: Very high complexity
• Example: Sales Order Fulfillment Monitoring (SOFM)
• 197 DBMS objects accessed up to 21 times in a single query

© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 7

Core Data Services (CDS)
Motivation

• Application Developers are skilled domain experts

MATNR
join predicate Runtime down from 4s to <100 ms
• Imperative programming languages but not SQL missing
• Writing SQL is hard
• Often no clue about relational algebra
• Not familiar with DBMS-level optimization
• Danger to miss important details
• Example: MBVMBEW (again)

• Necessity for simpler DBMS interface

© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 9

 Core Data Services

• Pull data modeling, retrieval, and processing to a semantic level close to the domain experts

CDS = SQL (+ careful extensions)

• Key concepts
• Entities with structured types (instead of flat tables)
• Custom-defined/Semantic Types (instead of primitive types)
• Associations for foreign key relations with cardinalities and simple path filter expressions
• Annotations to enrich the data models with additional metadata – e.g. for Analytics

© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 10

CDS Concepts: Data Types

• Supported Types:
• „built-in“ Primitive Types (like String, Integer, DecimalFloat, Date)
• Custom-defined Simple- and Structure Types

• Examples:
type Derived : String(111);

type AddressType : String(7) enum {

home; business = ‘biz’;
}

type Structured {
descr : Derived; // reusing a custom-defined type
amount : Decimal(10,2);
grossAmount : Decimal(10,2) = amount * (1.00 + taxrate()); // calculated element
kind : AddressType default home;
}

© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 11

CDS Concepts: Entities

• Entities
• Define the persistence layer of an application
• Structured types with an underlying persistency and a uniquely identifying key
• Entity is defined like a structured type, just with a leading keyword entity instead of type

• Examples: type Amount {

value : Decimal(10,2);
currency : String(3);
}

entity Address {
key streetAddress : String(77);
key zipCode : String(11);
city : String(44);
}

entity Employee {
key ID : UUID;
name : String(77);
salary : Amount; // Amount is a structured type
addresses : Association to Address[0..*] via entity Employee2Address;
}

© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 12

entity OtherAddress : Address; // has a separate persistence
CDS Concepts: Associations

• Associations define relationships between entities

• Which key to use
• Additional filter conditions (up to a complete join condition)
• Information regarding cardinality

• Examples:

© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 13

CDS Concepts: Annotations

• Domain-specific annotations to enrich/extend objects without changing the core model

CDS view example
• Example: @EndUserText.label: 'Financial Statement @EndUserText.label = <label> label for visualization/UI
sFIN'
@Analytics: { dataCategory: #FACT } @Analytics: { dataCategory: #FACT } fact table for BI tools
define view WSFinancialStatementQuery as
select
from WSFinancialStatement
{
key ChartOfAccounts,
key GLAccount,
…,
@Semantics.currencyCode: true @Semantics.currencyCode element is a currency code
key CompanyCodeCurrency,
@Semantics.amount.currencyCode:
@Semantics.amount.currency: indicates where to find the currency
'CompanyCodeCurrency'
@DefaultAggregation: #SUM @DefaultAggregation: #SUM default aggregation behavior for BI tools
@EndUserText.label: 'Amount In (other options are AVG, MIN, MAX, …)
Company Code
Currency'
AmountInCompanyCodeCurrency, … }

© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 14

Example: Data Model in CDS DDL and Mapping to Standard SQL

• CDS DDL • Standard SQL

© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 15

Example: Queries (QL)

• Superset of standard SQL (SQL + QL extensions)

• QL extensions to leverage enhancements provided by the data models

• Examples: (structured Types (salary.value) & Associations (orgunit.costcenter))

More Examples

© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 16

Example: Queries using QL enhancements and Standard SQL

• Retrieving a list of all ordered materials per companies

• CDS QL (heavy use of associations) • Standard SQL

SELECT FROM BSEG {bkpf.mandt, SELECT BKPF.MANDT, T001.BUTXT, MARA.MATXT,

bukrs.butxt, mara.matxt, SUM((menge)) SUM(BSEG.MENGE)
AS menge2 } FROM BKPF
WHERE bkpf.txkrs <> 0 AND menge > 0 JOIN BSEG ON BKPF.MANDT = BSEG.MANDT
GROUP BY bkpf.mandt, bukrs.butxt, AND BKPF.BUKRS = BSEG.BUKRS
mara.matxt; AND BKPF.BELNR = BSEG.BELNR
AND BKPF.GJAHR = BSEG.GJAHR
JOIN MARA ON BSEG.MANDT = MARA.MANDT
AND BSEG.MATNR = MARA.MATNR
JOIN T001 ON BSEG.MANDT = T001.MANDT
AND BSEG.BUKRS = T001.BUKRS
WHERE BSEG.MENGE > 0 AND BKPF.TXKRS <>0
GROUP BY BKPF.MANDT, T001.BUTXT,MARA.MATXT
© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 17
What are others doing?

• Many platforms ease the use of SQL through some kind of persistence framework
• Microsoft’s Entity Framework (EF) and LinQ
• Apple’s Core Data (CD)
• Force.com’s SOQL CDS Also supported in…

• JPA / Hibernate in JEE Parameterized Views

HANA, SQL Server, …
via table functions
• Active Records (AR) in Ruby on Rails
Annotations of Data Models EF, JPA, OData
Associations – all –, OData
• As CDS, most borrow and combine concepts Path Expressions + Infix Filters – all –, OData, XPath
from Entity-Relationship Modeling, XPath, … Calculated Attributes EF, CD, JPA
Custom-defined & Struct.Types – all –, SQL:1999
Structured Result Sets ( SQL:1999 impls )
Intrinsic Extensibility SOQL
Predicated Privileges JPA, …, Sybase ASE

© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 18

Where are the Differences to other Approaches?

• Other frameworks are bound to a particular application stack

• CDS is designed as an extension to SQL, independent of the application stack

• CDS stays in the relational model, instead of hiding it behind object-relational mappers
• Preserves advantages such as declarative/functional approach

Client Client
Object-Relational Relational / SQL

Application Stack Application Stack Entity Framework, LinQ

Core Data SQL + CDS

Persistence Fwk
JPA / Hibernate Force.com SOQL

Active Records
SQL +CDS
Database Database

© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 19

Summary

• HTAP creates data model / query challenges

• Need to increase productivity / ease of use for application developers

• CDS can be used to simplify modelling and view creation

• Results so far in S/4HANA application stack

• Over 20.000 CDS views
• Over 2,1 mio associations
• Average complexity:
• 12 tables reference (max: 1593)
• 3 levels of view stacks (max: 31)
• 96 associations (max: 30.236)

© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 20

Thank you.
Contact information:
Dr. Alexander Böhm

[email protected]