7/9/23, 12:48 Delta loading in SAP Datasphere based on the ABAP CDS view | SAP Blogs

Delta loading in SAP Assigned Tags

Datasphere based on the SAP Datasphere

ABAP Development
ABAP CDS view BW Operational Data
Provisioning (ODP)

4 7 259 SQL
#ABAPCDS
In applications where there is a high volume of data #CDS
creation, changes, and deletion happens, such as sales
#cdsAnnotations
orders, it is crucial for the application to have a mechanism
that provides delta records and for the extractor that needs View more...
to provide delta loading. This means that the application
should be able to identify and extract only the new,
modified or deleted data, ensuring efficient and accurate Similar Blog Posts
data loading.
SAP Datasphere – New
Replication Flow
In some of the cases nightly full uploads are not what we
By Mastan Reddy
exactly want, as time windows for data extractions are Veeramreddy Apr 07, 2023
limited. Solution for that is coming with ODP framework.
SAP Datasphere -
The ODP framework for CDS extraction provides delta Implementation Best
capabilities. We have actually two options for delta Practices
handling that we can chose from: By Prasanna Lakshmi
Kona Jun 23, 2023

Change Data Capture (CDC) Delta

Introducing Remote
Authorizations from
Generic Timestamp / Date based Delta SAP BW/4HANA for
SAP Datasphere
Let’s check the two delta mechanisms in detail below.
By Heiko Schneider Sep 23,
2021

Change Data Capture (CDC) Delta Related Questions

The easiest way to implement delta loading is to use CDC-
enabled CDS views by importing them as Remote tables in S/4 HANA CDS VIEW
which table join cannot
Datasphere and enabling Real-time access to them. real time access from
SAP Datasphere
Let us see how it is done: By Yuya Murade May 10,
2023

@AbapCatalog.sqlViewName: 'ZSQL_SALES_DELT Using delta non sap

@AbapCatalog.compiler.compareFilter: true sources to SAP
@AbapCatalog.preserveKey: true Datasphere
@AccessControl.authorizationCheck: #NOT_RE By Fabio José Perel Apr 27,
2023
@EndUserText.label: 'CDS View - Delta load

SAP Datasphere:
@Analytics.dataExtraction.enabled: true I_CustSalesPartnerFunc
@Analytics.dataCategory: #FACT CDC enabled but not
@Analytics.dataExtraction.delta.changeData working - Why?
define view ZCDS_SALES_DELTA as select fro
https://blogs.sap.com/2023/09/06/delta-loading-in-sap-datasphere-based-on-the-abap-cds-view/ 1/11
7/9/23, 12:48 Delta loading in SAP Datasphere based on the ABAP CDS view | SAP Blogs

inner join vbak on vbap.mandt = vbak.mandt By Martin Kreitlein Aug 29,

and vbap.vbeln = vbak.vbeln 2023
{

key vbak.vbeln as SalesDocument,

key vbap.posnr as SalesDocumentItem,
vbap.matnr as MaterialNumber,
vbap.kwmeng as CumulativeOrderQuan
vbap.vrkme as SalesUnit,
vbap.netwr as NetValue,
vbap.waerk as SalesDocumentCurrenc
vbap.arktx as TextforSalesOrderIte
}

This is our CDS view for delta loading on Sales

Documents for the purpose of an example.

If we want to declare a Dimension, Fact, Aggregation

Level or Cube we must to include the classification of the
CDS view in the header of our CDS view with the
following annotation:

@Analytics.dataCategory: #VALUE

Replace #VALUE by one of the categories commented

before:

#CUBE
#AGGREGATIONLEVEL
#DIMENSION
#FACT

A CDS view can be enabled for data extraction by just

adding the following annotation:

@Analytics.dataExtraction.enabled: true

After adding this annotation, the CDS view is available for

extraction by the ODP framework and visible for its data
consumers.

The Change Data Capture (CDC) recording mechanism

uses database triggers to record any changes to the tables
that belong to an ABAP CDS view. For example CDS
views such as CDS projection views, this can be done
automatically by using the following annotation:

Analytics.dataExtraction.delta.changeDataCapture.aut
omatic

https://blogs.sap.com/2023/09/06/delta-loading-in-sap-datasphere-based-on-the-abap-cds-view/ 2/11
7/9/23, 12:48 Delta loading in SAP Datasphere based on the ABAP CDS view | SAP Blogs

Please refer link for more

details: https://help.sap.com/docs/SAP_DATA_INTELLIG
ENCE/3a65df0ce7cd40d3a61225b7d3c86703/55b2a17f98
7744cba62903e97dd99aae.html

Adding the CDS view to the Datasphere as a Remote table

Choosing the SQL view name of CDS view

Import and Deploy as a Remote table of CDS view

Deployment has been done as a Remote table in

Datasphere

To be able to load initial data to the remote table we have

to run first Snapshot for it.

https://blogs.sap.com/2023/09/06/delta-loading-in-sap-datasphere-based-on-the-abap-cds-view/ 3/11
7/9/23, 12:48 Delta loading in SAP Datasphere based on the ABAP CDS view | SAP Blogs

Running Snapshot

Let’s take a closer look at some of the data that we initially

loaded first, because after using the Real-Time Access
feature, we will see that the data that changes in the source
system will also change in the Datasphere without
reloading or scheduling the data.

The data we are looking at as an example

We will see that the description in the “Short text for sales
order item” field of the Sales Documents changes in the
source system and is updated in Datasphere using the Real-
Time Access feature.

In order to load the changes (delta records) created in the

source system into Datasphere, we need to enable the data
replication feature for the remote table as “Real-Time
Access“.

Enabling the Real-Time Access feature

https://blogs.sap.com/2023/09/06/delta-loading-in-sap-datasphere-based-on-the-abap-cds-view/ 4/11
7/9/23, 12:48 Delta loading in SAP Datasphere based on the ABAP CDS view | SAP Blogs

Enabled the Real-Time Access feature

When we look at the details from the Data Integration

Monitor menu, it should be as follows:

Data Integration Monitor – general

Data Integration Monitor – details

Switching replication type for remote table

‘ZSQL_SALES_DELTA’ from batch replication to real-
time replication.
Subscribed to source system data changes.
Successful initial data transfer for real-time replication
of remote table ‘ZSQL_SALES_DELTA’. Now real-
time replication will automatically add delta data to
remote table ‘ZSQL_SALES_DELTA’.

Now let’s change the “Short text for sales order item” field
description of the Sales Documents in the source system
and see how it is reflected in the Datasphere.

https://blogs.sap.com/2023/09/06/delta-loading-in-sap-datasphere-based-on-the-abap-cds-view/ 5/11
7/9/23, 12:48 Delta loading in SAP Datasphere based on the ABAP CDS view | SAP Blogs

In the source system we change the data we specified as an

example above

All data updated in the source system are as above

The refresh frequency cannot be adjusted but my

observation is it takes place within 1 hour to get updated
records in Datasphere.

All data updates that we have as an example

Generic Timestamp / Date based Delta

In order to be able to load data on a date-based basis, it is
necessary to have date/time information in the relevant
application tables that is updated based on changes in the
application data.

“upd_tmstmp” is the field in the VBAK table which will

trigger delta records in our example.

https://blogs.sap.com/2023/09/06/delta-loading-in-sap-datasphere-based-on-the-abap-cds-view/ 6/11
7/9/23, 12:48 Delta loading in SAP Datasphere based on the ABAP CDS view | SAP Blogs

To define which field will trigger delta, we need to use

annotation

@Analytics.dataExtraction.delta.byElement.name

This element can be date (ABAP type DATS) or a time

stamp (UTC).

Lets update CDS view code with delta element annotation.

Updated CDS view

Please be aware of that: When converting a time stamp to a

date, we can use the tstmp_to_dats function as it up or we
can use the cast keyword as below.

cast(substring(cast(upd_tmstmp as abap.char(32)),1,8) as
abap.dats) as LastChangeDate

Both coding structures will give the same result.

After we have fulfilled our need, we will create a Data

Flow and limiting LastChangeDate = CurrentDate-1 at
Projection node in Datasphere.

This is our Data Flow – general view

The operation performed at the TO_DATE node: Since the

“LastChangeDate” field we created in the CDS view is of

https://blogs.sap.com/2023/09/06/delta-loading-in-sap-datasphere-based-on-the-abap-cds-view/ 7/11
7/9/23, 12:48 Delta loading in SAP Datasphere based on the ABAP CDS view | SAP Blogs

string type, need to be converted it to date type here in

node.

TO_DATE node

The operation performed at the FILTER node: The code

applied to obtain the changes (delta records) from a day
ago.

FILTER node

There are two methods we can apply to run the Data Flow
we created. They are:

The first one is: to run directly the Data Flow by

pressing the run button manually or
The second one is: to schedule the Data Flow to run
daily basis at specified time automatically.

In both of methods, all data that changed (delta records) the

day before in the source system will be uploaded to the
Datasphere.

As we have chosen to use the second method in our

example, the delta records obtained as a result of running
this data flow in the target table node one day later are
shown below.

https://blogs.sap.com/2023/09/06/delta-loading-in-sap-datasphere-based-on-the-abap-cds-view/ 8/11
7/9/23, 12:48 Delta loading in SAP Datasphere based on the ABAP CDS view | SAP Blogs

The result of changes (delta records) in Datasphere

The result of Scheduled Data Flow in Datasphere

Best Regards.

Alert Moderator

4 Comments

You must be Logged on to comment or reply to a post.

Julian Juraske
September 7, 2023 at 7:47 am

Hello Adem,

thanks for the great summary!

Do we not transfere the whole datatable in the second option from sourcesystem to
targetsystem ?
Shouldn't the purpose of Delta be, to transfere only the required data recors?

Is there a way to bypass the Value (Changedate) as a Filter to the Source ? (e.g. in a
BW Extractor)
Otherwise I could also just drop the Table and load full in the Dataflow, or implement my
https://blogs.sap.com/2023/09/06/delta-loading-in-sap-datasphere-based-on-the-abap-cds-view/ 9/11
7/9/23, 12:48 Delta loading in SAP Datasphere based on the ABAP CDS view | SAP Blogs

own Deltalogic within Datasphere(using SQL View) or use the Append Option in
combination with UPSERT.

Best Regards
Julian

Like 0Share

Adem GülerBlog Post Author

September 7, 2023 at 9:46 am

Hello Julian,

For the first question:

I did not see the need to mention this separately in order not to make the blog
too long. Of course, at each option, you must first initially load all records from
the source system.
The purpose of using delta logic is to exclusively retrieve newly created,
modified, and deleted records from the source system. This allows for a more
efficient and streamlined process of transferring data to the target system.

For second question:

I do not know. While the first two options are viable, I haven't tested the logic for
setting it as UPSERT. However, it's important to note that when a primary key is
present in our target table, we can employ that option to avoid any potential
duplicate key errors.

Best Regards.

Like 0Share

Julian Juraske
September 7, 2023 at 10:19 am

Hello Adem,

https://blogs.sap.com/2023/09/06/delta-loading-in-sap-datasphere-based-on-the-abap-cds-view/ 10/11
7/9/23, 12:48 Delta loading in SAP Datasphere based on the ABAP CDS view | SAP Blogs

In your first Option (CDC) you only Transfer the changed Records
(after Initial Full Load) = Delta.

In the second Option (DF) you ALWAYS(not only Initial) transfere the
whole Data from Source to Target.
Basicly what you are doing is a Full Load from Source to Target, and
Filter out the required Records in the Targetsystem (by setting a
Change Timestamp in Source).
The second Option (filter in Target System) can be achieved by many
different aproaches (e.g. SQL View comparing Data from today vs
yesterday, or the APPEND UPSERT Mode)
However for me this is not a real Delta, since you do not bypass a Filter
to the Source to reduces the amound of Records transfered.

Best Regards
Julian

Like 0Share

Martin Kreitlein
September 7, 2023 at 8:00 am

Hello Adem,

thanks a lot for that Blog.

What is missing here, for my understanding... how exactly is the Delta transfer
Followtechnically?
happening

I mean, there are two possibilities,

Like
the one you showed, where you need the Data Provisioning Agent
the other with Dataflows or Replication Flows, where you need the Cloud Connector.
RSS Feed
What exactly is happening if there is a network issue or the respective server is down
(DPA or CC) before the next Data replication?

Where is the information stored which records have been transferred successfully and
which not?

I tried to find the details about this in the Online Help, but I could not really find the
answers.

Thanks, Martin

Like 0Share

https://blogs.sap.com/2023/09/06/delta-loading-in-sap-datasphere-based-on-the-abap-cds-view/ 11/11