Scribd
Upload
35 views

Note 2620830 - How to Record SAP HANA Memory Allocator Traces to Analyze Memory Leaks

SAP Note 2620830 provides guidance on recording SAP HANA memory allocator traces to analyze memory leaks. It outlines symptoms of memory consumption issues, reasons for potential leaks, and detailed steps for collecting allocator traces and reports. The document also references related notes and components for further assistance and context on memory leak issues in SAP HANA.

Uploaded by

Mitchel Neyra
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
35 views

Note 2620830 - How to Record SAP HANA Memory Allocator Traces to Analyze Memory Leaks

SAP Note 2620830 provides guidance on recording SAP HANA memory allocator traces to analyze memory leaks. It outlines symptoms of memory consumption issues, reasons for potential leaks, and detailed steps for collecting allocator traces and reports. The document also references related notes and components for further assistance and context on memory leak issues in SAP HANA.

Uploaded by

Mitchel Neyra
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 4

SAP Note

2620830 - How To Record SAP HANA Memory Allocator Traces to Analyze Memory
Leaks
Component: HAN-DB (SAP HANA > SAP HANA Database), Version: 4, Released On: 14.05.2019

Symptom
You observe one or multiple of below situations:
1. You observe that the overall memory consumption slowly increases on the system even though there was no obvious
change to the load in the system.
You can see that a certain allocator is occupying a big amount of memory, e.g. using below query:

SELECT TOP 10 HOST, PORT, CATEGORY, EXCLUSIVE_SIZE_IN_USE FROM M_HEAP_MEMORY ORDER BY


EXCLUSIVE_SIZE_IN_USE DESC;

and there is no obvious culprit or query that it can be tracked back to.

2. You observe that the system is constantly running out of memory, even under a normal load.
The out of memory dump contains a section listing the top allocators ordered by exclusive_size_in_use, following below
pattern:

Top "M_HEAP_MEMORY" allocators (component, name, size). Ordered descending by exclusive_size_in_use.


1: <TYPE>: <ALLOCATOR_NAME> <SIZE> (ALLOCATOR_SIZE_IN_BYTE)
2: <TYPE>: <ALLOCATOR_NAME> <SIZE> (ALLOCATOR_SIZE_IN_BYTE)
3: <TYPE>: <ALLOCATOR_NAME> <SIZE> (ALLOCATOR_SIZE_IN_BYTE)
...

Below is an example which demonstrates that allocator 1 dominates memory consumption as it is much larger than all
others:

1: System: Pool/malloc/libhdbcswrapper.so 812.62gb

2: Statement Execution & Intermediate Results: Pool/JoinEvaluator/JERequestedAttributes/Results 55.48gb

3: Column Store Tables: Pool/ColumnStoreTables/Main/Dictionary/RoDict 12.03gb

By checking the statistics for this allocator in _SYS_STATISTICS.HOST_HEAP_ALLOCATOR, you see that the allocator
seems to be increasing consistently over time without ever decreasing.

3. You observe that after executing a certain query, the memory consumption on the system increases by a certain amount,
and it does not decrease after a certain amount of time.
You suspect a leakage for a certain allocator by monitoring the allocator sizes in M_HEAP_MEMORY before/after
executing the query, e.g. using query:

SELECT TOP 10 HOST, PORT, CATEGORY, EXCLUSIVE_SIZE_IN_USE FROM M_HEAP_MEMORY ORDER BY


EXCLUSIVE_SIZE_IN_USE DESC;

Reason and Prerequisites


Reason:
If a programming error in SAP HANA affects memory deallocation, it is possible that it causes allocations booked for a certain
allocator not to be deallocated properly.
If you suspected a leakage and further traces need to be collected for verification and root cause analysis, follow the solution
part.
Prerequisites:
Initial analysis indicates that the system is affected by a memory leakage issue
You have identified a certain suspicous allocator
Solution
Please follow the steps below in order to collect the allocator trace.
The hdbcons commands need to be executed as <sid>adm user from OS level.
The commands below are given for Single Database Container system.
If you are running MDC, please use hdbcons -p <tenant indexserver process id> to execute sub commands for tenant DB.
In case you are running MDC in high isolation mode, please refer to SAP Note 2410143.
1. If possible, clear the SQL Plan Cache or restart the system:

ALTER SYSTEM CLEAR SQL PLAN CACHE;

2. Reset possible existing trace entries. Please replace the allocator with the complete name including the hierarchy, e.g.
the full string returned as CATEGORY from M_HEAP_MEMORY

hdbcons "mm resetusage -r <COMPLETE_ALLOCATOR_NAME>"

3. Enable astrace for the specified allocator

hdbcons "mm flag <COMPLETE_ALLOCATOR_NAME> -sr astrace,dstrace"

4. Create initial Allocator Trace report, and write down the current size of the allocator. In case the SQL does not return
any record, it means that there is currently no allocation on this allocator.

hdbcons "mm top -l 20 <COMPLETE_ALLOCATOR_NAME>" > report_0_$(date +%y%m%d%H%M%S).txt

SELECT NOW(), HOST, PORT, CATEGORY, ROUND(EXCLUSIVE_SIZE_IN_USE/1024/1024) AS "SIZE(MB)" FROM


M_HEAP_MEMORY WHERE PORT LIKE '%03' AND CATEGORY = '<COMPLETE_ALLOCATOR_NAME>';

5. In case there is a suspected query/application transaction, execute it to reproduce. Otherwise, wait and monitor the size
of the allocation, until you see a noticeable growth compared to the starting point.
Save the result of the current allocation size, create a report of Allocator Trace and generate a callgraph.

SELECT NOW(), HOST, PORT, CATEGORY, ROUND(EXCLUSIVE_SIZE_IN_USE/1024/1024) AS "SIZE(MB)" FROM


M_HEAP_MEMORY WHERE PORT LIKE '%03' AND CATEGORY = '<COMPLETE_ALLOCATOR_NAME>';

hdbcons "mm top -l 20 <COMPLETE_ALLOCATOR_NAME>" > report_$(date +%y%m%d%H%M%S).txt

hdbcons "mm cg -r <COMPLETE_ALLOCATOR_NAME>" > mmcallgraph_$(date +%y%m%d%H%M%S).dot

Note, it is the best to check the allocator size and collect the report when there is no load (or relatively
low) on the system if possible. Only in that case, we can make sure that the stacks recorded in allocator trace are
pointing to the leakage, instead of meaningful allocations for the active queries.

6. Repeat step 5. and create 3-5 reports & callgraphs with the allocation size returned from the SQL.

7. Disable astrace

hdbcons "mm flag <COMPLETE_ALLOCATOR_NAME> -dr astrace,dstrace"

8. Cleanup

hdbcons "mm resetusage <COMPLETE_ALLOCATOR_NAME>"

9. Zip the allocator size returned by SQL, Allocator Trace reports and callgraphs generated in step 4 and 5 and contact SAP
Support for root cause analysis.

Software Components
Software Component From To

HDB 1.00 1.00+

HDB 2.00 2.00+


This document refers to
SAP Component Title
Note/KBA

2637828 HAN-DB Memory Leak on Pool/malloc/libhdbbasement.so When Collecting Performance Trace/Planviz/Plan


Trace with Function Profiler

2601475 HAN-DB- Memory Leak in Pool/malloc/libhdbcsapi.so When Running Enterprise Search Queries
ENG

2597818 HAN-DB Memory Leak in Pool/ESX When Using PlanViz Execution

2580435 HAN-DB Memory Leak in Pool/RowEngine/Session

2535110 HAN-DB Memory Leak on Pool/parallel/compactcol and Pool/parallel/aggregates or Pool/itab

2533352 HAN-DB- Memory Leak on "Pool/JoinEvaluator/JERequestedAttributes/Results"


ENG

2527251 HAN-DB Memory Leak in Pool/RowEngine/QueryCompilation

2410143 HAN-DB Using hdbcons Commands in MDC Environment With High Isolation Level

2405237

2402318 HAN-DB Memory Leak in Pool/RowEngine/RSTempPage

2398507 HAN-DB Memory in Pool/ICT is Constantly Increasing and Does not get Released

2374935 HAN-DB- Memory leak in Pool/malloc/libhdbcalcengine.so and Pool/malloc/libhdbcalcengineapi.so with


ENG hierarchy view or MDX query

2373932 HAN-DB Memory leak in Pool/malloc/libhdbolap.so

2368929 HAN-DB Memory Leak in Pool/RowEngine/RSTempPage when Row Store Temp Tables without Variable
Length Columns are Used

2362759 HAN-DB- Row table memory leak on SPS11 (Rev111~112.05) and SPS12 (Rev120~122.01)
ENG

2343177 HAN-DB- Memory leak in Pool/RowEngine/MonitorView in scale-out system


ENG

2340582 HAN-DB- Memory leak in Pool/RowEngine/MonitorView


ENG

2313013 HAN-DB Memory leak in Pool/Statistics when querying via XS engine

2312994 HAN-DB Memory leak in Pool/TransientMetadataAlloc when calling a procedure

2312983 HAN-DB Memory leak in Pool/parallel/aggregates when querying on distributed environment

2312976 HAN-DB Memory Leak in Pool/JoinEvaluator/JERequestedAttributes/Results In SAP HANA 1 SPS10 and


SPS11 When Executing DML Statements

2312948 HAN-DB Memory leak in Pool/SQLScript/Execution when a procedures is called via XS engine

2275252 HAN-DB Memory Leak in "Pool/malloc/libhdbrskernel.so" When SqlScript Procedures or UDFs Get Compiled

2271235 HAN-DB Memory Leak in Pool/RowEngine/QueryExecution when Using Batch Inserts on Row Store tables

2253121 HAN-DB Memory Leak in Pool Pool/malloc/libhdbrskernel.so when Using Batch Inserts Into Rowstore Tables

2088349 HAN-DB memory leak when querying calculation views which are executed in

2083100 HAN-DB Memory Leak in Combination With Hybrid LOBs

2000792 HAN-DB SAP HANA: High Memory Consumption and constantly growing on Pool/RowEngine/QueryExecution

This document is referenced by


SAP Component Title
Note/KBA

3114746 Unexpected Memory Growth in Pool/JoinEvaluator/JERequestedAttributes/Results When Using


Temporary Table

2637828 HAN-DB Memory Leak on Pool/malloc/libhdbbasement.so When Collecting Performance Trace/Planviz/Plan


Trace with Function Profiler

You might also like