STRESS TEST OBSERVATIONS

CPU Utilization: CPU usage had reached to a maximum of 36% and rest of the time the usage looks
normal.

Memory Utilization: Memory usage was normal during stress test, and it never crossed 67% of total
memory.
No of FLEXCUBE Sessions

No of FC sessions for every 15mins interval.

No of FC
Time Sessions
6.00PM 735
6:15PM 820
6:30PM 820
6:45PM 840
7:00PM 822
7:15PM 696
7:30PM 668

No of ACTIVE Sessions

No of Active sessions for every 15mins interval.

No of
Time ACTIVE Time Node1 Node2
Sessions
6.00PM 71 6.00PM 34 37
6:15PM 73 6:15PM 39 34
6:30PM 114 6:30PM 60 54
6:45PM 104 6:45PM 58 46
7:00PM 112 7:00PM 61 51
7:15PM 88 7:15PM 51 37
7:30PM 82 7:30PM 38 44

Blocking sessions: The below blocking session has been observed during the stress test and it got
released in few seconds.

Standby sync status

FCXHO: This standby was in sync with Exadata FCXPPT.

FCPPTOFF: This standby was in sync with Exadata FCXPPT.

Instance Availability: Both the instances were available during stress test without any issues.
Redo Generation during Stress Test:

Redo generation while stress test is 184 GB.

Alert log: Alert logs were monitored for both the instances and didn’t see any issues.

No Timeout error occurred in database alert log file.

FRA USAGE: It is being monitored continuously and it was under threshold values.

Tablespaces Usage: Tablespace usage has been monitored and added datafiles to the required ones.

Stress Test Delay –

Stress test was started late as app servers were not able to connect to database services via fcchost
user. This was later identified that ABA DBA created some services with same name as production on
other database.

Action – Services with same name as production were deleted on other database and issue got fixed.

RT screen issue –

Users reported some slowness in RT screen opening, app logs were checked by ATA team and
consulting team and no issue was identified.

To reverify the issue, ATA team worked with bank users however transactions were already
discarded and users were logged out. ATA team will check this separately with bank team.

AWR review of stress test -

Node 1 - Total cpu count on server is 128. Usage of cpu has maximum gone to 39 cpu per second.
Enough cpu is available on server.

Node 2 - Total cpu count on server is 128. Usage of cpu has maximum gone to 41 cpu per second.
Enough cpu is available on server.

1) Foreground Wait Event -

Network Wait class event - TCP Socket (KGAS) is still being observed in few AWR reports:
Solution - From the database point of view, these waits can safely be ignored; the wait event does
not represent a database issue.

From an application or network point of view, delays in establishing a network connection may
produce unwanted delays for users. You should make sure that the application makes network calls
efficiently and that the network is working well such that these delays are minimized.

2) Background Wait Event:

Ges Generic Event wait is observed in all AWR reports and Oracle support has already given patch
for this. This can be ignored and later can be supressed via patch.

3) SQL Ordered by Elapsed Time:

Please find the below sql_id where it is being observed in most of the AWR reports and tuning
this will help in the performance of the DB.
 Bank Team needs to check if this can be further tuned. Query is shown in above screen shot.

4) Killed the below query sessions which are running for last 10 min:

73r99snt9bb9q ---- SELECT * FROM ( SELECT Lovalias.*,rownum Rno FROM (SELECT * FROM ( SELECT
BRANCH_CODE, CUST_AC_NO, AC_DESC, CCY, CUST_NO, NULL PHY_ACC, NULL PHY_NAME FROM
STTM_CUST_ACCOUNT A WHERE RECORD_STAT = :"SYS_B_00" AND AUTH_STAT = :"SYS_B_01" AND
ACCOUNT_TYPE <> :"SYS_B_02" AND ACCOUNT_CLASS NOT IN (SELECT VALUE FROM LMTM_TYPE_VALUES
WHERE TYPE = :"SYS_B_03" AND CODE = :"SYS_B_04") AND NOT EXISTS (SELECT :"SYS_B_05" FROM
STTB_ACCOUNT WHERE AC_GL_NO = A.CUST_AC_NO AND (AC_STAT_NO_CR = :"SYS_B_06" OR
AC_STAT_FROZEN = :"SYS_B_07")) UNION ALL SELECT PHY_BRN BRANCH_CODE, VIR_ACC_NO AC_GL_NO,
VIR_ACC_NAME AC_GL_DESC, CCY AC_GL_CCY, VIR_CUST_ID CUST_NO, PHY_ACC, (SELECT AC_DESC FROM
STTM_CUST_ACCOUNT WHERE CUST_AC_NO = A.PHY_ACC) PHY_NAME FROM STTM_VIRTUAL_ACCOUNTS A
WHERE RECORD_STAT = :"SYS_B_08" AND AUTH_STAT = :"SYS_B_09" AND STATUS = :"SYS_B_10" ) WHERE
(CUST_AC_NO) LIKE :1 ORDER BY :"SYS_B_11" ) Lovalias ) WHERE Rno > :"SYS_B_12" AND Rno <= :"SYS_B_13"

Informed to consulting team and they are checking it.

5) Few SQL_ID profiling done with better plan –

6jwtq2k28mds4 - sql profile (execute dbms_sqltune.accept_sql_profile(task_name

5za8mnx4knxbr - sql profile(execute dbms_sqltune.accept_sql_profile(task_name

6) Sys job blocking issue –

update sys.job$ set this_date=:1 where job=:2

This session created enq: TX - row lock contention and session was killed multiple times.

EOD Details:

Posting 18 M entries took 30mins with 94 sessions on both the nodes.

EODM took 31 minutes with 4 users.

Here is the EOD Timing.