APM PLATFORM

Date: 25 Oct 2023

Department: Engineering
Division: Technology
AGENDA

1. Monitoring with APM

2. Evaluation Approach
3. Comparison
4. Key Terms
5. Risk & Mitigation
6. Recommendation / Conclusion

ACS Digital Berhad 2

 Monitoring with APM
● APM monitors ADBs infrastructure and web/backend application services in near real-time on
performance metrics such as uptime, latency, error rate
● The selected APM platform will be integrated with Zenduty, an incident on-call management
platform, to notify the relevant team members of potential incident according to the on-call duty
roster

Metrics
Application 1

APM Zenduty Identificatio

Infrastructure n Incident
2
Continuo Automate Mgmt
us d Classificatio
Assessme Notificati n
nt on
APM Alert Escalation
Condition(s) Policies 3

Alerts Trigger

ACS Digital Berhad 3

 *June 2023

Evaluation Approach
1 ADB issued RFQ with Year 1 and Year 2 Usage Estimate to the shortlisted bidders who are considered APM solution leaders

2 Bidders Profile Selected Clients Gartner Magic

Quadrant*

Datadog Datadog is an observability service for cloud-scale applications, providing Plaid, Sofi, Zendesk, Leader
monitoring of servers, databases, tools, and services, through a SaaS-based FICO, MOX, GXS, GX
data analytics platform. It is listed in Nasdaq and headquartered in NYC, USA Bank

Dynatrace Dynatrace is technology company listed on the NYSE that provides a software Bank of New Zealand, Leader
observability platform based on artificial intelligence (AI) and automation. KBC, KeyBank, Virgin
Dynatrace technologies are used to monitor, analyze, and optimize application Money
performance, software development and security practices, IT infrastructure,
and user experience for businesses and government agencies throughout the
world. It is headquartered in Waltham, Massachusetts, USA

NewRelic New Relic is a US-based web tracking and analytics company. The company's Bangkok Bank Leader
cloud-based software allows websites and mobile apps to track user
interactions and service operators' software and hardware performance. It is
headquartered in San Francisco, USA

3 Bidders came back with:

● Estimated usage fee
● Optional implementation fee (not applicable as ADB engineering will implement in-house)

4 Evaluation of RFQs carried out by Head of Engineering (Xing Yao, Benny, Chris), Head of TechOps (Leonard) and CTO

ACS Digital Berhad 4

 Usage Estimations (1/2)
Year 1 Year 2

APM hosts for API/Services 80 120

Infra hosts for Infra monitoring 60 90

288,000 hours per month 432,000 hours per month

- 80 API/services x 5 instances x 24 hours x 30 - 120 API/services x 5 instances x 24 hours x 30
Container hours
days days

Ingested logs: 600GB per month Ingested logs: 1200GB per month
- 20GB per day x 30 days - 40GB per day x 30 days
Logs Indexed logs (30 days retention): 200GB per Indexed logs (30 days retention): 400GB per
month month
- 20GB per day x 30 days x 1/3 - 40GB per day x 30 days x 1/3

Synthetics API test runs: 691,200 per month Synthetics API test runs: 1,036,800 per month
- 80 services x (12 x 24 x 30) 5mins interval for - 120 services x (12 x 24 x 30) 5mins interval for
1 month 1 month
Uptime Synthetics browser test runs: 432,000 per Synthetics browser test runs: 432,000 per
month month
- 10 websites x (12 x 24 x 30) 5mins x 5 - 10 websites x (12 x 24 x 30) 5mins x 5
locations interval for 1 month locations interval for 1 month

ACS Digital Berhad 5

 Usage Estimations (2/2)
Year 1 Year 2

Real User Monitoring (RUM) RUM sessions: 550,000 per month RUM sessions: 1,600,000 per month

Network Devices Monitoring 25 Wireless Point, 2 x L2 Network Switch, 2 x Network Firewall

Databases 5 AWS RDS Databases 10 AWS RDS databases

Users with Access to APM 60 97

ACS Digital Berhad 6

 Features Comparison (1/2)
Recommended
DataDog New Relic DynaTrace
Covered as part of the following solutions: Covered as part of the following solutions: Covered as part of the following solutions:
Performance 1. APM (Java applications) 1. Application Monitoring (Java) 1. Microservices monitoring
Monitoring - 2. Container monitoring
Backend Services
Module details (overall)
Covered as part of following solutions: Covered as part of the following solutions: Covered as part of the following solution:
Performance 1. RUM (ReactNative, min 0.63.4) 1. Mobile Monitoring (React Native) 1. Digital experience
Monitoring - Mobile
Application Module details (overall, React Native, min
0.59)
Covered as part of following solutions: Covered as part of the following solutions: Covered as part of the following solution:
Performance
1. RUM (Browser Monitoring) 1. Browser Monitoring (Nextjs) 1. Digital experience
Monitoring - Web
Application Module details (overall)
Application Covered as part of the following solutions: Covered as part of the following solutions: Covered under the following solution:
Dependency 1. APM (Service Map) 1. Service Maps 1. Dependency Mapping
Mapping 2. Dependency Visualization - Smartscape

Covered as part of the following solutions: Covered as part of the following solutions: Covered under the following solution:
1. Infra Monitoring 1. Infra Monitoring 1. Cloud monitoring
Infrastructure
2. AWS monitoring
Monitoring
Module details (overall, AWS)
Covered as part of the following solutions: Covered as part of the following solutions: Covered under the following solution:
1. APM (Tracing) 1. Application Monitoring (Tracing) 1. Distributed tracing, OpenTelemetry
Distributed Tracing
Module details (overall)

ACS Digital Berhad 7

 Features Comparison (2/2)
Recommended
DataDog New Relic DynaTrace
Metrics and Monitors Alerts Alerting

Notification Notification Notification

1. JIRA 1. Email 1. Incident Management (JIRA, PagerDuty)
Alert & Notification
2. PagerDuty 2. OpsGenie 2. ChatOps (Teams, Slack)
3. Slack 3. PagerDuty 3. Service Management (ServiceNow)
4. Webhook 4. Slack 4. Custom (Email, Webhook)
6. Webhook
Further clarification may be needed on the Further clarification may be needed on the Provide both SaaS and self-hosted solutions.
scalability of the solution scalability of the solution For SaaS, claims to be highly scalable in
Scalability supporting growing number of components to
be monitored. Further clarification may be
needed on the scalability of the SaaS solution
1. APM (Java applications) 1. APM (Java applications) 1. Java, Spring Boot
2. RUM (ReactNative, min 0.63.4) 2. Mobile (Android API 24+, iOS 10) 2. React Apps
Compatibility
3. RUM (Browser monitoring) 3. Web (NextJs Agent) 3. React Native
4. Python
Supported: Supported: Supported:
Dashboarding and 1. Dashboard 1. Dashboard 1. Dashboard
Reporting 2. Reporting based on dashboard 2. Notebooks
3. Reporting based on dashboard
Supported AIOps Supported:
Root Cause 1. AI-based root cause analysis 1. AI-based root cause analysis
Analysis
Module details (overall)
Available hosting sites: Available hosting sites: Available hosting sites: hosting zones
Available Hosting 1. US 1. US
sites 2. EU 2. EU
3. AP (Japan)

ACS Digital Berhad 8

 Cost Comparison
Recommended
DataDog New Relic DynaTrace

Pricing Granular, product-based pricing User + Data based pricing Broad category-based pricing

Estimated Usage Year 1 and Year 2 Estimated Usage Year 1 and Year 2 + Estimated Usage Year 1 and Year 2
(Pro Plan) Full Platform Licence for 60+97 RM 1,268,011 + RM 1,726,111 = RM
RM 365,676.48 + RM 598,865.28 users 2,994,121
= RM 964,541.77 USD 544,429.00
Total Fee

Total Total Total

USD 544,429.00
RM 964,541.77* RM 2,722,145.00* RM 2,994,121.00*

Setup Cost Not Applicable Not Applicable Not Applicable

RM 96,454.18 (10% Withholding RM 272,214.50 (10% Withholding

Tax) Tax) RM 239,529.68 (8% SST)
Tax
RM 77,163.34 (8% SST) RM 217,771.60 (8% SST) RM 29,941.21 (1% Stamp duty)
RM 9,645.42 (1% Stamp duty) RM 27,221.45 (1% Stamp duty)

Grand Total RM 1,147,804.70 RM 3,239,352.55 RM 3,263,591.95

Currency / Rate* Malaysian Ringgit(RM) USD1=RM5

ACS Digital Berhad 9

 Key Terms(1/2)
Clause Content

● Between Datadog and ACS Digital Berhad (Customer) from Nov 2023 to Nov 2024 (1
Contract Party and Period
year). Plan to renew for another year after Nov 2024.

● Pricing is in USD
● Contractual period, software license will be billed annually in advance with baseline fee of
Fee and Payment RM182,838.24(50% of the Year 1 estimate)
● Services is billed according to actual usage and offset with baseline fee. Any extra of the
month will be billed separately at the end of the month

● Data is protected in transit across public networks and encrypted at rest, all data
transmitted between Datadog and our users is protected using Transport Layer Security
(TLS) and HTTP Strict Transport Security (HSTS)
● Datadog utilizes encryption at various points to protect customer data and Datadog
secrets, including encryption at rest (e.g., AES-256), asymmetric encryption (e.g., PGP) for
Cybersecurity and Data Protection system backups, KMS-based protections for the protection of secrets
● Datadog utilizes RBAC where user roles can be configured such that only authorized users
with appropriate permissions have access to specific parts of the platform
● Datadog is SOC 2 Type II compliant, HIPAA-compliant, ISO-27001 certified, and GDPR
compliant
● For more info https://www.datadoghq.com/security/

● 99.8% uptime SLA. An uptime status webpage is available for the platform uptime status
update subscription
SLA/ BCP
● Minimal impact if APM is not accessible for a short period. It would not affect the
continuation of business operations, so the impact is expected to be negligible

Private and Confidential

ACS Digital Berhad 10
 Key Terms(2/2)

Clause Content

Governing Law The laws of Singapore

Below is the termination clause in the standard Datadog Master Subscription Agreement.
● Termination for cause
■ Either Party may terminate all Orders and this Agreement, effective on written notice to
the other Party, if the other Party materially breaches this Agreement, and such breach
remains uncured 30 days after the non-breaching Party provides the breaching Party
with written notice of such breach.
■ In addition, either Party may terminate all Orders and this Agreement, effective on written
notice to the other Party, in the event that the other Party (i) becomes insolvent, enters
Termination into bankruptcy or any similar financial reorganization or legally binds itself to any such
Clause reorganization or (ii) makes any assignment of its rights or assets for the benefit of its
creditors.
● If termination due to cause,
■ ADB will be refunded via pro-rate for services not yet rendered by Datadog
● For other terminations, there is no refund
● For more info https://www.datadoghq.com/legal/msa/#term-and-termination

Private and Confidential

ACS Digital Berhad 11
Risk & Mitigation
Risk Severity Mitigation

Data Privacy and Security: ● Datadog utilizes RBAC where user roles can be configured such that only authorized
Unauthorized Access and users with appropriate permissions have access to specific parts of the platform
Low
Leakage of Information on ● Datadog is SOC 2 Type II compliant, HIPAA-compliant, ISO-27001 certified, and GDPR
ADB's Data compliant https://www.datadoghq.com/security/?tab=compliance

● Datadog uses Agent based integration where software agents are deployed within an
Vendor Lock-in: Tight
application or on the host system to collect performance and monitoring data, which is a
coupling with Datadog
Low common approach for APM integration that is also adopted by New Relic and DynaTrace
preventing ADB from
● Datadog supports OpenTelemetry, which is a vendor-neutral open-source observability
changing APM platform.
framework, allowing easy migration of observability data between monitoring backends

● 99.8% uptime SLA. An uptime status webpage is available for the platform uptime status
Uptime: Service unavailable update subscription
Low
due to server down. ● Minimal impact if APM is not accessible for a short period. It would not affect the
continuation of business operations, so the impact is expected to be negligible

ACS Digital Berhad 12

 Thank You

ACS Digital Berhad 13

 Appendix

ACS Digital Berhad 14

Appendix 1: Incremental Usage

Incremental usage based on releases

Usage F&F Launch Year 1 Year 2

APM hosts 70 80 120

Infra hosts 50 60 90

Real User
1,500 550,000 1,600,000
Monitoring

ACS Digital Berhad 15

 Appendix 2: Datadog licenses
Feature Free Pro Enterprise

Basic Monitoring Limited Comprehensive Comprehensive

Real User Monitoring (RUM) Not Included Included Included

Synthetic Monitoring Not Included Included Included

Log Management Not Included Included Included

Anomaly Detection Not Included Included Included

Network Performance Monitoring Not Included Included Included

API and Application Scanning Not Included Not Included Included

APM (Application Performance Not Included Limited Comprehensive

Monitoring)

Custom Dashboards and Alerts Limited Comprehensive Comprehensive

Custom Integrations and Plugins Limited Comprehensive Comprehensive

SLA and SLO Monitoring Not Included Not Included Included

Full Detail : https://www.datadoghq.com/pricing/

ACS Digital Berhad 16

 Appendix 3: New Relic licenses
New Relic
Feature New Relic Free New Relic Pro
Enterprise

Application Performance Monitoring Limited to 24-hour data Comprehensive APM Comprehensive APM

Server Monitoring Limited to 4 hosts Up to 200 hosts Scalable Server Monitoring

Real User Monitoring (RUM) Not included Included Included

Synthetic Monitoring Not included Included Included

Infrastructure Monitoring Limited to basic features Advanced server and cloud Advanced server and cloud

Custom Dashboards Not included Included Included

Alerting and Notifications Limited capabilities Comprehensive alerting Comprehensive alerting

Transaction Tracing Not included Included Included

Full Detail : https://newrelic.com/pricing

ACS Digital Berhad 17

 Appendix 4: DynaTrace licenses
Support and
Licensing Type Features Pricing Model
Maintenance

Based on monitored
Dynatrace SaaS Cloud-hosted monitoring and analytics
entities
Included

Based on nodes or
Dynatrace Managed On-premises monitoring and analytics
clusters
Included

APM License Application Performance Monitoring Pricing based on usage Included

Infrastructure
Server and infrastructure monitoring Pricing based on usage Included
Monitoring

Real User Monitoring

End-user experience monitoring Pricing based on usage Included
(RUM)

Synthetic Monitoring Scripted tests for user interaction Pricing based on usage Included

Logs and Tracing Log management and distributed tracing Pricing based on usage Included

Cloud-Native Pricing based on

Specialized for container monitoring Included
Environment metrics

Full Detail : https://www.dynatrace.com/pricing/

ACS Digital Berhad 18