![Business Impact of Application Quality
• Poor software quality puts your investments at risk
• Cost to fix an error found after product release was up to 100 times more
than one identified in the design phase [Source: National Institute of Standards & Technology,
2002]
• About 80 percent of development costs are consumed by software
developers identifying and correcting defects [Source: IBM: The business value of software
quality.]
• Testing typically accounts for 30 to 50% of a software project budget &
non-reproducible or ad-hoc testing is of little or no use [Source: Pullam & Doyle, 1998]
[Source: W.R. Blischke, D.N.P. Murthy. "Reliability", Section 9.5.1, pp. 311.]](https://image.slidesharecdn.com/applicationtestingsuitevirtualclassroom-100312112440-phpapp02/85/Application-Testing-Suite-5-320.jpg)
Ad
More Related Content
What's hot (20)
Viewers also liked (13)
Ad
Similar to Application Testing Suite (20)
Ad
More from oracleonthebrain (7)
Recently uploaded (20)
Application Testing Suite
- 1. Application Performance Management – Virtual Classroom Mayur Palta Senior Technical Consultant
- 2. The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. © 2009 Oracle Corporation – Proprietary and Confidential
- 3. Agenda • Application Quality Management • Application Testing • Oracle’s Approach to Application Quality • Test Data Management • Application User Experience Management • Managing Real User Experience • Application Operations Management • Oracle Enterprise Manager • Next Steps
- 4. What is Application Testing? • A set of activities conducted through scripts with the intent of finding errors in software. • An activity being part of the software development process aimed at evaluating a software item (system, subsystem, unit etc.) features (functionality, performance etc.) against the given set of system requirements. • Process of verifying that a program functions properly.
- 5. Business Impact of Application Quality • Poor software quality puts your investments at risk • Cost to fix an error found after product release was up to 100 times more than one identified in the design phase [Source: National Institute of Standards & Technology, 2002] • About 80 percent of development costs are consumed by software developers identifying and correcting defects [Source: IBM: The business value of software quality.] • Testing typically accounts for 30 to 50% of a software project budget & non-reproducible or ad-hoc testing is of little or no use [Source: Pullam & Doyle, 1998] [Source: W.R. Blischke, D.N.P. Murthy. "Reliability", Section 9.5.1, pp. 311.]
- 6. Common Testing Cycles Traditional Ways 15% Tested Application Groups Own Testing Used Application Testing Tools Simulation Queries Very Scripting & Labor Intensive 85% Untested Problems Limits the Testing Not a Production Test Application Performance Problems Deploying an Untested System predominately link back to the SQL Queries in the Database Vulnerable to Poor Performance
- 7. A Lifecycle Approach to Quality 1. Design Test Plans 2. Develop Manual Test Based on Application Cases and Automated Requirements Test Scripts 4. Execute Load Tests 3. Execute Functional and Tune Application Tests to Validate Performance Application Requirements
- 8. Application Performance Testing & Management Real User Experience Insight Application Testing Suite Enterprise Manager Application Lifecycle
- 9. Application Performance Testing & Management Application Testing Suite Application Lifecycle
- 10. Oracle Testing History Acquired e-Test Suite from Empirix in June 2008 • 10+ years of automated testing experience • Currently Working with Internal Application Developers • Specialist in testing web-based application • .NET • JAVA • Coldfusion
- 11. Oracle Application Testing Suite A powerful, integrated testing solution Functional Testing for Web Apps Test Manager for Web Applications Load Testing for Web Apps
- 12. Oracle Application Testing Suite A Lifecycle Approach to Quality Oracle Test Manager for Web Applications 1. Design Test Plans 2. Develop Manual Test Based on Application Cases and Automated Requirements Test Scripts 4. Execute Load Tests 3. Execute Functional and Tune Application Tests to Validate Performance Application Requirements
- 13. Oracle Test Manager for Web Applications: Test Process Management • Manage test process from centralized Web-based console • Define test requirements • Develop manual & automated test cases • Document and track defects • Create reports
- 14. Oracle Application Testing Suite A Lifecycle Approach to Quality Oracle Test Manager for Web Applications 1. Design Test Plans 2. Develop Manual Test Based on Application Cases and Automated Requirements Test Scripts Oracle Functional Testing for Web Applications 4. Execute Load Tests 3. Execute Functional and Tune Application Tests to Validate Performance Application Requirements
- 15. Oracle Functional Testing for Web Applications: Automated Functional & Regression Testing • Automate transactions for Web applications & Web Services • Execute rigorous functional test cases • Create automated regression test suites • Identify and report on functional application failures
- 16. Oracle Application Testing Suite A Lifecycle Approach to Quality Oracle Test Manager for Web Applications 1. Design Test Plans 2. Develop Manual Test Based on Application Cases and Automated Requirements Test Scripts Oracle Load Oracle Functional Testing for Web Testing for Web Applications Applications 4. Execute Load Tests 3. Execute Functional and Tune Application Tests to Validate Performance Application Requirements
- 17. Oracle Load Testing for Web Applications: Load and Performance Testing and Tuning • Create realistic load test scenarios that simulate end-user behavior • Scale to thousands of concurrent users • Perform functional content validation under load • Monitor server-side performance & correlate to end-user response times • Isolate and resolve performance bottlenecks
- 18. Need for Data Masking Test Data Management • Key Drivers • Privacy and compliance LAST_NAME SSN SALARY • HIPAA, Breach Notification Laws AGUILAR 203-33-3234 40,000 • EU Data Privacy Directive BENSON 323-22-2943 60,000 • Application testing D’SOUZA 989-22-2403 80,000 • Offshore application development FIORANO 093-44-3823 45,000 • Offshore / In-house software QA • Key Requirements LAST_NAME SSN SALARY • Support database and application ANSKEKSL 111—23-1111 40,000 referential integrity BKJHHEIEDK 111-34-1345 60,000 • Minimal performance impact KDDEHLHESA 111-97-2749 80,000 • Protect against reverse FPENZXIEK 111-49-3849 45,000 transformation
- 19. Data Masking Pack Oracle Enterprise Manager Production • Automates production data masking • Mask data from a production database • Define rules once • Data relationship discovery • Automates data relationship enforcement using existing foreign keys Clone/ • Enter custom data relationships known Mask Staging to the application • Rules repository • Format library, masking definitions • Testing • View sample data before masking Test Test
- 20. Application Performance Testing & Management Real User Experience Insight Application Testing Suite Enterprise Manager Application Lifecycle
- 21. Challenges Business Challenge IT Challenge Line of Business Apps Manager / IT Operations (LOB) Manager Apps Support • Low transaction • Isolating problems • Reactive: Issues effectiveness – lost to applications and identified when user calls revenue, high modules • No visibility into support costs • No insight into each application & operational • Poor customer user’s experience errors satisfaction • Providing reports • Time consuming diag. • No visibility into and visibility to • Inability to anticipate customer trends stakeholders impact of change
- 22. Analysts Agree: User Experience is Critical Consistent end-to-end application and service 87% performance guarantees Unplanned infrastructure changes resulting in 73% incidents and downtime Unanticipated infrastructure effects from 49% consolidation & new application projects Mis-configuration of network objects 28% Source: Forrester Research, Top Five Challenges For Enterprise IT Infrastructure Managers — And How To Resolve Them, Thomas Mendel, March 2005 “Real life experience shows: 74% of problem alerts come from end users complaining to the service desk about performance problems, not from monitoring infrastructure components!” Jean-Pierre Garbani Forrester Research 2004
- 23. TomTom Delights customers and recovers revenue lost because of IT issues “There was a point in time when we were more or less driving in the dark…Now I’m convinced that for us, this product will pay back within the first six months, easily.” -Oscar Diele, Global vice president of e-commerce • Marketing removed user manual from the box, not telling IT! Challenge • Product was a big hit last Christmas, many people went on the website to download user manuals & performance went down, including the online shop • 27% of customer about to check out canceled their transaction • Oracle Enterprise Manager Solution – Real User Experience Insight • IT gets alerted, understands and reacts very quickly, cutting down the 27% loss. • IT captures all the 27% customers emails , Marketing immediately sends an Benefits email with an apology and a coupon to attract those customers to shop again. • TomTom recovered most of the 27% loss http://www.oracle.com/profit/features/091908_tomtom_.html
- 24. Solution Overview Optimizing end-user experience through real end-user monitoring Business Reporting KPI Monitoring Data Export Real User Experience Insight key highlights: - Replay of poor performance Transaction Performance - Executive dashboards Executive dashboard - Extensive KPI and SLA monitoring - Full alerting capabilities - Quick bottle-neck analyses - Transaction performance analyses - Customized reporting - Trend analyses - Full data integration via XML
- 25. Real User Experience Insight For: Line of Business executives Satisfaction Reports Business Transaction Funnel Usage Reports: • Data Browser • Sales by Country/State
- 26. Real User Experience Insight For: Application Owners and Application Support Dashboards & SLM Business Service Monitoring User Session Tracking, (e.g. Order Entry, Buy Stock) Page Replay
- 27. Real User Experience Insight For: IT Operations KPI configuration, monitoring, & alerting Server, network, & object performance diagnostics Data Export Error reports & analysis
- 28. Application Performance Testing & Management Real User Application Experience Insight Testing Suite Enterprise Manager Application Lifecycle
- 29. Application Operations Management Top Challenges & Risks • How to identify and resolve business issues? • How to manage entire application lifecycle and reduce compliance risks? • How to reduce operational costs? • How to prioritize issues that impact business user experience?
- 30. Solution: Integrated, Top-Down Application Management Application Users Application Siebel, PeopleSoft, EBS… Java, .NET, Composite Exchange, Legacy Software Oracle Applications Custom Applications Non-Oracle Applications Application Web Server, Portal Database AS, ESB, SOA Components Presentation Layer Management Systems Middleware Infrastructure Components Virtual Environments Servers Databases Storage Oracle Enterprise Manager
- 31. Comprehensive Applications Management Oracle Enterprise Manager • Manage Oracle applications and technology from a single console • Application Management Packs for E-Business Suite, Siebel, PeopleSoft, JDE • Optimize User Experience for Oracle Applications • E-Business Suite Accelerator • PeopleSoft Accelerator • Siebel Accelerator
- 32. Oracle Application Management Packs Complement Application Tools Application Management Pack Application Management Pack Application Management Pack for PeopleSoft Enterprise for Siebel for Oracle E-Business Suite Server Manager PS/Admin Oracle Applications Manager SARManalyzer, SARMquery Performance Monitor iSetup Diagnostic Data Collector PeopleTools Rapid Install Environment Verification Tool Change Impact Manager Rapid Clone Application Deployment Mgr
- 33. Application Operations Management Managing Packaged Applications Oracle E-Business Suite – PeopleSoft – Siebel E- Common Capabilities • 1-step Discovery • Out-of-box transactions • Reports • Topologies • System monitoring • Configuration Management • Service Level Management • Application monitoring • Transaction Tracing • Root-cause/Impact Analysis • Dashboards • Out-of-box service models Embedded Application-Specific Capabilities E-Business Suite PeopleSoft Siebel • Topological root-cause views • 500+ config metrics & • 50+ business metrics • 100+ configuration metrics PSAdmin admin tasks • Out-of-box processes • Clone automation for test • 25+ “tribal” knowledge- • HI Component support production environments based policies • SARM integration – real • Cross-instance comparisons • Centralized Log Mgt end user metrics, • Key bus flow monitoring transaction diagnostics
- 34. <Insert Picture Here> Application Management Pack for Oracle E-Business Suite
- 35. Application Management Pack for EBS E-Business Suite-Specific Management • Full Visibility into E-Business Suite Components • E-Business Suite System Parameter Change Tracking • Cloning Automation • Seamless Integration with vs Oracle Applications Manager • Support R12 and R11i ATG RUP4 or above
- 36. <Insert Picture Here> Benjamin Cabanas Program Manager GE Infrastructure “Implementing Application Management Pack for Oracle E-Business Suite has allowed GE Infrastructure to realize $200K annual savings, 84% reduction in clone cycle time, and 75% reduction in resources”
- 37. <Insert Picture Here> Application Change Management Pack for Oracle E-Business Suite (ACP)
- 38. Application Change Management Pack for Oracle E-Business Suite: Version 3.0 Simplify Change Improve User Reduce Human Orchestration productivity Errors
- 39. Product Overview Summary of Key Features Customization Manager Patch Manager Setup Manager • Create AD compliant • Configurable Patch Process: • Central console for all activity customization packages Step-by-step interview process • Migrate functional • Central console for all custom • Deploys custom and standard configuration/setup data across patches Oracle patches from a central different instances of EBS • Custom patches can be applied console using Patch Manager • Group extracts, loads, and • Simultaneous, multi-patch, reports into projects for easier • Over 200 file types supported multi-target deployment management and reuse. • Creating build lists with • Integration with Customization • Recognize and enforce the manifests Manager dependencies between different • Build or compile Java or Form • Multiple Patch Options: Support selection sets and provide libraries for wide variety of patching sequencing recommendations. • Validate the custom files based options such as Hot Patching, • Automated, simultaneous loads on a set of standards disable archive mode….etc across multiple instances • Interface with Version Control • Centralized Patch Deployment • Perform loads across different systems (CVS, Subversion) Tracking: In Progress, RUP levels • Capture additional Metadata Completed, and Scheduled • Share the extracts, loads, and around customization • Integrates with MetaLink to reports content with other • Standard and Comparison download patches users. reports
- 40. <Insert Picture Here> Application Management Pack for Siebel
- 41. Application Management Pack for Siebel Siebel-Specific Management • Full Visibility into Siebel Server Components • SARM Integration • Enterprise / Server / Server Component Parameters • Siebel Business Metrics
- 42. <Insert Picture Here> Application Management Pack for PeopleSoft
- 43. Application Management Pack for Peoplesoft Peoplesoft-Specific Management • Full Visibility into PeopleSoft Components • PeopleSoft System Parameter Change Tracking • Out-of-box Configuration Policies • Centralized Domain Administration vs • Support PeopleTools 8.48 & above
- 44. Projected Cost Savings • Reduce User Reported Incidents to Application Administrators by Over 50%* Costs • Reduce Problem Resolution Times by Over 50%* • Increase Administrator Productivity by over 25%* • 10 – 18 months payback* * actual savings, productivity improvements, and payback period may vary
- 45. Summary
- 46. Application Quality Management Pre-Deployment Post-Deployment Performance Testing Performance Monitoring Real User Monitoring Real Application Testing • Capture user performance • Database Testing • Identify key spot in applications • Database Capture/Replay • Alerting/Reporting • SQL Queries Capture/Replay Service Level Management Load & Stress Testing • Synthetic transactions • Any web application • Active Monitoring • Server Performance metrics • Collaborative testing Application Management Packs Functional/Regression Testing SOA Application Management • Onescript creation • Easy record/playback Diagnostic and Tuning Test Management Provisioning, Configuration Management Data Masking • Testing prod data securely Change Tracking and Synchronization END-TO-END PERFORMANCE MANAGEMENT
- 47. Application Performance Testing & Management Real User Experience Insight Application Testing Suite Enterprise Manager Application Lifecycle
- 48. Oracle Helps You Maximize Customer Value Deploys SOA Saves 80% time and Avoids online revenue infrastructure 92% effort for managing losses up to 25% faster Databases Cuts configuration Improves IT Drives asset utilization management effort by productivity by 25% up by 70% 90% Saves $1.9 million Saves $170,000 per Replaces manual with Oracle Enterprise year with Oracle tools with automation; Manager Enterprise Manager saves time by 50% Saves weeks on Reduces Database Reduces provisioning application testing testing time by 90% effort by 75% time Cuts application Delivers 24/7 uptime Reduces critical testing from weeks to with Oracle Enterprise patching time by 80% hours Manager
- 49. Next Steps
- 51. Architecture: Network Protocol Analysis Collected data: Page request •Client-ip, Server-ip, Gateway-ip Network Response Time •Requested URL (Page,Object) •Cookie/Session-id Internet •Referrer •GET & POST requests • User agents (phone type) Firewall Sniffing Switch Server Response Time •Response? •Server error, web site Web Web Web error, Hit OK server server server •Content of page (i.e order details, functional errors) •Delivered? •Aborted? DB DB •Network timeout?
- 52. Key Customers 250+ … and growing