Exposé: Testing Practices in Enterprise IT: Presenting the 2017 State of Functional Testing Survey Report

Editor's Notes

• #4: In June 2017, Panaya commissioned Spiceworks to survey the key testing executives in over 150 leading enterprise organizations throughout the US and the UK. The survey explored current functional testing trends, needs, challenges and future outlook. The survey's objective has been twofold. First, to provide a reliable source of industry benchmarking data. Secondly, to identify and represent - as accurately as possible - any solution gaps that may impact the industry or otherwise hinder the success of functional testing today. Today, we will present key trends that were uncovered in the survey as well as our insights on those trends and recommendations for testers. At the end of this webinar you will be better equipped to select a testing methodology as well as test automation and management solutions that will keep your organization ahead of the curve.
• #6: With change delivery to enterprise applications becoming instrumental in the competitive race to better win, serve and retain customers, our survey results suggest that testing activities are also becoming more pivotal, as business process quality is deemed more important than time-to-market. Beginning with the business-driven change request, through to project scoping and on to realization in development - the infrastructure for technical and functional testing must be present and connected to every phase. The last World Quality Report predicts that Testing will represent 40% of the IT spend in 2018. This is certainly due to the increasing number of organizations becoming agile, releasing change faster without compromising on quality, as well as on the increasing complexity of regulations. ‘The Age of the Customer’ is impacting testing practices as improving customer experience is now the highest priority for testers. Other priorities such as releasing on time - which is also a key tenet of the Quality-at-Speed strategy for continuous delivery of change – and increasing test automation follow respectfully. Test automation is coveted by all but still too costly (even for regression testing) to implement without risk, perhaps due to the lack of dedicated developers. However, automation is a top priority for testing executives - the lack of which identified as the second largest test execution challenge. To meet the demand and the apparent shortage of automation resources, it is expected by 31% of participants that QA and developers’ roles will eventually merge.
• #7: With change delivery to enterprise applications becoming instrumental in the competitive race to better win, serve and retain customers, our survey results suggest that testing activities are also becoming more pivotal, as business process quality is deemed more important than time-to-market. Beginning with the business-driven change request, through to project scoping and on to realization in development - the infrastructure for technical and functional testing must be present and connected to every phase. ‘The Age of the Customer’ is impacting testing practices as improving customer experience is now the highest priority for testers. Other priorities such as releasing on time - which is also a key tenet of the Quality-at-Speed strategy for continuous delivery of change – and increasing test automation follow respectfully. Test automation is coveted by all but still too costly (even for regression testing) to implement without risk, perhaps due to the lack of dedicated developers. However, automation is a top priority for testing executives - the lack of which identified as the second largest test execution challenge. To meet the demand and the apparent shortage of automation resources, it is expected by 31% of participants that QA and developers’ roles will eventually merge.
• #8: Test automation is coveted by all but still too costly (even for regression testing) to implement without risk, perhaps due to the lack of dedicated developers. However, automation is a top priority for testing executives - the lack of which identified as the second largest test execution challenge. To meet the demand and the apparent shortage of automation resources, it is expected by 31% of participants that QA and developers’ roles will eventually merge. This means that a mode practical and affordable way to test automation is required
• #9: We believe there is no need to choose between quality and speed. Testers can have both. But how? Remember that automation is the means, not the end and your goal is not to automate more test cases but to automate smarter. This includes workflow processes such as test steps handover and defect lifecycle management. For IT to stay viable in tackling both the complexity and the competitiveness of the marketplace, the way forward for organizations is to really extract value from test automation, rather than just automate the most simple or static business processes. Automating smarter may sometimes even mean automating less - leveraging machine learning to create and maintain automated test cases that are based on real-life production data means your resgression test suite, for example is always up-to-date and in-sync with the business.
• #11: In accord with the Digital Transformation trend, we see a growing demand for synergies between IT and Business. This includes more collaboration, ongoing business requirements awareness, joint planning and transparency as well as an overall business process-oriented approach to application lifecycle management. For example, the survey shows that testers’ key performance indicators (KPIs) have shifted, with Understanding business requirements seen as most critical to defining test methodology; followed closely by Improving customer experience. This trend is expected to intensify in the future, as over one third of testing leaders believe that business domain expertise will become a requirement for testers with the growing effects of agile and continuous delivery. This trend is expected to intensify in the future, as over one third of testing leaders believe that business domain expertise will become a requirement for testers with the growing effects of agile and continuous delivery. (graph #3 - 35% emphasis)
• #12: However, the new business-driven approach to testing exposes gaps in IT organizations’ readiness for continuous delivery of change, as almost half (47%) of IT leaders surveyed believe their test do not accurately reflect real-life production scenarios. In fact, creating testing scenarios that accurately reflect business processes is currently the top test planning challenge.
• #13: To meet delivery with quality, IT and Business become inherently linked from defining the change through delivery and into production. All the while, the two must work together to enable the right changes without compromising business process quality. This partnership between IT and Business is where Quality Assurance (QA) extends from technical development and testing to business testing and validation. To accommodate this, Your testing tool should support collaboration between various stakeholders. Features like system-wide defect tracking that enable communication with developers, testers, users and even third-party applications partners are a plus. choose a testing solution that mirrors the way business is conducted in the real world, to ensure no glitches occur in production.
• #15: BUT IT IS MORE THAN VISIBILITY INTO RISK. IT IS VISIBILITY INTO QUALITY – WERE ALL REQUIREMENTS DEVELOPED, DO WE HAVE TEST CASES ASSIGNED TO EVERY REQUIREMENT ? WHT S THE STATUS OF TEST EXECUTION ? ARE MY TESTING RESOURCES BALANCED TO ADDRESS ALL TESTS ? ARE TESTS PRIORITIXZED ACCORDING TO THE BUSINESS IMPORTANCE ? The lack of progress visibility and production data available for test case creation leads testing executives to take an extra cautious approach and as a result, test more and delay delivery.
• #16: FOCUS HERE ON RISK MANAGEMENT - Any potential change to the software landscape could cause downtimes and malfunctions that will result in lost revenue, bad customer experience and damage the brand equity. LACK OF VISIBILITY INTO THE IMPACTS OF CHANGE IN A COMPLEX ENTERPRISE IT ENVIRONMENT (CUSTOMIZATIONS, SPAGHETTI CODE) LEADS TO A CAUTIOUS APPROACH In line with the Business-IT convergence trend, we’ve also found that test managers lack visibility into test readiness. In other words, they lack the confidence TO INDICATE THUMBS UP BEFORE GOING TO PRODUCTION. The lack of progress visibility and production data available for test case creation leads testing executives to take an extra cautious approach and as a result, test more and delay delivery. We see that 31% of testers admit to always test more than needed to ensure quality, causing delays and other forms of waste.
• #17: AND ON TOP OF THAT – WHAT IS THE STATUS OF DEFECTS FOUND DURING TEST EXECUTION ? AS YOU CAN SEE HERE Going back to the fact that eliminating defects was found to be the top priority for testing, an inability to recreate or duplicate defects was identified as the most common pain point with regards to defect management. AND THIS IS ALSO RELATED TO COLALBORATION AND COMMUNICATION
• #18: With customer experience in the forefront of all businesses today, testing must support business process validation. Testing is no longer just about verifying that technical aspects of application changes work correctly (e.g., load and performance) but also, the validation of customer experience against business requirements. To gain the confidence that you can meet both time and quality standards, before embarking on your change project, make sure your testing solution provides full visibility into the upcoming project. That is, a wholistic overview of system interfaces as well as information on which transactions are used for each business process, how often and the significance of those transactions. To meet business requirements with speed without compromising quality, choose a testing solution that provides real-time visibility and tracking of all testing stages, the results, and the defects, as well as workflow management capabilities. Invest in a testing solution that supports seamless communication and collaboration between different and globally dispersed stakeholders, such as business users, testers and developers.
• #20: Our survey findings further demonstrate that while testing decisions and processes are driven by the understanding of business requirements and the need to improve customer experience, execution is hindered by the lack of appropriate tools, In fact, when asked about their top test execution challenges, most testing leaders reported tools that are unfriendly to non-technical testers and require intensive training. WHY IS THAT – BECAUSE MOST TESTING TOOLS ARE TARGETING THE TECHNICAL TESTERS AND IGNORE THE BUSIENSS USERS
• #21: The need for better solutions can account for the low percentage of organizations (12%) relying solely on existing centralized testing solutions such as HP QC, in contrast with the median percentage (42%) of organizations who have purchased such solutions. THIS IS VERY INEFFICIENT. AFTER INVESTING MILLIONS OF $$$ IN ALEGEDLY MODERN TOOLS YOU FIND YOURSELF MANAGING THE MOST CRITICAL ASPECTS OF BUSIENSS PROCESS QUALITY WITH EXCEL SPREADSHEETS AND STICKY NOTES
• #22: Capgemini’s World Quality Report 2016-17 cites “inadequate test automation” as a top barrier to agile adoption. It recommends investing in “intelligent self-learning QA and testing platforms for all areas of the application landscape”. Today’s legacy test management solution like HP QC, meant to support QA and testing, tend to deliver technically-oriented user interface and experience, consistent with the IT side of testing, rendering it useless to the Business side of testing. The result is a test management tool that lacks key components like collaboration, business process centricity, documentation, visibility and automated process handling – all conducive to delivering application changes faster. Moreover, there is a greater, unfulfilled scope for test automation that most companies are not aware of. Automating test design, allotment, workflow management and other process orchestration activities can reduce global test efforts by 30%. Features like the automatic sequencing of manual tests, automatic capturing of test results, auto-creation of test scripts and on-demand availability of the above via a central platform can make test automation an integral part of the entire execution process. Finally, the fact that 54% of business process testing is still done manually is a testimony to an immature ROI model. As organizations look for a practical and affordable way to increase levels of automation, capabilities such as Autonomous Testing, that provides an always-up-to-date mirror of production as automated test cases (based on business knowledge of the relevant cases to automate) is the way forward.