Quality Assurance and Testing is a bottleneck to implementing DevOps for many organizations
The practice of Continuous Testing – the process of fast and efficient validation of software releases in agile developments through highly automated tests – is gaining ground in large enterprises, with almost a third of IT executives (32%) stating that their IT departments had ‘fully embraced Continuous Testing’.
However, with 58% of enterprises deploying a new build daily (and 26% at least hourly), companies must work to improve their continuous testing effectiveness by streamlining their test environment within an agile development ecosystem.
The Continuous Testing report, released by Capgemini and Sogeti, in collaboration with Broadcom Inc.’s Enterprise Software Division, is based on 500 interviews with senior-level IT executives from large and medium-sized companies (over 1,000 employees) in industries including financial services, high tech, healthcare and life sciences, telecommunications, media and entertainment, and manufacturing. It found:
Companies must unlock the potential of automation in testing
The study strongly highlights that there is a significant scope to optimize Continuous Testing processes through technology. It found that automation was only being used to execute 24% of test cases, 24% of end-to-end business scenarios, and to generate 25% of the required test data.
Greater use of automation could significantly improve the velocity of testing activities in the agile teams; for example, over a third (36%) of those surveyed said that more than 50% of testing time is spent searching, managing, maintaining and generating test data. The report recommends that to respond to customer and market needs, all enterprises need to take significant action.
Use cases, detailed in the report, include a leading Australian bank, which had over 5,000 builds for more than 100 applications. By harnessing a single platform for automated build and deployment, integrated with testing and automation tools, it was able to reduce build cycle time by 40%, improve time to market and significantly increase environment uptime.
Enterprises need smarter orchestration and enablement for testing
The self-empowerment of autonomous teams resulted for many enterprises in an uncontrolled landscape with a broad diversity in QA and test automation approaches. To regain control, cites the report, organizations need to improve the central quality enablement of the agile teams with clearer QA guidelines and smarter QA technology provisioning.
A promising development is to make the test orchestration and execution much smarter via artificial intelligence (AI) technologies which provide “smart” test orchestration. With the addition of machine learning capabilities, systems will be able to automatically determine the tests that are required in the release and production cycles.
The report showed a clear need for improved transparency and orchestration in agile testing. Among the executives surveyed, 35% identified a ‘complete audit trail of testing activities’ and a ‘consolidated test and release pipeline’ as the most important test orchestration capabilities, with 32% highlighting a need for a ‘single-place for cross-team collaboration’ and ‘continuous delivery pipeline visibility’.
The shortage of centralized enablement on technology provisioning becomes most apparent when investigating the challenges with test environments. Teams wasted too much time procuring complete test environments. Four in ten (40%) of the respondents said their teams spend more than half of their time building and maintaining their test environments.
“The next two to three years will be a critical time for Continuous Testing as organizations need to solve the dilemma of transitioning to self-empowered autonomous teams where quality is everyone’s responsibility, while improving the central quality support and central quality oversight to create an agile development environment, that is truly able to deliver value in an increasingly complex IT landscape.” said Mark Buenen, Global Leader, Digital Assurance and Testing for the Capgemini Group. “This report clearly demonstrates the need to expand the levels of test automation, make provisioning of test environments and test data smarter, implementing meaningful quality metrics and smart quality dashboards as the core focus areas.”
Inter-disciplinary teams require new skills and support
In the last three to five years, the roles of both developers and testers have evolved significantly. According to the report, developers are now much closer to the customer, with a prominent role in shaping the user experience, while testers have moved out of siloed teams to work in parallel with developers and business teams, meaning that they are involved much earlier in the development life cycle.
The roles and responsibilities of developers and testers are blurring, but clearly it remains important to have QA and test focused experts in the agile teams.
These inter-disciplinary teams mark a move forward, but also create challenges, details the report. They require every team member to have a holistic understanding of the entire process, with testers needing to upgrade their technical skills. Companies must address the requirement for upskilling and a new, integrated approach to truly achieve Continuous Testing’s full potential.
“It’s difficult to fully capitalize on the benefits of Agile and DevOps without Continuous Test automation, which enables organizations to leverage data and intelligence to accelerate the creation of high quality software at the pace of digital business,” said Ashok Reddy, Senior Vice President and General Manager, Enterprise Software Division at Broadcom. “As the chasm between IT and business leaders continues to close, new DevOps practices and techniques, such as automating continuous testing processes, will begin to emerge as necessary tools to drive digital transformation in a hyper-competitive marketplace.”