4 continuous testing challenges organizations must address

How much time do you spend on your cell phone in a single day? According to one set of research, it’s more than five hours. From making calls to using things like banking and health care apps, today there is an app for just about everything. In today’s world, our lives very much revolve around digital technologies.

But what would happen if the apps that help run our lives and keep our businesses on track suddenly stopped working? Think of it this way: You wouldn’t have easy access to things that affect your daily life, like your checking account or health-monitoring systems. From a business perspective, processes would be delayed, and tasks would take longer to complete.

That’s why it is more important now than ever before for developers to assure continuous quality throughout the entire software development life cycle (SDLC). And it all starts with testing.

Once considered a bottleneck holding up different activities throughout the devops process, continuous testing (CT) is becoming a staple of the SDLC. Organizations can use these techniques in a variety of ways such as identifying, reacting and responding to bugs in real-time. Automation is helping to drive these implementations but not all tests are automated. In fact, according to a 2017 report from Sogeti, only 16 percent of test activities are automated.

While automation is a key factor to making continuous testing a reality, there are hurdles that dev teams need to overcome. Let’s look at the top four challenges organizations must address and how you can tackle them to assure rapid release turnarounds that are glitch-free.

1. Moving both faster and smarter

In today’s digital age, app developers are expected to move both faster and smarter. A good example is the iOS 11 release back in September 2017. Within its first few months of general availability, iOS 11 underwent multiple iterations and updates. With every new release, app developers had to ensure their apps would still work. This proved to be challenging as app developers often find it difficult to automate new tests within existing cycles. As a result, they are held back from continued maintenance and continued innovation. 

As consumers’ expectations continue to rise, more platforms will be introduced and new technologies like internet of things (IoT) devices and wearables will take center stage. Release cycles will also only continue to accelerate. To help overcome these obstacles, app developers should look to adopt acceptance test-driven development (ATDD) and behavior-driven development (BDD) practices, which drives the test automation code prior to the code itself. In addition, adopting faster test execution frameworks like Espresso and XCUITest for mobile, or headless browsers frameworks like Puppeteer and PhantomJS for web, accelerate setup time, and test execution speed.

2. Devops teams aren’t on the same page when it comes to test automation

Across devops teams, there unfortunately is a stigma of “it’s not my problem” when it comes to test data. Explained further, most believe that it’s the test data team’s job to provide all the data and insight. However, as devops continues to evolve within agile organizations, these silos are starting to disappear.

To implement a testing automation team that keeps the engine running, strong discipline is needed throughout the SDLC.

The first step is ensuring all teams agree when it comes to the process you choose, whether it’s BDD, ATDD, or agile. In addition, perhaps most important, leave shadow continuous integration (CI) processes in the past to ensure teams have access to a single view of the quality per code commit. Finally, keep in mind that each department has a different persona and set of needs. Because of this, it is important to provide them with the tools that best match their skill set, while still integrating with the overall CI and continuous development (CD) stack.

3. There is too much data to act on

The good news is that today, we find that tests are primarily doing what they’re meant to—generating new data and insights. However, the influx of information is making it nearly impossible for teams to act upon the new data and insights that are being generated. Lack of intelligence is partly to blame here however we do find that most organizations struggling to determine the best way to implement machine learning and artificial intelligence throughout the entire devops life cycle.

To start, focus on test analytics, leveraging current in-house tools to determine test suite efficiency and platform usage. This provides insights such as which tests are detecting the most bugs and which tests are constantly failing. It can also stabilize overall CT activities and reduce the overhead associated with “cleaning the noise” per each cycle. From a platform usage perspective, organizations that use such data should look to optimize their test labs for maximum platform utilization by revealing shortage in platforms per cycle, or unused platforms per cycle. In addition, matching platform usage throughout the testing cycle to test data that shows pass/fail results can point teams to focus their testing efforts on the most error-prone platforms. Prioritizing platforms saves time and stabilizes product quality.  

4. Lack of clarity makes open source a time suck

Although open source tools are still a relatively new concept throughout the industry, they are becoming an important element of the dev team’s stack. Most organizations are however struggling to maximize productivity with them mainly because they are limited in their abilities. For example, they are unable to support advanced features. They also produce detailed reports and optimize intelligently, making it difficult to enhance debugging capabilities or scale test automation itself. What’s more, in the case of mobile specifically, open source frameworks just can’t keep up.

Instead of using open source tools individually, teams should look to start combining them with commercial tools. Using the right open source test automation framework in conjunction with an enterprise commercial tool provider makes it easier to execute automation at scale – especially in the cloud.

At the end of the day, no organization wants their application to fail, and ultimately, let customers down. To achieve continuous quality and keep customers coming back for more, implementing CT throughout the entire SDLC will not only achieve continuous quality and keep customers coming back for more, it will also ensure teams catch all bugs in real time. This enables teams to speed up the process in which they can patch holes and return to business as usual.