Why testing automation is the key to enterprise mobile app adoption

1 Comment

At about the same time that Apple published information depicting a thriving iOS ecosystem that employs some 627,000 workers, IDC speculated that the number of enterprise applications optimized for mobility will quadruple by 2016. Even though “23 of the world’s top 25 retailers, 92 of the top 100 banks, and the 10 largest insurers all entrust core operations to Cobol programs running on IBM mainframes”, IBM still decided to forge an alliance with Apple in 2014 to help transform enterprise mobility.

And if you follow latest ranking of program languages monitored by RedMonk, the language most likely to be on developer hit lists will be #22 Swift and not #10 Objective-C, as the growth that Swift experienced is essentially unprecedented in the history of their rankings. Android has seen similar growth on RedMonk’s rankings with Java holding on at #2 this year and Go moving up to the #17 spot following its 1.4 release.

But growth in enterprise app numbers, alliances between technology leaders, and language adoption are not the only indicators that enterprises are starting to embrace mobile development more and more. The tooling and software support ecosystem around mobile enterprise is also maturing. Traditionally enterprises make platform decisions that last years or even decades. Much longer than the typical half-life of a mobile app in the app store which could be a short as three months.

With such long-term investment decisions in app development at stake, making sure the code you develop today will continue to work well into the future is important. That is where testing automation comes into play.

Testing iOS apps

Testing native iOS apps has had a bad rap in the past, especially when it comes to test automation. It has proven to be difficult, time-consuming, and the tools were not as capable as they are on other platforms (most notably Java). On a more positive note, the round trip from making an edit in the code, to compiling, to deploying and debugging is extremely fast when using the iOS Simulator. This helps cut test execution time down considerable compared to Android. Great strides have been made since iOS 8 and Xcode 6 have been out, so if you have not looked into automated testing of your iOS apps in a while, you may want to take a second look.

XCTest – Out of the box, iOS developers have access to a respectable unit testing framework know as XCTest. Think of unit testing as testing the code from the inside out. Looking at the internal operations under the covers, a well written suite of XCTests can quickly point to where problems are in your code base as you update to newer iOS versions. By default, all projects created from within Xcode generate an initial test suite that developers can expand upon.

OCMock/OCMockito – Mocking is a testing technique used to mimic situations that are difficult to reliable create in a repeatable and predictable fashion during the execution of a test. An example would be to replicate a trade on Wall Street or a bank transaction, something you may not want to do for real over and over again all day long. OCMock and OCMokito are two popular testing frameworks that allow developers to create more reliable testing environments that do not depend on the availability of complex back-end systems in order to provide an assessment of the overall health of the code.

Xcode Service and Continuous Integration – Continuous Integration refers to a technique in development that all code changes that are saved to the system (referred to as the code repository) are compiled and tested on an automated and continuous basis. Monitoring the overall health of your code can alert developers when a potential problem is first introduced to the code. There is typically a very visible and accessible report that allows developers to know exactly when something has gone wrong during development. That way there is nothing that any one developer can do without every developer instantly knowing about it; good or bad. XCode Service has made monitoring the health of iOS project as easy as possible.

Automating UI Testing with Instruments – Apple has always provided a rich set of instruments that helped developers know what was going on inside its apps. From memory leaks, random crashes, and battery drains, instruments have proven to be an invaluable tool when it comes to writing high quality and dependable apps. Automating the testing of the user interface is no exception. Where unit tests test the app from the inside out, user interface tests test the app from the outside in. Now developers can create test scripts that will execute tests over and over again against the user interface to ensure that changes they make do not break anything they thought they were done developing.

Testing Android

The biggest challenge to automated testing on Android has been test execution time. Even when leveraging the maturity of testing technologies that have evolved on the Java platform, it can still take hours for a suite of tests to complete. This is due primarily to the use of emulators in testing: emulators can slow development to a crawl at times.

Tethering a device to an automated test server can take a test device away from a developer or tester, but does prove to run things a lot faster. There are faster emulators out there like GenyMotion, but even then, looking for an even faster solution can make the difference in how often the tests get executed by members of your development team.

Another difference you will find when comparing iOS to Android is that for iOS, there is typically one and only one way to do things: the Apple way. With Android there are almost always more than two or three ways to accomplish about the same thing; unit testing, mocking, continuous integration and user interface testing.

JUnit/JUnit 4 – While the default Android testing API for unit testing, AndroidTestCase, is based on the JUnit 3 API, you can now consider switching to JUnit 4 to unit test your Android apps by using AndroidJUnitRunner. Like XCTest on iOS, JUnit is a unit testing framework meant to test how your app works on the inside. This is a developer’s first level of defense when trying to keep the quality of code that worked perfectly well when it was first written, from breaking some time down the road.

Mockito/Robolectric – While Mokito is more of a traditional mocking framework that can be used on the device or within an emulator, Robolectric is more of a replacement of the emulator or device entirely. Almost an “uber mock” if you will, Robolectric allows developers to execute tests almost anywhere. The advantage that Robolectric brings to the table is speed, you will find that your tests perform much faster, sometimes an order of magnitude faster. You can almost consider it more of a simulator replacement for Android’s emulator that provides just enough support to create a testing environment stable enough to provide value to your testing activity. Value that typically degrades when tests take way too long to execute.

Jenkins/Bamboo – Continuous integration servers that check the health of your Java code have been around for years. Technologies and servers have come and gone many times over. Two of the more popular choices for introducing a Continuous Integration environment to your Android projects are Jenkins and Bamboo. Both provide a means of monitoring your code base, automatically executing tests, and alerting developers when something has gone terribly wrong.

Calabash/Appium – While not entirely exclusive to Android as a User Interface testing harness, both Calabash and Appium provide a means of testing apps from the user interface. The ability to test the app from the outside in complements a unit testing strategy providing a more comprehensive strategy when it comes to monitoring the health of your apps code. One without the other will prove to leave gaps in your automated testing strategy that you will need to compensate for with your manual testing strategy.

1 Comment


Great stuff here. I realize a lot of what you list is back-end stuff, but there are other tools, like Appsee (www.appsee.com) which allow for testing visually. You can check heatmaps and actually view user sessions to see how they interact with the app and where the problem areas are, avoiding any issues for the launch. (Though they work after the release of an app as well so if there are any ongoing issues, you can track those too, as well as determine why abandonment rates are high or where there are technical issues and crashes.) I think the visual side of things is what can ultimately make or break an app’s success.

Comments are closed.