Android’s fragmentation woes may be fading, but they’re not gone yet. The fact that apps can run inconsistently across different Android devices has helped spawn a mini-industry in its own right, with companies such as TestDroid, AppThwack, DeviceAnywhere and TestCloud all offering app testing ‘in the cloud’, and outfits such as uTest effectively crowdsourcing app testing.
However, these services all have something in common: they test the apps on real devices. And they rightly advertise the fact, because they want to demonstrate the superiority of that method over the standard emulator found in the Android SDK. But the question is, is this approach always necessary?
TestObject, a German startup that this morning announced €1.1 million ($1.47 million) in funding from the Investitionsbank Berlin and S&S Media, would say not. The outfit is planning to use that cash partly to build up a bank of real devices later this year, but in the meantime it’s approaching the task differently.
“At the moment we’re running on virtual devices,” co-founder Hannes Lenke told me. “You can upload your application and see a device in the browser and just record a test, and we play the test back on multiple devices in multiple situations – with a good internet connection, a not-so-good connection, higher screen resolutions and so on.”
Keeping it cheap
What TestObject is doing is effectively to host Google’s Android emulator and automate testing on it across different device profiles. The efficiency benefits are clear – features include the ability to automatically run old recorded tests on new versions of the app, for example – but it still begs the question of why a developer would choose to do this rather than test across real smartphones and tablets.
The answer, according to Lenke, is cost. Because the service is still in beta, its actual pricing is yet to be determined, but even when TestObject steps up to its real-device competitors the company intends to maintain its virtualized-device service as a cheaper second option for early-stage testing.
“Real devices are more expensive. What we say is that virtual devices are 90 percent reliable,” Lenke said. “Our customers know 90 percent that their app will work on these devices, but if you want 100 percent reliability you can also test your app on real devices.”
The other big difference between TestObject and the majority of its competitors lies in its pricing model, which is pay-per-use rather than subscription-based — in that sense, it’s more of a genuine cloud outfit than most in this space. With that and its two-pronged approach to testing, the company may find itself able to stand out.