Do you always need real devices to test Android apps in the cloud? TestObject thinks not


Credit: Horia Varian

Android’s fragmentation woes may be fading, but they’re not gone yet. The fact that apps can run inconsistently across different Android devices has helped spawn a mini-industry in its own right, with companies such as TestDroid, AppThwack, DeviceAnywhere and TestCloud all offering app testing ‘in the cloud’, and outfits such as uTest effectively crowdsourcing app testing.

However, these services all have something in common: they test the apps on real devices. And they rightly advertise the fact, because they want to demonstrate the superiority of that method over the standard emulator found in the Android SDK. But the question is, is this approach always necessary?

TestObject, a German startup that this morning announced €1.1 million ($1.47 million) in funding from the Investitionsbank Berlin and S&S Media, would say not. The outfit is planning to use that cash partly to build up a bank of real devices later this year, but in the meantime it’s approaching the task differently.

“At the moment we’re running on virtual devices,” co-founder Hannes Lenke told me. “You can upload your application and see a device in the browser and just record a test, and we play the test back on multiple devices in multiple situations – with a good internet connection, a not-so-good connection, higher screen resolutions and so on.”

Keeping it cheap

What TestObject is doing is effectively to host Google(s goog)’s Android emulator and automate testing on it across different device profiles. The efficiency benefits are clear – features include the ability to automatically run old recorded tests on new versions of the app, for example – but it still begs the question of why a developer would choose to do this rather than test across real smartphones and tablets.

The answer, according to Lenke, is cost. Because the service is still in beta, its actual pricing is yet to be determined, but even when TestObject steps up to its real-device competitors the company intends to maintain its virtualized-device service as a cheaper second option for early-stage testing.

“Real devices are more expensive. What we say is that virtual devices are 90 percent reliable,” Lenke said. “Our customers know 90 percent that their app will work on these devices, but if you want 100 percent reliability you can also test your app on real devices.”

The other big difference between TestObject and the majority of its competitors lies in its pricing model, which is pay-per-use rather than subscription-based — in that sense, it’s more of a genuine cloud outfit than most in this space. With that and its two-pronged approach to testing, the company may find itself able to stand out.


Trent Peterson

Thanks for mentioning AppThwack. It’s exciting to see new players entering the mobile QA space. It’s a HUGE market and only getting bigger.

It’ll be interesting to see how their pricing and emulator vs. devices mantra changes over time. We started with emulators and quickly moved off of them, and I’d be interested to see where the 90% reliable number comes from. For anything but layout issues this seems much too high, and even those don’t always reproduce. For performance and carrier/manufacturer bugs, it’s essentially impossible with emulators.

For pricing based on use measured by time, DeviceAnywhere and Perfecto actually do just that. AppThwack has both subscriptions and a pay-as-you-go model that measures usage by tests instead of time.

Thanks again for spotlighting companies in Android and general mobile testing!


Not sure if there is such a service but maybe they could market it as a way for consumers to test apps and even phones before buying and without installing .Some developers and device makers must be willing to pay for a good simulator , ofc some could do it themself too.It’s one thing to read reviews,watch a video,look at screenshots and quite another to use it and explore the app yourself and doing it in the browser is pain free.

Comments are closed.