For as long as Android has grown in the mobile market, developers have faced a huge challenge: How to test their apps across a wide — and constantly growing — range of devices? Some dev shops test across the most popular Android handsets and hope for the best, ; a less than ideal solution. Having users beta test mobile apps can help. But managing all of that test data is no easy task.
That’s where TestFairy comes in.
The service, which launched on Tuesday, aggregates app testing data in the cloud, providing developers with real-time testing information. Developers upload their .apk app file to TestFairy which sends invites to beta tests.After the apps are installed, Test Fairy captures key metrics from handsets such as CPU, memory, GPS, network usage, and phone signal to name a few.
TestFairy can even provide screenshots or videos of how testers are interacting with the application under test, which could help developers replicate certain issues. Here’s a quick peek at how the service works:
This solution sounds brilliant to me. I spent years managing a corporate software testing team and wished for tools such as this. That was in support of standard Windows apps; I can’t imagine testing apps across hundreds of mobile devices with different screen sizes, processors, graphics chips and radios.
TestFairy is launching as a free service, but it intends to add premium pay services in the future. From a developer standpoint, I can’t see why you wouldn’t consider using it. Testers might be a bit shy, however: I noticed that a device’s exact location is also tracked and reported on. I can understand why: Any location-based app would need to test for this. For internal betas that’s fine, but public beta testers may balk.