17 Comments

Summary:

While nothing short of a fantastic delivery vehicle for all things iPhone Application, the App Store still has had its share of nuisances. (Even Apple doesn’t nail everything right out of the gate.) But with the unparalleled popularity of the iPhone and the ability to pair […]

While nothing short of a fantastic delivery vehicle for all things iPhone Application, the App Store still has had its share of nuisances. (Even Apple doesn’t nail everything right out of the gate.) But with the unparalleled popularity of the iPhone and the ability to pair it with 3rd party applications, growth is a necessity, and the App Store has already evolved in a very short period of time.

The reviewing process was flawed from the get go. Initially Apple had the idiotic notion that people who had never purchased or used an application should be able give it a review and star rating. Seriously, how could that work out well? The result was people who had no first hand knowledge of a [paid] application or game leaving scathing reviews because they didn’t like the price point. Apple has since fixed this, checking the ability to review an app against the user’s purchase history.

The solution is only partially effective, which is to say, it only works on paid iPhone applications. With the pricepoint of free, the apps that cost nothing are still susceptible of ridicule. Who wants to complain about something that’s free? Well, you’d be surprised. It’s not so much a situation of complaining about the price, but maybe the app, or the social commentary around the app — as was the case with the Barack Obama campaign application. The ‘review’ thread devolved into a Red vs Blue (unfortunately not the Halo machinima Red vs Blue) debacle (before and after the election). Thinking about the silliness of this particular situation, I can’t help but feel like the SNL Weekend Update segment, “Seriously!?!” But I suppose free speech is free speech, be it in a public venue, or an App Store application review thread.

On the brighter side, when Apple released the 2.2 version of the iPhone firmware, they instituted a smart idea to drive more star ratings for apps. If you’ve upgraded to 2.2, when you opt to delete an application from your iPhone directly (rather than removing via a sync with iTunes), you are prompted to give it a rating. It’s similar in feel to the prompt you get from a Customer Service IVR phone system, but with much less fuss — just tap the rating you want to give, and you’re done. If you want to go the extra mile, you can give a wordy review too. But I think the nice part of this feature is that more ratings are rolling in all the time. In fact, if you’ve been keeping score at home, the most popular of apps in the App Store were once maybe around 3000 ratings. Now the in-crowd of apps are surpassing 10000 reviews and beyond. So something Apple’s instituted is driving more people to sound off on the software they use (or don’t use) on their iPhones, which is great, because it only leads to better informed decisions before you click the ‘BUY’ button.

Are there any features of the App Store review process that I’ve missed? Or if you have some interesting ideas on how Apple could further educate potential buyers of App Store software, sound off here.

  1. Just as a comment, I surf the app store all the time and I still find a lot of people “gaming” the reviews. Nothing can be proven of course, but one can just tell that with some apps that this is definitely the case.

    Since one has to purchase the app to review it, I think what’s happening is that this new situation has skewed things to much in favour of the developers (at least the more immoral ones). it’s now possible to have a big batch of early reviews (from only half-believable email addresses), rating the app with five stars and containing no words beyond, “great!”. Me thinks the developers are using their ability to put paid versions in the hands of their friends to at least start off the comments in a winning direction.

    Not much can be done to stop this kind of false rating, but as a wordy person, I would like to see the actual reviews be forced to pass a threshold of some minimum number of characters, or to have low word count reviews cycled to the back. The reasoning here is that the rating is global anyhow, so the number of stars is not as important to the ranking order as the actual information contained in the review. Apple attempts to do the same with the “did this review help you?” buttons.

    There really is no worth in a review that says “love it” or “great” if that’s all it says. To have to go through ten pages of such dreck before finding any kind of actual review of the product is not helpful at all. If people want to just give an app five stars and say “great,” … I say we should keep the rating but relegate their oh-so-interesting “comments” to the back of the queue.

    Share
  2. I’ve been purchasing apps for my ipod touch for a few months now and have just started downloading apps for tmobile’s g1. The reviews in the Android Market store are 1000x worse than reviews have ever been in the Apple App Store. The Android Marketplace’s reviews are filled with hate speech and idiotic youtube level drivel.

    Share
  3. If the App Store asks you to review an app after you deleted it, well, won’t that be skewed towards an unfavourable response? Afterall, you’re deleting it for a purpose – most likely because you thought it was crap. The number of times I’ve deleted an app off the iPhone because i’ve completed it and was fully satisfied is the grand number: ONE. And even then I had already left a great review (with words!) on the iTunes App Store. (FYI, it was Soul Trapper ep1. Fantastic take on interactive audiobooks!)

    Share
  4. I think the “review when deleting” does cause a bias towards bad reviews. Maybe they need to prompt you for a good review sometime – like when you order an upgrade to an application, or something.

    Also, the Weekend Update phrase you’re looking for is “Really?”.

    Share
  5. I thought I’d throw my thoughts in as an App developer. Actually I’ve just blogged about this very subject -

    http://chillix.wordpress.com/2008/12/09/some-thoughts-about-reviews/

    Gazoobee talks about ‘gaming’ the reviews. I think it’s only natural for a developers friends and family to give positive reviews to a new app especially if they’ve followed it’s development. In a lot of cases they’re eagerly awaiting the release of the app – what amazes me is how many apps don’t get any reviews at all! Don’t these developers have any friends?

    The two Michaels talk about the negative bias that the ‘review when deleting feature’ might cause and this is something that concerns me as well. It’s too easy to give a bad star rating – if you want to leave a positive rating you have to make a conscious decision to go to the App Store, log on, write a review… I’d love to see them implement a ‘this is the 10th time you used this app – you must really like it’ feature as well so that you can easily give positive ratings to the Apps you actually use all the time.

    Another irritation is that in many cases the review content and star ratings obviously don’t match – I’ve seen loads of ‘fantastic app, great value’ 1 star reviews. You don’t need that many 1 star reviews to start dragging your average down…

    My final point is that a lot of reviewers expect Apps to be able to do things that Apple just doesn’t permit – a couple of our Apps have come in for unfair criticism for not offering a feature that’s currently impossible to implement and that we’ve never claimed to do anyway – specifically Apps booting up automatically rather than having to be manually launched…

    Share
  6. The ratings system may as well be designed to draw in harsh reviews. The only time you’re encouraged to rate an app is when you’re deleting in – probably not the way to get representative reviews from people who actually like the app. For those reviews that are left, there’s no way of telling when a review was left – how old it is or which version of the app it relates to – so, as the store gets older, more and more of the reviews are outdated, and the scores unrepresentative of the current version. This is both unfair on the developer, and unhelpful to the user.

    Frankly, the whole review system is plain shoddy.

    Share
  7. I think we also have to keep in mind when talking of reviews that people are basically idiots 80% of the time. I know that sounds harsh, but anyone who’s lived more than a decade or so knows this is true. Most reviews will therefore be inane, off-topic etc.

    I don’t know exactly how (although I raised one possible tweak above), but the task has to be to somehow shuffle the idiotic, spiteful, empty reviews to the back of the queue while simultaneously promoting the “better” reviews.

    The standard process for this on most sites, and also used in the app store is that “did you find this review helpful” stuff, but that simply doesn’t go far enough. When you have fools voting on the relevancy (or not) of similarly foolish reviews you’re not going to get a useful result.

    As a footnote, I would argue that most people are not that critical and are generally enthused about something after they purchased it, so the majority of the reviews are actually skewed *positive* although very shallow in terms of reasoning or content. The slight skew to the negative when deleting an app seems to me to be a way of balancing the number of reviews written moments after downloading by someone who gives it a thumbs up even though they have hardly used the thing or tested it at all.

    Share
  8. I didn’t have a problem with reviewing apps that hadn’t been purchased. I found Hahlo from the overpriced Twitterific’s reviews from someone who hadn’t bought it. There are tons of BS glowing reviews that are obviously fake.

    They should take everyone’s reviews and normalize them around 3 stars. That would avoid a lot of pumping of reviews.

    Share
  9. THESE APPS ARE GOING TO MASSES OF PEOPLE FROM AGES OF 6 TO 80′S. WE HAVE A WIDE RANGE OF OPINIONS. MY 9 PAGES OF APPS IS COMPLETELY FULL PLUS I USE DELICIOUS AND THATS ANOTHER 300 OF WEB APPS. I FEEL THE SYSTEM WORKS, THE APP STORE IS FAR MORE THEN I EVER THOUGHT IT WOULD BE, IN THIS SHORT TIME. I HAVE BEEN SWAYED BY REVIEWS, MOST TIMES HAPPILY. ITS BECOME A HOBBY BUT NOW I MUST DELETE AN APP TO ADD ONE AND THATS VERY TOUGH. MOST APPS COST LESS THAN A DRINK AND THE USEFULNESS AND ENJOYMENT LASTS A LOT LONGER THAN THE DRINK. I USE THE APPS A LOT MORE THAN THE IPHONE, SO I CARRY MY LIFE IN MY POCKET, I USED TO HAVE TO CARRY A LARGE BREIF CASE, I’M HAPPIER THIS WAY, YES I KNOW ALL CAPS LIKE I’M TYPING, IS CONSIDERED SHOUTING!!! WELL, I AM SHOUTING, BE CONSTRUCTIVE!!!!!!!!!!!!!!

    Share
  10. Since, we can not download demos of apps, it might be helpful to have a link to YouTubes review site videos in the description of the app by the developer. Granted, they can be biased. It is up to the viewer to sort this out.

    I also think the reviews should show the date they were written. This would be more useful to me than the sorting by latest reviews rather then most helpful?

    Share

Comments have been disabled for this post