25 Comments

Summary:

Back in the desktop software era, magazines ran software reviews in which the side-by-side comparisons of features took up an entire page. Buyers used these reviews to shortlist vendors, trying to anticipate which features they’d need over the next five years. Typically, the software with the most features won. Feature-itis ruled. But with software as a service, the focus has become whether the tool is good enough on day one and how well it will adapt over time. Indeed, in order to evaluate SaaS, those page-long feature comparisons can be whittled down to just seven critical questions.

Back in the desktop software era, magazines ran software reviews in which the side-by-side comparisons of features took up an entire page. Buyers used these reviews to shortlist vendors, trying to anticipate which features they’d need over the next five years. Typically, the software with the most features won. Feature-itis ruled.

No more. With software as a service, the focus has become whether the tool is good enough on day one and how well it will adapt over time. Take, for example, the Family Service Agency of San Francisco, which replaced its ailing paper-based system with SaaS donated by the Salesforce.com Foundation, improving productivity and accountability along the way. Speaking today at the SIIA’s eGov event in Washington, D.C., Bob Bennett, the agency’s CEO, explained how the agency turned a salesforce automation tool into a social services management tool.

The point here is that the initial feature set didn’t matter much. Indeed, in order to evaluate SaaS, those page-long feature comparisons can be whittled down to just seven critical questions:

  • Adaptability: How easily can you modify the application? This can be as simple as adding fields or building dashboards, or as advanced as a programming platform.
  • Reliability: How much can you depend on the system to function well? This boils down to four things: Performance, availability, scalability and security.
  • Task productivity: How effectively can your users accomplish their goals? How many cases-per-minute or entries-per-day can workers do, and how many errors do they make?
  • Price: How much will it cost — really? Because SaaS offerings are so varied in pricing, it’s hard to compare them. A better model is to create several benchmark subscribers (a 10-, 100-, and 1,000-person organization) and compare upfront and ongoing costs for them.
  • Back-end integration: Can you plug it in to other things? Any enterprise SaaS offering will have to work with other systems, for everything from authentication to data sharing.
  • Longevity: How long will the SaaS company be around, and what’s your exit strategy? With ISVs, you could ask for software in escrow. But as the sudden disappearance of Coghead shows, when a SaaS provider closes down, your entire IT systems can vanish with the flick of an “off” switch. Offers from Intuit and others to help stranded customers notwithstanding, this is a big problem.
  • Ecosystem: How many third-party developers and integrators surround a particular platform with plug-ins and add-ons, and how active are they? A vibrant ecosystem means a more extensible, flexible solution.

The key point, however, is that features on day one don’t matter as much as the efficiencies and cost savings you can squeeze out of the SaaS tool within 30 days of adoption — and how confident you are that those efficiencies and cost savings will endure.

You’re subscribed! If you like, you can update your settings

  1. Great post. If a vendor can’t fulfill these 7 basic criteria, an RFI/RFP should never even be issued.

  2. Whether it’s old or new technology like cloud computing, business needs to factor in long-term effects before making decisions.

    Posts like this will surely help companies with their plans adopting SaaS. May I just add that exit strategy can be made more simple if the platform relies on ‘open standards’ and not proprietary solutions that tend to lock customers in. Companies should always remember that control of their data and apps should not be ceded in any way despite the allure of low cost and shiny, easy to use apps or risk being ‘scr*wed’. Really.

    Best.
    Alain Yap
    Morph Labs

  3. Don’t forget switching costs.
    How difficult is to leave the service and go for another one?
    This is often the major hidden cost that locks in customers forever, regardless of performance.

  4. 7 Questions to Evaluate SaaS | Digital Asset Management Wednesday, February 25, 2009

    [...] Continues @ http://gigaom.com/2009/02/25/7-questions-to-evaluate-saas/ [...]

  5. Also… regional factor should be considered… one player can be good service provider in a certain regional area other cannot.

  6. great post.

    Tying-in price and longevity, there is also an appeal to small companies and startups. In a depressed market, small companies can buffer risk by knowing that “if all else fails”, at least they are on a subscription that can be immediately terminated, vs. having paid for a 2-3+ year license agreement that won’t be fully utilized.

    Of course, this is a Yin/Yang: from the SaaS vendor perspective, we have yet to see what the walk-away numbers will look like in this same scenario.

  7. Good post.

    Is there any survey/study report that presents the SaaS adoption challenges and what matters most to a customer when choosing a SaaS provider over the big players?

  8. OStatic Buffer Overflow | google android os blog Thursday, February 26, 2009

    [...] 7 questions to evaluate SaaS. Software as a Service is on the rise, in and out of the open source arena. Here’s a checklist for evaluating SaaS apps. [...]

  9. Alistair Croll: 7 Questions to Evaluate SaaS « SaaSkatoon: All Things SaaS! Saturday, February 28, 2009

    [...] Alistair Croll: 7 Questions to Evaluate SaaS By deliveredinnovation Categories: General Alistair Croll: 7 Questions to Evaluate SaaS [...]

  10. digitalideamedia Monday, March 2, 2009

    I work with an online database company, TrackVia, who put out some tips for customers to consider when shopping for SaaS solutions. At the end of the day, like with traditional software, you need to know that the product will be around for a while and that you will have the support you need when an outage or problem occurs. With the recent outage at Gmail and closing of Coghead, the following blog/podcast highlights a few tips companies should consider when evaluating SaaS offerings:
    http://www.trackvia.com/blog/2009/03/02/saas-tips/

  11. Analyzing web-based systems, must say one should take into account the niche of the market the vendor belongs to, if talk about the lock in and how not to get in a bind.The thing is there are different levels of commitment the customer must accept in exchange for advantages of the particular system.

  12. EDunigan – TrackVia Thursday, March 5, 2009

    In addition to the seven critical quesitons identfied in the article above, I believe that individuals evaluating SaaS options should take advantage of the free trial offered by SaaS providers. Fully test the SaaS application and see if it receives a five star crash test rating. To begin with, Total your data and see what sort of back-up the provider has. Check out the following requirements for a five star rating:

    http://www.trackvia.com/blog/2009/03/05/evaluating-saas-free-trial/

  13. Puntos a tener en cuenta para evaluar una saas | Saasmania Friday, March 6, 2009

    [...] después de casi un año escribiendo, creo que se quedaron cortos. El otro día llegue hasta este post donde a parte de lo que señalé en mi post, había otros que me parecen importantes destacar y [...]

  14. Great post by Alistair. Any business that is considering SaaS for its critical business processes must consider these 7-items, including, the free-trials as indicated by EDunigan.

    At my company Treova (www.treova.com), we spend a great deal of time managing customer expectations. A few practical lessons after having used and implemented several SaaS products since 2005:

    1. Determine how easy it is to get the data into the hosted solution.
    2. Find out how easy (or hard) it is to get the data out of the hosted solution.
    3. Presence of a robust user group and the var channel is important; and finally
    4. Compare the delta between the way things are done currently and the process modeled by the SaaS
    5. New features offered by SaaS that are currently not available to you

    The last two items indicated above are key factors to long-term satisfaction with the hosted solution, all else being equal. See our blog at http://www.treova.com/wordpress

  15. GroupSwim Reading List – 04/15/2009 « The GroupSwim Diving Board Wednesday, April 15, 2009

    [...] Alistair Croll – 7 Questions to Evaluate SaaS [...]

  16. Good article. Companies can certainly do with this advice, and most don’t have experience subscribing for and running SaaS solutions. It is something they cant even get away from now.

    We had done a similar article recently, and delved a little deeper into various factors to be considered while analyzing different SaaS vendors. It is called “SaaS Vendor Selection Manual”. As a follow up, we shall also be hosting a webinar on the subject by Rusty Weston, ex head of Information Week Research, on 5th June 2009.

  17. With SaaS, “features on day one don’t matter.” « The Scheduling Insider Monday, June 22, 2009

    [...] 22, 2009 Found this userful article on how to evaluate whether SaaS is good for your [...]

  18. Thanks for the article. I think it’s important that customers make contingency plans with their SaaS vendors in case those vendors should go out of business. At the least, the customer should insist on a full copy of their data in a standard format such as XML. Ideally the customer would also get a copy of the source code so they could install the SaaS app in-house if needed.

  19. Clean Tech Open Names 6 Cali Finalists: Low Cost Is King Tuesday, October 20, 2009

    [...] from the thermostat and the web, said CEO John Steinberg. The company, which is working with a software-as-a-service model, says it has a utility deal and other major announcements coming up soon, so stay [...]

  20. SaaS Development Thursday, October 29, 2009

    I really like your artcile on SaaS. In today era it is very important to move faster with latest technologies and software as a services is a best way. Many companies are now looking for full web based OS which will not only save their monthly expenses on hardware but also save time.

  21. So You Wanna Be a SaaS Provider? Sunday, May 23, 2010

    [...] a sustained market leader in SaaS, software companies need to focus on building and delivering a highly efficient service to their customers without spending significant money on the underlying infrastructure (cloud or [...]

  22. So You Wanna Be a SaaS Provider? | Digital Asset Management Sunday, May 23, 2010

    [...] a sustained market leader in SaaS, software companies need to focus on building and delivering ahighly efficient service to their customers without spending significant money on the underlying infrastructure (cloud or [...]

  23. Requirements for SaaS companies « Raja Jasti’s Blog – Renaissance Thinking Monday, May 24, 2010

    [...] a sustained market leader in SaaS, software companies need to focus on building and delivering a highly efficient service to their customers without spending significant money on the underlying infrastructure (cloud or [...]

  24. Well written and good information. Thanks for the post.

Comments have been disabled for this post