MemberHub provides places for group members to connect and communicate, as well as tools for managing those members. It offers more features than such services as Convos and Tgethr, which we’ve looked at in the past.

Its core features are focused on easily sharing information and working collaboratively within groups, whether working online or via email. It is built around discussions and messaging, integrated file management and calendar functionality.

MemberHub is not a project management system, but it has functions not found in other group communication systems, like the ability to send out text messages from the app. This could be tremendously useful for both reminders and emergency notifications.

Thanks to the recent upgrade, administrators can now add members to a group without users needing to pre-register.

For larger groups, the process of managing members and hubs (or groups) within a business or organization has been simplified. Improvements to this area make working with the product much easier. MemberHub can manage members individually and as a part of multiple groups, which is one of the key differences between MemberHub and other collaboration tools I have seen.

As a user, I can now add multiple email addresses to my profile, to better facilitate being a part of multiple hubs or groups. Administrators can add members to the larger organization, and assign them later to individual sub-groups like Finance or Membership. Administrators can create subdomains with custom branding for each hub.

An organization of any size should see many benefits from implementing MemberHub. In my first review, I was impressed with its functionality as a communication platform. I still think that MemberHub is a great option for businesses, schools, non-profits, and churches. It’s now an even better option for larger groups.

MemberHub remains one of my favorite and most-recommended web services. I’m not aware of any other service that combines member management and online communications features as well.

MemberHub offers a free single-hub plan, and paid options ranging from $5 per month for 3 hubs and 30 people, to $149 per month for 1,000 people and unlimited hubs.

How are you keeping your group connected?

Related content from GigaOM Pro (sub. req.):

• Enabling the Web Work Revolution
• Report: The Real-Time Enterprise
• Social Media in the Enterprise

The web conferencing market has changed considerably since our 2007 WWD roundup, but here are a few options that we’ve covered in the past.

• Fuze Meeting. This service is entirely browser-based, and offers many features, although it doesn’t have a free option.
• Skype. Simon reminded us that this is a good alternative for small meetings, and it has both video and screen-sharing capabilities.
• Yugma. This app includes some sophisticated features, but still offers a free version.
• Yuuguu. Doriano liked this IM-based service because it doesn’t require those you’re sharing screens with to download or install anything.

Here are some of the other solutions available. Prices, features and usability vary greatly.

• Adobe Web Conferencing
• Live Meeting
• GoToMeeting
• WebEx
• Zoho Meeting

What will you use to replace Dimdim?

Image courtesy Flickr user vivevans

Related content from GigaOM Pro (sub. req.):

• Enabling the Web Work Revolution
• Report: The Real-Time Enterprise
• Social Media in the Enterprise

A shared wiki is the logical place to store this type of information. A wiki is accessible to anyone on the team; a good wiki will become the clearinghouse for any and all information that needs to be documented. While there are many wiki packages, after a diligent search we chose editme.

I had identified a few very specific needs when looking for our wiki solution:

WYSIWYG Editor. Even as a tech-savvy company, my experience has shown that most users don’t easily adapt to using specialized wiki syntax. My goal was to make the platform as easy and accessible as possible; if people don’t understand it or the process is difficult then they won’t take the time to use it. I found editme had an easy-to-use visual editor but with the option of using some more powerful syntax as well. The ability to add images, formatting and links to other items and pages is crucial for maintaining an appealing and useful environment. editme does this quite well.

Permission Levels. Even though our team is very open, there is certain information that needs to be segmented and made available to a more limited audience. I liked that editme let me keep the wiki overall private and secure within our team but then also gave me the functionality to create groups for users and then easily assign pages to specific groups. Overall, I am happy with the way that I can assign and manage these permissions. (However, on a related security issue, I’m not fond of the way that that it sends password reminders and logins via plain text in an email. I would much prefer a reset link or other reset or reminder options.)

It is also possible to make the wiki site public, or a mix of public and private; the ability to segment access and permissions in those scenarios becomes even more critical.

Affordability and Support. As a software company ourselves, we have no problem supporting other products and services that we find useful and that contribute to our own success. I know that there are also open-source alternatives that we can install and maintain but as a business critical service we decided that going with a hosted and maintained offering was our best option. With unlimited users at even the cheapest plan and pricing ranging from $5 – $79 month, it was well within our budget, and as new employees join the team we simply point them to the wiki to get them started.

In the six months we’ve had our editme site up and running we’ve found that the adoption has been good, particularly in our Customer Experience group where we focused our initial efforts.

Initially, we had some trouble with the menu creation and the page organization features; they were functional but confusing. That has since been revamped and the new Page Organizer feature is really slick. We can now drag-and-drop pages around to easily change the structure of the site.

Billed as both a wiki and a collaboration platform, editme straddles the line between wiki and web site creator, comfortably living in both worlds. While we were primarily looking for a data organization tool, the additional web site creation and styling features are easy to use and a nice bonus.

Pricing for editme starts at $5 per month, and 30 day trials are available on all plans. It’s a hosted solution and works across all major browsers.

Find out more about how software is enabling companies to improve collaboration — check out our Net:Work conference, coming to San Francisco in December.

Related content from GigaOM Pro (sub. req.):

• Are You Empowering Your Mobile Workforce?
• Report: The Real-Time Enterprise
• How to Manage Consumer-Grade Collaborative Tools in the Workplace

It’s a bit ironic that such a closed platform as Apple’s iOS, and its accompanying App Store, welcomes open-source software at all. Yes, the Apple iPhone SDK states, “If Your Application includes any FOSS, You agree to comply with all applicable FOSS licensing terms.” But the platform itself, and the App Store rules, effectively block adherence to this stipulation by removing the ability to access, read, and redistribute underlying source code for apps.

It’s a core part of Apple’s mantra, as repeated by Apple CEO Steve Jobs this week, that “integrated will trump fragmented [Android] every time.”

This integrated approach is at odds with the more free-spirited, open-source approach, as reflected in new skirmishes between open-source advocates and Apple or app developers. For example, the Free Software Foundation recently filed a complaint against Apple for its distribution of the open-source GNU Go game without providing its underlying source code, as required by the GNU General Public License (GPL). The FSF believes Apple’s App Store rules violate Section 6 of the GPL, which indicates that “a redistributor of the licensed program may not impose further restrictions on the recipients to copy, distribute, or modify the program.” Apple doesn’t allow any of those freedoms.

Unsurprisingly, rather than confront the question, Apple simply dumped the GNU Go from the App Store.

But the issue won’t die, thanks to a similar concern arising over The Battle for Wesnoth. It’s a tricky situation, because in this case, Wesnoth’s developers — at least, many of them – do not favor the FSF’s position, and believe that the App Store regulations do not violate either the spirit or letter of GPL licensing.

As an interesting corollary to this legal argument, Wesnoth developer David White defends open-source applications on the App Store because they funnel money and interest back into open-source development:

• Wesnoth has obtained significant revenue streams which have been used to fund the development of art and other content in areas in which Wesnoth needs improvement to compete with commercial offerings.
• FLOSS [Free, libre, open-source software] gaming has been exposed to a significant audience which might be otherwise unaware of it.
• Other FLOSS games are now also interested in distribution on the same platform.

In large part, these funds are possible precisely because the App Store regulations make it onerous to get Wesnoth on the iPhone/iPad in any other way other than through the original Wesnoth development team. Apple doesn’t allow multiple entries for the same app within the App Store, and Apple’s policies make it difficult to replicate the open-source code.

Can developers get the source code from the app developer’s website? Absolutely. Can they recompile it and distribute it to jailbroken iOS devices? Of course. It’s open source. It’s the platform that’s closed, and closure provides the opportunity to better monetize the code.

The platform effectively makes the open-source app proprietary without a single line of code licensed under a restrictive license.

This isn’t actually new. Think of Google. As Om Malik has written, infrastructure is Google’s key competitive advantage. Guess what? That same infrastructure that makes it possible to run open source at dramatic scale and efficiency is the very same infrastructure that makes it virtually impossible for anyone else to make any use of this open-source software.

Because Google runs this software internally, it isn’t required to release it.  Even if Google released every single line of code, small competitors couldn’t replicate its services, if for no other reason than they couldn’t afford the hundreds of thousands of servers necessary.

More fundamentally, even big competitors couldn’t replicate Google, for the reasons Tim O’Reilly pointed out at the Open Source Business Conference in 2009:

Tim asked a question to the audience: “Could anyone in the Open Source community build the infrastructure to deliver Google Voice Search?” The response: a stony silence. The implication? Vendor lock-in is [no] longer about proprietary source code. It’s about massive, hard-to-replicate data sets — making Google a potential Microsoft of the next decade.

Hardware and data make Google’s source code essentially useless to anyone but Google. Does this make it less valuable as a member of the wider open-source community? Hardly. Indeed, as I’ve said before, Google is the world’s largest “open-source company,” and a better open-source citizen than most. Why? Because Google can afford to give away mountains of code because its sales aren’t threatened by competitors (or customers) taking the services enabled by its code without paying.

Smaller software developers lack Google’s scale, of course, but may be able to achieve some of the same effects by writing applications to run on closed platforms like Apple’s.

Wesnoth’s developers don’t appear to have been motivated by the desire to sell more games because of Apple’s restrictive policies. Rather, they simply wanted the maximum audience for their creation. But the closed nature of the App Store is helping them to both find more users and more paying customers, all while leaving source code fully available for those who actually want it: developers.

Disclosure: I work for Canonical, a Linux vendor.

Related content from GigaOM Pro (subscription req’d):

• Who Will Be Impacted if Windows Phone Thrives?
• App Developers: Are You Ready for HTML5 and Metered Data?
• Is the Future of Enterprise Completely Open Source?

Apple first announced the new Apple TV broadband set-top box on September 1, pricing it at $99 and announcing a new business model for streaming rentals of TV episodes and movie files. With the new streaming model, Apple rents TV episodes from Fox and ABC for 99 cents, compared to the $1.99 price for standard definition episode downloads and HD downloads for $2.99.

To put that into perspective: Roku reported at the beginning of this year that it had sold 500,000 of its set-top devices after about a year-and-a-half on the market, and hopes to have sold more than a million of its box by the end of the year. So Apple is tracking well ahead of other, similar devices already in the market.

That said, Apple TV is still very much a hobby when compared to the other products it makes. Apple sold 4.2 million iPads and 3.9 million Macs in the third quarter, as well as 9 million iPods and 14 million iPhones. While those products were available all quarter, compared to just the last several weeks like the Apple TV, it’s clear that it could be some time before Apple’s video set-top box becomes a significant portion of Apple’s business.

Related content on GigaOM Pro: (subscription required)

• Three Reasons Over-The-Top TV Apps Will Beat Big-Cable
• Strategies for the Future of Digital Content Storage
• How to Market Your iPhone App: A Developer’s Guide

Users come to love services like Facebook and Twitter (even Foursquare) so much that it resembles a national crisis among some circles when one of their sites goes down for a few hours. How do they try to avoid these occurrences in the future? Well, they spend some of those millions improving their physical infrastructures and creating specialized software to address unique needs. Greenpeace aside, I don’t recall hearing any complaints about Facebook’s first data center, nor about Twitter’s planned data center, and these companies aren’t even making money.

So why is it that financial analysts and shareholders get so up in arms when massive public web companies like Amazon or Google embark on spending kicks? After all, they bring in billions in revenue each year, and Google is responsible for more web traffic than Facebook and Twitter combined. Because it detracts from near-term profitability, that’s why. Om calls such analysts “idiots and short-term thinkers,” and he’s spot-on with his analysis of the situation.

Imagine if Google’s search engine, or its paid Google Apps service, went down as frequently as Twitter. What about Amazon Web Services, which hosts a good number of popular web sites and relatively important enterprise applications? What if these companies never rolled out new services because that would require spending more money on infrastructure? The answer is that they wouldn’t be too popular for too long. Companies delivering services via the web have to spend money – on infrastructure – to make money.

I can only imagine the number of cloud computing providers who’d love to have the problem of needing to scale to meet demand, and actually having the cash to do so. That day will never come if they don’t do infrastructure right in the first place.

When we’re talking about companies for which millions today might mean billions tomorrow, it might behoove shareholders and analysts to lighten up a little bit on the bottom line.

Related content from GigaOM Pro (sub req’d):

• Analyzing Cloud Revenues: Look at the Growth, Not the Money
• How Twitter is Re-Engineering to Address Always on Usage
• Lessons from Facebook: How Google Can Reach One Billion Users

What separates New Relic from other APM providers is that its RPM service monitors live software from the inside instead of simply monitoring external web application metrics. RPM comes in five editions, ranging from the free Lite version, which gives basic monitoring capabilities, up to the Enterprise version, which provides a range of capabilities to discover and cure performance issues. According to founder Lew Cirne, the internal view helps customers identify problems early and avoid “the thorniest issues,” citing the recent Foursquare outage as a problem that might have been avoided if the company could have been proactive in addressing the problem.

Formerly, New Relic supported only Java and Ruby web applications, but as of today, it also supports .NET and PHP applications. This is nothing new for cloud-based services, which tend to start with a focused offering and expand their customer bases as they grow by adding additional language support. Even larger companies like Google and Salesforce.com took this approach with their cloud offerings. More applications mean more money, after all.

New Relic’s already-expansive footprint would seem to underscore the value of its service and of the SaaS model. The company counts just about every cloud provider as a technology partner (Joyent and Heroku offer it as an add-on in their offerings), and the company has experienced 200 percent growth annually since launching in 2008. Cirne says the company presently has 5,600 customers running the service in production, with 900 of them paying New Relic directly. Among them are large enterprises running New Relic within their data centers.

Perhaps more important than the product, however, is how the economics of New Relic underscore the power of the SaaS model –- specifically, the freemium model prevalent among SaaS 2.0 vendors. Nine hundred paying customers is no small feat when you consider that the company just increased its sales force to three from two a couple of weeks ago.  Cirne contrasts New Relic’s approach to that of Salesforce.com, which customers can’t start using without speaking with multiple salespeople, as well as to the last company Cirne founded, Wily (an on-premise APM vendor), which he sold to CA for $375 million.

When CA bought Wily, Cirne says it had 500 customers, 270 employees (about 150 of them in sales) and had raised $45 million. Keep in mind, New Relic just closed its latest round. Most of what it has accomplished was with about $10 million. Cirne says $20 million is way more than New Relic needs to achieve profitability, but it’s good to have some extra cash to capture opportunities in what he thinks will be a billion-dollar business.

Image courtesy of Wikipedia Commons user KlausFoehl.

Related Research about on Big Data from GigaOM Pro:

• Mo’ Money: Life is Good for Cloud Vendors
  
• VMforce: Who’s the Biggest Winner?
• Could SaaS + PaaS Equal Cloud Computing Gold?

One is its evolution from an internally developed virtualization-management solution at megabank Credit Suisse into, a spinoff company selling that software to the greater IT community in 2008. Credit Suisse was managing thousands of virtual servers and desktops while DynamicOps’ underlying Virtual Resource Manager software was developed, so scalability isn’t an issue. In fact, said VP of Marketing Rich Bordeaux, one customer currently manages 30,000 VMs and virtual desktops and is looking to have more than 60,000 within 18 months.

Furthermore, because the bank wasn’t about to move certain application from physical servers, physical-resource management is inherent in the Cloud Automation Manager. The new features in Cloud Automation Manager are what make it truly cloudy, though: multitenancy, self-service provisioning, and the ability to provision and manage resources from Amazon EC2.

DynamicOps’ other ace is that Cloud Automation Manager is an integral part of Dell’s recently announced Virtual Integrated System (VIS) offering. The DynamicOps software is the foundation of the VIS Self-Service Creator component. VIS is Dell’s attempt to combat competitors’ converged infrastructure solutions (e.g., Cisco UCS and HP BladeMatrix), so such an OEM deal should help validate DynamicOps’ claims of enterprise-readiness (if the Credit Suisse heritage doesn’t already do so).

Of course, it’s hard to get too excited about any cloud software right now before organizations actually start buying it. Web hosts and MSPs have been buying up software from vendors like VMware, Cloud.com and 3tera, but most other businesses appear to be doing no more than dipping their toes in the water right now.  DynamicOps has a strong virtualization-management history, though, and an existing customer base to tout (and sell), so perhaps it can break in where others appear to be left outside.

Related Research about NoSQL Databases from GigaOM Pro:

• Report: NoSQL Databases — Providing Extreme Scale and Flexibility
  
• Defining Internal Cloud Options: From Appistry to VMware
• Let the Private-Cloud Management Race Begin

“Clustrix has built a truly impressive Clustered Database System from which Internet-scale businesses can benefit,” said Jonathan Heiliger, Facebook VP of technical operations in a press statement. When writing about the company back in May 2010, Stacey pointed out that this clustered database system was webscale’s Holy Grail.

Paul Mikesell — CEO of Clustrix and the former co-founder of storage system success story Isilon — told Stacey the appliance being built by his company would be ideal ” for companies managing large amounts of data, such as big travel, e-commerce and social websites.” Heiliger believes that CDS can “make it very easy for fast-growing companies to scale their infrastructure, and enhance reliability, while driving down cost and complexity.”

Clustrix is backed by Sequoia Capital, U.S. Venture Partners (USVP), and ATA Ventures.

Related Research about NoSQL Databases from GigaOM Pro:

• Report: NoSQL Databases – Providing Extreme Scale and Flexibility
• What Cloud Computing Can Learn From NoSQL
• The Red-Hot Data Warehouse Market: Who’s Buying Next?

For this round, I’ll look at companies that share these common themes:

• They market to application and infrastructure developers.
• They have a simple sign-up button on their home page.
• They integrate through simple web-based mechanisms that turn conventional deployment times of days or weeks into minutes or hours.

Transactional email. Most applications, consumer to enterprise, rely on email as a communications mechanism with users. With a large user base, sending users something as simple as notification emails can be challenging with spam filters, blocked IP addresses and other email hurdles that complicate the process. SendGrid helps improve the chances that the email you send will reach a user’s inbox. SendGrid does this by using its email servers as a proxy, and maintaining the purity of the IP addresses, one of the biggest email management challenges

Compute-intensive cycles. While it’s getting easier to “dial up” virtual machines in the cloud, some applications still need instant access to compute cycles for CPU-intensive functions: for example, in the software development process and with math-centric calculations. Without having to actually spin up your own server, PiCloud allows application developers to send functions to the cloud where they are executed and returned. The company is initially focusing on the Python programming language, but I see no reason why this couldn’t expand later on.

Network services. Network services have been hosted for years, but the integration and simplicity of deployment has dramatically increased with cloud computing. Managed DNS is one area that is seeing rapid innovation. The Domain Naming System (DNS) maps web address domain names to network IP addresses, and companies using a DNS service benefit from improved security. OpenDNS and Zerigo provide ready-to-go DNS services.

Database as a service. Every application needs a data store, and if you know what you need at that level, why not let someone else manage it? That’s the premise behind the service offering of Cloudant, which offers a version of CouchDB in a shared cluster that’s ideal for development and small- to mid-sized applications. Instead of buying and configuring a database server, you can dial one up in the cloud.

Indexing and search. Our real-time world demands up-to-date search, and Solr, the “blazing fast open-source enterprise search platform from the Apache Lucene project” helps companies provide just that. Websolr, a company that provides support for Solr, also offers managed indexes. In this case, they host the index of your content in their shared cluster and can manage and optimize performance.

More segments are adopting the infrastructure apps model every day, and when people think outside the consumer web box, there are significant investment areas. Are there more infrastructure app areas that we’ve missed?

Gary Orenstein is the host of The Cloud Computing Show.

Photo courtesy of Flickr user michaeldbeavers.

Related content from GigaOM Pro (subscription req’d):

• How Regulated Industries Can Move Toward the Cloud
• Is the Future of Enterprise Completely Open Source?
• Why Labor as a Service is as Cloudy as it Gets

However, there’s frequent misunderstanding of the layers at which companies are operating, which leads to misconceptions over which collaborate and which compete. Matt Asay’s recent article, for example, gave a good overview, but made the too-common assumption that companies compete simply because they both do something with Hadoop. Another article about the emerging big data “SMAQ” stack nailed the concept, but didn’t explain how to think about each vendor, where they fit in relation to one another and how they interconnect.

It can also be said that lack of a vendor ecosystem definition is hurting adoption. A recent survey revealed that the largest challenge facing enterprises considering Hadoop is the steep learning curve. Misunderstanding the richness of layers and categories of a vendor ecosystem contributes to the steepness.

As we roll up to the next big Hadoop event, it’s time to formalize the emerging Hadoop-based Big Data solution ecosystem as it is today and set the stage for where it going. At Karmasphere, we define and delineate the layers and categories of product in a pretty familiar ecosystem of three layers: Infrastructure, Data Management and Big Data Applications.

All three layers are necessary to handle the unprecedented volumes of data that need to be turned into meaningful results. The results are extremely varied, whether looking for a friend, getting a movie, book or professional colleague recommendation, understanding the spread or triggers of disease, detecting fraud, comprehending the behavior and buying pattern of a customer; the opportunity and competitive advantage inherent in burgeoning data is vast. The Big Data solution ecosystem already handles these and many other analytics today.

For the ecosystem to function, it first needs to address the different layers of Hadoop. As a collection of Apache Foundation projects, Hadoop includes the core MapReduce and distributed file system projects. It also includes Chukwa, HBase, Hive, Pig and ZooKeeper. Such a mesh of projects forms the essential cylinders of Hadoop-related power. To ultimately deliver value to today’s enterprise, these projects each need to work well within private data center or public cloud infrastructures (or both simultaneously). Cooperation with existing data repositories is key, and tools are required that turn collections of open-source projects into highly valuable, prime-time manipulators of vast amounts of data.

Vendors within the ecosystem’s Infrastructure Layer provide hardware and software that do commodity processing and base storage. This includes cloud providers and private data centers. The Data Management Layer in the Hadoop ecosystem includes more “under-the-hood components.” In initial Hadoop deployments, this layer often contains some, many, or all of the Hadoop open-source projects sourced from the Apache Foundation itself or from companies that make their own distribution of Hadoop like Cloudera, IBM and Yahoo.

Alternatively, a cloud provider like Amazon Web Services offers Hadoop as a service on top of its basic compute and storage infrastructure. Hadoop’s incorporation within the enterprise data fabric is being further enabled and validated by connectors to traditional RDBMS and data warehouse products, which are also actively becoming important members of the ecosystem. Admin tools, some coming from Hadoop distribution providers, some likely to come from other vendors, enable provisioning and management of Hadoop clusters for enterprise operators and administrators.

In combination, the Infrastructure and Data Management layers of the Hadoop ecosystem create a giant data power grid for the twenty-first century enterprise. To manage and exploit that power, the Big Data Applications layer is where you find the software for developers and analysts. Developer tools, software for technical analysts or business intelligence applications are all coming to market to help data professionals harness the power of the Hadoop to transform, analyze and speedily deliver meaningful results. Karmasphere, Datameer, IBM and traditional business intelligence vendors are all adding value at this layer.

Each new vendor product and service announcement adds more richness and choice for Hadoop users. This ecosystem is helping accelerate adoption and better support of real-world business requirements and the ability to harness the power of big data today and for many years down the road.

Martin Hall is a co-founder and CEO of Karmasphere.

Source: Karmasphere

From the I-was-going-to-conduct-this-research-but-someone-beat-me-to-it department, commercial Hadoop startup Karmasphere today released the results of a survey (PDF) of 102 Hadoop developers regarding adoption, use and future plans. The results provide some interesting insights into how Hadoop grows within organizations and underscore its status as an extremely valuable, but none-too-simple analytics tool. Of course, the latter characterization is why ISVs like Karmasphere, Cloudera and Datameer exist: to make millions by reducing the Hadoop learning curve.

Among the key results:

• Sixty-eight percent of deployments begin as skunkworks projects, with 86 percent advancing to active development or production environments within a year.
• The top three reasons for using Hadoop are data mining for business intelligence (19 percent), lowering the cost of data analysis (15 percent) and performing log analysis (13 percent), although uses like ETL (11 percent), scientific research (10 percent) and better utilizing unstructured data (9 percent) aren’t far behind. The longer organizations use Hadoop, the more valuable they find it and the more uses they find for it.
• The number of Hadoop developers looks to rise by between 50 and 60 percent within the next year.
• Java is the dominant language (86 percent), with Pig and Hive sharing the No. 2 spot at 44 percent each (multiple responses were allowed).
• The steep learning curve (44 percent) and hiring qualified people (34 percent) top the list of general challenges, while debugging Hadoop jobs (63 percent) and monitoring Hadoop jobs (47 percent) top the list of programming challenges. Seventy percent of respondents feel that these challenges will have a major-to-moderate effect on growing or expediting their Hadoop deployments.

Based on what I’ve seen and heard about Hadoop, these numbers seem accurate. They’re also the reasons why the above-mentioned startups are receiving a lot of attention from all types of users and vendors, and why the ecosystem of commercial products supporting Hadoop keeps on growing. Hadoop is arguably the most mature tool for analyzing large volumes of unstructured data, and these numbers suggest it’s also the most capable. When commercial products evolve enough to mitigate the learning curve and overall lack of skills, watch out.

Related content from GigaOM Pro (sub req’d):

• Why the Hoopla About Hadoop?
• The Incredible, Growing, Commercial Hadoop Market
• Why the Time Is Right for Hadoop

This isn’t because Microsoft is somehow smarter than Apple, but rather because the underlying dynamics of the technology industry have fundamentally changed. In brief, the technology world is increasingly embracing “write” communities, as Jono Bacon calls them, not simply “read” communities. Open source may have kickstarted this trend, but open APIs and open data are taking it to new heights.

Read communities aren’t characterized by a dearth of developers, but rather by what those developers can do on a given platform. After all, few can claim to sing to developers as eloquently as Microsoft CEO Steve Ballmer does, but there’s a (big) difference between talking to developers and letting them talk back. In your code. On your platform.

As noted, it’s telling that the shelf life of Apple’s dominance is much shorter than Microsoft’s decades-long dominance. Microsoft, after all, never had to deal with competing write communities, as Apple does with Google Android. Major developers like Facebook find Android more flexible: It allows them to write into and draw from the platform the capabilities they need.

Hence Apple, once the no-brainer first choice for developers despite its heavy hand on the development process, is increasingly losing out to the more free-spirited Android, which analysts see claiming over 50 percent of the smartphone market in just a few short years. Apple has responded by loosening its grip on iOS application developers, but it may be too little, too late.

Android isn’t perfect, of course, and still suffers from a worsening fragmentation problem. But its comparatively open nature makes it an inviting alternative to the closed iOS development. As but one example, try to get meaningful analytics data out of the iPhone. If you’re Apple, you can do that. If you’re anyone else, particularly Flurry, you’re out of luck.

Apple giveth, and Apple taketh away.

Contrast that with Google Android, which has an open-source logging/analytics tool developers can use called Logcat. Android is open source, which prevents Google from exercising control over how developers collect analytics data on Android devices. While one can make an argument that it’s good to have potentially sensitive analytics information guarded well by a responsible party like Apple, given Apple’s record of somewhat arbitrary and heavy-handed control over its platform, I’d vote for freedom on this one.

This isn’t just an Apple vs. Google story, either. It’s just one example of how innovation happens generally, no matter the industry. As Steven Johnson points out in The Wall Street Journal:

[I]deas are works of bricolage. They are, almost inevitably, networks of other ideas. We take the ideas we’ve inherited or stumbled across, and we jigger them together into some new shape. We like to think of our ideas as a $40,000 incubator, shipped direct from the factory, but in reality they’ve been cobbled together with spare parts that happened to be sitting in the garage.

The problem, as Johnson goes on to highlight, is that governments have largely pursued innovation in the past 100 years by doing the exact opposite of what is actually required to foster such innovation. The same is equally true of individual corporations like Apple or Microsoft:

[I]ntellectual property, trade secrets, proprietary technology, [and] top-secret R&D labs…share a founding assumption: that in the long run, innovation will increase if you put restrictions on the spread of new ideas, because those restrictions will allow the creators to collect large financial rewards from their inventions. And those rewards will then attract other innovators to follow in their path.

The problem with these closed environments is that they make it more difficult to explore the adjacent possible, because they reduce the overall network of minds that can potentially engage with a problem, and they reduce the unplanned collisions between ideas originating in different fields. This is why a growing number of large organizations—businesses, nonprofits, schools, government agencies—have begun experimenting with more open models of idea exchange.

It’s this sort of open exchange of ideas and code that leads to economic historian Eckhard Höffner to conclude that Germany closed the gap on England’s industrial revolution in a short span of time due to the wide-open nature of the country’s publishing market in the mid-1800s. Weak copyright law enforcement sent innovation into overdrive in Germany, while a comparative monopoly on publishing in England stymied that country’s early industrial lead.

Eventually, Germany followed England’s lead, and innovation slowed there, too, but ramped up in the United States, where “borrowing” the works of Dickens and other great European authors, not to mention technological inventions, was standard operating procedure. European creators didn’t like the Yankee “thieves,” but loose IP protection led to greater adoption of their works, industrial and cultural progress, and the authors still managed to get paid.

Since then, the industrialized West, including the United States, has increasingly clamped down on intellectual property in the interest of fostering it, but with the opposite effect. As numerous studies attest, patents and other intellectual property tools have slowed innovation, not accelerated it. Industrial innovation has accordingly moved to areas like Brazil and China where IP protection is light.

This isn’t just a matter for economists, but also for business strategists. It’s possible, for example, that Oracle’s integrated approach to product development will prove successful, but likely not over the long term. Such an all-consuming, go-it-alone approach breeds powerful enemies, including within one’s own customer base. It certainly creates distrust within the developer ecosystem.

Oracle may profess not to care, but competitors like Microsoft increasingly recognize that they must care. Software developer Dave Newman declares that “The .Net community operates in a non-collaborative vacuum,” and then announces he’s abandoning .Net. Microsoft can’t afford to lose too many Dave Newmans.

Neither can Oracle.

In today’s market, companies need community. They need adoption of their APIs. No company is smart enough to come up with all innovation on its own, so the best companies will create read/write platforms through which third-party developers have the flexibility and distribution to reach customers.

Open source is an essential part of this, but isn’t sufficient of itself to crown any particular vendor or technology king. Linux is rapidly taking over in the mobile market, but has yet to make a dent on the general consumer desktop. But the fact that open source isn’t sufficient of itself to decide a winner is no reason that platform vendors, specifically, and technology vendors, generally, shouldn’t be making the most of open source to enhance their attractiveness to third-party developers.

Related content from GigaOM Pro (subscription req’d):

• How to Market Your iPhone App: A Developer’s Guide
• The Real Impact of Facebook’s New Approach to Gaming
• Why Apple Should End Its Fight Against iPhone Jailbreaking

After all, it’s not as if office suites are playing center stage in technology innovation. Not even Microsoft, which has owned the office productivity suite market for decades, has bothered to release meaningful changes to the desktop version of Office this century. So why should we expect more from The Document Foundation?

Supporters will likely cite Microsoft’s dominance as the very reason to look elsewhere for innovation. Indeed, The Document Foundation has declared its aim to channel innovation back into the office productivity market:

The Foundation will be the cornerstone of a new ecosystem where individuals and organizations can contribute to and benefit from the availability of a truly free office suite. It will generate increased competition and choice for the benefit of customers and drive innovation in the office suite market.

Perhaps. But why start from the paradigm of 1980s technology? Nothing on the Foundation’s new website, or in any of its press materials, suggests that the Foundation’s purpose is to do anything more than free OpenOffice development from the control of one company, Oracle. There’s no discussion of the possibilities of integration with the web. Screenshots look an awful lot like the OpenOffice suite that LibreOffice claims to leave behind.

This isn’t surprising, given that the new LibreOffice has only recently divorced itself from OpenOffice, not to mention the Foundation’s own proclamation that it’s not looking to fork OpenOffice, but rather for “continuity” with its OpenOffice past. Given that it starts from the same client-heavy code base and mentality, how can it hope to truly liberate OpenOffice from the shackles of the desktop on which it was born?

If anyone is advancing the office productivity market, it’s Google Apps  or Zoho Office, which were born on the web. It’s unclear what a web-light, client-heavy Microsoft Office clone can hope to achieve in terms of real innovation. And why are we worried about replicating Microsoft Office functionality, which has long been the aim of the OpenOffice community? While some Excel spreadsheet jocks may live in Microsoft Office, very few of the rest of us give it more than a cursory glance on a regular basis. It’s not that we’re not engaged in “office productivity,” either. We just work differently now.

We email. We SMS. We Facebook. We IM. Or perhaps we crop photos in iPhoto  or make movies in iMovie. What we don’t do, or rarely do, is open a Word document to create a stale relic of communication. Business moves too fast these days to open attachments. Again, yes, there are people who live in documents, spreadsheets, and presentations. But these people are not you, most of the time.

Real innovation today is occurring at the intersection between cloud data and client-side code, as TripAdvisor demonstrates. And it’s happening in the very definition of rich-client applications, as such applications become more mobile and more web-friendly through the innovations of Strobe Inc. and others.

In short, there are far better uses of The Document Foundation’s developer talents than replicating Microsoft’s tired Office legacy. I think a better OpenOffice is a worthy goal, and support that. After all, enterprises will rely on Office and documents for years to come, even as they keep the green-screen terminals around to support outdated applications.

But the future belongs to the web, and The Document Foundation’s very name suggests a backward-looking focus, not the future focus that will keep it relevant. The web is built upon open source, and many of its most interesting innovations have arisen from the open-source community. I’d love to see The Document Foundation help move the conversation around “documents” to the web that is supplanting the need for relics of the way we once worked.

Note: My company, Canonical, supports The Document Foundation. The views expressed here are completely my own.

Related content from GigaOM Pro (subscription req’d):

• How to Manage Consumer-Grade Collaborative Tools in the Workplace
• Forget Syncing, Let’s Put Music in the Cloud!
• Cloud FUD Goes Mainstream, But It’s Still Misguided