The company serves 14 million web sites and reaches 1.3 billion unique users a month. On any given day sites using the service see 3 billion views. Together, those visits create 10 terabytes of data each day, leading the folks at AddThis to realize their data was the asset. As Hooman Radfar, founder and executive chairman of AddThis, put it, “Every week, we see the same amount of information that’s stored in the Library of Congress.”

For now, the company is selling its customers the ability to tap into the social graph those terabytes of data can generate, so site owners can see relationships between their content and how people react to their content. The end result will be new products that AddThis will be able to offer, as well as a data set that tracks sharing and connections over a large portion of the Internet. Such social data is attracting a lot of interest at the moment.

The sheer size of the data set and the real-time nature of tracking shares in real-time puts a lot of demands on the database, so the engineers at AddThis have managed to build something pretty cool: A data store that can handle both batch processing and real-time analytics. They call it Hydra, and it’s possible that in the near future they will open it up as a service, said Radfar.

Much like Google’s BigQuery or 1010data does, offering Hydra as a service would allow other companies to put their own information on Hydra and query it for analysis. As more and more of these on-demand and cloud-based analytics services become available, the question will soon be what kinds of data and queries will fit best with each. Generally moving terabytes of data from one cloud to another will cost a lot and take hours if not days.

Clearly, social graph data is what Hydra does best at the moment, but I talked to Matt Abrams, who is an engineer at AddThis, to understand how Hydra works and how it scales to understand what else might mesh.

Hydra is a custom-build distributed file system and distributed database. It’s stored on four clusters of about 80 servers each in co-location facilities. When asked about the decision to go in-house, Abrams said that at a certain point it just became too expensive to keep everything on Amazon Web Services. The service needed fast IO and ensuring that became difficult and costly to do when AddThis didn’t control the infrastructure.

The data is sharded in multiple ways to ensure that a query for different elements can be performed quickly. So a single piece of data is replicated across different servers to make it faster to find in specific queries and is also replicated for backup in case the storage fails –something Abrams says happens to about 30 percent of his storage hardware right out of the box.

The Hydra platform obtains speed by having the right data ready in a variety of places for the query, much like you or I might keep a flashlight in every room of the house so we can get to it if the power goes out. To keep all these pieces of data stored in various hash sets requires 1.8 petabytes of storage.

For the deep dive, check out this three-part series Abrams wrote a year ago when AddThis had half the data it has now, or check out the more recent post on High Scalability.

But the result is a platform that AddThis uses to build out a social graph that rivals Facebook’s in breadth — and one it makes available to its customers so they can one day do cool things such as offer personalized web sites for users with little effort on their part.

Image courtesy of Flickr user tpholland.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• A near-term outlook for big data
• 10 ways big data changes everything
• NewNet Q2: Google closes the quarter with a bang

Content-sharing toolmaker Clearspring today introduced OExchange, a way for services to inspire link-sharing without establishing formal relationships to get their logos next to Facebook’s widespread “like” buttons. OExchange is an open protocol that helps content sites dynamically discover relevant sharing services, send them content in a format they can accept and interpret user sharing preferences as explicitly stated.

On board for OExchange are Clearspring, Digg, Echo, Google, Instapaper, LinkedIn, Microsoft, Posterous, PrintFriendly, Springpad, StumbleUpon, Webs.com and yfrog. Google Buzz and Posterous already have parts of the protocol live on their sites. As for the remaining big players, Clearspring CEO Hooman Radfar said his company is in talks to bring on Twitter and Yahoo, and has also approached Facebook, though the latter seems much less likely to join in.

OExchange does not dynamically personalize user sharing preferences based on where they’re logged in like XAuth, the similar initiative from Meebo (see disclosure below) that also gathered a us-vs.-them non-Facebook posse. Nor does it carry a user message along with the link (in the vein of Facebook’s “like” and “recommend”), but that is in the works.

OExchange seems like a further-along version of OpenLike, the grassroots effort that was set up in direct opposition to Facebook’s launch of its web-wide “like” buttons at f8. However, OpenLike is a bit simpler with much less adoption so far; rather than a full protocol, it’s an open-source alternative to Facebook’s tool. Radfar insisted that unlike OpenLike, OExchange had been in the works far before f8 and isn’t a reactionary play.

“The timing lined up that Facebook was making this move to colonize the web, but we couldn’t have pulled this coalition together if it was reactionary,” said Radfar. “The beauty is there’s no central arbiter like Facebook; this is a completely distributed system.”

“Facebook is setting user experience expectations today, but it will be interesting to see how third-party standards break apart the monolith,” added Clearspring community manager Justin Thorp, comparing Facebook to AOL’s centralized system before the web overtook it. Thorp argued that Facebook’s long-term dominance is far from inevitable. “IE still has 60 percent market share, right? Imagine if Microsoft put identity in the browser,” he said.

Related content from GigaOM Pro (sub req’d):

There’s No Stopping Facebook

Disclosure: Meebo is backed by True Ventures, a venture capital firm that is an investor in the parent company of this blog, Giga Omni Media. Om Malik, founder of Giga Omni Media, is also a venture partner at True. Please also see the disclosure in Liz’s bio with regard to Facebook.

But do we really need the option of sharing on so many random services? What if those sharing widgets had a bit of intelligence about what a certain user already prefers, and just showed her the options she’s likely to use? That’s the aim of a new open platform called XAuth that’s being released on Monday by Meebo, with the support of Google, Microsoft, MySpace, Yahoo, JanRain, Disqus and Gigya.

XAuth detects whether or not a user is logged into a service, or has recently done so, by looking at his browser history by checking for a local XAuth token left using the XAuth JavaScript API. Then a publisher can prominently display those preferred social networks and communication sites, in the hope that seeing their favorite logos will make users more likely to connect and share.

The tool will clearly be useful for Meebo, which offers a service to encourage sharing called the Meebo Bar that runs alongside the bottom of hundreds of sites, reaching an estimated 120 million users per month. But XAuth could be all the more useful if many web services choose to use the same code, which Meebo plans to open-source and place under the care of the Open Identity Exchange or the OpenID Foundation. It’s promising that so many large web services have already signed on, if only for the inaugural press release.

However, Clearspring is absent from that list of partners, as is Facebook, which would prefer that everyone used Facebook Connect. Facebook is expected to launch its own sharing toolbar that would compete with Meebo’s as early as this week at its f8 developer conference in San Francisco.

I think visiting a web site that knows I’m logged in elsewhere could get a little creepy. Meebo CEO Seth Sternberg said that XAuth won’t know anything about a user’s email address or login, and that it’s a way to pass data around with privacy restrictions and white-listing intact. Further, he said, XAuth could be extended in ways that make it more worthwhile for users. For instance, if you were reading an article on a newspaper’s site, it might be nice to see a feed of your social gaming activity pop up, alerting you to head back to another site to resume playing.

Related content from GigaOM Pro (sub req’d):

Social Advertising Models Go Back to the Future

Meebo is backed by True Ventures, a venture capital firm that is an investor in the parent company of this blog, Giga Omni Media. Om Malik, founder of Giga Omni Media, is also a venture partner at True.

1. Ease of integration — Is it easy enough for you to embed the widget code into your web site or blog so you can offer sharing functionality to your readers?
2. Ease of use/performance – Is it easy and fast for your readers to share your content with others via multiple options such as email, their favorite social networks and microblogs?
3. Analytics — Does it offer you robust insight into the sharing activity on your site or blog?

Having been a longtime user, I can tell you that my initial response to the above questions about the ShareThis service would be “yes” on No. 1 and No. 2, and “sort of” on No. 3. But then I learned about the changes to the AddThis service, and I think that it may win me over as a customer. Here’s why.

At the time I started using ShareThis, AddThis was an underfunded tech startup that looked more ad hoc than professional. I was swayed by ShareThis’ polish when it entered the content sharing space. Today, AddThis gets the added professional muscle of Clearspring, a tools and platform sharing company; this acquisition bodes well for the AddThis application.

AddThis is a community-driven platform with the goal of letting people “share anything to anyone anywhere.” It no longer limits visitors to sharing content only to a small hand-picked list of sites. It now wants to be a “switchboard for the web,” which means that the providers and networks that show up when your visitors click on the AddThis button are dynamic and flexible.

The new AddThis menu is a “smart menu.” If you are visiting sites using AddThis, you’ll get tagged with cookies that the AddThis application can read at the next AddThis-powered site you visit so it gets to know you and makes a note of your network preferences. So if you tend to share content mostly on Evernote and Posterous, instead of Facebook and Twitter, AddThis will now customize the menu you see to see your most frequently used networks instead of a pre-selected hierarchy of services.

Another smart aspect to the newly enhanced AddThis service is the ability for any social network to get added to a vast and growing list of potential places where visitors can distribute content to, which makes AddThis more open. Users can also recommend additional services; AddThis will then contact those services to get listed in the directory.

AddThis believes these enhancements to the application are good for the user and good for the publisher. As a content publisher, I see its point. As a user, I like the idea of a smarter menu so I don’t have to sift through additional screens to find my favorite sharing forums. And smaller service providers can also benefit by not being automatically relegated to the “last screen” of a sharing menu.

Another addition to the new and improved AddThis is translation of the tool into 50 languages.

Some other benefits of using AddThis over ShareThis include pure market share. The company says that according to Comscore, AddThis has about three times the reach of ShareThis in terms of sheer unique users.

On the analytics front, the company says it has the competitive horsepower to crush the competition over time. I’ve just signed up for an AddThis account to start testing it out; analytics are a big deal to my company, so we’ll see how that measures up over time.

So which one do you use? AddThis or ShareThis? And why?

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Millennials in the enterprise, part 1: strategies for supporting the new digital workforce
• A 2011 NewNet Forecast
• What Enterprise Software Vendors Could Learn from the Consumer Space

However, Adobe is tardy in its effort to offer this capability, as companies like Clearspring already tout widgets and applications in Flash.  Plus, Facebook opened up its API to developers in 2007, so the rush of developers looking to build apps on the social network has dried up over the years. Because Flash doesn’t interact well with the various characteristics of each social network platform, a company called Transpond (formerly iWidget) has offered a service for the last year that helps developers make applications that accommodate the languages of each.

Flash isn’t supported on the iPhone, one of the hottest mobile platforms out there, so iPhone app developers who want to use Adobe’s new service will have to build special versions of their applications that don’t use Flash. The service is supported on Symbian and Windows Mobile phones, however.

Today’s launch of Adobe’s distribution service is another sign of the San Jose, Calif.-based company trying to strengthen its position in the advertising market. Last week, Adobe acquired web analytics company Omniture for $1.8 billion, a move that many viewed as a way for Adobe to better track advertising sales.

Gigya and Adobe announced their partnership back in March, saying they joined forces to work on a solution for publishers and advertisers dubbed Project Radiate that was set to release this summer. Gigya received $1 million in Series C funding led by DAG Ventures in October 2008. While Adobe is late to the game with its distribution service, making things easier for developers is always a positive.

Image courtesy of Adobe

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• The state of cross-platform media measurement
• Connected world: the consumer technology revolution
• Will Games Help Google Figure Out How to Be Social?

All this negative thinking is a result of pondering over the $18 million in Series C funding raised by Mclean, Va.-based Clearspring from New Enterprise Associates (NEA), Novak Biddle Venture Partners and other current investors. Competitors include Musestorm and Gigya, that have come up with platforms that allow publishers to publish, distribute and monitor their widgets.

Their total funding to date is $35 million. Other investors include Steve Case, former CEO of AOL who is said to have put in a big dose of cash. One of the other investors is said to be Adobe Systems, mostly because Clearspring is a big proponent of Flash-based widgets, and at one time ADBE was rumored to be buying the company.

More than the funding, the pre-money valuation is what made me gulp hard. It is said to be in high double digit millions – somewhere north of $60 million. Why is that? Because the company is said to have some astonishing numbers…

* served over 34 billion widgets to date; that’s up from 3 billion in May of 2007
* serves nearly 4 billion widgets per month
* has 126 million unique views per month worldwide
* had 50 million unique views per month in the U.S. alone; that’s up 6.5% from February

…. not that they really matter. Because they really don’t tell the story about Clearspring. First time I met Hooman Rafdar, CEO & Co-Founder of Clearspring was back in Fall 2006 at the Widgets Live conference that was co-orgnaized by me. His idea was that as widgets phenomenon spread, the widget makers would need infrastructure – platform and analytics – to publish and distribute the widgets as they grew in numbers. Then later they started old media companies and sports leagues with development of widgets and then they launched a widget ad network. So from that perspective this company is a infrastructure provider + consultant + ad network. In other words it doesn’t really own the widget impressions, per se. Not that the company has to!

Radfar didn’t give me any revenue numbers (typically no start-up talks about it) or divulged the percentage of widgets on his network that carry ads. He did mention $5 CPMs, which is much higher than the CPMs being reported by some of the other social media operators. Many of the widgets that carry CPM ads, have low CPMs – somewhere in the $0.15-to-$0.20 range, down from nearly 40 cents in late 2007.

I think there in lies the bigger problem with the widget sector as a whole: no one seems to have come up with a business model that makes sense. The widgets have become too successful, and have created too much inventory that keeps growing but doesn’t perform as well, forcing advertisers to re-adujust their efforts. Yet you have companies like Sprout getting funded, which tells me that there is a mismatch between market reality and VC expectations and lack of understanding of this sector.

The big opportunity is in analytics that lead to a better widget advertising system does a great job of behavioral targeting. The other opportunity is in developing new kind of ad formats that can be scaled. Radfar’s company plans on building better analytics that will improve the widgets and widget-based advertising. Lets hope for the sake of Radfar’s investors he is right and gets there. That is before Google does.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• The future of mobile: a segment analysis by GigaOM Pro
• Fantasy Football’s Very Real Digital Business
• Survey: How apps can solve photo management

Economic Slump Hurting Cable, Not Satellite; perception of satellite’s HD programming as superior helping growth. (Reuters)

New TV Series Scarlet a Marketing Hoax; elaborate ad campaign promotes LG’s new TV sets. (The Wall Street Journal)

Can DVD Sales Rebound? DVD sales held steady last quarter at $3.5 billion, but high gas prices are putting the squeeze on the shiny discs and could impact Blu-Ray adoption. (Variety)

National Geographic Channel Changes Up; site now allows video previews of upcoming episodes and makes 800 videos available. (MediaWeek)

Sling Media Updates Mobile Player; new versions add support for more handsets, improve streaming quality and add support for more set-top boxes. (TV Squad)

TVGuide.com Sees Traffic Growth; site says it is getting 12 million unique visitors a month, up 40 percent over 2007. (TVWeek)

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Who and what to watch in the new era of the living room
• Smart TV forecast: gigabit Wi-Fi in the living room
• New Business Models For Pay TV Services

The story has consistently been framed as an issue of getting users rather than making money. For a while the excitement was around MySpace, then Facebook opened its APIs and the widget makers rushed into a promised land of adoption and media coverage. But as social networks and widget platforms, such as Clearspring, begin to explore monetization strategies, and as widget makers such as Slide pull in large amounts of capital, the time to prove that widgets can make money is here.

There are few ways companies are doing this, with the most obvious one centered around using a widget as a billboard. If you already make money from your content and can offer something interesting, a widget is a good way to extend your brand and get people to watch your new television show, read your content or see your movie.

Examples of this type of widget includes the film widgets from the likes of MovieFone, or the widget tools The Wall Street Journal and New York Times offer via Netvibes. In fact, Netvibes is looking to take this a step further by charging people for better placement of their widgets on its personalized home page in a manner similar to Google’s AdSense program.

If you buy into the idea of widgets as a billboard, or that some type of advertising could eventually be shown on widgets, then the idea of knowing how many people are snagging or looking at your widget — and how often — becomes interesting. Services such as ClearSpring, MuseStorm and Triggit provide these sorts of tracking and analytics services and already charge — or plan to charge — the widget makers a fee. Those that charge widget makers serving advertisers such as major publishers and sports leagues are in luck. Those that charge consumer widget makers without a clear revenue model might find themselves wondering what to do if their customers don’t make money.

But running analytics and platforms can be a challenge when there are no set standards for widget monetization. Much like the early days of banner advertising, when people just sold whatever unused space they had, the world of widget ads is a Wild West with no set sizes, rules about placements, or even defined success metrics.

Many company executives and their venture backers in the space believe that the Interactive Advertising Bureau, widget makers, social networks and publishers will get together in the near future to hammer out standards, from something as simple as how to determine the appropriate pixel height and width of an in-widget ad to how to figure out when a widget ad is successful.

Some of the metrics currently being considered include charging advertisers based on: the number of people who see the widget (cost per thousand); the number of people who actually interact with the widget (cost per click); and the cost per install, which measures how many people snag the widget for their own page. As much as analytics can tell people about what happens with a widget and where it goes, it’s still hard to say how successful it has been unless it’s tied directly to a purchase. And frankly, even those numbers can be tweaked.

This standardization will likely emerge once a successful model does. So far, Beacon isn’t winning rave reviews and it’s worth waiting a bit to see what some of the more consumer focused widget-creation platforms like RockYou and Slide can contribute with their revenue models, but my bet is this will happen sooner rather than later (like 2009).

Much like Google monetizes their users by giving them something that enough people want (search results, free directory services via GOOG411) and then painlessly taking something Google can use (money from advertisers, a sampling of accents for its voice recognition efforts), a widget monetization strategy needs to keep users happy while also giving something of value to someone.

The challenge facing most widget makers that target consumers is figuring out what it is their widget-watchers have that’s valuable — and who might want it. Those focused on a more ad-centric model are going ahead with their plans to use the widget as a more interactive billboard, but companies like Slide and RockYou may find that strategy more difficult.

Even with the uncertainly, I agree with Satya Patel of Battery Ventures (and a former Google executive) who says, “While widgets have generated a lot of hype, they are certainly an important vehicle for content, commerce, communications and advertising and they are here to stay. There will be several large businesses built in and around the widget market.” But I think we’ll also see a lot of them fail.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• 12 tech leaders’ resolutions for 2012
• Survey: How apps can solve photo management
• Sector RoadMap: Content personalization in 2013

[Widgets] are all the rage on the Web. Marketers are thinking of ways to use them to sell ads, and venture capitalists are mulling investments in the hottest widget makers. The stampede reached a roar this month, when market leader MySpace and Facebook expanded their services for targeted ads, including widgets designed by marketers.

But look at the numbers for investments in widget distribution platforms and widget suppliers, and it becomes clear that this not a stampede but more like a Sunday stroll through the widget park.

A widget is just an embeddable bit of web or desktop content wrapped up in a nice user interface. You can put widgets on blogs, social networking profile pages, or on an Ajax start page, though not all widgets can be embedded on all kinds of pages.

Though many companies might use widgets to promote and distribute their content or features, there are two categories of startups that, to me, represent relatively pure bets on widgetization. The first is widget development, distribution, and packaging platforms like Clearspring or Widgetbox; the second, widget suppliers like RockYou or Slide.

I’m leaving out many other kinds of widgets and widget startups, of course. You can have desktop dashboard widgets, widgets to be embedded on blogs like Blogburst or MyBlogLog or Lijit, Ajax start pages that support widgets, video syndication widgets, and more.

But taking a look at widget platforms and social networking widget purveyors, we can get an idea of whether 2007 has seen a stampede towards widgets or perhaps something more genteel. I’ve gathered funding data for widget distribution platforms and widget distributors in the table below.

This doesn’t represent all the funding for such companies. Fox Media is building SpringWidgets, so that investment doesn’t appear here. Widgetbox has been rumored to have taken additional funding from Sequoia. And some companies have reported investments but declined to provide specific figures. Still, this gives an idea of the magnitude of interest in the space.

You could also argue that companies like YouTube and PhotoBucket are widget companies — and if you added in their acquisition prices the numbers do start to get really big, really quickly (partly, of course, because acquisition prices represent the entire value of those companies). But I think those two are examples of how embeddable web content can help promote a broader web service. iLike might fit better with those, too.

The total funding for these companies is just under $60 million, a respectable number but by no means a stampede towards widgets. This could reflect the uncertainty around how to make money with widgets and widget platforms. Or perhaps it’s more evidence of investor sobriety.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• 12 tech leaders’ resolutions for 2012
• Fantasy Football’s Very Real Digital Business
• Survey: How apps can solve photo management