How Internet Content Distribution & Discovery Are Changing

55 Comments

[qi:073] Every few years, the Internet — and, by extension, the web — gets bigger and better. As publishing tools get better, we share more content online. As we publish more content, more services emerge to help us find and consume that content. In the early days of the commercial web, it was magazine-like entities such as Hot Wired. Then came search-engine directories and portals such as Lycos and Yahoo.

Towards the end of the last century, digital content started to grow exponentially, and with that, arose the need for a super-search engine like Google. Larry Page and Sergey Brin helped changed user behavior by making it easy to seek, search and consume any content. They spent billions on their infrastructure and made search better, faster and easier — so easy that, like a drug, we got hooked on Google’s search. (Related: Google at 10: Larry, Sergey & me.)

And now we are seeing yet another subtle change in people’s behavior and how content is discovered online. It is happening because of three major reasons, as I’ve detailed in previous posts:

1. The web is transitioning from mere interactivity to a more dynamic, real-time web where read-write functions are heading towards balanced synchronicity. The real-time web, as I have argued in the past, is the next logical step in the Internet’s evolution. (read)
2. The complete disaggregation of the web in parallel with the slow decline of the destination web. (read)
3. More and more people are publishing more and more “social objects” and sharing them online. That data deluge is creating a new kind of search opportunity. (read)

When I first met Dave Winer, he explained to me the concept of “river of news” — where stories would flow as a river and you would dip in and drink what you could. It made absolute sense — after all, as bandwidth (and connectivity) grew, we would be using the web more, including sharing more and more objects. A few years later, the guys at 30 Boxes showed off a time line that essentially aggregated the life of my entire “social network.” As they say, the future always takes a little longer to arrive. In 2009, it has. These trends are showing up as Facebook’s status feed or Twitter streams. John Borthwick of Betaworks defines the stream as:

…a real time, flowing, dynamic stream of information — that we as users and participants can dip in and out of, and whether we participate in them or simply observe, we are  a part of this flow. Overload isn’t a problem anymore since we have no choice but to acknowledge that we can’t wade through all this information.

Today we are sharing links, text messages and photos in these streams. Videos or video life streams come later. (About two years ago, I wrote a column for Business 2.0, “Reach out and Twitter someone“, pointing out that we will all be streaming life moments as more and more bandwidth is available both at home and on the go.) These streams are more relevant mostly because the context comes from our social graph.

I find that when folks share stories, links, photos and videos on Facebook, a majority of them are useful. The idea of social shame acts as a a way to share better to overcome the problem of plenty that comes with the rapid growth of the Internet. Sharing better means a higher likelihood that people will actually visit the links being shared on their social networks.

A few months ago, Liz Gannes, writing for NewTeeVee, pointed out that Facebook and Twitter are becoming major sources of traffic for bloggers, such as Perez Hilton, and online video sites, and are growing really fast. This means that content can “go viral” more quickly than ever before, as we saw recently with the tremendous trajectory of views for unlikely international hero Susan Boyle.

We have also seen Facebook, Twitter and FriendFeed become referrers of traffic to our network of blogs. Mark Cuban (of the Dallas Mavericks) and Borthwick (of Betaworks) have noticed similar trends. “For the first time ever, more people are finding my blog from Twitter and Facebook referrals than via Google. The total number of people coming to my blog is increasing. The percentage of people who find it via Google is declining. Significantly,” writes Cuban.

Like Borthwick and Cuban, I believe that we are seeing a disruption of behavior in how people use the web. For now, it is still an early adopter phenomenon, but with 200 million Facebook subscribers and Twitter’s rocket-like growth, it is only a matter of time before these two sources become major web content discovery engines. How these changes will impact Google’s business remains to be seen, but one thing is for sure — many Web content discovery engines that exist as destinations (Digg, for example) are going to face challenging times.

55 Comments

Article Distribution

We also have a client that is receiving more traffic from NON-Google sources. We get more traffic from Facebook and other social media sites than we do from Google. In our article marketing we have started writing the articles not for the purpose of building backlinks to the website, but rather to get direct traffic from the articles themselves.
Times they are a changin!

Nestor

Great article!

I agree with Adam about the social bookmarking (Delicious, Digg, Redit, etc) because it is about community and not about destination. I found your article because I’m following a person in delicious and he saved your post.

What I like from these sites is the following:
1) Article’ sentiment comes from humans and not from machines
2) There is a qualified fisherman who fishs the relevant content for me. I follow people that i consider subject matter experts in a specific area and consume the content that they read.
3) I agree with your assesment about Twitter and Facebook because the community component, what makes relevant the content flowing in the social media river is that it comes from people that I know and in some way I have sort type of affinity with :)

Thanks for sharing

Andrew @ Personal PPC Coach

Very interesting quote. We have to admit to ourselves that we can’t process all the information so we don’t worry about overload. I guess I’ve always been reluctant to become a facebook junkie because of this overload. Interesting.

James

Om – it seems as if many people miss the connection between technology adoption and generational traits.

Twitter and Facebook adopters are, primarily, 11-34 (netGen, Trophy Gen, Baby Boom Echo Gen). This group is very socially aware and status conscious, which drives initial adoption. A secondary trait is that they trust their social network more than their ability to do objective analysis. So, your thesis that media distribution will be driven by social networks is founded on solid market traits.

Google requires more work – pogo sticking to find effective keywords, analyzing link / pages to meet needs, etc. Referrals are fast and have the benefit of inherent context. They work offline too ;-)

Excellent post.

Ina

It’s an interesting observation, the one that you make.
Indeed the way we use, share and create content on and for the Internet is changing continuously. And it will continue to do so as Internet is part of our habitats now and we need to keep things dynamical if we want to survive.

Maddy

Really enjoyed this post. I am also interested to see how the shift away from the conventional search engine will impact our future search habits. The fact that search.twitter.com offers a great deal of value in Twitter’s relative infancy is truly astounding. Google’s reaction to this growing trend will be very interesting.

linekill

Great post.

I’ve always wonder how the Internet and the web will change together with the advancement of techniques and ways. And I would agree that Facebook, Twitter, etc now direct more traffic than google. Specially on blogs. And at the same time, blogs are starting to define the web. I can still remember when blogs are nothing but personal thoughts, posts and articles but now, it seems to be molding the web into something I have no idea of.

The Internet is slowly evolving, like a recently-discovered continent. It started out, then people went to visit this land with hopes of a better life, the turf war, the gold rush, and now it seems that countries are starting to form in the continent. The next would be cities, clear divisions, rules and everything…

Stream may be the start of web 3.0. The Internet – Less Like The Wild West.

Ken

The whole problem the river of news idea is that that river can become an engorged beast running so fast you can’t keep up. If I’m not checking my Facebook feed every few minutes or hours (depending on the day), I’d have to dig back into the previous pages (no time for that) and try to catch up. It might be that this is the new reality but it doesn’t mean it’s the correct one. These newer (Web 2.5?) feed systems share the biggest fault that RSS feeds have: it’s one big generic feed. Just like everyone loves to have a tag cloud sitting off to the site with different sized words based on popularity, these feeds need to accentuate the links/blogs/video/whatever that are more likely to be of interest.

Thing is, I don’t know if I want to keep setting up my preferences with each new “hot” social site that springs up. The idea that “overload” isn’t a problem is bs. It’s not that it’s no longer a problem. It, like every nuisance human-made or otherwise, is that we as humans have adapted in the best way we can. We go apathetic and start to ignore. Oh, there’s another video of a kid being cute with spaghetti all over himself. Oh, look, another news article about the death of yet another company? Great, file it under Whatever. Not everyone in my feeds has a relevant or otherwise interesting article out there. Maybe you’re lucky and you have a lot of people with something interesting to say but the majority of what’s out there, shame or no shame, is a lot of noise. I think the assumption is that you’ve surrounded yourself with relevancy when in fact you’re an outlier. For the general public, much of that river is filled with junk.

Give it a couple years and I’ll bet there’ll be a new site out there that people start flocking to (because they’re tired of the “old” sites). The question will be though, after all the friends we’ve added and tweets (or twits as I like to call them) we’ve logged, will the barrier to entry for new social sites be that we just don’t want to deal with starting the info overload all over again? Most likely not…unless there’s an import tool or a feed that ties into the feed that ties into some other feed so that what started with a stream ended up as the Amazon…pirana and all.

Maybe it’s finally time for that AI to be the one taking care of the info overload river by making it a stream of relevancy mixed with the ability to open the flood gates and peer into the river behind if you’re really bored.

swag

Do you honestly think Susan Boyle was a Facebook- or Twitter-generated phenomenon? If so, I think you’re dosing the Kool-Aid a bit much there.

Neil

The ‘real-time web’ meme really is all the hotness right now – but I’m sure we’re all hoping it doesn’t lead to home pages fueled by auto-updating newsfeeds. We might not realise it just yet but the HTTP request is a beautiful thing; I like visiting a ‘page’ safe in the knowledge that the content won’t scroll out of view without me giving it the go ahead to do so. Batch processing definitely has a place. Facebook could be on the right track with that ‘4 new stories’ ajax link.

On the other hand, Google results are starting to feel a little stale, and Twitter’s search engine is fast becoming a more preferable first port-of-call for finding stuff. If only Firefox would allow us two search engine boxes in the chrome customisation options….

Todd Spraggins

“Overload isn’t a problem anymore since we have no choice but to acknowledge that we can’t wade through all this information.”

Wow, IMHO this is going to introduce a form of psychosis for many, where decisions and relations are formed on temporal snippits of information. Acceptance is a large step in mental processing and I have to think that this is not going to be a trivial undertaking for many to acknowledge that there are gaps in our ability to acquire or absorb all the relevant information; especially for information-aholics.

Jonathan Mendez

i got here from twitter.

great post. it has become clear that Google is recovery – 40% of queries are repeat, 70% are navigational, etc. Just like the great library it was built to be you don’t walk in a library and get handed books by librarian. You go to a section and find what you are looking for.

as you rightly point out social is discovery… and has been from the first cave drawings.

The distinction is clear to me and both will coexist and even support one another. the question becomes is there an economy in place that can support social discovery and the answer today is no.

discovery will need to be seamlessly integrated with recovery tools and/or new modes of discovery monetization will need to be built. These are daunting challenges that at the moment too few are focused on solving. It requires something as revolutionary as the very changes taking place in content. as revolutionary as ppc was. let’s not forget how Google was able to spend those billions. they earned it

Jonathan Mendez

i got here from twitter.

great post. it has become clear that Google is recovery – 40% of queries are repeat, 70% are navigational, etc. Just like the great library it was built to be you don’t walk in a library and get handed books by librarian. You go to a section and find what you are looking for.

as you rightly point out social is discovery… and has been from the first cave drawings.

The distinction is clear to me and both will coexist and even support one another. the question becomes is there an economy in place that can support social discovery and the answer today is no.

discovery will need to be seamlessly integrated with recovery tools and/or new modes of discovery monetization will need to be built. These are daunting challenges that at the moment too few are focused on solving. It requires something as revolutionary as the very changes taking place in content. as revolutionary as ppc was. let’s not forget how Google was able to spend those billions. they earned it.

Sachin

very interesting read…I agree that nowadays “Twitter and FriendFeed become referrers of traffic to our network of blogs” but I feel this is only “to an extent and for a few” and there are still lots and lots of people who feel that twitter is not converting tweets to traffic and if it is converting it is not worth it, but any ways we cannot at all underestimate Google search in this case. What I believe is that there is no substitute for Google search (although I agree that twitter searches MOST recent, fresh content(news if any) better than Google) and I don’t see any in near future. Google is in the blood.

jasonspalace

social media is evolving. what we are seeing in terms of unique discovery based internet traffic is not defining much for us. we are merely seeing our curiosities of this internet unfolding. we are still confined to consuming our information based on the internet providing mediums made available to us. this will change as we enter an age where the medium won’t be so important as the delivery mechanism for that information. right now we have facebook, google, twitter, start pages, and the aggregators that bring internet headlines to the mainstream’s attention (digg, slashdot, youtube, etc) yet these mediums are still static computer web page interfaces. it is when these entities act more as “channels” will we know what and how content delivery is occuring for us. another great topic Om, thanks!

Yonni Harif

Great post, Om –

To take it a step further, this transformation is also impacting the business world. With the growing popularity of Twitter, Facebook, iPhone, iGoogle, AIR and their distant cousins from the social media/Web 2.0 world, companies are embracing some of these tools and associated culture, driven by employees (for internal usage) and changing consumer behavior.

David Winer’s “river” often overflows also when it comes to companies interacting with customers, employees and suppliers. There is a lot of business-critical information to be exchanged, and often customers and distributors must navigate to a host of portals/websites to get biz done. This is not very efficient over time, and so social business models now emerge – extending business tasks to secure widgets, social networks, desktops, mobile, etc.

It will be interesting to track how this transformation continues to unfold.

Baratunde Thurston

Great post Om,

One thing I’ve been thinking about for years is how to filter these streams or rivers and then ensure that our investment in such filters is portable across services.

For example, I’ve put lots of time into my facebook friend lists. Now I can get a friend stream focused around my “Political” friends or “Social Media Homies” or “Comedy” colleagues. This is like tuning the antennae on the web. The problem is, that investment is isolated to facebook.

Similarly, I invested lots of time into creating twitter user groups on Tweetdeck. Then my laptop was stolen, and I lost those settings because Tweetdeck doesn’t sync this data anywhere.

We will need at least some of the following to help us manage the stream

– ability to structure or filter the river both by hand and with some level of automation
– filter redundancy so we can protect our investment in this curating process
– filter portability across services so we can take it with us from place to place

In the age of facebook connect and google friend connect there’s lots of excitement about porting ones social graph from site to site, but along with this, I think it is as important to port the filters we apply to these networks.

pruett

Really enjoyed this post. This is such a huge topic and everybody is writing about this “river of news” and its implications on the future of web content consumption.
You brought up Digg at the end of the post. Do you think they will be able to transform their service to become what Rose explains, “a more living and breathing site?”
I am also interested to see how the shift away from the conventional search engine will impact our future search habits. The fact that search.twitter.com offers a great deal of value in Twitter’s relative infancy is truly astounding. Google’s reaction to this growing trend will be very interesting.

Jason Nazar

Om,

great post. I’ve been reading a lot of folks writing on similar themes lately. I think there is an important additional analysis… why are people sharing all this data? In my opinion still the largest reason people contribute any “user generated content” on the web is Ego. We want to be seen/heard, and we want our content discovered.

But ego is not a sustainable variable, over time and by itself for all this content. And while those of us at the forefront of this content sharing may consider it a more pure form of sharing and discovery, it will likely be supplanted by a market necessity and the ability to make money around it.

For me one of the most interesting questions is: What are going to be the different models to charge and/or monetize content that will eventually have to be driven by a profit motive instead of an ego motive.

Om Malik

Jason,

I think ego is part of the equation but I see it as a the networked-life essentially imitating life in general. User generated photos or sharing of links and videos is just part of our human behavior. We share books, photos and often tips with our friends and family. That is essentially is what is happening on the web. It will happen more and more.

Sure ego is not a sustainable variable, for vanity is a deadly sin as the good book tells us. But I think if you really took a birds-eye point of view, the network (Internet) is becoming more and more human.

On the money part — I will leave you and other smart guys to figure that out. :-)

Adam Singer

Great piece Om, one thing hit me though:

“but one thing is for sure — many Web content discovery engines that exist as destinations (Digg, for example) are going to face challenging times.”

I would argue that Digg is about the community and fostering a culture that you can’t get on large sites Facebook which include the world – it’s about the community, not the destination. Facebook is less interesting to me because it is “everyone,” which sort of destroys the unique culture that fosters on niche specific sites. Macro networks are less interesting than niche networks, which foster a group of like-minded users. All of those same users are on Facebook too, but the two destinations are not the same, and with good reason. I’m with you that content distribution and discovery are changing, but don’t discount passionate communities who gather around a destination like Digg or Reddit. Those communities would not be the same if housed inside larger entities. Plenty of forums from the 90’s still exist, Slashdot is still around…

I’m not discounting the power of realtime web and in fact am publishing something about it tomorrow having to do with marketing. I guess it boils down to the question of how committed are users to specific networks or destinations. You can get content more efficiently somewhere else, but users might not flock away from their old nesting grounds depending on how deep the connection is with that destination.

Om Malik

Adam

You raise some valid points. Let me be specific about your thoughts on Facebook. I see them as a meta-platform that is eventually going to evolve into an authentication system. Facebook Connect is allowing people to build those niche communities, the very kind you think are going to become more useful. I think Facebook’s macro nature is what will make it crucial to the new web infrastructure.

On the point of passionate communities, I think Digg is facing the problem where wisdom of crowds is now replaced by wisdom of the hordes. In comparison, Hacker News is a much better destination site, which exists only to send you somewhere. Digg wants you on their “destination”. Sort of like come to their bar and then talk with others while you are it. Twitter/Facebook are more like a house party.

Slashdot and Well are two communities – they were always meant to be just that and content/destinations later.

Comments are closed.