Why Email Clients Need to Change

97 Comments

My inbox is broken.

Not in an I-can’t-check-my-messages kind of way, but in a fundamental, inboxes-will-never-be-the-same-again kind of way.

Analysis of inbox traffic for 30 daysWith every birthday reminder, bill confirmation, new friend, direct message, password recovery, and mailing list, the content of our inboxes becomes less and less a means of communication and more and more a record of all we do online. Email is the lowest common denominator of digital identity. It’s our web keychain. It’s the catch-all of our online lives.

But if inboxes don’t fundamentally change in order to adapt to their new role as the keeper of myriad transactions across the entire web, they’ll be obsolete.

Have a look at your inbox. Chances are much of what’s in there isn’t just traditional email conversations between you and someone else. A few hours of unscientific inbox querying and a quick analysis with Mail Trends showed that less than half of my messages in the last month consisted of such simple correspondence. The other half were records of things I’d done, people who’d followed me on social networks, bookings I’d made, confirmations of sites I’d signed up for, and so on.

Companies like Xobni and Xoopit offer email analytics, as do some Firefox plug-ins, but everyone still assumes that what’s in an inbox is predominantly conversations with people. It’s not.

Inboxes need to get smarter. My perfect email client would:

  • Scan incoming messages and build a list of all the companies I’ve paid, and those with whom I have recurring payments, showing spending history.
  • Keep all my logins and password recoveries for online accounts in one place, safely encrypted.
  • Group and track mailing list digests, and give me controls to unsubscribe from them.
  • Show me all interactions with each of my friends in one place, regardless of whether they happened on email, Facebook, Friendfeed, or Twitter. OpenID holds promise here, but has yet to be properly integrated into inboxes.
  • Track and analyze transactions semantically, from upcoming travel to events I’m attending.

Today, I have to visit dozens of other sites and services to make sense of my online life. This is a waste: I already have a record of all these transactions in my inbox. I just need a better way to look at them.

Gmail offered a tantalizing glimpse of what inboxes could be, but stopped short of recognizing this shift from conversations to a digital record of our online lives. The inbox of the future looks more like logfile analysis and aggregation and less like an email platform. Today, you can hack some of this together with Greasemonkey scripts, clever Gmail filters, or add-ins from Gmail labs. But it’s not enough: We need an inbox that embraces its new role as the universal record of our online lives.

97 Comments

Mike

Whilst it is unlikely, surely online services should allow you to control what it will remind you about in its profile/settings area. In a common frame of reference: facebook, you can set it to email lots about what happens or just the rarities such as birthdays or messages (assuming you close friends prefer to email!) I must admit whilst getting 50% less emails a day makes me feel less popular I am more productive for not having to delete so many fb notifs.

John

One interesting take on next generation email is at gtdinbox.com. I stopped using their tool a long time ago because it’s way too unstable. (It’s based on gmail and every time Google tweaks anything behind the scenes of gmail gtdinbox breaks.) But their core idea is important:

***Every email is either junk, reference information, or a badly formed task.***

For me, a large part of that 50% “normal” slice is tasks: People asking me to do things, me reminding myself to do things, or me asking other people to do things. Although gtdinbox’s implementation is fragile, I very much like their approach.

Arjun

Ouch no. This is where I prefer divergence over convergence. Here is what I feel:

a) Fancy inboxes are fads. Give it a few months and the only thing you will mostly see is your main inbox. mail that gets autoarchived to other folders have a much lower frequency of check. I am talking about an average trend here.

b) A compelling part of many social networking sites is the “UI environment” not just an element of receiving or sending messages. Moving SNS to your inbox takes away the experience.

c) Email is still considered ‘primary’. other identities are secondary. Mixing the two makes for a heady cocktail.

Marc Eisenstadt

These self-report analyses are actually quite useful IMHO, especially when done as part of a longitudinal study, although inevitably people will arrive at wildly different conclusions (including the conclusion that you can already do the appropriate filtering with off-the-shelf tools, which is partly true).

The landscape keeps changing, though, and you might therefore want to compare your breakdown with a more detailed (pre-Facebook, pre-Twitter) analysis I undertook based on a retrospective categorization of EIGHT YEARS of email (!!!) back in February 2005, particularly in ‘Table 3’ at http://getreal.corante.com/archives/2005/02/11/eight_years_of_email_stats_pass_1.php.

My analysis back in 2005 raised similar issues, although I argued that I did NOT want bots or intelligent agents sorting my email. Generally, your own discipline is the answer. Reason? Any smart filtering/sorting tool may buy you several orders of magnitude ‘improvement’ (think how great ‘feed aggregators’ were when they first appeared), but the signal-to-noise ratio eventually plummets as the noise level skyrockets once again (consider all your unread aggregator feeds). See my related thoughts on the signal-to-noise problem here.

-Marc

Blubadger

I think Andreas has identified the ideal-world solution.

But for the real world, surely the very first step is to have 2 e-mail accounts.

Account 1 you use for real people. This is what IMAP-SMTP-POP3 was designed for.

Account 2 you pimp with filters and tags and whatever else you need to deal with bots and admin. (With the Multiple Inbox addon I’ve now got a pretty smooth GTD machine up and running.)

iampariah

My inbox (Outlook 2007) looks very much like Alistair’s when graphed out. I currently manage it with a ton of Outlook and Auto-Mate rules, but that management is awkward and time-consuming when it comes to ongoing management. Moreoever, the organization of my mail is limited to Outlook’s internal categorization and folder capabilities (with Gmail it would be effectively the same with tags and folders).

What we need is a lifestreaming application, something that, at it’s heart, is an e-mail client, but collects and processes nearly any type of user-configurable, user-subscribable electronic data. Everything from RSS feed items to file attachments, URLs to phone call records, should be given equal treatment by the application. It should contain powerful rules and actions that can be performed on any type of content in order to categorize, tag, and otherwise organize that data, as well as relate pieces of data together regardless of their format or means of retrieval. And, the client should allow the user to pull in any content; it shouldn’t operate from a limited list of predefined services like Twitter, Facebook, LinkedIn, and so on. Too many new and interesting services are launched while application developers are celebrating and promoting their last big integration with a third-party service. The client should offer the user the ability to connect to a variety of data streams in common formats–RSS, ATOM, microformats, Twitter Oauth, FB Oauth, OpenID, etc.

I use Xobni, and like it, but–and I’m sorry Matt–it’s analytics features are nearly useless while Xobni’s “Conversations” list is just mildly above useless. Much of that has to do with the fact that Xobni is yet another column that takes up space in an application already crowded with columns. If Xobni were a full-fledged view, replacing the main Inbox/mail folder views of Outlook, and if it managed conversations, it could be vastly more useable.

I like Xobni’s ability to show me the photo, phone number, and other contact info of the author of an e-mail message I’m reading. That is very useful to me, particularly when Xobni pulls that data from my contacts folder as well as Facebook and LinkedIn. For that feature alone I love Xobni (and recommend it), but that’s the utility of Xobni in my opinion–as a contact integrator. It isn’t an e-mail manager or analytics aid.

Pies

Try Gmail. It has great conversation threading, full author info on mouse-over, and tags are far more versatile and useful than folders — if you don’t know if a particular e-mail should go into the “Work”, “To Do”, or “[Client name]” folder, it should probably go into all three.

Bob Finch

Many of us simply can’t use Gmail for work purposes. It’s a complete non-starter.

While I have a Gmail address, I don’t give it to clients and I don’t send business email through it. Plus, if someone sends me a proposal or an introduction through Gmail, it comes across the same way as AOL, Hotmail, Yahoo mail, etc. does: unprofessional.

I know this feeling differs among people who work in tech fields, but for those of us who court business in other industries a lack of branded domain mail is bad form.

Andrew

Finch,
You’re right: gmail/hotmail/yahoo accounts scream “SOHO/consultant/personal.” I love startups, but unless you are an individual “building the brand of you” (sooo FastCompany circa 2001), having a branded domain email account is a requirement. Fair? Maybe not. True? Yep.

Scott Kendall

This post reminds me a lot of this infographic on snail mail: http://www.flickr.com/photos/threadsy/3474179018/
Suspected that personal mail was just a small percentage of total mail, but 3.8%… ridiculous.

Agree, there’s a lot of opportunity to harness all of these disparate types of email messages and intelligently display the aggregate information with new viewing paradigms. And BTW, don’t agree that IM replaces the need for email entirely. Instead it just splinters off the short-form synchronous personal communication use case which wasn’t well-suited to the email paradigm (long-form, asynch).

peter.bravado

otherinbox is a very good solution although i think its in the early stages and needs much more functionality, configurability and speed. haven’t seen better as a concept though. xobni seems limited to outlook.

peter.bravado

otherinbox is a very good solution although i think its in the early stages and needs much more functionality, configurability and speed. haven’t seen better as a concept though. xobni seems limited to outlook.

Roswell Real Estate

We all need to “check” what is actually being delivered to our inboxes. Flooding your inbox with worthless face-book and Twitter junk usually will get you no where.

thegeniusfiles

Gosh, I stopped using email clients a couple years ago. And the last year before that, I was solely using it for backup purposes. Then one day, it occured to me that I could just use another webmail account for that purpose. Since then, I have not looked back.
I agree with the author that email has a lot of undeveloped potential. To my mind, Gmail Labs is the most ambitious large-scale project. You can see that they are exploring new ideas. To me, functionality is of course important, but interface is what enables functionality. Interfaces should be more fluid, dynamic, context-sensitive, and customizable.

Alistair Croll

Thanks for the feedback (and pointers to other services.)

@Sam — admittedly, I’m not a typical email user. I suspect that the contents of a mainstream inbox will look more and more like mine, however, because people are too lazy to turn off notifications. Similarly, enrollment in websites requires somewhere to confirm identity and somewhere to send recovered passwords; and my bank needs to tell me when a transfer has occurred. While this may not be the 50% of my inbox, it will be an important part of anyone’s inbox that needs to be analyzed differently.

I really like your suggestion for plug-ins for email (that’s what I hoping for with Greasemonkey) and this could be done much as WordPress has a plug-in ecosystem — each social service would provide an email plug-in to handle/aggregate/protect its specific data.

@Rob — yes, that’s the kind of thing I’m looking for. While I might set up filters (and frankly I’m too disorganized to do so properly) as this becomes a more mainstream problem it’s less likely that consumers in general will do so.

Also, an interface like that looks surprisingly like a Facebook home page…

@Welson — agreed, search is a crutch. Sometimes if I’m doing an expense report I find myself using Gmail search for the amount of the expense that’s in my credit card in order to find the original invoice. That will only work until I use up the namespace of three-digit numbers. Then I’m hosed. ;-)

@Joshua, @arthur — appreciate the link. I didn’t know about you guys, and you’re definitely trying to tackle the problem. Will check it out.

@Andreas — Yes, the email client of the future doesn’t need to stick to the IMAP/POP/SMTP stack. But I do firmly believe this: Much as Twitter is an API for human attention, email is the Syslog of web users. Most consumers want it all in one place, and that’s a new breed of email client.

Pies

I think you’re overcomplicating the whole thing. Set up filters for the 10 most common annoyances and then consider if you need a better inbox.

I’m speaking from experience: a lot of my incoming email is automatic, mostly from my own scripts that send newsletters or pre-compute some stuff for websites. I set up filters for the most common stuff, and my inbox is _much_ better now. I set up filters to tag those e-mails as “Scripts” or “Newsletters” and skip the inbox.

Very simple, very effective.

Jessica

I use OtherInbox as well, and it does pretty much everything you wanted. I think the passwords are the only thing it isn’t already doing automatically, but with separate organized inboxes for everything, it’s simple enough to click the inbox for the website you want and just look at the first message.

You should seriously try it out…I think it was made with people like you in mind.

By the way, I’m a fan, not an employee or anything.

Andreas

I think your proposal leads to the wrong way. Instead, we should work on reducing the mis-use of email. We should

* use NNTP/Usenet for “mailing lists” (gmane.org does this already)
* use e.g. XMPP/Jabber for instant messaging (direct messages, …)
* use OpenID for identity, which should remove the need for password reminders etc.
* use WebCal/iCal for events/appointments

Those are all protocols much more appropriate for their purpose than email.

John

I agree with this comment, at least for the most part. But I still want one place to hold all this information. Maybe you don’t call that email, maybe we should call it a “messaging repository” or something.

The problem isn’t that email is broken. The problem is that inboxes have very limited intelligence. Filters are nice, but they’re not intelligent.

Joshua Baer

Hi Alistair,

Great article! I couldn’t agree more – that’s what drove us to create OtherInbox. It automatically finds the messages in your inbox that aren’t from real people – all the messages you are complaining about – and creates a list of all your merchants and websites. We track receipts, shipping notices, and other info and build you a calendar on the fly with useful information. We automatically identify mailing lists and groups and put all of the messages into a separate folder for each mailing list. We currently have a way to manage all of our account logins and are planning on adding password management support in the future.

Joshua Baer
Founder of OtherInbox
http://oib.com

Zeljko

I’m using “Postbox” for some time and it pretty much gets the the job of integrating ma Gmail and social networking done.

Rob Kingston

Sam makes a useful point – most people who have this problem are people who would setup filters in Gmail.

But that’s not good enough. We need something more seamless. For instance, when you login to your email account you could be faced with an interface that shows:

Inbox (20) (Normal Emails)
Recent Transactions (30)
Social Actions (30)
Alerts & Updates (30)

Maybe, maybe not…

Trace

I second Sam: Gmail filters will immediately remove 20% in the form of twitter follows…. likely much of the other unnecessary notifications you get as well….

Ouriel Ohayon

i could not agree more. Email starts to become unsmart. That s why we are trying to bring back some smartness to email with Topify.com and make notifications more meaningfull and actionnable

Weldon Dodd

I absolutely use my Google Apps account as a searchable database of logins, passwords, account #’s, etc. More and more of that stuff is going into 1Password where it is encrypted and protected, but there is lots of reference stuff in my email. Fortunately, searching in gmail/google apps is really fast (much faster than on my laptop).

Still, I’m looking for something better. Today I signed up to check out otherinbox.com which appears to be very close to what you are asking for. I became interested because of the co-founders is speaking at RailsConf on SproutCore and how Cocoa (Mac/iPhone) programming principles in SproutCore have helped him become a better developer.

Rick

Maybe you need to get smarter. There’s no good reason you “need” to receive Facebook notifications in your email inbox. Turn off the email alerts, dork!

Sam

Eh–I’m not really buying this.

First of all, you’re looking at a VERY specific segment. Very few people would have 21% of their inbox comprised of Twitter follows. More importantly, the type of person who would have 21% Twitter follows would know how to use Gmail filters. For me, all my Twitter notes are filtered to skip my inbox and go into the Twitter label. Facebook skips and goes to the Facebook label. I only sign up for accounts (bank, registrations, etc.) using a a junk version of my gmail (putting an extra period in the gmail address and filtering). My e-mail is my primary source of information–my inbox is clean, relevant, and efficient. I scroll through Facebook and Twitter notifications at my leisure. So, for people like me/us, e-mail is still relevant and doesn’t need to change.

More importantly, for the 98% of e-mail users who don’t really use Twitter and don’t mind getting their Facebook notifications, e-mail is still great and doing its job.

This isn’t to say the emergence of social networks shouldn’t be integrated more thoroughly into gmail. There really should be Facebook/MySpace/Twitter plugins to Gmail, which leverages APIs and has its own section. It would help with the clutter you mention, as well.

B

Agreed. The author is assuming that 90% of people online are internet savvy and who bounce around on different social networks. I turn off all email notifications from facebook, twitter, brightkite and myspace, that way I will check them in my off time and am not constantly stuck to my phone browsing at every available opportunity. If somebody needs to contact me instantly they can send me a (pushed) email notification if the app isn’t running.

My inbox is always clear and everything is sorted into appropriate folders. No need for some kind of inbox revolution here.

kdawg

yes…filters have been around for ages….but why can’t there be something better?

take a look at personal finance packages like Mint, Quicken etc…
they use collective intelligence to categorize your transactions and allow them to be visualized, browsed more efficiently…

mail clients do this sort of thing with SPAM…

…but how about applying the wisdom of the crowds to stuff that ISN’T SPAM…
how many millions of users use facebook? why do I need to create a specific filter to deal with this? Can’t there be a client that can categorize all of your email BY DEFAULT (based on how most TYPICAL behave)…from there you can decide what to do with your already categorized email?

Xobni and Xoopit sounded like they were putting some focus in these areas…so far it’s been a bit disappointing…and Xoopit actually adds to the SPAM! wtf?

…of course I should just get off my ass and build this myself…but I know I’m pretty lazy, so here’s to hoping someone else will :)

Niraj

“why do I need to create a specific filter to deal with this? Can’t there be a client that can categorize all of your email BY DEFAULT (based on how most TYPICAL behave)…from there you can decide what to do with your already categorized email?”

Simple…because over 90% of people wouldn’t know what’s going on and would end up confused/frustrated. Think about how much effort Google has made in explaining what tags are and why they’re better than folders, and people are still confused and not making the best use of them. To start auto-filtering things would be confusing and you’d never match the filtering preferences of many of your users correctly (i.e. some might want different folders for groups of friends, or want messages to stay unread, or flagging of different kinds of notifications from the same service, etc.)

I think for the vast majority of people, filters and tags like those in Gmail are sufficient to managaing email, though many don’t know how to use them or set them up. It’s always tough to make things like that simple enough for the least technical users. This article is talking about solutions that probably affect less than 5% of email users, so it’s not surprising that there hasn’t been a lot of movement on it.

vivek

I’ve been talking to friends and colleagues about such enhancements to email clients for a while, and its exciting to see your article on the same topic!
You’re absolutely correct in saying that our email inboxes have become unstructured repositories of all kinds of information, and this presents the need and opportunity to perform all kinds of information analysis and ‘grooming’, to help users tag and structure their data. A feature I’d add to your list above, would be for email clients to keep track of things like “returned emails, email address change notifications, and new email id’s from existing names”, to intelligently distill out such info and help reflect the same in the address book (perhaps interactively).

Take care,
-v

Matt Brezina

awesome article Alistair, and thanks for mentioning Xobni. I will offer that while deep under the hood, xobni is processing a lot of email analytics, the utility of our product has much more to do with organizing your inbox around people and companies; we help you quickly find information relating to both.

Additionally, we have partnered with Facebook, Yahoo Mail, and LinkedIn to bring those communication channels into the inbox. There will be much more coming from us in that direction.

As for semantic analysis of your transaction, travel, and event emails – that is part of our dream, and we hope to provide more of that soon.

I hear your pain.

Best,
Matt Brezina
Founder, Xobni

Ian Rae

If only Xobni wasn’t tied to Outlook. On one hand this makes sense since Outlook is broken and needed Xobni to make it useable again. On the other hand these are obvious innovations that shouldn’t be tied to a dead end software client, set them free!

pwb

Newsflash for bloggers and twitterers: Outlook is, by far, the most used email client. And yes, the fact that it has some major deficiencies, especially around search, leads to major opportunities.

Nicholas

Is Outlook used because of its features or because it is forced upon the corporate user? What do people use personally?

John

Outlook sucks. The people who use it either are forced to use it by their employers or don’t know any better. The latter group is hopelessly lost anyway and not much will help them.

Fattom Boy

Chalk it to clueless employers. I tried Outlook Express ’cause it came with Windows and talked to Hotmail. Microsoft fixed that — I ditched it years ago. At work, I’m stuck with shiny Outlook 2007 (check your calendar, yup?, ok…).

Outlook not only cries out for plug-ins, it cries out for a replacement. Work doesn’t give me “admin”, so even plug-ins are … (God bless Evolution …)

Stephen

I like Outlook, as a suite. my Company forces me to use Lotus Notes on my work computer I transfer everything I can to personal Outlook, it has a superior Calendar, and the address book and journaling are great, the e-mail client might be weak, but in the grand scheme of usage e-mail is only a portion of what Outlook can be used for. I am glad Xobni chose outlook and not the likes of Thunder Bird or other.

Comments are closed.