Web Tablets and Text Entry — How To Do It?
The post I did recently about web tablets and text entry made one thing clear to me — that nothing about it is clear. The point I was trying to make is a valid one, that we no longer passively sit back while web surfing and consume the vast amount of information the web dishes out to us. More than ever before we now interact with that content, and we do it by entering our thoughts and ideas. We do that with Twitter, Facebook, MySpace, FourSquare and many other web apps. We tell people where we are, what we are doing, how we are doing it and share our lives with others.
We have seen the focus on web tablets shift into high gear recently, with talk about products we are told will be coming soon (CrunchPad), and others we expect to see but have no confirmation we ever will (Apple Tablet). There are a number of web appliances appearing that run the Android OS, and it is apparent that we will see such tablets continue to hit the scene next year.
I stick by my assertion that our interaction with the web means that for any web tablet to appeal to the masses it must be able to handle data entry naturally. The key here is “naturally”, as consumers end up shying away from input methods that do not feel right. We want to get our thoughts into the tablet, and have the device know what we meant and put it in a form that others can consume. This is not as easy as you might think, and it has been a stumbling block for tablet acceptance for years. So what options might be used for web tablets to handle text entry the way consumers will feel comfortable using?
Physical keyboard
The keyboard is certainly one entry tool that cannot be overlooked, even for web tablets. This method hasn’t changed in a long time, but it is the most familiar to consumers. The QWERTY keyboard is still the method used by most for rapid entry of gobs of text. The problem is that for a web tablet to be a thin and light tablet it will not have one of these keyboards. That creates a bit of a dilemma for the genre.
The best way to go to allow the use of a keyboard with a web tablet may be to have a detachable wireless one. Say a thin, special keyboard that attaches to the back of the tablet, yet pops off when needed for entry. This would give users the input method they are most familiar with, but it would have to be done carefully to keep things simple and light. This would also add a cost factor to the web tablet, something that might not work.
Touch keyboard
The input method that will most likely be used is the onscreen touch keyboard. Think of the iPhone keyboard made larger to fit the expected screen size of the web tablet. A multitouch screen could support a keyboard on the screen that is a decent size for typing, and would require no additional hardware nor engineering to use.
What is not clear to me is how good these keyboards will be for extended entries. I have tried using such a keyboard on a 12-inch touch screen, and it is not a very productive method. It’s hard to get any good typing speed with a touch keyboard, and I am not sure how the average user will accept them. It is good enough for very short entries that typically make up a common web session, but falls short with longer entries.
Digital ink
Since the post I made about web tablets was done in digital ink, some folks assumed I was implying that this is the preferred method of entry for these devices. While I am comfortable with using digital ink, it is the result of years of using a Tablet PC. I have to admit that I am in the vast minority when it comes to regular consumers, most will not find inking on a tablet screen to be natural nor comfortable.
This method also adds a significant technical problem to overcome for use in consumer products like web tablets. Users will quickly get frustrated when they write something on the screen and it is not interpreted correctly. As good as handwriting recognition is today on Tablet PCs, I don’t think it is good enough for consumer products. Remember, the point of having good text entry is for longer written pieces, and this is the very type that digital ink has problems producing error-free.
PDA input
Remember the Graffiti input method that the original Palm Pilots used? Believe it or not, there are still quite a few folks who think this method is the best way to enter lots of text on a digital screen. Once the user gets practiced writing letters in a small box on the screen, I have to admit it is amazing how fast text can be entered with great accuracy.
The average consumer is not going to give this method a fair chance, however, so I don’t see this working on a web tablet. Consumers don’t want to practice to make text entry efficient, they just want it to work out of the box. Putting a new interface for entering text is going to meet stiff resistance.
Speech
The biggest obstacle to providing a good text entry method for web tablets centers around how they will be used. Tablets are used in the hands, that is the nature of the device. This makes it difficult to enter text using any of the methods mentioned so far; the user has to enter the text while holding the tablet in one hand. This is the biggest limiter affecting how lots of text can be entered on a tablet. One method that works well in this scenario is speech recognition.
Current speech recognition technology is better than most realize. It is possible to get 98 percent recognition accuracy today, making this a viable method for text entry. The user simply dictates the entry into the tablet, and it converts it to digital text for input on the web. It is the easiest way to handle the entry problem, but it is not without its drawbacks.
The two biggest obstacles to using speech with web tablets are background noise and looks. To get high accuracy converting speech to text requires a controlled noise environment, something tablet users will not necessarily have all the time. Accuracy goes down quickly when dictation is done in noisy environments, and this will be a common scenario. The other obstacle to using speech is more insidious, and that is that users do not like to look silly.
Talking to your computer makes you look silly, and that just won’t fly. Users will be reluctant to do so in front of others, no matter how much they might like Star Trek. Talking to the web tablet in the presence of others will also increase that background noise problem.
So what do you think will work?
These methods to provide text entry on web tablets are the most obvious ones, but maybe there are other ways to do it that are not readily apparent. What do you think? Is there another way to get text into a web tablet that are not mentioned here? Leave your thoughts in the comments, let’s have a good discussion about this.
Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.
I think everything but the physical keyboard should be on these tablets. As a tablet owner, there is certainly a use for on screen keyboard, digital ink, and speech. I know you have many posts over the years about how each one of those inputs methods is beneficial in its own way.
My only concern would be the inking experience on non-windows devices. The evolution of inking from windows xp tablet to windows 7 has been incredible. Can another company create just as great an inking experience first try?
Plenty of good thoughts here. Part of the reason a touch keyboard works on the iPhone and other small phone tablets is because they’re small. Once you get to 5 or 6 inch screens, you start to require two handed operation, which for typing means two thumbs. If screens get bigger than than that, say 8 to 10 inches, it becomes difficult to use an on-screen keyboard while holding the device — unless you want to hunt and peck with one hand or finger while holding the device in the other hand.
Speech is definitely an solid option but is hindered by factors outside of the user’s control, i.e.: background noise, so I see it more as a secondary option rather than a primary one for input. It’s great for use here and there, but not for all-day needs.
The ideal method (in my mind, anyway) is one that’s the most liberating to the user. I’m thinking of having the user be able to write anywhere on the screen — no stylus, just using a finger — and having the properly interpreted writing inserted into the appropriate text area. As impressed as I am with the Tablet Input Panel in Windows 7, I don’t want to be hindered by a specific confine where I have to write.
Hey James, great post.
I’ve been looking at digital ink a bit differently lately as a result of asking myself this simple question: when do I really need handwriting recognition? I think part of the failure of digital ink has been the focus on handwriting recognition, which has never been good enough to make people comfortable. The human brain is so much better at reading ink and most (not all, but most) of what we do with text is write it and read it with our eyes. If your colleagues at work got occasional ink emails or IMs would this be a problem? Blog posts or tweets in digital ink form raise some issues when it comes to indexing the content for search, but might not the “best effort” handwriting technology we have today be good enough to make this content pretty indexable? The main thing would be to get that HWR processing out of the user’s face so they didn’t have the unsatisfying and distracting experience of it’s imperfect operation.
The tasks I consider that I can’t reasonably do with pure ink are things like creating spreadsheets and writing code. By and large they’re things I wouldn’t normally do when I’m genuinely mobile. I’d love to see more mobile devices that takes ink seriously as a “first-class citizen” for communication of information rather than something that is bolted on in service to ASCII text, which even in the best case imposes so many unnecessary limitations. Livescribe’s Pulse SmartPen seems to have great potential and it will be interesting to watch it evolve (adding wireless, a good desktop SDK, etc). But I believe that there are millions who are still looking for something like Mike Mace’s InfoPad concept.
I think speech is interesting in a few scenarios, but is ultimately quite unnatural as well. A lot of the excitement over it seems to be from people who aren’t really considering the real-world texture of the mobile use context: there are way too many situations where speaking to your device is awkward, rude, or simply won’t work because of the amount of other voices and sounds around. I’m much more optimistic that a rethinking of digital ink will be the Next Big Thing in mobile input/output.
“Think of the iPhone keyboard made larger to fit the expected screen size of the web tablet.” — No, don’t think of that. The iPhone’s onscreen board doesn’t work because the device is small but because the user can reach the keys. A larger board would put some keys out of reach. To put it another way, the board doesn’t need to fit the screen; it needs to fit the user. A better choice would be the same size set in one corner or edge or split in two.
James, interesting thoughts, as always. I’ve been using a tool called ‘slide-it’ on my windows mobile phone, and find it by far the easiest text input method. There’s a similar product coming out for multiple platforms called Swype, check it out:
http://www.swypeinc.com/
Gizmodo are now reporting that the Crunchpad is now dead so I guess that’s one less tablet to worry about:
http://gizmodo.com/5415292/the-crunchpad-is-dead
In the shorter term I think we’ll doodle along pretty much as we are now. In the longer term I expect something totally different than what we have–a breakthrough. I think it will be an either/or/combination of biofeedback/virtual/motion and camera technologies. I look for the beginnings of the breakthrough to come from the video game market (xbox, wee, playstation) phones, or from the biotech industry as they try to help paralyzed people to communicate better.
I’ve been a tablet user for many years. In fact, I wrote one of the first posts for a series on GBM called “My Life with Ink” about how I use it (and still do) in my home inspection business.
I’ve moved from a slate to a convertable primarily for price issues. My Motion LE 1600 died suddenly and I needed to preplace it right away but did not have the extra $2000 or so to get another Motion. I went with an HP tx2510us but am not happy with it.
I had been using ink and handwriting recognition up till then with great results. I got the tablet primarily to be able to sign digital docs and annotate photos for my business. I now use the keyboard. When I write articles I use Dragon Naturally Speaking 10 wit great results. I write software reviews twice a month and maintain 4 blogs and this is the main input method for these longer pieces.
With improvements in noise canceling mic technology, I think this would be the best solution for input to a keyboardless slate tablet.
I agree that handwriting recognition is not yet good enough for general consumers, although it has come on in leaps and bounds.
To give an example, when you write a proper name on a tablet PC, or an abbreviation, or some other term that’s out of the ordinary and that is not going to be recognised by the converter, you have to go into the mode where you can tap out the word letter by letter. It’s a nuisance, but no surprise. What is a surprise is that even in Windows 7 making this transition between input modes is clunky.
Another handwriting-recognition irritation I’m finding in Windows 7 is that recognised-and-converted text is still liable to change as you continue to write subsequent words. It’s so frustrating to see words that have been correctly recognised mysteriously changing to incorrect words.
Issues like these stand in the way of general acceptance of handwriting recognition, I think, and make some sort of keyboard essential. But, as you say, how can you attach a keyboard elegantly to a device that’s basically a type of slate?
Great post, and on a subject that must be one of the most important for the continued miniaturization of computers!
A couple of points I’m not sure I entirely agree with. One is that new entry methods must require no learning of new techniques by consumers.
The mass market was texting on phones with numeric keyboards just a couple of years ago, no easy thing to learn to do! The incentive was great enough, so the effort was made and now people walk around typing on their phones with their thumbs. The qwerty models that came later needed another adaptation with their tiny keys, as have touch screen keyboards- and consumers have bought them and adapted.
Another point, that people don’t want to be seen talking to their computers- we’ve all gotten used to using and seeing others use bluetooth headsets while in public. To me, using speech commands to operate a portable computer has hurdles to overcome, but the social one doesn’t look like a deal breaker except places where speaking would be unacceptable.
As for pen input, I’ve been using it on an Apple with an external tablet for a couple of years now, so Apple’s maligned and long abandoned Inkwell system. I like it, enough to use it over the keyboard sometimes because I can enter simple commands with one hand. For longer entries, the keyboard is king- as I imagine it would be with a tablet computer.
I use the “write anywhere” option, meaning that there is no dedicated input panel onscreen. So the issue of having part of the screen obscured by a panel or onscreen keyboard isn’t there. The downside, as implemented back in ≈2005 or so, is that there isn’t an opportunity to edit the text before it is entered.
If I could learn to use that input system, surely others can as well- especially with years of improvements, applications designed with the pen in mind and a dedicated device.
Personally, even if it improves, I would use voice and handwriting input only “a little”. I like the idea of an application being able to read my handwriting for indexing the text, but I prefer to type than write.
So what I really want is a better variety of easy to use keyboard input mechanisms to choose from. I hate to use my tablet as a tablet because the basic things I want to do are not intuitive – from changing the display brightness to text input for a text field to even managing OneNote pages. It should not be so hard.
Maybe a little work hunting and pecking is to be expected, but the rest should be natural.
My thinking is that it’s a catch-22… once it’s popular, text input and other controls on the tablet will become easier. But it won’t become popular until it’s easier.
So I think it will take one of the following:
1) A genius developer that just makes it work well.
2) Borrowed ideas from consumer phones or a future Apple tablet
3) A long slow evolution of improvements
Sorry if this sounds pessimistic, but I think it’s also just admitting the reality of the situation. I don’t deny that there are expert tablet users out there that can make it work as-is. But for the average user, it’s just not there overall (not just text input).
For me…
I’m fine with a virtual keyboard for _light_ input. Especially if the screen is big enough that I have a reasonable amount of the app still visible even when I’m using a full thumb sized keyboard. Imagine a 9″ Android Tablet, in portrait mode, with ConnectBot at the top, and a virtual keyboard at the bottom. I should have pretty close to a full 80×24 terminal visible in that use case.
For heavy typing, I have (in my gadget bag) a Matthias folding USB keyboard (no need to worry about connections or batteries, like with my older iGo folding Bluetooth keyboard). It’s full size, and quite usable. It just requires that I have a table, and that the device has USB ports. Luckily, the Entourage eDGe looks like it’ll work with this. That, and a video out port, gives you KVM support, as well.
One thing I think would greatly help Android (or any tablet OS) is the “Swipe” input method. You basically have an on screen qwerty keyboard, but instead of “tapping” the keys, you sort of play “connect the dots” with them. To type “send”, you put your finger down on the s, keep it down while moving to the e, keep it down while moving to the n, and keep it down while moving to the d. Then you raise your finger (for the first time since putting it down on the s). That would help a lot.
I don’t know how well Apple’s inkwell technology has improved upon (or even just preserved) the Newton handwriting recognition, but that’s another one that might be useful. But, for just taking notes, simple e-ink writing with a stylus, into a graphical format (no automated analysis of what you wrote) would be “useful enough”. You’d just have to go back and transcribe it yourself (with your USB keyboard, perhaps while at a KVM switch), later.
I personally never really liked Grafiti on the Palm. I know some people liked it a lot … and maybe it’s just that I didn’t use it enough to get used to it. But, yeah, I wouldn’t be attracted to that as an imposed solution. I also never use the Grafiti-like input on Maemo.
So, there you go: for the foreseeable future, on an Android tablet, I plan on using the on-screen/virtual keyboards for light text input. Then I plan to use a USB keyboard of some sort for heavier text input. If I get the Entourage eDGe, I could also use the e-paper screen for “written notes” that I transcribe via keyboard, later. Last, I’d find it useful to see the “swipe input” method available for the Android virtual keyboards :-}
(forgot to hit the subscribe checkbox)
I share the pessimism. I don’t think there is a good way, not unless people are willing to invest in leaning a new method. And we know from the PDA input results that they are not willing. (That one was decent.) If people are willing I am optimistic about the swype though I have concerns about multi-language, acronyms, etc. I was going to bring it up but Brian beat me to it. I would like to see apple implement this on my phone by default to see how much I am willing to learn it. :) (I know of the third party software, but I hated it. It is not convenient.)
I do see some hope. Apple has been good at telling us what we want. Maybe they will do that again. Then we’ll learn and the problem will be solved. Palm told us to learn graffiti and nobody cared. Maybe when Jobs will tell us to learn, learn we will.
This could be one solution:
when I had my Fujitsu slate, I found the virtual keyboard satisfactory for a number of browsing situations. Since I have been on various boards and blogs for a number of years, it never really bothered me that much. When the going got tough, I’d hit the desk with a Happy Hacking Keyboard.
I really do insist that the 10″ screen size is going to be optimal. It is professional size, akin to my larger Moleskines. With a bluetooth headset it allows for notes. It is useful for browsing. In a vertical mode, the thumbs are well positioned. The only reason I would want a larger tablet is for sketching, but I am not the average consumer or professional.
All of the above are coming around one day soon, so hopefully we’ll have our choice!
I use the S5′s virtual keyboard quite a bit, and use a pair of old ThinkOutside keyboards (Bluetooth & USB). For middlin’ input, I stick to the tried, true and trusty FITALY soft keyboard.
It seems crazy to me that I can write faster on my HTC Touch than on my Samsung Q1U. Zero weight keyboard is outstanding for a simple keyboard, bit it suffers in comparison to the Touchpal keyboard on my smartphone because it doesn’t have predictive text. I’ve tried several on the Q1U that claim to have predictive text, but none does it anywhere near as effectively as Touchpal (or the iphone, apparently) or even T9. I agree with sumocat–the keyboard wouldn’t have to fill the screen; couldn’t one of the phone keyboards simply be ported over to PCs?
James–I imagine you’ve tried ritepen; it allows the user to write anywhere onscreen. I have the older version, and it’s not perfect, but it’s pretty good for writing longer text passages.
“Talking to your computer makes you look silly” or awesome! ;)
On a recent trip, we were wandering aimlessly through a business park searching for a Mexican restaurant. The driver admitted that she only vaguely knew its location.
So I whipped out my iPhone and said “Hacienda near me”. (a restaurant name I couldn’t spell)
The car laughed.
Moments later I was able to direct the driver to the restaurant.
The car murmured “Cool”.
Speech when it works well can be fast and not so embarrassing. Still I don’t want to dictate my every email, IM, and google search to the world around me. Even future flawless speech input will still need a more private backup method.
Very nice of James to mention Graffiti. As one long time user I really appreciate it for texting and limited text entry on my WinMo phone. Tried digital ink on my HP TC1100 just about drove me mad.Of course with Graffiti time has proven that the average user is not going to go for it despite the fact that compared to say texting on a numeric phone pad it is light years ahead. The sad reality for Graffiti is it never became trendy. If there had been some decent marketing or some well known American personalities using it then who knows, maybe it would have reached critical mass with average users discovering it is in fact dead easy to use. Anyway in the average user world I’m encouraged by snippets of news that Apple and HTC are coming with a stylus that will work on Capacitive devices. This may finally mean I can move on from Graffiti to another workable digital ink solution. Bring it on!
I agree with Rove-IT. Graffiti (or Block Recognizer on WinMo)is my text-input method of choice for texting & emailing. Has been for years since I bought my first Palm IIIx. I’m very comfortable with it to such an extent that I write essays on my WinMo phone regularly. Just makes sure you get a proper pen-sized stylus to go with it. Once learned, Graffiti is easy & comfortable to use. I pray that it will not fades away.
(Written via my TC1100 using the 1st gen TIP, which do has Block Recognizer)
For voice input, give ‘em a wireless microphone that looks like a cell phone (or perhaps, that *is* a cell phone). Then they won’t look silly, they’ll just look like another person talking into their cell phone.