Multitouch user input is the current “latest big thing” in mobile computing. With the runaway popularity of Apple’s iPhone and iPad, and the company’s pioneering multitouch laptop trackpads now being busily copied across the industry, some suggest that multitouch devices will soon displace the traditional mouse.
However, revolutions in user input technology can result in unforeseen consequences, an emblematic example being the spike in repetitive stress injury that resulted from the switch from traditional “springy,” raked typewriter keyboards to flatter, often “clicky,” and frequently hard-landing, computer keyboards back in the ’80s.
Analogically, little is yet known about long-term stresses that using multitouch input systems may inflict on our muscles, nerves, and tendons.
Arizona State University Biomedical Informatics Department assistant professor Kanav Kahol is team leader of a research project to measure the amount of stress placed on hands and wrists of individuals using multitouch electronic devices like Apple’s iPad. Researchers will use cyber-gloves to measure kinematic phenomena produced in users interacting with multitouch systems with finger-flick gestures.
Prof. Kahol’s team, supported by a $1.2 million grant from the National Science Foundation, includes computer interaction researchers, kinesiologists and ergonomic experts from both ASU and Harvard University, engaged in developing a tool kit that could be used by designers as they design and refine new multitouch systems.
The ASU project’s aim is to develop best practices and standards for human/machine interface interactions that are safe and cause minimal user stress, while allowing users to fully benefit from the new levels of immersion that multitouch interaction facilitates.
“We Are All Part of a Large Experiment”
“When we use our iPhone or iPad, we don’t naturally think that it might lead to a musculoskeletal disorder,” says Prof. Kahol commenting in an ASU media release. “But the fact is it could, and we don’t even know it. We are all part of a large experiment. Multitouch systems might be great for usability of a device, but we just don’t know what it does to our musculoskeletal system.”
In a project abstract, Prof. Kanol notes that the researchers’ principal focus will lie in developing a methodology and process for selecting ergonomically appropriate gestures and mapping them relative to tasks employed in human computer interaction, such as the multitouch technology that has reached maturity in products like the iPhone.
Prof. Kahol observes that as we move toward a world where human-computer interaction is based on various body movements that are not well documented or studied, we face “serious and grave risk” of creating technology and systems that may lead to musculoskeletal disorders (MSD), and that many of today’s multitouch systems give no consideration to eliminating gestures already known to lead to MSD injuries, or to eliminating gestures that are symptomatic of a patient population.
For example, he points out that the gesture for zoom function with the iPhone is exactly the same gesture used for detection of Parkinsons disease (PD), since people in early stages of PD can’t execute this gesture, which means that iPhones are not usable by PD patients or people who may go on to develop PD — just one example on how gestures wrongly chosen for multitouch interaction can alienate certain populations or cause muscle fatigue and other ergonomic issues. Kanov contends that it is important to address this issue before we create another man-made diseases like carpal tunnel syndrome — which he calls “a hallmark of bad interaction design.”
The overall methodology to develop ergonomic gestures involves development of accurate multi-digit hand movement simulations that can predict muscle fatigue due to gestures. This enables developers to select a vocabulary of gestures that can be mapped onto task hierarchies derived through task analysis.
The project’s initial focus will to evaluate the impact multitouch devices have on the human musculoskeletal system. Users will be fitted with electromyography (EMG) equipment to measure muscle forces, and cyber-gloves to measure kinematic features produced while users interact with multitouch systems. Researchers will then evaluate the impact of those stresses.
Part two of the study will develop biomechanical models where users will be able to, as Prof. Kahol explains, “enter the motion of a gesture, and the system will produce the forces being exerted through that motion, like a specific movement of the hand. We would then take this data back to the Microsofts, the Apples and other manufacturers so they could use it when they are designing new devices.”
The system the team develops is to be built with off-the-shelf components and provide device designers a new tool to use when developing new multitouch systems.
“The designers, the computer scientists, the programmers, they know little about biomechanical systems, they just want a system that they can employ in a usable manner and tells them if a gesture causes stress or not,” says Kahol. “So our major challenge is going to be developing the software, the tool kit and the underlying models that will drive the tool kits.” He notes that the last time designers developed a fundamental interaction system with computers they modified the standard keyboard — a transition that as noted above, was not without its share of drawbacks.
“When we developed the keyboard, we didn’t think through how working with it would affect the hands, arms, etc.,” Kahol said in a statement earlier this month. “As a result, it created a multimillion dollar industry in treating carpal tunnel syndrome. That is what we want to prevent with multitouch systems. We are going for the preventative, rather than the curative. Multitouch systems might be great for usability of a device, but we just don’t know what it does to our musculoskeletal system.”
Now, hopefully we will.