Nice Gesture, But What Does It Mean?

 

By Tim Kridel

One step forward, two steps back. That’s how Don Norman describes [1] today’s gesture-based user interfaces (UIs) for smartphones, tablets and a growing assortment of other devices.

Named one of the world’s 27 most influential designers by Business Week, Norman [2]laments the lack of standards, which have created a world where a finger-swipe on one device often doesn’t have the same effect on another. That inconsistency often makes using gesture-based UIs as much fun as folding a fitted sheet. Norman spoke to me about this and more from South Korea, where he was a distinguished visiting professor in the Korea Advanced Institute of Science and Technology’s Department of Industrial Design.

Q: Your colleagues from the Nielsen Norman Group [3] recently did usability tests on Apple’s iPad: “The first crop of iPad apps revived memories of Web designs from 1993, when Mosaic first introduced the image map that made it possible for any part of any picture to become a UI element. As a result, graphic designers went wild -- anything they could draw could be a UI, whether it made sense or not. It’s the same with iPad apps -- anything you can show and touch can be a UI on this device. There are no standards and no expectations.” Yet tablet and smartphone sales remain brisk.

A: It’s confusing and difficult for people. On the other hand, it’s so engaging and so much fun, that it in many ways compensates for its difficulties.

I’m about to give a series of talks about the way the field has evolved. In the beginning, we were so delighted with the technology. With the passage of time, we’ve come to take understandability, usability and function for granted. What we want now is a good experience, some fun and delight.

That’s where Apple has transformed itself as a company. It really is about experience, fun, delight and entertainment. That’s what the iPhone did: By this whole new way of interacting with strokes and gestures, it was so delightful and different.

Q: In “Gestural Interfaces: A Step Backwards in Usability,” you and Jakob Nielsen [4] identified two root causes for gesture UI inconsistencies: a lack of industry guidelines and “the misguided insistence by companies (e.g., Apple and Google) to ignore established conventions and establish ill-conceived new ones.” Will we ever have standards for gesture UIs?

A: There will be gestures that Apple will patent and refuse to let Microsoft and Google use. All of the companies will have competing methods that will just confuse the hell out of people. But we had that in the beginning of computers too.

When I was at Apple as vice president of the Advanced Technology Group in the ’90s, I was fighting hard for standards. We had this wonderful meeting, and we got all of the major companies together to argue for standards: Sun, IBM, Apple. Microsoft walks in and says, “This meeting is unnecessary. If you want standards, just use Microsoft’s.” That’s Apple’s attitude today.

Q: One downside of standards -- whether it’s a UI or a wireless technology -- is that they sometimes don’t leave companies with many opportunities for market differentiation, aside from price.

A: Every Android phone basically looks the same. So how do you compete in a world like that, where if you really follow Google’s guidelines, you can’t distinguish LG from Samsung from HTC from Motorola? Nobody wants to compete on price. That’s death.

With early technologies, as people are learning, it’s understandable that we don’t have standards and that sometimes different applications don’t follow the same rules. Jakob and I wrote our joint piece to be critical yet friendly, to say that we thought gestures and the new interfaces were exciting, powerful, adding a lot to our experience, and that this early confusion can be overcome. This confusion is compounded by the lack of differentiation among the vendors, who are desperate to do anything different, and the companies don’t wish to cooperate.

Q: For now, consumers and business users seem to be willing to put up with those inconsistencies, judging by how many iOS and Android apps and devices there are. How long before the gee-whiz wears off and they start to complain that app and device UIs aren’t as user-friendly as they could or should be?

A: It’s starting to wear off already. When the iPad came out, people just swooned over it. Now you’re starting to see articles, sometimes by the same journalists, saying, “You know, it’s got a lot of weaknesses. You can’t really type on it.” I think it’s becoming more realistic that the pad is not a substitute for a computer. It’s a wonderful device, but for its own purpose: You can read on the couch or answer a quick question. That’s a reality that wasn’t there at first.

Q: You and Jakob also bemoaned the lack of “Undo” in today’s gesture UIs. That absence is surprising, considering that PCs have conditioned us to expect it, and it’s useful.

A: I think because it comes a little out of the browser world, where the theory is that the “Back” button is the “Undo.” But it isn’t, not when you type something. On top of that, “Back” has always been very confusing. You’re never sure what’s going to happen when you hit “Back.”

I really hate this. There’s a stack of the previous locations, and “Back” takes you to the previous location. But I’m in some app, and I’m not sure where I am because it doesn’t tell me, and I want to get back to what I think is the home page. I hit “Back” and, whoops, I’m on the desktop.

It shouldn’t do that. You shouldn’t be able to back out of the application. In the browser, when you reach the end of its list, it doesn’t close the browser or take you to the desktop. It just dims the back button and doesn’t work anymore.

That’s what ought to happen when you hit “Back” in an application. I got really angry at one application developer. I kept saying, “Why are you doing this?” They explained that it wasn’t them, that the “Back” was programmed into the operating system.

Q: That means operating system companies have to provide a foundation that enables device vendors and app developers to make UIs more user-friendly.

A: Yes, it does. And they know that. That’s why Apple has usability guidelines. I’m not sure if Google does. There are people who have written them for Google.

Photo: @iStockphoto.com/studiocasper

--

Tim Kridel has been covering all things tech and telecom since 1998 for a variety of publications and analyst firms. Based in Columbia, Mo., he still enjoys the childhood hobby that led to a career writing about technology: ham radio. He is a frequent contributor to Digital Innovation Gazette [5].