The Touchy-Feely Future Of Technology

By NPR Staff

In 1975, when then-composer and performer Bill Buxton started designing his own digital musical instruments, he had no way of knowing he was helping to spark the next technological revolution. But nine years — and a master's in computer science — later, that all changed.

"I wasn't trying to make a computer interface, I was just trying to make a drum," Buxton tells NPR's Robert Siegel. "Did I envision what was going to happen today, that it would be in everybody's pocket — in their smartphone? Absolutely not. Did we realize that things were going to be different, that you could do things that we never imagined? ... Absolutely."

Today, Buxton is known as a pioneer in human-computer interaction, a field of computer science that has seen a spike in consumer demand thanks to a new, seemingly ubiquitous technology: Touch.

According to the technology, media and telecommunications company IHS iSuppli, global shipments of touch-screen cellphones and tablets have gone from 244 million units to 630 million units in just two years. This year, iPad sales nearly quadrupled compared to 2010.

But if you ask Bill Buxton, the touch explosion has been long in the making. It's part of a theory he calls The Long Nose of Innovation and it says that much of the innovation behind any technological breakthrough actually takes place over a long period of time.

Buxton became part of the long nose of touch technology when, in 1977, he signed up to study computer science at the University of Toronto. Today, Buxton is principal researcher at Microsoft Research, but he says the fact that he started out as a musician and not a technology insider has been invaluable to his work in computer science.

"It's just your imagination that's driving it and you're not trying to be so deliberate," he says. "That usually just makes you get uptight, constrained and it's far better just to find something you love doing, chase it down and the rest will just fall out."

Sci-Fi: Boldly Going Where No Science Has Gone Before

To many, the tablet computer seems new. But NPR's Laura Sydell reports that the idea for a flat, personal computer shaped like a book has actually been around for a long time. Just think of Arthur C. Clarke's 1968 novel 2001: A Space Odyssey in which space travelers follow news on Earth via a "Newspad" that downloads the world's major electronic papers.

Clarke's newspads also show up in Stanley Kubrick's 1968 film by the same name, where the fictional devices look so much like an iPad that today Samsung says it proves their Galaxy Tab isn't a rip off of Apple's iPad. They've even included a link to a YouTube clip from the film in their court documents.

Even before Space Odyssey, tablet computers had already appeared in 1966 on the original Star Trek. The first iteration was called an electronic clipboard and was used to control the ship. Around 1989, it was redesigned to ultimately do a lot of the same things iPads do today; the show's characters used it to read books, look at reports and send messages.

Usability expert Kevin Fox says he's not surprised science fiction writers came up with the tablet before science did.

"I think science fiction is the brainstorming part of science," Fox says. "Look at Jules Verne for example. He's talking about going to the moon; he's talking about submarines, that sort of thing. It's a lot easier to do that than it is to hold your tongue ... until you've actually made a rocket that can go to the moon."

How iPads Are Changing The Classroom

Now that the iPad does exist, people are finding a lot of practical applications for it. Jamestown Elementary School in Arlington County, Va., has a growing cache of iPads, about 100 for 600 students. The school uses its tablets for everything from writing to math to reading graphic novels. But NPR's Larry Abramson reports that in one classroom the iPad has been a real game changer.

Special education assistant Lesley McKeever uses an iPad to get her student, an affectionate autistic boy who can't speak, to learn to connect words with images by touching the right picture on the screen. Touch technology has been so helpful for students with autism that Arlington County provides enough iPads for every student in the special education classroom.

According to Apple, more than 2,300 school districts in the U.S. have iPad programs for students or teachers. But the benefits of having iPads in the classroom don't come free. Teachers say you have to invest time into the technology in order to get something out of it, which means much of the iPad's usefulness will depend on the applications both teachers and publishers discover as adoption grows.

The Jury's Still Out On Tablets In Hospitals

Hospitals are also exploring the usefulness of iPads. At the University of California, San Diego Hospital, physician's assistant Kate Franko uses an iPad 2 to update a patient who just received a brand new kidney on his recovery. She pulls up a graph of blood tests that charts how well his new kidney is working, then a chest X-ray from a few days ago.

It shouldn't come as a surprise that tablets are starting to find their way into hospitals, but Jenny Gold of NPR partner Kaiser Health News reports that their usefulness in the medical profession is still an open question. There are concerns about spotty wireless in hospitals, distracted doctors and the security of patient records.

Jonathan Mack of the West Wireless Health Institute, a nonprofit that works to lower the cost of health care through new technology, says another reason more hospitals aren't using tablets is that they've already invested millions into electronic record systems that aren't compatible with tablets.

"In order to go back around and deploy these on iPads with full functionality," Mack says, "it requires [that hospitals] cough up a lot more money."

And hospitals won't be willing to do that when they aren't even sure that tablets will make things easier. In the meantime, even pioneers like Franko are hedging their bets. In addition to her iPad and two iPhones, she also carries a good old-fashioned pager around — just in case.

Living The Fantasy Of Touch

Hospitals aren't the only ones expressing hesitation about touch. Professor Sherry Turkle, director of the MIT Initiative on Technology and Self, sees the rise of technology like smartphones as cause for concern. She says touch devices are more than just the latest in a sequence of high-tech distractions.

"I think that the touch pad is a very important moment because the touch pad makes our devices feel more like an extension of ourselves," Turkle tells Robert Siegel. "We feel anxious when we're not in contact [and] we feel a kind of psychic permission."

A psychic permission to, say, text in the middle of a class or even at a funeral. That's because whipping out your touch-screen smartphone is a lot more discreet then whipping out your laptop. And if touch-screen texting isn't subtle enough, Turkle says she knows of people who are trying to perfect the technique of texting without breaking eye-contact.

"This is becoming a new, highly valued social skill," she says.

(In keeping with Bill Buxton's Long Nose theory, so-called blind texting technology has actually been around for a while — since 1984, to be exact. That's the year Casio released the AT-550 watch which allowed users to write character over character on the watch in order to control calculator functions. According to Buxton, "Incredibly subtle texting is there just waiting for somebody to make a product. There's no impediment to that.")

Even without blind texting, there's something almost instinctual about the relationship people are developing with their technology. Turkle says that's because touch-screen devices appeal to a sentiment that pretty much everyone can relate to: the desire to be a kid again.

"[The] fantasy of using your body to control the virtual is a child's fantasy of their body being connected to the world," Turkle says. "That's the child's earliest experience of the world and it kind of gets broken up by the reality that you're separate from the world. And what these phones do is bring back that fantasy in the most primitive way."

And Turkle warns that living in that fantasy world could mean missing out on the real world around you.

Surface: The Next 'Bingo' Moment

Still, it's clear that the fantasy world of touch technology isn't going anywhere. Samsung and Microsoft have just released a new product that's bound to once again change the way many people interact with computers.

It's called the Samsung SUR40, or Microsoft's Surface, and while it looks a lot like a table, it's actually a multi-touch computer. The Surface at the Microsoft facility in Reston, Va., has a screen saver that makes it look like a pool of water. When you wave your hands over it, the water ripples in response. But when your hands approach the screen, a menu turns to face you. If someone across the table moves their hands to the monitor, the menu turns to face them. That's because the Surface can actually see your hand approaching it.

"Not only is it a display that can present graphical information to you, each pixel can be thought of as like a camera element in a digital camera, or scanner," says Bill Buxton, who came up with the idea for Surface.

Buxton explains the way Surface works as being similar to how two paper cups connected by a piece of string can serve as walkie-talkies. In the walkie-talkie scenario, each cup is used as both a microphone to speak into and a speaker to hear what the other person is saying. Somewhere around 1992, Buxton says he started wondering if the same thing could be done with light.

"I asked an engineer who was way smarter than me, 'Could that be done?' and the answer was 'Yes,'" Buxton recalls. "In essence Surface is the first device that really has taken that approach."

Buxton explains that the Surface monitor is made of up pixels that can both display images and read them. So if you hold a piece of paper face-down on the monitor, it can tell you what it says — and that's pretty groundbreaking.

"That opens up a whole new realm of interaction that is something that some of us have been working on for over a decade, and dreamed about maybe 20 years ago," Buxton says. "And bingo, it's starting to be manufactured."