Skip to main content

Roboticist sees improvisation through machine's eyes

By Todd Leopold, CNN
updated 5:12 PM EST, Fri February 3, 2012
Heather Knight, who specializes in social robotics, sees the robot Data as a way of investigating her area of study.
Heather Knight, who specializes in social robotics, sees the robot Data as a way of investigating her area of study.
STORY HIGHLIGHTS
  • Roboticist Heather Knight has a performing robot named Data
  • Data's abilities include joke-telling and dancing
  • Data's reactive abilities highlight true human improvisation, a key creative trait
  • Human brain is capable of so much -- and must be kept limber to be at peak

Editor's note: This is the fourth in a weekly series on characteristics of creativity. Next week's piece will focus on acting teacher Elizabeth Kemp and the role of mentors.

Pittsburgh (CNN) -- The comedian, a small robot named Data, takes the stage.

Rising from a crouch, as if psyching himself up for his audience, he waves. "It's an honor to be here," he says in a voice that makes him sound like a geeky adolescent boy with a helium-sucking addiction.

He launches right in to his routine, telling a doctor-patient joke, a Swiss army joke, old chestnuts from Fred Allen and Groucho Marx such as the line about why television is called a medium -- it's neither rare nor well done. (Rimshot, please.) His owner and programmer, roboticist Heather Knight, looks on with amusement.

To a human's eyes, Data is engaged in rudimentary improvisation: "the ability to respond effectively to the unexpected and unplanned," in the words of creativity consultant and DePaul University instructor Pamela Meyer.

The audience applauds (or doesn't) and flashes colored cards handed out by Knight before the performance -- green for approval, red for disapproval. Data adjusts accordingly, his sensors noting color and noise level, tapping into his programming to adopt from a variety of options -- changing jokes, expressing disappointment.

Knight is a social roboticist, and having Data improvise is a way for her to investigate how to connect people better with technology.

What's Next: A gal and her robot

A girl and her robot
Storytelling with robots

Improvisation is an activity humans often take for granted, the "prototypical creative behavior," writes researcher Shelley Carson in her book "Your Creative Brain." On a basic level, everybody improvises, because there are many daily situations that require spur-of-the-moment actions in which we draw from experience and choose unconsciously: avoiding a collision on a highway, for example, or throwing together a meal when you're missing an ingredient.

But on a more creative and mindful level, improvisation has "value," says creativity expert Robert Epstein, a former editor of Psychology Today and founder of the Cambridge Center for Behavioral Studies. It can help generate new ideas and new ways of doing things.

Computers have been able to do that for some time, he points out. "There have been programs that have been around for decades that will get a computer to generate new material that crosses that line into the significant new," such as a drawing program created by artist Harold Cohen.

So why are we so much better at improvising than robots -- which are essentially mechanical devices controlled by computers? You'd think that, by now, some form of artificial intelligence -- the iPhone's Siri or "Jeopardy's" Watson or perhaps some combination of IBM's Deep Blue and MIT's Kismet -- would have been able to meet the challenge.

But Data -- named for the "Star Trek" android that yearns to be human -- and robots like him lack certain human attributes, most notably experience. That training to think, react and do includes not just conceptual knowledge but muscular and sensory memory -- and starts the moment we're born. Moreover, we don't just draw from our own history, but from the histories and reactions of those around us and before us.

Computers are well short of having the memory power needed to acquire and employ that experience. The human brain, the organ that keeps track of our lives, remains one of the most remarkable creations ever known. In a 2011 article for the scholarly journal Science, researchers Martin Hilbert and Priscila Lopez estimated that the entire processing power of the world's stock of general-purpose computers in 2007 more or less equaled that of one human brain (about 6.4 quintillion instructions per second -- the standard measure of computer speed).

Stefan Schaal, a professor of neuroscience and biomedical engineering at the University of Southern California, stresses the importance of life experience in learning.

"One beauty about humans, and animals in general, is we learn developmentally," Schaal says. We may look at a robot as being incredibly adultlike and smart: Machines such as IBM's chess-playing Deep Blue and "Jeopardy!"-playing Watson, for example, defeated some of their human counterparts through sheer memory power. But they're still like small children compared with the vast scope of human ability.

"Take the manipulation skills -- working with tools and toys -- of a 3-year-old. We have no robot that comes close to that," Schaal says. "Or perceptual skills, vision and seeing and categorizing what's out in the world, of a 3-year-old -- we have no system that comes even close to that."

Improvising practice

Which isn't to say Data -- a prefab model called the Nao from the Paris-based company Aldebaran Robotics -- doesn't have at least some small capabilities. His hardware includes two cameras, an accelerometer, position sensors in his joints and electric current sensors so he knows if he's pushing hard against something. Knight has added jokes, monologues and gestures to Data's memory.

He's "basically a laptop computer, plus a huge amount of sensors," Knight says.

We can overprogram our time ... and that can essentially suck out that time for serendipity and improvisation.
Creativity consultant Pamela Meyer

A doctoral student at Carnegie Mellon University's Robotics Institute and owner of a firm called Marilyn Monrobot, Knight specializes in social robotics -- the interaction of humans and robots. Data is an entertainer who -- through his sensors, memory and programming -- provides a foundation for a future of routine human-robot relationships.

For Knight, putting Data on stage is a means to an end.

"I think that robotics has a tremendous amount to work with from theater," she said, her speech alternately full of girlish enthusiasm and steady with technical detail. Theater, she adds, is focused explicitly on character interactions -- and her research is engaged in how to help robots "read" people and vice versa.

Familiarity breeds improvisation ability. Think of the three-man weave, a popular basketball exercise. Three players run up and down the court, weaving to the inside or outside while passing the ball to one another. The players must learn to adjust the speed of their pace, the accuracy of their passes (which, optimally, are launched to a place a player is expected to be, not where he is) and any number of physical idiosyncrasies. In a game situation, all that practice can mean the difference between easy layups or passes that go sailing out of bounds.

A colleague of Knight's, new media artist Golan Levin, has helped create intriguingly similar tasks for robots. In one, four iCreate robots -- Roomba vacuum cleaner-type machines -- were put through the paces of Samuel Beckett's play "Quad." It's a wordless, scripted piece that focuses on movement, but there's enough randomness in the staging for the actors to create idiosyncrasies -- which, in turn, they must react to.

With enough experience, improvisation comes naturally -- which allows the improviser to combine years of expertise with a sense of freedom into the unknown. Think of a jazz musician embarking on a solo, or a chess player seizing on a sudden strategy.

The concept has become codified in the "10,000-hour rule," based on a study by the psychologist Anders Ericsson that Malcolm Gladwell popularized in his book "Outliers" -- the idea that raw talent is honed by constant practice. Through that process, a person gains expertise and comfort with the "rules" -- and a comfort with breaking them. Gladwell's examples include the Beatles, who put in their 10,000 hours through their nightly, hourslong Hamburg shows. In 1960, when the Beatles first traveled to Germany, the band was still finding its way. By 1963, the group was a tight unit -- and ready to take pop music in unforeseen directions.

"(W)hen a challenging activity is encountered for which you have developed expertise, you will respond impulsively and automatically with one of your implicitly stored expert training programs," writes "Your Creative Brain's" Carson. It's an area in which humans still have a big edge on machines -- even machines that play "Jeopardy!"

Creating space

How does improvisation make us more creative?

From a personal perspective, it opens us up, says Dave Buckman, who directs and performs with the Austin, Texas-based improv group the Frank Mills.

"A lot of people have hang-ups. They need a wall of protection to guard against other ideas," says Buckman, who frequently works with corporate clients. "If you get over the fear of being seen in a vulnerable light, it lends itself to other areas of life."

For companies, improvisation can lead to new discoveries that may not have even been on their radar, adds DePaul's Meyer.

"Valuing improvisation creates space for emergent creativity, (so we don't) become tightly fixed to our agendas," Meyer says. "If we allow room for improvisation, some of the wonderful, unexpected things may come up that take organizations in new directions."

There's a story, perhaps apocryphal, concerning a daydreaming staffer of Henry Ford. The auto magnate had hired an efficiency expert to improve operations at the company, and the expert suggested Ford fire the man. Ford refused.

But Mr. Ford, the stunned expert insisted, every time I walk by his office he's got his feet up on his desk and he's staring at the ceiling. He hasn't done a lick of work since I got here.

Sir, Ford replied, that man once had an idea that saved this company millions of dollars. And he was sitting in just that position when he had that idea.

Improvisation is also a way of recharging the brain. Sometimes we have to be willing to let the mind wander to take creative leaps. (It's no wonder so many good ideas come to us, almost unbidden, in the shower or while taking a walk; the mind is relaxed and free.)

With a constant stream of stimuli we face -- the Internet, television, music, all the noise of 21st-century life -- it may be harder today than ever before to create space for such mental relaxation. John Seely Brown, the former head of Xerox's Palo Alto Research Center lab, says he's concerned that recent generations, under the constant spell of activity, are losing that ability.

"I think kids today are afraid of being bored," he says. "And that's when you imagine something."

It's a topic that comes up with increasing frequency these days, most notably in Nicholas Carr's "The Shallows." Carr's thesis, memorably summarized by the title of his Atlantic article -- "Is Google Making Us Stupid?" -- is that by immersing ourselves in the Internet, humans are becoming less present: that we are browsing, not thinking, and possibly rewiring our brains in the bargain.

We have to elevate the value of unstructured time, says Meyer, who points out that the most effective improvisation happens in a sweet spot between the extremes of boredom (uninterest) and panic (freezing up) -- a groove, as a musician might put it. Within those extremes, improvisation can be thought of as a continuum between unthinking reactions and fully engaged playfulness.

"We can overprogram our time, whether we're younger or have full adult responsibilities, and that can essentially suck out that time for serendipity and improvisation," she says.

Away from HAL

Data tells jokes, dances to \
Data tells jokes, dances to "Thriller" and paraphrases Shakespeare -- with help from roboticist Heather Knight.

There may come a day when computers become more human. Futurists are fond of talking about the "singularity," the ultimate merging of human and machine. Perhaps eventually we'll create a computer as ingenious (but not as flawed) as "2001: A Space Odyssey's" HAL 9000.

But HAL isn't here yet, and neither are any of his sentient brethren. Seemingly brilliant machines such as Watson and Deep Blue are a mile deep but an inch wide. They can do one thing very, very well -- even going so far as to learn from their mistakes in their area of specialization -- but they don't have a wider context in which to view the world. A robot such as Knight's Data might be able to engage in improvisational behavior but doesn't have the combination of "knowledge" (programming) and experience to match a human.

Still, it's the journey that keeps Knight enthusiastic. She's programmed the "boyish and overconfident" robot to do a routine with her, attempting to impress her with wit and dance moves to "Thriller." He quotes Shakespeare and replaces it with robot-centric references: "If you prick us in our battery pack, do we not bleed our alkaline fluid?"

The sketch routine, like the stand-up, had its bugs: At a performance last summer, the little robot shut down as he was starting his act.

She hit his on-off switch and rebooted him: her own rudimentary improvisation.

Data is just the beginning, she imagines. She has plans ("Don't tell him this," she asks impishly) to work with other robots, investigating how robots and humans engage until robots become a common part of our lives. Asked if she's concerned about her work being a step in creating something malevolent, she scoffs.

"What's much more interesting is the idea of how we can work together," she says. "There's already ways technology has completely transformed the way we operate on a daily level and how we understand the world. Technology always transforms society, but it can also empower society."

Robots already work in factories; in years to come, they'll become personal assistants in ways Siri can't even comprehend.

And working with robots, she emphasizes, has actually highlighted what a complex and wonderful machine the human brain is -- capable of taking giant leaps, even imagining robots that tell jokes and dance.

"Things our brains have been specifically created to do over thousands and millions of years are difficult to duplicate with machines," she says. "But it has helped me, in working with systems that work with humans, to see how complex human behavior actually is."

ADVERTISEMENT
Part of complete coverage on
Spark
updated 5:20 PM EST, Tue January 17, 2012
Brian Wilson
Brian Wilson has long been labeled "the troubled musical genius" of the Beach Boys. You can try to put him in a box. But he won't stay there.
updated 10:13 AM EST, Sat January 28, 2012
Few would consider Pulitzer winner Jennifer Egan a failure. But she once did, oddly enough after getting her first story published in The New Yorker.
updated 10:50 AM EST, Mon January 30, 2012
Paul D. Miller, aka DJ Spooky, combines genres and styles to create something new. It's a characteristic key to spreading ideas -- and innovations.
updated 5:12 PM EST, Fri February 3, 2012
Data the robot
Heather Knight uses a stand-up comedian robot to look at the relationships between people and machines -- and what it means to be human.
updated 10:40 AM EST, Mon February 13, 2012
Elizabeth Kemp trained with Elia Kazan and played opposite Tom Hanks before she hit a wall. It would propel her into her greatest role: mentor.
updated 3:42 AM EST, Sat February 18, 2012
Kelly and Woods
Grace Kelly has dazzled listeners with her sax playing since she was 9. What is the secret to such young success -- talent, hard work, passion?
ADVERTISEMENT
ADVERTISEMENT
ADVERTISEMENT
ADVERTISEMENT