Teaching humans and robots to work together

Could you give me a hand with this project?" "No lo entiendo."

Sometimes it can be hard to work harmoniously when you don't speak the same language.

So what can happen when the stumbling block between partners is not language or background but the simple fact that their brains work in different ways—say, if some workers depend on biological brain matter, while others get their smarts from computer chips? Can these partners get the job done? And can they do it better together than alone?

Engineering researchers at Johns Hopkins and four other universities are about to find out. With an eye toward greater productivity and enhanced safety, the scientists have joined together in a project to create new ways for humans and robots to work cooperatively.

Although robotic technology has made enormous strides, the scientists say that more attention needs to be directed at developing cooperation between humans and robots. The idea that independent machines can do all the work in a hospital or manufacturing plant with no help from humans remains a science fiction pipe dream, they say.

During the current rush toward automation, Gregory Hager, chair of the Computer Science Department in Johns Hopkins' Whiting School of Engineering, wants to keep humans in the loop. Hager is a co-principal investigator in a new four-year $3.5 million human-robot interaction research project funded by the National Science Foundation.

Modern robots, he says, can surpass humans in some procedures, but in other tasks, humans still hold the edge.

"When it comes to heavy-lifting, repetitive motions, and work that requires an absolutely steady hand, robots perform really well," he says, "but they have trouble tying knots without human help. Humans are better at handling flexible materials like string or cable. People also excel in drawing on experience and making decisions. Our goal is to figure out how people and robots can learn to work together to complete tasks that neither can do alone."

To achieve this goal, Hager's team at Johns Hopkins is collaborating with researchers at Stanford University; the University of California, Santa Cruz; the University of California, Berkeley; and the University of Washington. The project—part of a larger federal effort called the National Robotics Initiative—will address manipulation problems "that are repetitive, injury-causing, or dangerous for humans to perform yet are currently impossible to reliably achieve with purely autonomous robots," the researchers said in their grant application.

The issues of concern, they say, are generally those that call for dexterity, complex perception, and complex physical interaction. "We conjecture that many such problems can be reliably addressed with human-robot collaborative systems, where one or more humans provide needed perception and adaptability, working with one or more robot systems that provide speed, precision, accuracy, and dexterity," they wrote.

In particular, the researchers will focus on improving human-robot teamwork in two key areas: manufacturing procedures that involve a small number of objects, and medical tasks, including suturing and dissection.

One company the robotic technology experts are working with assembles wiring harnesses for commercial airplanes. This process involves a tying operation now carried out by humans, work that results in uneven product quality and carries a risk of repetitive-motion injuries. Finding a smooth way to weave robots into this process could reduce such problems, increase work efficiency, and set an example for future manufacturing innovations, the researchers say.

The other focal point is robot-assisted surgery. Today's da Vinci surgical system uses robots that mimic the hand motions of an operator sitting at a console near the patient. The operator's role is now limited by the technology, but the researchers intend to work with the system's maker to expand and improve the surgical tasks that can be performed by a human-robot partnership. The goal is to improve speed, accuracy, and precision, and allow the robot to handle more of the repetitive motions.

In both the manufacturing and medical procedures, the researchers also will seek to enhance the verbal and tactile communication between the human and robot partners.

"It's hard to get robots and people to work together, and it's even harder to get multiple people and multiple robots to work together," says Allison Okamura, a Stanford associate professor of mechanical engineering, who is lead investigator on the NSF grant. "A main focus of our project is to get robots to understand what people are doing and be able to step in when necessary."