The Johns Hopkins University Applied Physics Laboratory has received $4 million in funding from the Mann Fund to develop a next-generation retinal prosthesis system. The Mann Fund was created by philanthropist Alfred E. Mann 15 years ago to support the development of biomedical technologies.
Image caption: Harmonie collaborator Second Sight’s artificial retina device, which bypasses damaged rods and cones in the retina.
Image credit: COURTESY SECOND SIGHT
This work will be conducted in close collaboration with Second Sight Medical Products, a Sylmar, California–based company that develops, manufactures, and markets implantable visual prosthetics for blind individuals.
APL will be developing the next-generation glasses with embedded vision- and eye-tracking sensors. Using these sensors and onboard hardware, the system will identify potential obstacles, doorways, hallways, and household objects and their relative positions. This information will be distilled into a format that can be projected into the retinal prosthesis, bypassing the damaged rods and cones in the retina.
The components of this retinal prosthesis will enable APL's broader vision of a semiautonomous controller for assistive robotic manipulators and remote devices, the Hybrid Augmented Reality Multimodal Operation Neural Integration Environment, or Harmonie.
Harmonie is the latest spinoff effort from the Laboratory's Revolutionizing Prosthetics Program. As part of that effort, researchers investigated a number of different control modalities for moving a modular prosthetic limb and were able to achieve impressive control. But they discovered that users had to remain very focused on what they were doing, creating a lot of cognitive burden.
Harmonie came about as a way to remove that burden. The basic idea behind the system is to combine elements of computer vision (i.e., identification of objects and where they are), autonomous manipulation (how do I move the arm to the desired object's position, what's the best grasp to use?), and a user interface (how do I tell what actions to perform on the object?).
The current implementation uses off-the-shelf components such as the Microsoft Kinect for the vision system, but Harmonie project manager Kapil Katyal says that the ultimate goal is to integrate these technologies into a small form factor glasses-type device.
The Harmonie system has been deployed in clinical settings at Caltech and Johns Hopkins and is undergoing additional testing as part of an internally funded project, the Clinical Evaluation of Emerging Rehabilitative Technologies. It also was demonstrated in December 2013 at the DARPA Robotics Challenge Exposition, where the Lab's RoboSally used the technology to semiautonomously reach and grasp a fire extinguisher.
Katyal says that the developers hope to have a prototype ready for market in three and a half years.
"This system would enable a broad base of users to efficiently and effectively control dexterous manipulators like the [modular prosthetic limb] and would also allow APL to develop innovative solutions in contemporary general purpose human-computer interaction technologies such as head-mounted displays, augmented reality, eye-tracking, and brain-computer interfaces," he says. "Moreover, the synergies between the Harmonie system and Second Sight's next-generation retinal prosthesis provide an opportunity for the Laboratory to expand its footprint in neuroprosthetic research and development and enhance our reputation in this field."
Michael McLoughlin, Harmonie program manager and principal investigator for the Revolutionizing Prosthetics Program, says that the Harmonie project is synergistic with ongoing, independent research and development efforts at APL. "It's an extension of what we've been trying to do with the prosthetics program from the beginning, and promises to address the needs of a whole new class of patients," he says.
Posted in Science+Technology
Tagged applied physics laboratory, prosthetics, vision
