Student Gyorgy Levay develops foot-operated game controller

Gyorgy Levay ordinarily doesn't have time to play video games. The Johns Hopkins biomedical engineering master's candidate is too busy working on controls for upper-limb prostheses to find time for running and jumping around the tops of buildings in the parkour game Mirror's Edge. Even if he did, Levay lost both hands to a meningitis infection five years ago, and operating the keyboard for a first-person shooter game is difficult. Over the 2015–16 school year, however, Levay spent considerable time running around a video game's virtual world.

Image caption: Johns Hopkins biomedical engineering grad students, from left, Adam Li, Nate Tran and Gyorgy Levay displayed their innovative video gaming shoes at a recent design competition.

Image credit: John Bidlack / Homewood Photography

He was product testing a foot-operated controller that he and two classmates invented. "When I was testing it, or at least I could call it testing, I actually played a lot," Levay says. "Now that I don't have a good excuse, I don't have the time."

The native of Budapest, Hungary, teamed up with classmates Adam Li and Nhat Tran in a biomedical instrumentation course, where a semesterlong assignment was to design an alternative-control device. Levay told them he has trouble controlling computers, specifically games, with his arms but knew he could use his feet, and suggested tailoring an idea around him as a case study. Their GEAR controller, short for Game Enhancing Augmented Reality, won a $7,500 grand prize in the 2016 Intel-Cornell Cup, a competition among student inventors, and was named a finalist in the inaugural Student Healthcare Design Competition, which invited participation from undergraduate and graduate students across Johns Hopkins.

While the finished product isn't retail-ready—the GEAR controller looks a bit like a pair of orthopedic sandals with thick plastic soles ("We're not exactly designers," Levay admits)—its functionality is impressive. Each foot controller has three buttons: one at the heel and two on each side of the ball of the foot. The two buttons at the ball can be pressed at the same time, giving each foot a total of four actions that can be programmed into any combination.

The team designed the controller with the natural motion of feet in mind. "If you want to start a step, you're going to put pressure on the forward part of your feet," Levay says. "If you go back, you're going to put more pressure on your heels. In that sense, [the controller] is very intuitive. It tries to simulate how you would move on your own." The students tested and demonstrated the controller with a variety of games, including Mirror's Edge, and they programmed different button combinations for the game's motions such as jump and crouch. Those movements are less intuitive but no more complicated than using the space bar to jump and the shift key to crouch. "The learning curve was pretty fast," Levay says of gamers testing the controller, adding that many took only about half an hour to grow accustomed to using it. "We were actually surprised at how effective it was. When we built the second prototype and started using it, it was just so incredibly easy to use, and it did exactly what we wanted it to do. It didn't take a long time to figure out how it worked."

Image caption: The GEAR team devised these game controller shoes that respond to different foot movements.

Image credit: John Bidlack / Homewood Photography

Levay knows a market exists for this product, and not simply for amputees who want to play games. Keyboard, joystick, trackball, mouse—perhaps thinking only in terms of two-hand computer controls is a limitation. Why not add feet to the design process? "We've been using hands to control computers for forever," Levay says, and brings up virtual reality. Currently, a standard VR setup includes a headset and a hand controller for movement. "When you're immersed in a virtual reality, you might as well control your movement with your feet, as you do in real life. One way or another, something like this is going to come out, whether this design or someone comes up with something even more effective."

GEAR may end up a mere footnote in a future innovative product's development story. Levay says he, Li, and Tran aren't in a position to create a startup around it, and his academic research has prevented him from pitching the idea to potential investors. At least they created a controller that allows Levay to play games again, should he ever find the time. "I played a lot before I lost my hands, and I can play most games now that don't involve a lot of movement," he says. "But this kind of forward, backward type of movement, I can't play those games at all without a device like this."