Johns Hopkins and Amazon collaborate to explore transformative power of AI

Johns Hopkins University and Amazon are teaming up to harness the power of artificial intelligence to transform the way humans interact online and with the world. The new JHU + Amazon Initiative for Interactive AI, housed in the Johns Hopkins Whiting School of Engineering, will leverage the university's world-class expertise in interactive AI to advance groundbreaking technologies in machine learning, computer vision, natural language understanding, and speech processing; democratize access to the benefits of AI innovations; and broaden participation in research from diverse, interdisciplinary scholars and other innovators.

Amazon's investment will span five years, comprising doctoral fellowships, sponsored research funding, gift funding, and community projects. Sanjeev Khudanpur, an associate professor of electrical and computer engineering at the Whiting School, will serve as the initiative's founding director. Khudanpur is an expert in the application of information-theoretic methods to human language technologies such as automatic speech recognition, machine translation, and natural language processing.

"Hopkins is already renowned for its pioneering work in these areas of AI, and working with Amazon researchers will accelerate the timetable for the next big strides," Khudanpur said. "I often compare humans and AI to Luke Skywalker and R2D2 in Star Wars: They're able to accomplish amazing feats in a tiny X-wing fighter because they interact effectively to align their complementary strengths. I am very excited at the prospect of the Hopkins AI community coming together under the auspices of this initiative, and charting the future of transformational, interactive AI together with Amazon researchers,"

Ed Schlesinger, dean of the Whiting School, said, "We are very excited to work with Amazon in this new initiative. We value the challenges that they bring us and the life-changing potential of the solutions we will create together, and look forward to strengthening our work together over the coming years."

Amazon's funding will support a broad range of activities, including annual fellowships for doctoral students; research projects led by Hopkins Engineering faculty in collaboration with postdoctoral researchers, undergraduate and graduate students, and research staff; and events and activities, such as lectures, workshops, and competitions aimed at making AI activities more accessible to the general public in the Baltimore-Washington region.

Prem Natarajan, Alexa AI vice president of natural understanding, says the partnership underscores Amazon's commitment to addressing the greatest challenges in Al, democratizing access to the benefits of Al innovations, and broadening participation in research from diverse, interdisciplinary scholars and other innovators.

"This initiative brings together the top talent at Amazon and Johns Hopkins in a joint mission to drive groundbreaking advances in interactive and multimodal AI," Natarajan said. "These advances will power the next generation of interactive AI experiences across a wide variety of domains—from home productivity to entertainment to health."

The two organizations have teamed up in the past, with four Johns Hopkins faculty members joining Amazon as part of its Scholars program: Ozge Sahin, a professor of operations management and business analytics at the Johns Hopkins Carey Business School, in 2019, and in 2020, Gregory Hager, Mandell Bellmore Professor of Computer Science; René Vidal, Herschel Seder Professor of Biomedical Engineering and director of the Mathematical Institute for Data Science; and Marin Kobilarov, associate professor of mechanical engineering.

The new initiative will build on Hopkins Engineering's existing strengths in the areas of machine learning, computer vision, natural language understanding, and speech processing. Its Mathematical Institute for Data Science conducts cutting-edge research on the mathematical, statistical, and computational foundations of machine learning and computer vision. The Center for Imaging Science and the Laboratory for Computational Sensing and Robotics conduct fundamental and applied research in nearly every area of basic and applied computer vision. The university's Center for Language and Speech Processing, one of the largest and most influential academic research centers of its kind in the world, conducts research in acoustic processing, automatic speech recognition, cognitive modeling, computational linguistics, information extraction, machine translation, and text analysis. CLSP researchers conducted some of the foundational research that led to the development of digital voice assistants.

"AI has tremendous potential to enhance human abilities, and to reach it, AI of the future will interact with humans the same way we naturally interact with each other. What endeared Amazon Alexa to users was the effortlessness of the interaction. I envision that the research done under this initiative will make it possible for us to use much more powerful AI in equally effortless ways, regardless of our own physical limitations," Khudanpur said.

Hager, a director for Amazon Physical Retail, and Vidal, currently an Amazon Scholar in visual search and AR, were instrumental in helping Amazon and JHU establish the collaboration.

"Computer vision and machine learning are transforming the way in which humans shop, share content, and interact with each other," Vidal said. "This partnership will lead to new collaborations between JHU and Amazon scientists that will help translate cutting-edge advances in deep learning and visual recognition into algorithms that help humans interact with the world."

Seth Zonies, a director of business development for Johns Hopkins Technology Ventures, the university's commercialization and industry collaboration arm, said, "This collaboration represents the opportunity to harness academic ingenuity to address needs in society through industry collaboration. The engineering faculty at Johns Hopkins are committed to applied research, and Amazon is at the forefront of product development in this field. We expect this collaboration to result in deployable, high-impact innovation."