CS Seminar: Brenden Lake

Description

Brenden Lake, an assistant professor of psychology and data science at New York University, will give a talk titled "Towards More Human-Like Learning in Machines: Bridging the Data and Generalization Gaps" for the Department of Computer Science.

Abstract:

There is an enormous data gap between how AI systems and children learn language: The best LLMs now learn language from text with a word count in the trillions, whereas it would take a child roughly 100K years to reach those numbers through speech. There is also a clear generalization gap: Whereas machines struggle with systematic generalization, people excel. For instance, once a child learns how to "skip," they immediately know how to "skip twice" or "skip around the room with their hands up" due to their compositional skills. In this talk, Brenden Lake will describe two case studies in addressing these gaps.

The first addresses the data gap, in which deep neural networks were trained from scratch, not on large-scale data from the web, but through the eyes and ears of a single child. Using head-mounted video recordings from a child, this study shows how deep neural networks can acquire many word-referent mappings, generalize to novel visual referents, and achieve multi-modal alignment. The results demonstrate how today's AI models are capable of learning key aspects of children's early knowledge from realistic input.

The second case study addresses the generalization gap. Can neural networks capture human-like systematic generalization? This study addresses a 35-year-old debate catalyzed by Fodor and Pylyshyn's classic article, which argued that standard neural networks are not viable models of the mind because they lack systematic compositionality—the algebraic ability to understand and produce novel combinations from known components. This study shows how neural networks can achieve humanlike systematic generalization when trained through meta-learning for compositionality (MLC), a new method for optimizing the compositional skills of neural networks through practice. With MLC, a neural network can match human performance and solve several machine learning benchmarks.

Given this work, we'll discuss the paths forward for building machines that learn, generalize, and interact in more humanlike ways based on more natural input.