CLSP Spring Seminar Series: Maha Elbayad

Description

Maha Elbayad, a senior research scientist at Meta AI, will present a Center for Language and Speech Processing seminar titled "Large Concept Models: Language Modeling in a Sentence Representation Space."

Abstract:

While large language models (LLMs) have revolutionized AI, their token-level processing contrasts sharply with human cognition's multi-level abstraction. This talk explores moving beyond token-based manipulation to reason in a latent space. We introduce the Large Concept Model (LCM), an architecture that operates on language- and modality-agnostic "concepts," represented as sentence embeddings within the SONAR space. Trained for autoregressive sentence prediction, the LCM learns to reason and generate at a higher semantic level. Evaluated on generative tasks like summarization and summary expansion, the LCM demonstrates impressive zero-shot generalization across multiple languages. Crucially, this concept-based representation within the SONAR space naturally facilitates multimodal extension, enabling the model to reason about and generate content grounded in diverse sensory inputs. This paves the way for more robust and human-like AI systems.