CS/CLSP Seminar: Jaemin Cho

Description

Jaemin Cho, a doctoral candidate in the Department of Computer Science at the University of North Carolina at Chapel Hill, will give a talk titled "Faithful Reasoning and Fine-Grained Evaluation for Multimodal Generation" as a joint seminar for the Department of Computer Science and the Center for Language and Speech Processing.

Abstract:

The paradigm of training large-scale foundation models has driven significant advancements in multimodal artificial intelligence (AI). However, pursuing further performance gains solely through model scaling is becoming impractical due to rising computational costs and resource limitations. Moreover, the reasoning and generation processes of these models remain mostly uninterpretable and uncontrollable, often leading to unfaithful outputs. In this talk, Jaemin Cho will discuss his efforts to make multimodal generative models more controllable and trustworthy without increasing their size. First, he will introduce faithful reasoning frameworks, in which the multimodal generation process mirrors how humans reason about and create content such as images and videos. Concretely, in these frameworks, models create a detailed plan that decomposes a complex generation task into simpler steps, as well as retrieve relevant information from multimodal knowledge bases before generating the final outputs. Next, Cho will describe fine-grained evaluation methods that assess model capabilities across multiple dimensions, such as object counting and spatial relation understanding, thereby providing a detailed understanding of the models' strengths and weaknesses. In turn, these evaluations enable targeted model improvements that address identified weaknesses through test-time guidance or by updating training environments. Together, these directions offer a pathway toward more intelligent, reliable, and efficient multimodal AI models.