CLSP Spring Seminar Series: Yulia Tsvetkov

Description

Yulia Tsvetkov, an associate professor at the Paul G. Allen School of Computer Science & Engineering at the University of Washington, will present a Center for Language and Speech Processing seminar titled "LLMs Under the Microscope: Illuminating the Blind Spots and Improving the Reliability of Language Models."

Abstract:

Large language models (LLMs) are pretrained on diverse data sources—news, discussion forums, books, online encyclopedias. A significant portion of this data includes facts and opinions which, on one hand, celebrate democracy and diversity of ideas, and on the other hand are inherently socially biased. In this talk. I'll present our recent work proposing new methods to (1) measure media biases in LLMs trained on such corpora, along the social and economic axes, and (2) measure the fairness of downstream natural language processing (NLP) models trained on top of politically biased LLMs. In this study, we find that pretrained LLMs do have political leanings which reinforce the polarization present in pretraining corpora, propagating social biases into social-oriented tasks such as hate speech and misinformation detection. In the second part of my talk, I'll discuss ideas on mitigating LLMs' unfairness. Rather than debiasing models—which, our work shows, is impossible—we propose to understand, calibrate, and better control for their social impacts using modular methods in which diverse LLMs collaborate at inference time.