CLSP Spring Seminar Series: Ioana Ciucă

Description

Ioana "Jo" Ciucă, an interdisciplinary Jubilee Joint Fellow at the Australian National University, will present a Center for Language and Speech Processing seminar titled "A Universe to Be Decided: Toward Specialized Foundation Models for Advancing Astronomy."

Abstract:

I discuss the application of Foundation Models in Astronomy through the collaborative efforts of the UniverseTBD consortium with a mission to democratize science for everyone. One of our key objectives is to overcome the limitations of general-purpose foundation models, such as producing limited information in specialized fields. To this end, we have developed the first specialized large language model for astronomy, AstroLLaMa-1. This model, enhanced by exposure to domain-specific literature from the NASA Astrophysics Data System and ArXiv, demonstrates improved text completion and embedding capabilities over existent generative pre-trained transformer (GPT) models. I further discuss the potential of large language models (LLMs) in generating complex scientific hypotheses and extracting meaningful insights from astronomy literature. Our findings, validated by human experts, demonstrate the LLM capability in informed scientific critique and uncover intriguing patterns in the embedding space, highlighting the potential of LLMs to augment scientific inquiry. I will also discuss preliminary work with the multi-modal model AstroLLaVA, which allows us to interact with astronomical images via natural language. Through the work of UniverseTBD, we aim to explore how artificial intelligence can assist human intelligence in astronomy and, more broadly, science.