CS Seminar Series: Beidi Chen

Description

Beidi Chen, a postdoctoral scholar in the Computer Science Department at Stanford University, will give a seminar talk titled "Randomized Algorithms for Efficient Machine Learning Systems" for the Department of Computer Science.

Please attend the event by using the Zoom link (Meeting ID: 988 8958 4440 | Passcode: 150174).

Abstract:

Machine learning (ML) has demonstrated great promise in scientific discovery, healthcare, and education, especially with the rise of large neural networks. However, large models trained on complex and rapidly growing data consume enormous computational resources. In this talk, I will describe my work on exploiting model sparsity with randomized algorithms to accelerate large ML systems on current hardware. I will start by describing SLIDE, an open-source system for efficient sparse neural network training on CPUs that has been deployed by major technology companies and academic labs. It blends Locality Sensitive Hashing with multi-core parallelism and workload optimization to drastically reduce computations. SLIDE trains industry-scale recommendation models on a 44 core CPU 3.5x faster than TensorFlow on V100 GPU. Next, I will present Pixelated Butterfly, a simple yet efficient sparse training framework on GPUs. It uses a simple static block-sparse pattern based on butterfly and low-rank matrices, taking into account GPU block-oriented efficiency. Pixelated Butterfly trains up to 2.5x faster (wall-clock) than the dense Vision Transformer and GPT-2 counterparts with no drop in accuracy. I will conclude by outlining future research directions for further accelerating ML pipelines and making ML more accessible to the general community, such as software-hardware co-design and sparse models for scientific computing and medical imaging.