CS SEMINAR

Resource-Efficient LLM Serving: Optimizing Compute and Memory

Speaker
Dr Haiying Shen, Associate Professor, Department of Computer Science at the University of Virginia

Chaired by
Dr HE Bingsheng, Professor, School of Computing
hebs@comp.nus.edu.sg

20 Nov 2026 Friday, 09:30 AM to 11:00 AM

MR1, COM1-03-19

Abstract

My research focuses on job scheduling and resource management for Machine Learning (ML) and Large Language Model (LLM) systems. With the rapid growth of deep learning models, minimizing monetary costs and maximizing the time-efficiency of inference-serving systems have become critical challenges. Addressing these challenges requires efficient task scheduling, as well as optimized resource management to ensure high resource utilizations, and adherence to Service Level Objectives (SLOs). However, current approaches often fall short of fully addressing these challenges. In this talk, I will present our novel methods for bridging these gaps and enabling efficient execution of LLM workloads. I will also briefly outline my ongoing and future research plans aimed at advancing LLM systems.


Biography

Dr. Haiying Shen is an Associate Professor in the Department of Computer Science at the University of Virginia. During her 2024 sabbatical, she served as a Consulting Researcher at Microsoft in Redmond, WA, where she focused on LLM systems. Her research area is distributed systems, with a focus on ML/LLM systems, cloud computing, edge computing, and cyber-physical systems (CPS). Dr. Shen has made significant contributions to her field, with an H-index of 53 and over 380 publications in top conferences and journals such as SIGCOMM, OSDI, EuroSys, SoCC, ASPLOS, CoNext, Infocom, IEEE/ACM Transactions on Networking (TON), IEEE Transactions on Parallel and Distributed Systems (TPDS), and IEEE Transactions on Mobile Computing (TMC). Her work has received the George N. Saridis Best Transactions Paper Award (2021), best paper awards at CloudCom (2016) and NAS (2018), a best paper runner-up award at ICCCN (2015), best paper award nominations at ICPP (2021), MASS (2011), and CCGrid (2009). She has also received several prestigious awards, including the Microsoft Faculty Fellowship Award (2010), IEEE TCSC Mid-Career Award (2015), IBM Faculty Award (2015), NSF CAREER Award (2013), and Sigma Xi Clemson Chapter Young Investigator Award (2013). Dr. Shen serves as an Associate Editor for TON, TMC, and IEEE Networking Letters (NL). She also has served as program co-chair and general co-chair for several international conferences and has participated in the program committees of numerous leading conferences.