CS SEMINAR

Predictable Multiplexing of AI Infrastructure

Speaker
Dr Danyang Zhuo, Hamelsky Family Assistant Professor, Duke University
Chaired by
Dr Jialin LI, Sung Kah Kay Assistant Professor, School of Computing
lijl@comp.nus.edu.sg

10 Dec 2026 Thursday, 03:00 PM to 04:00 PM

SR8, COM1-02-08

Talk abstract:

GPUs and inter-GPU communication bandwidth are the scarcest resources in AI, so serving more applications with the same infrastructure means sharing them: network interfaces, inference engines, and GPUs. But sharing creates hidden interactions that break what applications rely on. A noisy neighbor can starve a tenant's network, a busy inference server can change a model's output, and a background job can stall an interactive one. In this talk, I will show how to share AI infrastructure with isolation, determinism, and responsiveness by making these interactions explicit. Collie and Husky systematically test the shared RDMA networks that connect GPUs, uncovering isolation bugs that became three NVIDIA CVEs. Vosti provides the first formal definition of deterministic LLM inference and a formally verified inference engine that satisfies the definition. Nixie lets unrelated AI applications share a consumer GPU while keeping interactive ones responsive. I will conclude with future directions toward predictable, shared AI infrastructure.

Speaker bio:

Danyang Zhuo is Hamelsky Family Assistant Professor in the Department of Computer Science at Duke University. Before Duke, he was a postdoctoral researcher at UC Berkeley. He completed his Ph.D. in the Paul G. Allen School of Computer Science and Engineering at the University of Washington. His research has received NSF CAREER Award, IEEE/ACM MICRO Best Paper Award, USENIX Security Distinguished Paper Award, and USENIX FAST Best Paper Award.