Efficient and Green Code LLMs: Happier Software Engineers, Happier Planet
COM1 Level 2
SR10, COM1-02-10
Abstract:
For decades, researchers have explored methods to automate software engineering (ASE) tasks. Recently, many are excited about the potential of code Large Language Models (code LLMs) for ASE tasks. However, code LLMs are large, slow, and energy-hungry compared to traditional ASE solutions, which raises usability and sustainability concerns. This is especially true when we want to deploy them in IDEs on local devices, which is often the preferred setting. This talk will highlight a three-pronged approach to improve the efficiency and energy consumption of code LLMs. It will first discuss 'Avatar,' which combines constraint solving, metaheuristic search, and knowledge distillation to create a much smaller, more efficient, and energy-saving model. The talk then will present 'SimPy,' the first code LLM-oriented programming language grammar; its simple structure captures pertinent semantics succinctly, allowing code LLM to be more efficient while retaining similar efficacy. Finally, the talk will present `FrugalCoder,' the first solution that efficiently "appraises" the potential outcome before running code LLMs, preventing costly but unfruitful code LLM executions. The talk will conclude with a call for action for more research on non-functional properties of code LLM, which have received less attention in the literature but, in this speaker's opinion, are as important as functional properties.
Bio:
David Lo graduated with a PhD from SoC, NUS, in 2008 under the supervision of Prof. Khoo SiauCheng. He enjoyed his time at SoC being part of the Programming Language Lab. After he graduated, he joined Singapore Management University, first as a Lecturer, and currently the OUB Chair Professor of Computer Science and Director of the Center for Research in Intelligent Software Engineering (RISE). Championing the area of AI for Software Engineering (AI4SE), he has demonstrated how AI —encompassing data mining, machine learning, information retrieval, natural language processing, and search-based algorithms —can transform software engineering data into actionable insights and automation. Through surveys and interviews, he has identified practitioners' pain points and explored the acceptance thresholds for AI-powered tools, effectively performing requirements engineering for AI4SE research. His contributions have led to over 20 awards, including two Test-of-Time awards and ten ACM SIGSOFT/IEEE TCSE Distinguished Paper awards, and accumulated more than 30k citations. An ACM Fellow, IEEE Fellow, ASE Fellow, and National Research Foundation Investigator, Lo has also served as a PC Co-Chair for ASE'20, FSE'24, and ICSE'25. For more information, visit: http://www.mysmu.edu/faculty/davidlo/