Safe & Productive High-Performance Programming with User-Schedulable Languages
Abstract:
In the twilight of Moore's law, computational performance is limited by locality (the distance over which data has to move, e.g., from nearby caches, far away main memory, or across networks), parallelism, and hardware specialization. Because of this, I argue that we should think of the performance and efficiency of an application as determined not just by the algorithm and the hardware on which it runs, but critically also by the organization of its computations and data. For algorithms with the same complexity--even the exact same set of arithmetic operations--the order, granularity, and placement of computations and data can easily change performance by an order of magnitude because of locality, parallelism, and hardware specialization. Because explicit control over the organization of computation is below the level of abstraction of common high-level languages, high-performance code traditionally has to be written at much greater cost in low-level languages like C/C++, with safety and correctness also offloaded to the programmer.
My group has been working to develop a new paradigm for high-performance programming--user-schedulable languages--where the specification of computation is decoupled from choices about its organization, but the organization is still controlled explicitly through "scheduling directives." The compiler guarantees the safety and correctness of the program after scheduling, eliminating a large class of bugs and significantly improving programmer productivity, while still enabling performance engineers to deliver state-of-the-art performance on complex modern hardware. In this talk, I will introduce user-schedulable languages through the example of Halide, a high-performance language we developed for image and tensor processing, before discussing newer directions including richer support for specialized hardware and end-to-end verified optimization and compilation.
Bio:
Jonathan Ragan-Kelley is Associate Professor of Electrical Engineering & Computer Science at MIT. He works on high-efficiency visual computing, including systems, languages, and compilers for graphics, vision, and machine learning. His work has been recognized by a Sloan fellowship as well as the ACM SIGGRAPH Significant New Researcher, NSF CAREER, and PLDI 10-year most influential paper awards. He was previously Assistant Professor of EECS at UC Berkeley, a visiting researcher at Google, a postdoc in Computer Science at Stanford, and earned his PhD in EECS from MIT in 2014. He co-created the Halide language and has helped build more than a dozen other DSL and compiler systems, the first of which was a finalist for an Academy technical achievement award.