Enough with the Glue Code! Let's Build Snap Fit Data System Architectures
Abstract:
Data management systems have grown increasingly powerful—but also increasingly entangled. Over decades, we have “extended” databases not by principled design, but by gluing on new features: new operators, new access methods, new execution backends, new hardware support, new data models. The result is a generation of systems that are impressive in capabilities yet fragile in structure: difficult to extend, hard to reason about, and even harder to maintain. Glue code holds everything together, until it doesn’t, and a developer finds themselves changing code in 30 files to add a new optimizer rule. In this keynote, I argue that it is time to move beyond this era of ad hoc extensibility and embrace snap fit architectures: database systems composed of interoperable, well specified, and independently evolvable components that click together cleanly—without bespoke bridges, duplicated logic, or deeply buried assumptions. I will outline the principles that make snap fit designs possible, drawing on lessons from operator level composition, modular execution kernels, and partial query evaluation. I will show how such architectures make it easier to innovate, easier to tailor systems to workloads and hardware, and easier to integrate new capabilities such as compression-aware processing and multi modal AI. Using examples from recent work, I will illustrate how snap fit thinking enables us to replace brittle stacks with systems that are transparent, extensible, and built to evolve. Finally, I will offer a vision for a research agenda that treats composability not as an implementation detail—but as a first-class architectural goal for the next generation of data systems. These systems will be leaner, faster and easier to maintain. It’s time to stop gluing. It’s time to start building.
Bio:
Holger Pirk is an Associate Professor in the Large Scale Data and Systems group at Imperial College London and, currently, a Visiting Associate Professor at NUS. His research spans all things data: analytics, transactions, systems, algorithms, data structures, processing models, and everything in between. While some of his work targets “traditional” relational databases, his broader aim is to expand the applicability of data management techniques. To this end, Holger studies Composable Database Systems—systems that are extensible to support heterogeneous workloads, data models, and hardware. This naturally leads to research at the intersection of data management, compilers, and computer architecture, with applications in areas ranging from generative modeling and graph processing to classic analytical workloads. Before joining Imperial, Holger was a Postdoctoral Associate in the Database Group at MIT CSAIL, a PhD student in the Database Architectures Group at CWI in Amsterdam, and an undergraduate in Computer Science at Humboldt Universität zu Berlin. Holger knows how to speak and write, as evidenced, respectively, by a CIDR Gong Show Award and a VLDB Best Paper Award.