Structure-Aware Learning for Scalable AI Systems: Training, Inference, and Post-Training Alignment
COM3 Level 2
MR21, COM3 02-61
Abstract:
Recent progress in artificial intelligence has been driven largely by scale, with advances often achieved through larger models, larger datasets, and greater computational resources. While this paradigm has delivered remarkable empirical success, it has also encouraged increasingly blunt homogeneous algorithmic designs across multiple stages of the AI pipeline, including model training, long-context inference, and post-training alignment. In many real-world AI systems, however, the underlying computational and statistical structure exhibits substantial heterogeneity: different components of the input or training process contribute unequally to optimization difficulty, informational relevance, and reasoning complexity. Existing methods often fail to account explicitly for such heterogeneous structure, which can limit both efficiency and performance. This thesis studies how structured heterogeneity arises across these three stages and how accounting for it can lead to more effective and scalable AI systems.
At the training stage, I study click-through rate prediction, where sparse categorical features follow highly skewed frequency distributions. I show that feature frequency is strongly correlated with Hessian sharpness, revealing a previously underexplored source of optimization difficulty, and propose Helen, a frequency-aware optimizer that improves CTR model training through frequency-wise Hessian eigenvalue regularization. At the inference stage, I study long-video understanding with multimodal large language models, where only a small subset of frames is relevant under strict visual token budgets. I propose FOCUS, a training-free and model-agnostic keyframe selection method that identifies informative temporal regions through confidence-guided exploration and improves the efficiency of long-video understanding. At the post-training alignment stage, I study reward modeling for large language models, where preference instances vary substantially in difficulty. I propose CAMEL, a confidence-gated reflection framework that selectively invokes additional reasoning only for uncertain cases, yielding a better accuracy-efficiency trade-off in reward modeling.
Taken together, these works present a unified perspective on scalable AI systems through the lens of structured heterogeneity. Across training, inference, and post-training alignment, they show that scaling AI systems is not only a matter of increasing model size or computational budget, but also of designing algorithms that better match the heterogeneous structure of the underlying problem.