Ensemble-Based Few-Shot Learning

Abstract:

Few-shot learning commonly generalizes to unseen classes from only a small labeled support set, and predictions are especially sensitive to the distribution shift. Ensemble methods can reduce the sensitivity by aggregating complementary evidence, but conventional ensembles often involve much overhead. This thesis develops ensemble algorithms that improve trained few-shot classifiers efficiently.

The first contribution, FSL-QuickBoost, complements an off-the-shelf few-shot classifier with an alternative encoder and a random-forest-based one-vs-all classifier. The auxiliary classifier compares pairs of examples through elementwise feature differences, and its output is ensembled with the original few-shot classifier output. This design supplies complementary model-side evidence at modest cost and yields up to 21% relative accuracy improvement across the various few-shot models and datasets. The second contribution, FSL-Rectifier, shifts from model ensemble to test-time data augmentation. A GAN-based image combiner transfers class-defining appearance from a test example to poses selected from training data, after which the original and generated representations are combined for bias reduction. Given a pretrained combiner, the method is model-agnostic and training-free, yielding up to 13.7% relative few-shot accuracy improvement.

Nevertheless, the FSL-Rectifier is limited in its generative component stability. To address the limitation, the third contribution, 1S-DAug, develops a stabler generative pipeline. From a single test image, it combines class-preserving geometric perturbation, partial noise injection, and attention-conditioned diffusion denoising to produce diverse image augmentations while retaining their class fidelity. Aggregating these generations with the original images also yields a training-free plugin for off-the-shelf few-shot classifiers. Across four datasets and multiple backbone families, 1S-DAug consistently improves classification and yields up to 20.5% relative accuracy improvement.

Lastly, 1S-DAug intentionally keeps the inference steps low for computation considerations, though this would cap the image generation quality and potentially deteriorate the performance of test-time data augmentation; to address this, PEEK corrects exposure bias in deterministic diffusion sampling by making a provisional update, querying the denoiser at the look-ahead state based on the higher-signal input, and inject the "peeked future" prediction back to the current denoising step. A greedy search selects the correction positions, while prefix and noise-schedule-rebasing caches reduce the cost of a 9-step search by 65.3%. Across pixel-space and latent-space models in unconditional, class-conditional, and text-conditional settings on CIFAR-10, LSUN Bedroom, ImageNet, and MS-COCO, PEEK yields up to 49.3% relative improvement in Fr'echet Inception Distance over baselines matched by the number of inference steps, and it is also training-free. Together, these contributions suggest that the cost-aware ensemble algorithm designs can improve few-shot learning via ensemble effectively and efficiently.