Image and video restoration

Abstract:

In the field of image and video restoration, advancements in super-resolution (SR) and deblurring have been driven by deep neural networks. Traditional SR methods upscale entire images, which can be computationally intensive when only specific regions, like faces or license plates, require high resolution. To address this, we propose a novel approach that focuses on super-resolving selected regions of interest (ROI) by leveraging the surrounding image context. Our method efficiently restores ROIs with minimal computational cost, outperforming approaches that target only the ROI.

In parallel, video deblurring has seen remarkable progress, yet different frame regions exhibit different characteristics and should be provided with corresponding relevant information. To achieve fine-grained deblurring, we propose a memory-based video deblurring technique that accounts for the varying characteristics of different frame regions by storing blurry-sharp feature pairs in a memory bank. This memory bank is further enriched using a bidirectional and multi-scale strategy, maintaining low model complexity and fast inference times.

Moreover, blurring especially degrades high-frequency (HF) information, yet this aspect is often overlooked by recent models that focus more on enhancing architectural design. Recovering these fine details is challenging, partly due to the spectral bias of neural networks, which are inclined towards learning low-frequency functions. To overcome this, we introduce a video deblurring method that employs adaptive high-pass kernels, dynamically predicted from a combination of basis kernels, to specifically target and recover HF features. This strategy is highly efficient, resulting in low-memory footprints for training and fast run times for inference, all while achieving state-of-the-art when compared to low-budget models.

In summary, we present novel, efficient methods for targeted super-resolution and deblurring, enhancing the quality of images and videos while maintaining computational efficiency. Experimental results demonstrate the effectiveness of these approaches.