WaveFace: Authentic Face Restoration with Efficient Frequency Recovery

Yunqi Miao1, Jiankang Deng2, Jungong Han3,
1University of Warwick, 2Imperial College London 3University of Sheffield
Paper arXiv Dataset
Left: Illustration of our frequency-aware BFR scheme. Restoration is performed in the frequency domain instead of the pixel domain. Right: Comparisons with SOTA methods. Previous methods struggle to restore facial details or the original identity while our method achieves a good balance of realness and fidelity with fewer artifacts.

Abstract

Blind face restoration via diffusion prior is a highly ill-posed problem, which suffers from 1) slow training and inference speed, and 2) failure in preserving the original identity and recovering fine-grained facial details. In this work, we propose WaveFace to solve the problems in the frequency domain, where low- and high-frequency components decomposed by wavelet transformation are considered individually to maximize authenticity as well as efficiency. The diffusion model is applied to recover the low-frequency component only, which presents general information of the original image but 1/16 in size. To preserve the original identity, the generation is conditioned on the low-frequency component of low-quality images at each denoising step. Meanwhile, high-frequency components at multiple decomposition levels are handled by an unified network, which recovers complex facial details in a single step. Evaluations on four benchmark datasets show that: 1) WaveFace outperforms state-of-the-art methods in the authenticity, especially in terms of identity preservation, and 2) authentic images are restored with the efficiency 10x faster than existing diffusion model-based BFR methods.

Method

Interpolate start reference image.
Overview of our model structure. We develop a hybrid-guidance identity-preserving image synthesis framework. Our model, built upon StableDiffusion, utilizes text prompts and reference human images to guide image synthesis while preserving human identity through an identity input.

Results

We provide the synthetic CelebA-Test used in our paper and our restoration results. Please download to check more results.

Interpolate start reference image.
Comparison with state-of-the-art methods on CelebA-Test.

Interpolate start reference image.
Comparison with state-of-the-art methods on Real-world datasets. Row 1-3: LFW-Test. Row 4-6: WebPhoto-Test. Row 7-9: WIDER-Test.

BibTeX

@article{miao2024waveface,
          author    = {Miao, Yunqi and Deng, Jiankang and Han, Jungong},
          title     = {WaveFace: Authentic Face Restoration with Efficient Frequency Recovery},
          journal   = {CVPR},
          year      = {2024}
        }