FRAN: feature-filtered residual attention network for realistic face sketch-to-photo transformation

Abstract

Face sketch-to-photo transformation aims at generates face photo images from sketched face images. Although transformations have progressed significantly with the development of deep learning techniques in recent years, generating face photos with realistic photo styles and rich facial details is still challenging. In this paper, a new realistic face sketch-to-photo transformation method is proposed based on the feature-filtered residual attention network (FRAN), which is able to propagate more precise feature information in the deep network. Specifically, a feature-filtered residual module is constructed by filtering feature maps in the residual block to filtrate short-term feature information. In addition, a decoder-guided attention module is designed to integrate and filtrate the long-term feature information. Moreover, to synthesize face photo images with more facial details, a Sobel operator-based detail loss is proposed to constrain the network training. The experimental results on the public datasets demonstrate that FRAN generates more realistic face photo images than state-of-the-art approaches in terms of visual perception and quality evaluation. Furthermore, the face photo images generated by FRAN obtain higher face recognition accuracy than those created by the compared methods.