Reversible anonymization for privacy of facial biometrics via cyclic learning

Abstract

Facial recognition systems have emerged as indispensable components in identity verification. These systems heavily rely on facial data, which is stored in a biometric database. However, storing such data in a database raises concerns about privacy breaches. To address this issue, several technologies have been proposed for protecting facial biometrics. Unfortunately, many of these methods can cause irreversible damage to the data, rendering it unusable for other purposes. In this paper, we propose a novel reversible anonymization scheme for face images via cyclic learning. In our scheme, face images can be de-identified for privacy protection and reidentified when necessary. To achieve this, we employ generative adversarial networks with a cycle consistency loss function to learn the bidirectional transformation between the de-identified and re-identified domains. Experimental results demonstrate that our scheme performs well in terms of both de-identification and reidentification. Furthermore, a security analysis validates the effectiveness of our system in mitigating potential attacks.

1 Introduction

Facial recognition technology is designed to identify and authenticate individuals [1,2,3,4,5]. This process involves analyzing facial features, including the position, shape, and spacing of the eyes, nose, and mouth, which are then compared with pre-stored data in the biometric databases to verify the individual’s identity. Nowadays, this technology has found widespread applications across various domains, including security surveillance, mobile phone unlocking, financial transactions, and more [6,7,8,9,10]. However, the widespread adoption of facial recognition systems has raised concerns regarding unauthorized access and potential misuse of sensitive biometric data stored in the associated databases. In response to these challenges, facial anonymization has emerged as a viable solution [11].

Anonymization is the process of removing or altering personally identifiable information (PII) from data sources to safeguard individuals’ privacy. Techniques include eliminating direct identifiers such as names and social security numbers [12], employing data generalization to reduce specificity [13], and introducing noise to obfuscate sensitive information [14]. These methods are employed to ensure that the data cannot be traced back to specific individuals, thus preserving their privacy. Within this context, facial anonymization emerges as a specialized subset aimed at removing or altering facial features in images or video, making it challenging or even impossible to identify individuals. Through facial image anonymization techniques, identity information within the facial region is effectively erased, preventing unauthorized third parties from discerning the identity of any specific individual from a protected face.

While the primary objective of facial anonymization techniques is to safeguard individuals’ privacy, the potential reversibility of these procedures also warrants consideration [15]. Certain scenarios, particularly within law enforcement or medical research, may necessitate the restoration of anonymized data to its original form for investigative or analytical purposes. Consequently, the development of reversible facial anonymization methodologies is crucial, enabling the restoration of facial features while preserving privacy. It is noteworthy that reversing the anonymization process involves converting obscured or altered data back into its original form, potentially reintroducing sensitive information. Hence, within the framework of reversible facial anonymization, robust measures should be implemented to ensure that only authorized individuals can execute the reversal process.

Combining the application requirements mentioned above, we propose a reversible anonymization for privacy via cyclic learning. The contributions of the proposed scheme are summarized as follows:

• (1) We present a facial anonymization system that effectively obscures facial features within the facial region, thereby safeguarding the privacy of underlying identity information.

• (2) Our system incorporates reversibility, enabling the restoration of anonymized facial features as needed. Additionally, the recovered images retain a high level of fidelity, ensuring accurate representation.

• (3) The proposed method demonstrates a certain level of security and provides resistance against potential security attacks.

The organization of this paper is as follows. In Sect. 2, we introduced the existing face anonymization methods for privacy protection. Section 3 presents the proposed privacy-preserving face anonymization scheme. Section 4 details the implementation of our reversible anonymization scheme. Section 5 discusses the performance of the proposed model and compares it with the advanced method.

2 Related works

Numerous researchers have been engaged in developing face anonymization techniques to protect identity information in facial images. As the easily manipulated anonymization elements, masking [16], blurring [17], and mosaicking (pixelization) [18,19,20] are extensively used in face privacy-preserving commercial applications. To evaluate the effectiveness of these three techniques in balancing privacy protection and image intelligibility, a study conducted a subjective evaluation through crowdsourcing [21]. The study revealed that masking tends to have a more negative impact on visual experience. Moreover, it was observed that blurring provides weak privacy protection and may expose sensitive information. Following that, a subsequent study [22] further investigated the effectiveness of blurring and mosaicking. In this study, researchers evaluated human observers’ ability to extract information from images processed with varying degrees of blurring and mosaicking. The results demonstrated that mosaicking generally provides better privacy protection compared to blurring across most levels. Therefore, we can conclude that compared to masking and blurring, mosaicking offers a better trade-off between intelligibility and privacy.

Recent advancements have introduced facial anonymization techniques based on deep neural networks (DNNs). In 2017, Meden et al. proposed a facial de-identification method that utilizes generative neural networks (GNNs) to synthesize artificial surrogate faces [23]. In their approach, each generated face is composed of multiple identities selected from a predefined gallery. Subsequently, Sun et al. introduced a hybrid approach that combines a data-driven method with a parametric face model [24]. Within their methodology, they replaced the identity-related components of the facial parameter vectors. Moreover, generative adversarial networks (GANs) are employed to further refine image quality. Building upon the face attribute transfer model (FATM), Li and Lyu transfer non-identity-related facial attributes from the original faces to donor faces [25]. However, this approach necessitates a computationally demanding face alignment step. Later, DeepPrivacy was introduced, which anonymizes facial images while preserving the original data distribution [26]. While these anonymization techniques effectively obscure personal identity information within the face images, they also result in irreparable damage to the original data. In certain scenarios, there might emerge a requirement to restore the original data, rendering such methods inadequate for those contexts.

As a consequence, the notion of reversibility has been introduced. Based on a spatial relocation algorithm, Cichowski and Czyzewski presented a reversible anonymization method that permutes the pixel ordering within the region of interest (ROI) for privacy protection [27]. Later, Yamac et al. combined a multi-level encryption scheme with compressive sensing for a reversible privacy-preserving method [28]. In 2020, Gu et al. introduced a face identity transformer network where identity alteration is conditioned by a password. This network generates multiple anonymized face images corresponding to different passwords and reconstructs the original face images only when the correct password is provided [29]. Following that, Cao et al. proposed a personalized and invertible de-identification method with deep generative models [30]. Within their approach, they introduced a user-specific password to control the direction and degree of identity alteration by adjusting parameters. In 2022, a reversible face de-identification method for video surveillance data was presented [31]. This method focuses on optimizing two main components: a public module and a private module. The public module is responsible for receiving the original data and generating the de-identified stream. In contrast, the private module, intended for legal or security authorities, analyzes the public stream and reconstructs the original data. In these methods, despite authors claiming anonymization of identity-related biometrics, the remaining biometrics could still be collected and utilized for other purposes.

3 Methodology

Reversible facial anonymization has emerged as a promising solution for effectively balancing privacy preservation and data utility in facial protection. However, many existing methods preserve partial facial biometrics in anonymized images that may still contain information the image owner wishes to keep undisclosed. To address this issue, we propose a novel reversible anonymization system that globally anonymizes facial biometrics, thereby preventing access to underlying identity information. Moreover, the de-identified facial features can be re-identified to provide data utility for authorized users. Further details regarding our proposed system will be presented in subsequent sections.

3.1 Reversible anonymization system

Figure 1 depicts the schematic diagram of the proposed reversible anonymization system. The system consists of two sub-networks: the de-identification network (De-ID network) and the re-identification network (Re-ID network). To protect the ID image $X$, the De-ID network de-identifies the facial biometrics and generates the De-ID image $Y=\text{De}-\text{ID}(X)$, where the face region is globally pixelized. Conversely, the Re-ID network reidentifies the de-identified facial biometrics and generates the re-identified (Re-ID) image $\widetilde{X}=\text{Re}-\text{ID}(Y)$. The primary objective of our reversible anonymization system is to optimize the De-ID and Re-ID networks, ensuring that $X\ne Y$ and $X\approx \widetilde{X}$ at the identity level.

Fig. 1

Schematic diagram of the proposed reversible anonymization scheme

3.2 System properties

This section explores the key properties of the proposed reversible anonymization system. It is widely recognized that faces contain numerous biometric features, such as the eyes, nose, mouth, eyebrows, chin, cheeks, and other facial structures. Each of these features contributes to shaping an individual’s distinctive appearance. Notably, apart from features directly related to an individual’s identity, other indirectly related features can also be integrated and analyzed to determine individual identity. Therefore, merely obscuring biometric features directly related to identity is insufficient. In our system, we globally de-identify facial features by pixelating with De-ID networks. This process, illustrated in Fig. 1, renders facial features unrecognizable, thereby safeguarding the privacy of the face image. Consequently, the ID image and the De-ID image generated by the De-ID network do not share the same identity, as depicted in Fig. 2.

Fig. 2

Anonymity describes a mismatch of identity between De-ID and ID images

However, as a consequence of this de-identification process, the face image loses its inherent usability. To maintain the usability of face images, our Re-ID networks restore the De-ID image to its original form, as shown in Fig. 1. The Re-ID image not only matches the ID image in terms of identity but also demonstrates a high level of fidelity, as illustrated in Fig. 3. It is crucial to acknowledge that the reversible process could also result in the re-leakage of sensitive information. Hence, stringent access restrictions, such as the use of cryptographic keys, must be enforced on the Re-ID network in practical applications.

Fig. 3

Reversibility describes a match of identity between Re-ID and ID images

Although the proposed reversible face anonymization system effectively achieves the goals of de-identifying and reidentifying face images, it is crucial to recognize the inherent value of such data and the susceptibility of privacy-protected systems to potential third-party attacks. Hence, the consideration of security as an evaluation factor becomes imperative in refining our scheme. Among the various third-party attacks, model theft attacks are the most prevalent and damaging. In this type of attack, the attacker employs API requests to obtain the output of the proposed system, enabling them to acquire a local training set. With this local training set, the attacker can retrain a local system capable of performing the same task. Notably, if the attacker possesses knowledge of the proposed system’s structure, the locally trained system can be equally effective. However, even with an identical network model and training data, we anticipate that the forged Re-ID network cannot accurately recover the genuine Re-ID image from the De-ID image generated by the proposed system. In other words, the Re-ID image generated by the forged Re-ID network cannot match the identity of the Re-ID image generated by the proposed system, as illustrated in Fig. 4.

Fig. 4

Security describes a mismatch between Re-ID and ID images, where the Re-ID image is generated via an attacker’s model

4 Model implementation

In this section, we present the details of the system implementation. First, we present the system architecture of our scheme. Next, we explain the loss function employed in our scheme. Finally, we present the dataset used for training and testing the proposed system.

4.1 Architecture

Cyclic learning involves learning features from different domains through the circulation of data within them. It is consistent with the concept of the proposed system, i.e., learning the properties of the face image domain and the face mosaic domain and implementing data transformation. Consequently, most neural networks based on cyclic learning can be utilized to implement the proposed system. Cycle-consistent adversarial networks (CycleGAN) [32] have garnered considerable attention as a prominent application of cyclic learning in computer vision. They facilitate image style transformation between different domains. Figure 5 shows the schematic diagram of CycleGAN. It comprises four sub-networks, $\mathcal{G}$, $\mathcal{F}$, ${\mathcal{D}}_{(\mathcal{X})}$, and ${\mathcal{D}}_{(\mathcal{Y})}$, where $\mathcal{G}$ and $\mathcal{F}$ serve as two generators, while ${\mathcal{D}}_{(\mathcal{X})}$ and ${\mathcal{D}}_{(\mathcal{Y})}$ function as two discriminators. Both $\mathcal{G}$ and $\mathcal{F}$ are utilized to generate images in the target domain, each comprising two convolutional layers, nine residual layers, and two deconvolution layers. Meanwhile, ${\mathcal{D}}_{(\mathcal{X})}$ and ${\mathcal{D}}_{(\mathcal{Y})}$ adopt a network structure of $70\times 70$ PatchGAN to further facilitate the generated images to approximate the target domain, i.e., ${\mathcal{D}}_{(\mathcal{Y})}$ makes the generated image $Y$ from $\mathcal{G}$ more like the real image in the domain $\mathcal{Y}$, and ${\mathcal{D}}_{(\mathcal{X})}$ makes the generated image $X$ from $\mathcal{F}$ more like the real image in the domain $\mathcal{X}$. Moreover, to stabilize the network performance, cyclic consistency loss constrains the images generated through two consecutive generators back to the original domain, i.e., $\widetilde{X}=\mathcal{F}(\mathcal{G}(X))$ is constrained to $X$ domain and $\widetilde{Y}=\mathcal{G}(\mathcal{F}(Y))$ is constrained to $Y$ domain. On this basis, CycleGAN is employed for face anonymization in this study. In the implementation, the face images and pixelated face images are mutually transformed, with each serving as the target domain for the other.

Fig. 5

The schematic diagram of CycleGAN

4.2 Loss function

The overall objective of our reversible face anonymization system based on CycleGAN is expressed as follows:

$${{L}_{A}=LGAN}_{(\mathcal{X}\to \mathcal{Y})}+{LGAN}_{(\mathcal{Y}\to \mathcal{X})}+{L}_{cyc},$$

(1)

where ${{L}_{A}=LGAN}_{(\mathcal{X}\to \mathcal{Y})}$ and ${LGAN}_{(\mathcal{Y}\to \mathcal{X})}$ are adversarial losses and ${L}_{cyc}$ is cyclic consistency loss.

Based on the system architecture described above, there are two mapping between $\mathcal{X}$ domain and $\mathcal{Y}$ domain. For the mapping $\mathcal{X}\to \mathcal{Y}$, the adversarial loss ${{L}_{A}=LGAN}_{(\mathcal{X}\to \mathcal{Y})}$ is defined as follows:

$${LGAN}_{(\mathcal{X}\to \mathcal{Y})} = [\mathit{log}{\mathcal{D}}_{\mathcal{Y}}(y)] + [\mathit{log}{(1-\mathcal{D}}_{\mathcal{Y}}(\mathcal{G}(x)))].$$

(2)

In this mapping, $\mathcal{G}$ devoted to generating image $Y=\text{De}-\text{ID}(X)$ like the real image in the domain $Y$ and ${\mathcal{D}}_{(\mathcal{Y})}$ as far as possible to distinguish the generated image $Y$ from the real image in domain $\mathcal{Y}$, i.e., $y$. Therefore, $\mathcal{G}$ aims to maximize this adversarial loss ${LGAN}_{(\mathcal{X}\to \mathcal{Y})}$, while ${\mathcal{D}}_{\mathcal{Y}}$ does the opposite. The adversarial loss ${LGAN}_{(\mathcal{Y}\to \mathcal{X})}$ is defined as follows:

$${LGAN}_{(\mathcal{Y}\to \mathcal{X})} = [\mathit{log}{\mathcal{D}}_{(\mathcal{X})}(x)] + [\mathit{log}(1-{\mathcal{D}}_{(\mathcal{X})}(\mathcal{G}(x)))],$$

(3)

which is similar as ${LGAN}_{(\mathcal{X}\to \mathcal{Y})}$.

Cyclic consistency loss is employed to ensure the cyclic consistency between the original image and the reconstructed image. To achieve the desired outcome of $X\approx \widetilde{X}$ and $Y\approx \widetilde{Y}$, we leverage the utilization of the cyclic consistency loss ${L}_{cyc}$, which is formulated as follows:

$${L}_{cyc} = {||\widetilde{X} -X ||}_{1}+ {||\widetilde{Y}- Y||}_{1}.$$

(4)

In this equation, the distance between the original and Re-ID images is computed using the pixel-wise L1 norm ${||*||}_{1}$, which measures the absolute difference between corresponding pixels, where $\widetilde{X}=\mathcal{F}(\mathcal{G}(X))$ and $\widetilde{Y}=\mathcal{G}(\mathcal{F}(Y))$.

4.3 Train dataset

To train our system, we initially selected 924 images from the CelebA [33] dataset. These images were divided into two distinct sub-training datasets: one for natural face images and another for pixelated face images. The images within the pixelated face dataset were further processed by pixelating the face region identified by the “CV.CascadeClassifier” function. The size of each pixelated block was determined as $\text{min }(M, N)/20\times \text{min}(M, N)/20$, where $M$ and $N$ represent the length and width of the face region. Our system was evaluated using two test datasets, CelebA and FFHQ [34]. It is important to clarify that the CelebA dataset used for testing purposes does not contain any data that was used for training the system.

5 Experiments

In this section, we conduct a series of experiments to evaluate the performance of the proposed reversible face anonymization system. Our experiments begin with intuitive perception and quantitative criteria to measure the anonymity and reversibility of the system. Furthermore, we analyze the system’s capability to withstand potential attacks. The proposed reversible anonymization scheme is implemented using PyTorch, with acceleration provided by an NVIDIA RTX 3090 GPU.

5.1 Anonymity analysis

We first evaluate the anonymity performance by visual inspection. Figure 6 presents the de-identification results obtained by our De-ID networks. The face regions in the four De-ID images are globally pixelated, and none of the facial features is exposed, thus preventing the retrieval of identifying information from the images. Next, we assess the anonymization accuracy. Complex regions in images tend to contain more detailed information than smoother regions. In face images, complex regions typically contain facial features like the eyes and the nose, which may reveal individual identities. Conversely, smoother regions, like the skin, usually exhibit fewer distinctive facial features. It is important to highlight that our system implements de-identification through pixelation, resulting in reduced complexity within each pixelated block. Thus, we can quantify the anonymization accuracy by examining changes in complex pixels before and after the anonymization process. In the implementation, we first compute the complexity of each pixel in both the ID and De-ID images. The pixel complexity is computed by the following:

$${P}_{(x)} =\sum_{i=1}^{8}{({d}_{(xi)}-{\overline{d} }_{(xi)})}^{2} .$$

(5)

where ${P}_{(x)}$ denotes the complexity of the pixel $x$, ${d}_{(xi)}$ represents the gradient of the $i$-th neighboring pixel relative to pixel $x$ within a $3\times 3$ matrix, and ${\overline{d} }_{(xi)}$ denotes the average gradient. Pixels are then classified as complex or smooth based on a predefined threshold value $T$. Subsequently, the complexity of each pixel in the ID image is compared with its corresponding pixel in the De-ID image. In the facial region, accurate anonymization occurs when the pixel in the ID image is complex, while its counterpart in the De-ID image appears smooth. Conversely, in the background region, accurate anonymization is achieved when both the ID and De-ID images appear smooth. Lastly, anonymization accuracy can be determined by calculating the ratio of correctly anonymized pixels to the total number of complex pixels.

Fig. 6

The de-identification results of our reversible anonymization scheme. For each pair, the first one is ID image, and the second one is De-ID image

Figure 7 illustrates the anonymization accuracy results for the four images depicted in Fig. 6, with corresponding values of 61.15%, 52.66%, 42.71%, and 59.63%. Moreover, the average accuracy results for the CelebA and FFHQ datasets are 65.27% and 58.43%, respectively. Upon analyzing the data alongside the corresponding images, it becomes apparent that most pixels in the facial region undergo a transition from complexity to smoothness. However, it is noteworthy that certain pixels do not undergo the anticipated transformation, particularly those concentrated in the hair and background areas. This discrepancy is primarily attributed to slight blurring in the de-identified image, resulting in improper changes in complexity. Additionally, variations in values among pixel blocks contribute to inaccuracies. Nevertheless, if we overlook these two factors, the system achieves accurate anonymization.

Fig. 7

Quantitative results of the anonymization accuracy. Each image pair consists of a top image depicting the pixel complexity of the ID image and a bottom image illustrating the quantified anonymization accuracy. Smooth pixels are depicted by white dots, complex pixels by blue dots, and yellow dots indicate pixels inaccurately anonymized

5.2 Reversibility

Reversibility analysis is vital to determine if the De-ID images generated by our scheme are suitable for other face-related tasks. Figure 8 illustrates the reidentification outcomes of our reversible anonymization scheme. Upon comparison and analysis, it becomes evident that although some blurring is observed in the generated Re-ID images, the facial features are nearly recovered without significant loss.

Fig. 8

The reidentification results of our reversible anonymization scheme. For each pair, the first one is ID image, and the second one is Re-ID image

To evaluate the impact of our de-identification and reidentification processes on the identifiability of facial images, we utilize FaceNet [35] to assess the identities of image pairs at various stages. FaceNet projects facial images into a high-dimensional space, where the distances between face embeddings indicate the similarity or dissimilarity between the depicted faces. In our assessment, ID images undergo de-identification and reidentification to generate corresponding De-ID and Re-ID images. These De-ID and Re-ID images, along with their respective ID images, are then fed into FaceNet to determine their similarity. Furthermore, we also assess the distances among different ID images to provide an objective evaluation. The distance metric employed is cosine similarity [36], defined as follows:

$$\text{Cosine similarity}(a,b)=\frac{a\cdot\,b}{\left|a\right|\left|b\right|},$$

(6)

where the vectors output by FaceNet for image pairs are denoted as $a$ and $b$. In our experimental setup, we established the face similarity threshold as 0.8. This threshold denotes that face images with a similarity score exceeding 0.8 exhibit significant resemblance in facial features. To visualize the results, we selected 100 ID images from the dataset and obtained their corresponding De-ID and Re-ID images for similarity evaluation. Additionally, we randomly selected 200 different ID images to assess their similarity. The outcomes are depicted in Fig. 9. Upon analysis, we observed that the distances between De-ID images and their corresponding ID images, as well as among different ID images, fall within the range of − 0.4 to 0.4, with overlaps. This distribution highlights a noticeable disparity between the generated De-ID images and their corresponding ID images. This outcome also indirectly indicates the security of our method. Conversely, the distances between re-identified (Re-ID) images and their ID counterparts predominantly range from 0.8 to 1.0, indicating a high degree of similarity between Re-ID and ID images. The dataset results in Table 1 further support our findings.

Fig. 9

The results of cosine similarity distance of different face image pairs

Table 1 The average cosine similarity distance of test datasets

Additionally, we also discuss fidelity, which is a higher-level requirement for reversibility. To evaluate fidelity, we utilize two commonly used metrics: peak signal-to-noise ratio (PSNR) [37] and structural similarity index (SSIM) [38]. PSNR measures the fidelity of a reconstructed image by pixel-wise comparison with its original version. It is defined as follows:

$$\text{PSNR}=10\cdot {\text{log}}_{10}(\frac{{MAX}^{2}}{MSE}),$$

(7)

where $\text{MAX}$ represents the maximum possible pixel value and $\text{MSE}$ denotes the mean squared error between the original and reconstructed images. Additionally, SSIM quantifies the similarity between two images by assessing their luminance, contrast, and structural features. It is expressed as follows:

$$\text{SSIM}=\frac{(2{\mu }_{x}{\mu }_{y}+{C}_{1})(2{\sigma }_{xy}+{C}_{2})}{({\mu }_{x}^{2}+{\mu }_{y}^{2}+{C}_{1})({\sigma }_{x}^{2}+{\sigma }_{y}^{2}+{C}_{2})}.$$

(8)

where ${\mu }_{x}$ and ${\mu }_{y}$ represent mean values, ${\sigma }_{x}$ and ${\sigma }_{y}$ donate standard deviations, and ${\sigma }_{xy}$ represents the covariance. In addition, ${C}_{1}$ and ${C}_{2}$ are constants included to prevent instability at low luminance or contrast levels. The PSNR and SSIM results for the De-ID images, Re-ID images with their corresponding ID images, and different ID images are presented in Table 2. Notably, the average PSNR between the De-ID images and their corresponding ID images is calculated to be 20.87 dB and 19.26 dB, accompanied by an average SSIM value of 0.4231 and 0.3648 for the two datasets, respectively. The results exceed those obtained when comparing different ID images, indicating that the proposed anonymization scheme primarily focuses on modifying the facial region while causing minimal alterations to the background region. In contrast, the average PSNR between the Re-ID images and their corresponding ID images reaches 37.44 dB and 35.21 dB, while the average SSIM value stands at 0.9017 and 0.8823, respectively, which signifies excellent fidelity.

Table 2 The average PSNR and SSIM of the images in the datasets

5.3 Security analysis

In this section, we provide a security analysis of our proposed system. As discussed, privacy-protected systems are susceptible to model theft attacks. In such attacks, adversaries exploit API requests to obtain a local dataset from the target system. Subsequently, this dataset is used to train local networks that mimic the functionality of the target system. Through this method, attackers gain access to sensitive data generated by the targeted system. For our system, adversaries may leverage trained local networks to decrypt the De-ID images produced by our system and access identity information within authentic Re-ID images.

To assess the system’s resilience against model theft attacks, we conducted a series of simulation experiments. In each experiment, we assumed that attackers had access to training data identical to ours but with variations in network architecture, training duration, and loss functions and optimization methods. Specifically, we adjusted the attacker’s local network by reducing the number of residual blocks from 9 to 6 to introduce differences in network architecture. Furthermore, we trained the attacker’s network for 150 epochs, whereas our system underwent training for 250 epochs to assess the impact of training duration. Additionally, we employed the Wasserstein Generative Adversarial Network with Gradient Penalty (WGAN-GP) in the attacker’s generator instead of the Least Squares Generative Adversarial Network (LSGAN) used in our system to introduce variations in loss functions and optimization methods. Moreover, we conducted simulation experiments under identical training conditions.

After training the simulated networks, we utilized the forged Re-ID networks to de-identify the De-ID images produced by our system. The resulting Re-ID images are depicted in Fig. 10. It can be observed that the forged Re-ID network introduces noticeable artifacts and distortions, resulting in a visibly distinct appearance compared to the Re-ID images generated by our system. Additionally, we found that even when trained under identical conditions, the Re-ID images produced by the forged Re-ID network differ from the authentic ones. To further investigate whether randomness in the training process affects the results, we conducted three additional training sessions under the same conditions. The De-ID images and Re-ID images from these experiments are shown in Fig. 11. To quantify the differences, we visualized the pairwise differences between the De-ID images and Re-ID images generated by these three models, as shown in Fig. 12. This observation leads to the conclusion that despite identical training conditions, randomness in the network results in variations in the generated images. Consequently, even if attackers possess comprehensive training knowledge, they cannot decrypt the De-ID images generated by our system without access to the network parameters.

Fig. 10

The results of the resistance model theft attacks. The first column displays the De-ID images generated by the proposed system, the second column depicts the corresponding Re-ID images, and the third column showcases the Re-ID images produced by forged Re-ID networks trained under identical conditions to the authentic Re-ID networks. The remaining columns exhibit Re-ID images generated by various forged Re-ID networks, which differ from the authentic networks in terms of architecture, training duration, loss functions, and optimization methods

Fig. 11

De-ID images and Re-ID images obtained under the same training conditions

Fig. 12

Pairwise difference images of De-ID images and Re-ID images in Fig. 11

5.4 Comparisons

In this section, we compare the key features of the proposed method with nine state-of-the-art methods [23,24,25,26,27,28,29,30], and [31]. Our analysis focuses on three main indicators: reversibility, integration of deep learning, and anonymity strategy. The findings of our comparison are presented in Table 3. From the comparison results, several key distinctions emerge. Firstly, [27,28,29,30,31], and our method provide reversibility, thereby enhancing data usability. Secondly, [29,30,31], and our approach utilize deep learning-based techniques, thereby enhancing efficiency. Lastly, unlike [29, 30], and [31], our approach embraces a global anonymity strategy, mitigating the risk of facial feature leakage.

Table 3 Comparison of key features

6 Conclusions

This work designs a reversible facial feature anonymization scheme that removes face features from face images through De-ID networks and recovers them through Re-ID networks. The experimental results demonstrate the effectiveness of the proposed De-ID network in achieving successful anonymization, rendering the identity information within the face images inaccessible. Conversely, the Re-ID network can restore face images with remarkable fidelity, preserving the utility of the original face image. Furthermore, our system demonstrates the ability to withstand a certain level of security attacks.

Although our model achieves excellent performance, it is worth noting that the generated images may exhibit tonal shifts and blurring artifacts. While achieving a completely lossless recovery of the original image is not the primary objective of our system, we acknowledge the necessity for further refinement of our facial trait anonymization technique in future research endeavors.