PIFR - a method of generating a 3D mask, regardless of the angle of rotation of the face
3r33333. 3r3-31. 
3r33333.
3r33333. We will bring you the translation of the article “ PIFR: Pose Invariant 3D Face Reconstruction ".
3r33333.
3r33333. In many real-world applications, including face detection and recognition, the generation of 3D emoticons and stickers, the geometry of the face needs to be restored from flat images. However, this task remains difficult, especially when most of the information about a person is unknowable.
3r33333.
3r33333. Jiang and Wu from Jiangnan University (China) and Kittler from the University of Surrey (United Kingdom) offer 3r317. A new algorithm for 3D-reconstruction of the face - PIFR 3r-3285. , which significantly increases the accuracy of the reconstruction, even in difficult poses. 3DMM models. developed by Branton and others;
3r33333.
3r350. 3D model of a person with a multi-resolution
provided by the University of Surrey (United Kingdom);
3r33333.
3r3355. large-scale model of the face (CMML) 3r3285. created by Imperial College.
3r33333.
3r33333.
3r33333. The article uses the BML model, which is the most popular.
3r33333.
3r33333. There are several approaches to recreating a 3D model from a flat image, including:
3r33333.
3r33333. 3r3196. 3r33333.
method cascading regression;
3r33333.
a combination of landmark face detection and 3D face reconstruction, as well as indexing of signs for constructing 3r380. tree regression model
;
3r33333.
method normalization of expression and position of the face ;
3r33333.
3DMM extension (3r390. E-3DMM 3r-3285.), which takes into account the change in facial expression;
3r33333.
3r395. weighted fit
LandMark 3DMM based on the traditional regression method.
3r33333.
3r33333.
3r33333.
3r33333. The article by Jiang, Wu and Kitler proposes a new algorithm for setting invariant 3D face reconstruction - POSE (Pose-Invariant 3D Face Reconstruction - PIFR), based on the 3DMM method.
3r33333.
3r33333. First, the authors propose to generate a frontal image, normalizing one input face image. This step allows you to recover additional identity information of the person.
3r33333.
3r33333. The next step is to use the weighted sum of the 3D features of the two images: frontal and original. This allows us not only to maintain the position of the original image, but also to expand the identification information.
3r33333.
3r33333. Scheme of the proposed approach:
3r33333.
3r33333. 3r3124.
3r33333.
3r33333. Experiments show that the PIVL algorithm has significantly improved the performance of 3D face reconstruction compared to previous methods, especially in complex poses.
3r33333.
3r33333. Consider the proposed model in more detail.
3r33333.
3r33333.
3r33333. The PIVL method relies heavily on the 3DMM fitting process, which can be expressed as minimizing the error in calculating the coordinates of 3D projections of key points. However, the face created by the 3D model has about 5?000 vertices, and therefore iterative calculations lead to slow and inefficient convergence.
3r33333.
3r33333. To overcome this problem, researchers suggest using key points (for example, the center of the eye, the corner of the mouth and the tip of the nose) as the main truth in the process of fitting the mask. In particular, weighted fit 3DMM fit is used.
3r33333.
3r33333. 3r3149.
3r33333. [i] Top row: original image and landmark. Bottom row: 3D-model of the face and its alignment on the 2D image 3r-3252.
3r33333.
3r33333. The next task is to recreate a 3D face mask on a close-up. To solve this problem, researchers use high-precision normalization of posture and expression (VNPV) , but to normalize only posture, not facial expressions. In addition, 3r3-3160. editing Poisson
used to restore the area of the face, closed due to the angle of view.
3r33333.
3r33333.
3r33333. The effectiveness of the PIVL method was evaluated to recreate the face: 3r-3288. 3r33333.
3r33333. 3r3174. 3r33333.
in small and medium poses;
3r33333.
close ups;
3r33333.
complex postures (deviation angles ± 90).
3r33333. 3r3185.
3r33333.
3r33333. For this, researchers used Three publicly available :
3r33333.
3r33333. 3r3196. 3r33333.
The AFW dataset, created using Flickr images, contains 205 images with 468 marked faces, complex backgrounds and facial poses.
3r33333.
An LFPW dataset containing 224 face images in the test set and 811 face images in the training set; each image is marked with 68 characteristic dots; 900 images from both sets were selected for testing in this study.
3r33333.
The AFLW dataset is a large-scale database of individuals that contains about 250 million images tagged by hand, and each image is labeled with 21 feature points. In this study, only images in complex positions of a person from this data set were used for qualitative analysis.
3r33333.
3r33333.
3r33333.
3r33333.
Quantitative analysis of r3r3241.
3r33333. Using the Average Euclidean Metric (CEM), the study compares the performance of the PIFR method with the E-3DMM and FW-3DMM in the AFW and lfpw data sets. The cumulative error distribution curves (RNO) are as follows: 3r-3288. 3r33333.
3r33333.
3r33333. [i] Comparison of the cumulative error distribution (PHO) curves in the AFW and LFPW 3r3252 data sets.
3r33333.
3r33333. As can be seen from these graphs and tables below, the WIVL method shows superior performance compared with the other two methods. Especially good is the effectiveness of recreation for close-ups.
3r33333.
3r33333.
3r33333.
3r33333.
Qualitative analysis 3r33232.
3r33333. The method was also evaluated qualitatively based on photographs of the face in a different position from the AFLW data set. The results are shown in the figure below.
3r33333.
3r33333.
3r33333. [i] Comparison of 3D face reconstruction: (a) source image; (b) FW-3DMM; (c) E-3DMM; (d) the proposed approach is 3r33252.
3r33333.
3r33333. Even if half of the landmarks are not visible due to non-trivial posture, which leads to large errors and failures of other methods, the PIFR method still works well.
3r33333.
3r33333. The following are additional examples of the effectiveness of the WAVL method based on images from the AFW dataset.
3r33333.
3r33333.
3r33333. Top row: enter 2D images. Middle row: 3D mask. Bottom row: alignment of the mask
3r33333.
3r33333.
3r33333. The new algorithm for the reconstruction of the face of the WIVL gives good results of reconstruction even in difficult poses. Taking both the original and frontal images for a weighted merge, the method allows you to recover enough information about the faces to recreate the 3D mask.
3r33333.
3r33333. In the future, the researchers plan to recover even more information about the face in order to improve the accuracy of the mask re-creation.
3r33333.
3r33333. Original
3r33333.
3r33333. Translated - Farid Gasratov
3r33333. 3r33333. 3r33333.
3r33333.
3r33333. 3r33333. 3r33333. 3r33333.
3r33333.
3r33333. We will bring you the translation of the article “ PIFR: Pose Invariant 3D Face Reconstruction ".
3r33333.
3r33333. In many real-world applications, including face detection and recognition, the generation of 3D emoticons and stickers, the geometry of the face needs to be restored from flat images. However, this task remains difficult, especially when most of the information about a person is unknowable.
3r33333.
3r33333. Jiang and Wu from Jiangnan University (China) and Kittler from the University of Surrey (United Kingdom) offer 3r317. A new algorithm for 3D-reconstruction of the face - PIFR 3r-3285. , which significantly increases the accuracy of the reconstruction, even in difficult poses. 3DMM models. developed by Branton and others;
3r33333.
3r350. 3D model of a person with a multi-resolution
provided by the University of Surrey (United Kingdom);
3r33333.
3r3355. large-scale model of the face (CMML) 3r3285. created by Imperial College.
3r33333.
3r33333.
3r33333. The article uses the BML model, which is the most popular.
3r33333.
3r33333. There are several approaches to recreating a 3D model from a flat image, including:
3r33333.
3r33333. 3r3196. 3r33333.
method cascading regression;
3r33333.
a combination of landmark face detection and 3D face reconstruction, as well as indexing of signs for constructing 3r380. tree regression model
;
3r33333.
method normalization of expression and position of the face ;
3r33333.
3DMM extension (3r390. E-3DMM 3r-3285.), which takes into account the change in facial expression;
3r33333.
3r395. weighted fit
LandMark 3DMM based on the traditional regression method.
3r33333.
3r33333.
3r33333.
The proposed method is PIFR
3r33333. The article by Jiang, Wu and Kitler proposes a new algorithm for setting invariant 3D face reconstruction - POSE (Pose-Invariant 3D Face Reconstruction - PIFR), based on the 3DMM method.
3r33333.
3r33333. First, the authors propose to generate a frontal image, normalizing one input face image. This step allows you to recover additional identity information of the person.
3r33333.
3r33333. The next step is to use the weighted sum of the 3D features of the two images: frontal and original. This allows us not only to maintain the position of the original image, but also to expand the identification information.
3r33333.
3r33333. Scheme of the proposed approach:
3r33333.
3r33333. 3r3124.
3r33333.
3r33333. Experiments show that the PIVL algorithm has significantly improved the performance of 3D face reconstruction compared to previous methods, especially in complex poses.
3r33333.
3r33333. Consider the proposed model in more detail.
3r33333.
3r33333.
Description of the method
3r33333. The PIVL method relies heavily on the 3DMM fitting process, which can be expressed as minimizing the error in calculating the coordinates of 3D projections of key points. However, the face created by the 3D model has about 5?000 vertices, and therefore iterative calculations lead to slow and inefficient convergence.
3r33333.
3r33333. To overcome this problem, researchers suggest using key points (for example, the center of the eye, the corner of the mouth and the tip of the nose) as the main truth in the process of fitting the mask. In particular, weighted fit 3DMM fit is used.
3r33333.
3r33333. 3r3149.
3r33333. [i] Top row: original image and landmark. Bottom row: 3D-model of the face and its alignment on the 2D image 3r-3252.
3r33333.
3r33333. The next task is to recreate a 3D face mask on a close-up. To solve this problem, researchers use high-precision normalization of posture and expression (VNPV) , but to normalize only posture, not facial expressions. In addition, 3r3-3160. editing Poisson
used to restore the area of the face, closed due to the angle of view.
3r33333.
3r33333.
Performance comparison with other methods
3r33333. The effectiveness of the PIVL method was evaluated to recreate the face: 3r-3288. 3r33333.
3r33333. 3r3174. 3r33333.
in small and medium poses;
3r33333.
close ups;
3r33333.
complex postures (deviation angles ± 90).
3r33333. 3r3185.
3r33333.
3r33333. For this, researchers used Three publicly available :
3r33333.
3r33333. 3r3196. 3r33333.
The AFW dataset, created using Flickr images, contains 205 images with 468 marked faces, complex backgrounds and facial poses.
3r33333.
An LFPW dataset containing 224 face images in the test set and 811 face images in the training set; each image is marked with 68 characteristic dots; 900 images from both sets were selected for testing in this study.
3r33333.
The AFLW dataset is a large-scale database of individuals that contains about 250 million images tagged by hand, and each image is labeled with 21 feature points. In this study, only images in complex positions of a person from this data set were used for qualitative analysis.
3r33333.
3r33333.
3r33333.
3r33333.
Quantitative analysis of r3r3241.
3r33333. Using the Average Euclidean Metric (CEM), the study compares the performance of the PIFR method with the E-3DMM and FW-3DMM in the AFW and lfpw data sets. The cumulative error distribution curves (RNO) are as follows: 3r-3288. 3r33333.
3r33333.
3r33333. [i] Comparison of the cumulative error distribution (PHO) curves in the AFW and LFPW 3r3252 data sets.
3r33333.
3r33333. As can be seen from these graphs and tables below, the WIVL method shows superior performance compared with the other two methods. Especially good is the effectiveness of recreation for close-ups.
3r33333.
3r33333.
3r33333.
3r33333.
Qualitative analysis 3r33232.
3r33333. The method was also evaluated qualitatively based on photographs of the face in a different position from the AFLW data set. The results are shown in the figure below.
3r33333.
3r33333.
3r33333. [i] Comparison of 3D face reconstruction: (a) source image; (b) FW-3DMM; (c) E-3DMM; (d) the proposed approach is 3r33252.
3r33333.
3r33333. Even if half of the landmarks are not visible due to non-trivial posture, which leads to large errors and failures of other methods, the PIFR method still works well.
3r33333.
3r33333. The following are additional examples of the effectiveness of the WAVL method based on images from the AFW dataset.
3r33333.
3r33333.
3r33333. Top row: enter 2D images. Middle row: 3D mask. Bottom row: alignment of the mask
3r33333.
3r33333.
The result is
3r33333. The new algorithm for the reconstruction of the face of the WIVL gives good results of reconstruction even in difficult poses. Taking both the original and frontal images for a weighted merge, the method allows you to recover enough information about the faces to recreate the 3D mask.
3r33333.
3r33333. In the future, the researchers plan to recover even more information about the face in order to improve the accuracy of the mask re-creation.
3r33333.
3r33333. Original
3r33333.
3r33333. Translated - Farid Gasratov
3r33333. 3r33333. 3r33333.
3r33333.
3r33333. 3r33333. 3r33333. 3r33333.
