mirror of
https://github.com/opencv/opencv.git
synced 2026-07-31 08:13:04 +04:00
Merge branch 4.x
This commit is contained in:
@@ -0,0 +1,95 @@
|
||||
# DNN-based Face Detection And Recognition {#tutorial_dnn_face}
|
||||
|
||||
@tableofcontents
|
||||
|
||||
@prev_tutorial{tutorial_dnn_text_spotting}
|
||||
@next_tutorial{pytorch_cls_tutorial_dnn_conversion}
|
||||
|
||||
| | |
|
||||
| -: | :- |
|
||||
| Original Author | Chengrui Wang, Yuantao Feng |
|
||||
| Compatibility | OpenCV >= 4.5.1 |
|
||||
|
||||
## Introduction
|
||||
|
||||
In this section, we introduce the DNN-based module for face detection and face recognition. Models can be obtained in [Models](#Models). The usage of `FaceDetectorYN` and `FaceRecognizer` are presented in [Usage](#Usage).
|
||||
|
||||
## Models
|
||||
|
||||
There are two models (ONNX format) pre-trained and required for this module:
|
||||
- [Face Detection](https://github.com/ShiqiYu/libfacedetection.train/tree/master/tasks/task1/onnx):
|
||||
- Size: 337KB
|
||||
- Results on WIDER Face Val set: 0.830(easy), 0.824(medium), 0.708(hard)
|
||||
- [Face Recognition](https://drive.google.com/file/d/1ClK9WiB492c5OZFKveF3XiHCejoOxINW/view?usp=sharing)
|
||||
- Size: 36.9MB
|
||||
- Results:
|
||||
|
||||
| Database | Accuracy | Threshold (normL2) | Threshold (cosine) |
|
||||
| -------- | -------- | ------------------ | ------------------ |
|
||||
| LFW | 99.60% | 1.128 | 0.363 |
|
||||
| CALFW | 93.95% | 1.149 | 0.340 |
|
||||
| CPLFW | 91.05% | 1.204 | 0.275 |
|
||||
| AgeDB-30 | 94.90% | 1.202 | 0.277 |
|
||||
| CFP-FP | 94.80% | 1.253 | 0.212 |
|
||||
|
||||
## Usage
|
||||
|
||||
### DNNFaceDetector
|
||||
|
||||
```cpp
|
||||
// Initialize FaceDetectorYN
|
||||
Ptr<FaceDetectorYN> faceDetector = FaceDetectorYN::create(onnx_path, "", image.size(), score_thresh, nms_thresh, top_k);
|
||||
|
||||
// Forward
|
||||
Mat faces;
|
||||
faceDetector->detect(image, faces);
|
||||
```
|
||||
|
||||
The detection output `faces` is a two-dimension array of type CV_32F, whose rows are the detected face instances, columns are the location of a face and 5 facial landmarks. The format of each row is as follows:
|
||||
|
||||
```
|
||||
x1, y1, w, h, x_re, y_re, x_le, y_le, x_nt, y_nt, x_rcm, y_rcm, x_lcm, y_lcm
|
||||
```
|
||||
, where `x1, y1, w, h` are the top-left coordinates, width and height of the face bounding box, `{x, y}_{re, le, nt, rcm, lcm}` stands for the coordinates of right eye, left eye, nose tip, the right corner and left corner of the mouth respectively.
|
||||
|
||||
|
||||
### Face Recognition
|
||||
|
||||
Following Face Detection, run codes below to extract face feature from facial image.
|
||||
|
||||
```cpp
|
||||
// Initialize FaceRecognizer with model path (cv::String)
|
||||
Ptr<FaceRecognizer> faceRecognizer = FaceRecognizer::create(model_path, "");
|
||||
|
||||
// Aligning and cropping facial image through the first face of faces detected by dnn_face::DNNFaceDetector
|
||||
Mat aligned_face;
|
||||
faceRecognizer->alignCrop(image, faces.row(0), aligned_face);
|
||||
|
||||
// Run feature extraction with given aligned_face (cv::Mat)
|
||||
Mat feature;
|
||||
faceRecognizer->feature(aligned_face, feature);
|
||||
feature = feature.clone();
|
||||
```
|
||||
|
||||
After obtaining face features *feature1* and *feature2* of two facial images, run codes below to calculate the identity discrepancy between the two faces.
|
||||
|
||||
```cpp
|
||||
// Calculating the discrepancy between two face features by using cosine distance.
|
||||
double cos_score = faceRecognizer->match(feature1, feature2, FaceRecognizer::DisType::COSINE);
|
||||
// Calculating the discrepancy between two face features by using normL2 distance.
|
||||
double L2_score = faceRecognizer->match(feature1, feature2, FaceRecognizer::DisType::NORM_L2);
|
||||
```
|
||||
|
||||
For example, two faces have same identity if the cosine distance is greater than or equal to 0.363, or the normL2 distance is less than or equal to 1.128.
|
||||
|
||||
## Reference:
|
||||
|
||||
- https://github.com/ShiqiYu/libfacedetection
|
||||
- https://github.com/ShiqiYu/libfacedetection.train
|
||||
- https://github.com/zhongyy/SFace
|
||||
|
||||
## Acknowledgement
|
||||
|
||||
Thanks [Professor Shiqi Yu](https://github.com/ShiqiYu/) and [Yuantao Feng](https://github.com/fengyuentau) for training and providing the face detection model.
|
||||
|
||||
Thanks [Professor Deng](http://www.whdeng.cn/), [PhD Candidate Zhong](https://github.com/zhongyy/) and [Master Candidate Wang](https://github.com/crywang/) for training and providing the face recognition model.
|
||||
@@ -3,7 +3,7 @@
|
||||
@tableofcontents
|
||||
|
||||
@prev_tutorial{tutorial_dnn_OCR}
|
||||
@next_tutorial{pytorch_cls_tutorial_dnn_conversion}
|
||||
@next_tutorial{tutorial_dnn_face}
|
||||
|
||||
| | |
|
||||
| -: | :- |
|
||||
@@ -26,6 +26,11 @@ Before recognition, you should `setVocabulary` and `setDecodeType`.
|
||||
- `T` is the sequence length
|
||||
- `B` is the batch size (only support `B=1` in inference)
|
||||
- and `Dim` is the length of vocabulary +1('Blank' of CTC is at the index=0 of Dim).
|
||||
- "CTC-prefix-beam-search", the output of the text recognition model should be a probability matrix same with "CTC-greedy".
|
||||
- The algorithm is proposed at Hannun's [paper](https://arxiv.org/abs/1408.2873).
|
||||
- `setDecodeOptsCTCPrefixBeamSearch` could be used to control the beam size in search step.
|
||||
- To futher optimize for big vocabulary, a new option `vocPruneSize` is introduced to avoid iterate the whole vocbulary
|
||||
but only the number of `vocPruneSize` tokens with top probabilty.
|
||||
|
||||
@ref cv::dnn::TextRecognitionModel::recognize() is the main function for text recognition.
|
||||
- The input image should be a cropped text image or an image with `roiRects`
|
||||
|
||||
@@ -10,6 +10,7 @@ Deep Neural Networks (dnn module) {#tutorial_table_of_content_dnn}
|
||||
- @subpage tutorial_dnn_custom_layers
|
||||
- @subpage tutorial_dnn_OCR
|
||||
- @subpage tutorial_dnn_text_spotting
|
||||
- @subpage tutorial_dnn_face
|
||||
|
||||
#### PyTorch models with OpenCV
|
||||
In this section you will find the guides, which describe how to run classification, segmentation and detection PyTorch DNN models with OpenCV.
|
||||
|
||||
Reference in New Issue
Block a user