1
0
mirror of https://github.com/opencv/opencv.git synced 2026-07-31 00:03:03 +04:00

Move objdetect HaarCascadeClassifier and HOGDescriptor to contrib xobjdetect (#25198)

* Move objdetect parts to contrib

* Move objdetect parts to contrib

* Minor fixes.
This commit is contained in:
WU Jia
2024-03-22 04:40:10 +08:00
committed by GitHub
parent 213e1a4d9d
commit aa5ea340f7
111 changed files with 22 additions and 741097 deletions
+4 -4
View File
@@ -255,10 +255,10 @@ if(DOXYGEN_FOUND)
endif()
# copy haar cascade files
set(haar_cascade_files "")
set(data_harrcascades_path "${OpenCV_SOURCE_DIR}/data/haarcascades/")
list(APPEND js_tutorials_assets_deps "${data_harrcascades_path}/haarcascade_frontalface_default.xml" "${data_harrcascades_path}/haarcascade_eye.xml")
list(APPEND js_assets "${data_harrcascades_path}/haarcascade_frontalface_default.xml" "${data_harrcascades_path}/haarcascade_eye.xml")
# set(haar_cascade_files "")
# set(data_harrcascades_path "${OpenCV_SOURCE_DIR}/data/haarcascades/")
# list(APPEND js_tutorials_assets_deps "${data_harrcascades_path}/haarcascade_frontalface_default.xml" "${data_harrcascades_path}/haarcascade_eye.xml")
# list(APPEND js_assets "${data_harrcascades_path}/haarcascade_frontalface_default.xml" "${data_harrcascades_path}/haarcascade_eye.xml")
foreach(f ${js_assets})
get_filename_component(fname "${f}" NAME)
@@ -1,107 +0,0 @@
Face Detection using Haar Cascades {#tutorial_js_face_detection}
==================================
Goal
----
- learn the basics of face detection using Haar Feature-based Cascade Classifiers
- extend the same for eye detection etc.
Basics
------
Object Detection using Haar feature-based cascade classifiers is an effective method proposed by Paul Viola and Michael Jones in the 2001 paper, "Rapid Object Detection using a
Boosted Cascade of Simple Features". It is a machine learning based approach in which a cascade
function is trained from a lot of positive and negative images. It is then used to detect objects in
other images.
Here we will work with face detection. Initially, the algorithm needs a lot of positive images
(images of faces) and negative images (images without faces) to train the classifier. Then we need
to extract features from it. For this, Haar features shown in below image are used. They are just
like our convolutional kernel. Each feature is a single value obtained by subtracting the sum of pixels
under the white rectangle from the sum of pixels under the black rectangle.
![image](images/haar_features.jpg)
Now all possible sizes and locations of each kernel are used to calculate plenty of features. For each
feature calculation, we need to find the sum of the pixels under the white and black rectangles. To solve this,
they introduced the integral images. It simplifies calculation of the sum of the pixels, how large may be
the number of pixels, to an operation involving just four pixels.
But among all these features we calculated, most of them are irrelevant. For example, consider the
image below. Top row shows two good features. The first feature selected seems to focus on the
property that the region of the eyes is often darker than the region of the nose and cheeks. The
second feature selected relies on the property that the eyes are darker than the bridge of the nose.
But the same windows applying on cheeks or any other place is irrelevant. So how do we select the
best features out of 160000+ features? It is achieved by **Adaboost**.
![image](images/haar.png)
For this, we apply each and every feature on all the training images. For each feature, it finds the
best threshold which will classify the faces to positive and negative. But obviously, there will be
errors or misclassifications. We select the features with minimum error rate, which means they are
the features that best classifies the face and non-face images. (The process is not as simple as
this. Each image is given an equal weight in the beginning. After each classification, weights of
misclassified images are increased. Then again same process is done. New error rates are calculated.
Also new weights. The process is continued until required accuracy or error rate is achieved or
required number of features are found).
Final classifier is a weighted sum of these weak classifiers. It is called weak because it alone
can't classify the image, but together with others forms a strong classifier. The paper says even
200 features provide detection with 95% accuracy. Their final setup had around 6000 features.
(Imagine a reduction from 160000+ features to 6000 features. That is a big gain).
So now you take an image. Take each 24x24 window. Apply 6000 features to it. Check if it is face or
not. Wow.. Wow.. Isn't it a little inefficient and time consuming? Yes, it is. Authors have a good
solution for that.
In an image, most of the image region is non-face region. So it is a better idea to have a simple
method to check if a window is not a face region. If it is not, discard it in a single shot. Don't
process it again. Instead focus on region where there can be a face. This way, we can find more time
to check a possible face region.
For this they introduced the concept of **Cascade of Classifiers**. Instead of applying all the 6000
features on a window, group the features into different stages of classifiers and apply one-by-one.
(Normally first few stages will contain very less number of features). If a window fails the first
stage, discard it. We don't consider remaining features on it. If it passes, apply the second stage
of features and continue the process. The window which passes all stages is a face region. How is
the plan !!!
Authors' detector had 6000+ features with 38 stages with 1, 10, 25, 25 and 50 features in first five
stages. (Two features in the above image is actually obtained as the best two features from
Adaboost). According to authors, on an average, 10 features out of 6000+ are evaluated per
sub-window.
So this is a simple intuitive explanation of how Viola-Jones face detection works. Read paper for
more details.
Haar-cascade Detection in OpenCV
--------------------------------
Here we will deal with detection. OpenCV already contains many pre-trained classifiers for face,
eyes, smile etc. Those XML files are stored in opencv/data/haarcascades/ folder. Let's create a face
and eye detector with OpenCV.
We use the function: **detectMultiScale (image, objects, scaleFactor = 1.1, minNeighbors = 3, flags = 0, minSize = new cv.Size(0, 0), maxSize = new cv.Size(0, 0))**
@param image matrix of the type CV_8U containing an image where objects are detected.
@param objects vector of rectangles where each rectangle contains the detected object. The rectangles may be partially outside the original image.
@param scaleFactor parameter specifying how much the image size is reduced at each image scale.
@param minNeighbors parameter specifying how many neighbors each candidate rectangle should have to retain it.
@param flags parameter with the same meaning for an old cascade as in the function cvHaarDetectObjects. It is not used for a new cascade.
@param minSize minimum possible object size. Objects smaller than this are ignored.
@param maxSize maximum possible object size. Objects larger than this are ignored. If maxSize == minSize model is evaluated on single scale.
@note Don't forget to delete CascadeClassifier and RectVector!
Try it
------
Try this demo using the code above. Canvas elements named haarCascadeDetectionCanvasInput and haarCascadeDetectionCanvasOutput have been prepared. Choose an image and
click `Try it` to see the result. You can change the code in the textbox to investigate more.
\htmlonly
<iframe src="../../js_face_detection.html" width="100%"
onload="this.style.height=this.contentDocument.body.scrollHeight +'px';">
</iframe>
\endhtmlonly
@@ -1,15 +0,0 @@
Face Detection in Video Capture {#tutorial_js_face_detection_camera}
==================================
Goal
----
- learn how to detect faces in video capture.
@note If you don't know how to capture video from camera, please review @ref tutorial_js_video_display.
\htmlonly
<iframe src="../../js_face_detection_camera.html" width="100%"
onload="this.style.height=this.contentDocument.body.scrollHeight +'px';">
</iframe>
\endhtmlonly
@@ -1,11 +0,0 @@
Object Detection {#tutorial_js_table_of_contents_objdetect}
================
- @subpage tutorial_js_face_detection
Face detection
using haar-cascades
- @subpage tutorial_js_face_detection_camera
Face Detection in Video Capture
-5
View File
@@ -22,11 +22,6 @@ OpenCV.js Tutorials {#tutorial_js_root}
In this section you
will learn different techniques to work with videos like object tracking etc.
- @subpage tutorial_js_table_of_contents_objdetect
In this section you
will object detection techniques like face detection etc.
- @subpage tutorial_js_table_of_contents_dnn
These tutorials show how to use dnn module in JavaScript
-36
View File
@@ -623,15 +623,6 @@
volume = {5},
pages = {1530-1536}
}
@inproceedings{Lienhart02,
author = {Lienhart, Rainer and Maydt, Jochen},
title = {An extended set of haar-like features for rapid object detection},
booktitle = {Image Processing. 2002. Proceedings. 2002 International Conference on},
year = {2002},
pages = {I--900},
volume = {1},
publisher = {IEEE}
}
@article{Lowe04,
author = {Lowe, David G.},
title = {Distinctive Image Features from Scale-Invariant Keypoints},
@@ -1042,25 +1033,6 @@
number = {3},
publisher = {ACM}
}
@inproceedings{Viola01,
author = {Viola, Paul and Jones, Michael J.},
title = {Rapid object detection using a boosted cascade of simple features},
booktitle = {Computer Vision and Pattern Recognition, 2001. CVPR 2001. Proceedings of the 2001 IEEE Computer Society Conference on},
year = {2001},
pages = {I--511},
volume = {1},
publisher = {IEEE}
}
@article{Viola04,
author = {Viola, Paul and Jones, Michael J.},
title = {Robust real-time face detection},
journal = {International Journal of Computer Vision},
year = {2004},
volume = {57},
number = {2},
pages = {137--154},
publisher = {Kluwer Academic Publishers}
}
@inproceedings{WJ10,
author = {Xu, Wei and Mulligan, Jane},
title = {Performance evaluation of color correction approaches for automatic multi-view image and video stitching},
@@ -1159,14 +1131,6 @@
year = {2013},
publisher = {Springer}
}
@incollection{Liao2007,
title = {Learning multi-scale block local binary patterns for face recognition},
author = {Liao, Shengcai and Zhu, Xiangxin and Lei, Zhen and Zhang, Lun and Li, Stan Z},
booktitle = {Advances in Biometrics},
pages = {828--837},
year = {2007},
publisher = {Springer}
}
@incollection{nister2008linear,
title = {Linear time maximally stable extremal regions},
author = {Nist{\'e}r, David and Stew{\'e}nius, Henrik},
@@ -1,4 +0,0 @@
Face Detection using Haar Cascades {#tutorial_py_face_detection}
==================================
Tutorial content has been moved: @ref tutorial_cascade_classifier
+1 -1
View File
@@ -48,7 +48,7 @@ OpenCV-Python Tutorials {#tutorial_py_root}
- @ref tutorial_table_of_content_objdetect
In this section you
will learn object detection techniques like face detection etc.
will learn object detection techniques.
- @subpage tutorial_py_table_of_contents_bindings
@@ -259,10 +259,10 @@ Next, create the directory `src/main/resources` and download this Lena image int
Make sure it's called `"lena.png"`. Items in the resources directory are available to the Java
application at runtime.
Next, copy `lbpcascade_frontalface.xml` from `opencv/data/lbpcascades/` into the `resources`
Next, copy `lbpcascade_frontalface.xml` from `opencv_contrib/modules/xobjdetect/data/lbpcascades/` into the `resources`
directory:
@code{.bash}
cp <opencv_dir>/data/lbpcascades/lbpcascade_frontalface.xml src/main/resources/
cp <xobjdetect_dir>/data/lbpcascades/lbpcascade_frontalface.xml src/main/resources/
@endcode
Now modify src/main/java/HelloOpenCV.java so it contains the following Java code:
@code{.java}
@@ -273,7 +273,7 @@ import org.opencv.core.Point;
import org.opencv.core.Rect;
import org.opencv.core.Scalar;
import org.opencv.imgcodecs.Imgcodecs;
import org.opencv.objdetect.CascadeClassifier;
import org.opencv.xobjdetect.CascadeClassifier;
//
// Detects faces in an image, draws boxes around them, and writes the results
@@ -2,6 +2,8 @@ Detection of ArUco boards {#tutorial_aruco_board_detection}
=========================
@prev_tutorial{tutorial_aruco_detection}
@next_tutorial{tutorial_barcode_detect_and_decode}
| | |
| -: | :- |
@@ -3,8 +3,7 @@ Barcode Recognition {#tutorial_barcode_detect_and_decode}
@tableofcontents
@prev_tutorial{tutorial_cascade_classifier}
@next_tutorial{tutorial_introduction_to_pca}
@prev_tutorial{tutorial_aruco_board_detection}
| | |
| -: | :- |

Before

Width:  |  Height:  |  Size: 29 KiB

After

Width:  |  Height:  |  Size: 29 KiB

Before

Width:  |  Height:  |  Size: 31 KiB

After

Width:  |  Height:  |  Size: 31 KiB

@@ -3,3 +3,4 @@ Object Detection (objdetect module) {#tutorial_table_of_content_objdetect}
- @subpage tutorial_aruco_detection
- @subpage tutorial_aruco_board_detection
- @subpage tutorial_barcode_detect_and_decode
@@ -1,148 +0,0 @@
Cascade Classifier {#tutorial_cascade_classifier}
==================
@tableofcontents
@prev_tutorial{tutorial_optical_flow}
@next_tutorial{tutorial_barcode_detect_and_decode}
| | |
| -: | :- |
| Original author | Ana Huamán |
| Compatibility | OpenCV >= 3.0 |
Goal
----
In this tutorial,
- We will learn how the Haar cascade object detection works.
- We will see the basics of face detection and eye detection using the Haar Feature-based Cascade Classifiers
- We will use the @ref cv::CascadeClassifier class to detect objects in a video stream. Particularly, we
will use the functions:
- @ref cv::CascadeClassifier::load to load a .xml classifier file. It can be either a Haar or a LBP classifier
- @ref cv::CascadeClassifier::detectMultiScale to perform the detection.
Theory
------
Object Detection using Haar feature-based cascade classifiers is an effective object detection
method proposed by Paul Viola and Michael Jones in their paper, "Rapid Object Detection using a
Boosted Cascade of Simple Features" in 2001. It is a machine learning based approach where a cascade
function is trained from a lot of positive and negative images. It is then used to detect objects in
other images.
Here we will work with face detection. Initially, the algorithm needs a lot of positive images
(images of faces) and negative images (images without faces) to train the classifier. Then we need
to extract features from it. For this, Haar features shown in the below image are used. They are just
like our convolutional kernel. Each feature is a single value obtained by subtracting sum of pixels
under the white rectangle from sum of pixels under the black rectangle.
![image](images/haar_features.jpg)
Now, all possible sizes and locations of each kernel are used to calculate lots of features. (Just
imagine how much computation it needs? Even a 24x24 window results over 160000 features). For each
feature calculation, we need to find the sum of the pixels under white and black rectangles. To solve
this, they introduced the integral image. However large your image, it reduces the calculations for a
given pixel to an operation involving just four pixels. Nice, isn't it? It makes things super-fast.
But among all these features we calculated, most of them are irrelevant. For example, consider the
image below. The top row shows two good features. The first feature selected seems to focus on the
property that the region of the eyes is often darker than the region of the nose and cheeks. The
second feature selected relies on the property that the eyes are darker than the bridge of the nose.
But the same windows applied to cheeks or any other place is irrelevant. So how do we select the
best features out of 160000+ features? It is achieved by **Adaboost**.
![image](images/haar.png)
For this, we apply each and every feature on all the training images. For each feature, it finds the
best threshold which will classify the faces to positive and negative. Obviously, there will be
errors or misclassifications. We select the features with minimum error rate, which means they are
the features that most accurately classify the face and non-face images. (The process is not as simple as
this. Each image is given an equal weight in the beginning. After each classification, weights of
misclassified images are increased. Then the same process is done. New error rates are calculated.
Also new weights. The process is continued until the required accuracy or error rate is achieved or
the required number of features are found).
The final classifier is a weighted sum of these weak classifiers. It is called weak because it alone
can't classify the image, but together with others forms a strong classifier. The paper says even
200 features provide detection with 95% accuracy. Their final setup had around 6000 features.
(Imagine a reduction from 160000+ features to 6000 features. That is a big gain).
So now you take an image. Take each 24x24 window. Apply 6000 features to it. Check if it is face or
not. Wow.. Isn't it a little inefficient and time consuming? Yes, it is. The authors have a good
solution for that.
In an image, most of the image is non-face region. So it is a better idea to have a simple
method to check if a window is not a face region. If it is not, discard it in a single shot, and don't
process it again. Instead, focus on regions where there can be a face. This way, we spend more time
checking possible face regions.
For this they introduced the concept of **Cascade of Classifiers**. Instead of applying all 6000
features on a window, the features are grouped into different stages of classifiers and applied one-by-one.
(Normally the first few stages will contain very many fewer features). If a window fails the first
stage, discard it. We don't consider the remaining features on it. If it passes, apply the second stage
of features and continue the process. The window which passes all stages is a face region. How is
that plan!
The authors' detector had 6000+ features with 38 stages with 1, 10, 25, 25 and 50 features in the first five
stages. (The two features in the above image are actually obtained as the best two features from
Adaboost). According to the authors, on average 10 features out of 6000+ are evaluated per
sub-window.
So this is a simple intuitive explanation of how Viola-Jones face detection works. Read the paper for
more details or check out the references in the Additional Resources section.
Haar-cascade Detection in OpenCV
--------------------------------
OpenCV provides pretrained models that can be read using the @ref cv::CascadeClassifier::load method.
These models are located in the data folder in the OpenCV installation or can be found [here](https://github.com/opencv/opencv/tree/5.x/data).
The following code example will use pretrained Haar cascade models to detect faces and eyes in an image.
First, a @ref cv::CascadeClassifier is created and the necessary XML file is loaded using the @ref cv::CascadeClassifier::load method.
Afterwards, the detection is done using the @ref cv::CascadeClassifier::detectMultiScale method, which returns boundary rectangles for the detected faces or eyes.
@add_toggle_cpp
This tutorial code's is shown lines below. You can also download it from
[here](https://github.com/opencv/opencv/tree/5.x/samples/cpp/tutorial_code/objectDetection/objectDetection.cpp)
@include samples/cpp/tutorial_code/objectDetection/objectDetection.cpp
@end_toggle
@add_toggle_java
This tutorial code's is shown lines below. You can also download it from
[here](https://github.com/opencv/opencv/tree/5.x/samples/java/tutorial_code/objectDetection/cascade_classifier/ObjectDetectionDemo.java)
@include samples/java/tutorial_code/objectDetection/cascade_classifier/ObjectDetectionDemo.java
@end_toggle
@add_toggle_python
This tutorial code's is shown lines below. You can also download it from
[here](https://github.com/opencv/opencv/tree/5.x/samples/python/tutorial_code/objectDetection/cascade_classifier/objectDetection.py)
@include samples/python/tutorial_code/objectDetection/cascade_classifier/objectDetection.py
@end_toggle
Result
------
-# Here is the result of running the code above and using as input the video stream of a built-in
webcam:
![](images/Cascade_Classifier_Tutorial_Result_Haar.jpg)
Be sure the program will find the path of files *haarcascade_frontalface_alt.xml* and
*haarcascade_eye_tree_eyeglasses.xml*. They are located in
*opencv/data/haarcascades*
-# This is the result of using the file *lbpcascade_frontalface.xml* (LBP trained) for the face
detection. For the eyes we keep using the file used in the tutorial.
![](images/Cascade_Classifier_Tutorial_Result_LBP.jpg)
Additional Resources
--------------------
-# Paul Viola and Michael J. Jones. Robust real-time face detection. International Journal of Computer Vision, 57(2):137154, 2004. @cite Viola04
-# Rainer Lienhart and Jochen Maydt. An extended set of haar-like features for rapid object detection. In Image Processing. 2002. Proceedings. 2002 International Conference on, volume 1, pages I900. IEEE, 2002. @cite Lienhart02
-# Video Lecture on [Face Detection and Tracking](https://www.youtube.com/watch?v=WfdYYNamHZ8)
-# An interesting interview regarding Face Detection by [Adam
Harvey](https://web.archive.org/web/20171204220159/http://www.makematics.com/research/viola-jones/)
-# [OpenCV Face Detection: Visualized](https://vimeo.com/12774628) on Vimeo by Adam Harvey
Binary file not shown.

Before

Width:  |  Height:  |  Size: 55 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 52 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 7.4 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 11 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 112 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 278 KiB

@@ -3,7 +3,7 @@ Introduction to Principal Component Analysis (PCA) {#tutorial_introduction_to_pc
@tableofcontents
@prev_tutorial{tutorial_barcode_detect_and_decode}
@prev_tutorial{tutorial_optical_flow}
| | |
| -: | :- |
+1 -1
View File
@@ -4,7 +4,7 @@ Optical Flow {#tutorial_optical_flow}
@tableofcontents
@prev_tutorial{tutorial_meanshift}
@next_tutorial{tutorial_cascade_classifier}
@next_tutorial{tutorial_introduction_to_pca}
Goal
----
@@ -1,4 +1,4 @@
Other tutorials (objdetect, photo, stitching, video) {#tutorial_table_of_content_other}
Other tutorials (photo, stitching, video) {#tutorial_table_of_content_other}
========================================================
- photo. @subpage tutorial_hdr_imaging
@@ -6,6 +6,4 @@ Other tutorials (objdetect, photo, stitching, video) {#tutorial_table_of_content
- video. @subpage tutorial_background_subtraction
- video. @subpage tutorial_meanshift
- video. @subpage tutorial_optical_flow
- objdetect. @subpage tutorial_cascade_classifier
- objdetect. @subpage tutorial_barcode_detect_and_decode
- ml. @subpage tutorial_introduction_to_pca
+1 -1
View File
@@ -10,7 +10,7 @@ OpenCV Tutorials {#tutorial_root}
- @subpage tutorial_table_of_content_features2d - feature detectors, descriptors and matching framework
- @subpage tutorial_table_of_content_dnn - infer neural networks using built-in _dnn_ module
- @subpage tutorial_table_of_content_gapi - graph-based approach to computer vision algorithms building
- @subpage tutorial_table_of_content_other - other modules (objdetect, stitching, video, photo)
- @subpage tutorial_table_of_content_other - other modules (stitching, video, photo)
- @subpage tutorial_table_of_content_ios - running OpenCV on an iDevice
- @subpage tutorial_table_of_content_3d - 3d objects processing and visualisation
@cond CUDA_MODULES