Block layout-based convolution in DNN #28585
merge together with https://github.com/opencv/opencv_extra/pull/1321
Some core parts of the new engine in DNN module have been revised substantially:
1. all tests seem to pass, except for `Test_Graph_Simplifier.ResizeSubgraph`, which has been disabled because it does not take the newly added `TransformLayoutLayer` into account. The test should be reworked perhaps.
1. convolution and related operations (maxpool/avgpool) now use so-called block layout (`DATA_LAYOUT_BLOCK`), where `NxCxHxW` tensors are represented as `NxC1xHxWxC0`, where `C1=(C + C0-1)/C0` and `C0` is a power-of-two (usually 4, 8, 16 or 32).
1. graph is now pre-processed and `TransformLayoutLayer` is inserted to convert data from NCHW or NHWC layout to the block layout or vice versa. The transformations are done in a lazy way only when they are really needed. For example, in the whole Resnet only 2 transformations are performed.
1. transformer-based models and other models that do not use convolutions will run as usual, without going to block layout.
1. there is yet another graph preprocessing stage added that embeds constant weights/scale and bias into convolution and batch norm layers.
1. 'batchnorm', 'activation' and 'adding a residual' are now fused with convolution, just like in the old engine. That brings some noticeable acceleration.
1. optimized convolution kernels have been added.
* depthwise convolution, as well as maxpool and avgpool support C0=4, 8, 16 etc. _as long as_ C0 is divisible by the number of fp32 lanes in a SIMD register of the target platform (e.g. on ARM with NEON there must be `C0 % 4 == 0`, on x64 with AVX2 `C0 % 8 == 0`).
* non-depthwise convolution only supports C0=8 for now. C0=8 seems to be a sweetspot for ARM with NEON, x64 with AVX2 or RISC-V with RVV (with 128- or 256-bit registers). For some platforms with dedicated matrix accelerators C0=16 or even C0=32 might be more efficient, but we could add the respective kernels later.
* only fp32 kernels have been added. fp16/bf16 kernels might be added a little later.
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
- [x] Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
Use YuNet of fixed input shape to fix not-supported-dynamic-zero-shape for FaceDetectorYN
* use yunet with input of fixed shape
* update yunet used in face recognition regression
Add DNN-based face detection and face recognition into modules/objdetect
* Add DNN-based face detector impl and interface
* Add a sample for DNN-based face detector
* add recog
* add notes
* move samples from samples/cpp to samples/dnn
* add documentation for dnn_face
* add set/get methods for input size, nms & score threshold and topk
* remove the DNN prefix from the face detector and face recognizer
* remove default values in the constructor of impl
* regenerate priors after setting input size
* two filenames for readnet
* Update face.hpp
* Update face_recognize.cpp
* Update face_match.cpp
* Update face.hpp
* Update face_recognize.cpp
* Update face_match.cpp
* Update face_recognize.cpp
* Update dnn_face.markdown
* Update dnn_face.markdown
* Update face.hpp
* Update dnn_face.markdown
* add regression test for face detection
* remove underscore prefix; fix warnings
* add reference & acknowledgement for face detection
* Update dnn_face.markdown
* Update dnn_face.markdown
* Update ts.hpp
* Update test_face.cpp
* Update face_match.cpp
* fix a compile error for python interface; add python examples for face detection and recognition
* Major changes for Vadim's comments:
* Replace class name FaceDetector with FaceDetectorYN in related failes
* Declare local mat before loop in modules/objdetect/src/face_detect.cpp
* Make input image and save flag optional in samples/dnn/face_detect(.cpp, .py)
* Add camera support in samples/dnn/face_detect(.cpp, .py)
* correct file paths for regression test
* fix convertion warnings; remove extra spaces
* update face_recog
* Update dnn_face.markdown
* Fix warnings and errors for the default CI reports:
* Remove trailing white spaces and extra new lines.
* Fix convertion warnings for windows and iOS.
* Add braces around initialization of subobjects.
* Fix warnings and errors for the default CI systems:
* Add prefix 'FR_' for each value name in enum DisType to solve the
redefinition error for iOS compilation; Modify other code accordingly
* Add bookmark '#tutorial_dnn_face' to solve warnings from doxygen
* Correct documentations to solve warnings from doxygen
* update FaceRecognizerSF
* Fix the error for CI to find ONNX models correctly
* add suffix f to float assignments
* add backend & target options for initializing face recognizer
* add checkeq for checking input size and preset size
* update test and threshold
* changes in response to alalek's comments:
* fix typos in samples/dnn/face_match.py
* import numpy before importing cv2
* add documentation to .setInputSize()
* remove extra include in face_recognize.cpp
* fix some bugs
* Update dnn_face.markdown
* update thresholds; remove useless code
* add time suffix to YuNet filename in test
* objdetect: update test code
Vadim Pisarevsky
Wwupup
Yuantao Feng