Include pixel-based confidence in ArUco marker detection #23190
The aim of this pull request is to compute a **pixel-based confidence** of the marker detection. The confidence [0;1] is defined as the percentage of correctly detected pixels, with 1 describing a pixel perfect detection. Currently it is possible to get the normalized Hamming distance between the detected marker and the dictionary ground truth [Dictionary::getDistanceToId()](https://github.com/opencv/opencv/blob/4.x/modules/objdetect/src/aruco/aruco_dictionary.cpp#L114) However, this distance is based on the extracted bits and we lose information in the [majority count step](https://github.com/opencv/opencv/blob/4.x/modules/objdetect/src/aruco/aruco_detector.cpp#L487). For example, even if each cell has 49% incorrect pixels, we still obtain a perfect Hamming distance.
**Implementation tests**: Generate 36 synthetic images containing 4 markers each (with different ids) so a total of 144 markers. Invert a given percentage of pixels in each cell of the marker to simulate uncertain detection. Assuming a perfect detection, define the ground truth uncertainty as the percentage of inverted pixels. The test is passed if `abs(computedConfidece - groundTruthConfidence) < 0.05` where `0.05` accounts for minor detection inaccuracies.
- Performed for both regular and inverted markers
- Included perspective-distorted markers
- Markers in all 4 possible rotations [0, 90, 180, 270]
- Different set of detection params:
- `perspectiveRemovePixelPerCell`
- `perspectiveRemoveIgnoredMarginPerCell`
- `markerBorderBits`
The code properly builds locally and `opencv_test_objdetect` and `opencv_test_core` passed. Please let me know if there are any further modifications needed.
Thanks!
I've also pushed minor unrelated improvement (let me know if you want a separate PR) in the [bit extraction method](https://github.com/opencv/opencv/blob/4.x/modules/objdetect/src/aruco/aruco_detector.cpp#L435). `CV_Assert(perspectiveRemoveIgnoredMarginPerCell <=1)` should be `< 0.5`. Since there are margins on both sides of the cell, the margins must be smaller than half of the cell. When setting `perspectiveRemoveIgnoredMarginPerCell >= 0.5`, `opencv_test_objdetect` fails. Note: 0.499 is ok because `int()` will floor the result, thus `cellMarginPixels = int(cellMarginRate * cellSize)` will be smaller than `cellSize / 2`
### Pull Request Readiness Checklist
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [ ] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The PR is proposed to the proper branch
- [x] The feature is well documented and sample code can be built with the project CMake
Replaces wasteful temporary object construction with in-place construction using emplace_back. Covers hot paths in objdetect, imgproc, and stitching modules.
Fixed issues identified by PVS Studio #28185
Partially fixes https://github.com/opencv/opencv/issues/28167
Paper: https://pvs-studio.com/en/blog/posts/cpp/1321/
Closed items: N2, N4, N5, N6, N7, N8, N10, N11, N13, N14.
To be continued...
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [ ] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
Remove floating point arithmetic from angle computation in QR codes #28157
### Pull Request Readiness Checklist
resolves https://github.com/opencv/opencv/issues/24646
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Fix QRCodeDetector::detectAndDecode crash #27877
### Pull Request Readiness Checklist
Fix#27807
The problem is that when we find closest points from hull, we can get same closest point for several different points
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
doc: fix doxygen warnings for imgcodecs, flann and objdetect #27730
Close https://github.com/opencv/opencv/issues/27729
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
Consider QRCode ECI encoding #24426
### Pull Request Readiness Checklist
related: https://github.com/opencv/opencv/pull/24350#pullrequestreview-1661658421
1. Add `getEncoding` method to obtain ECI number
2. Add `detectAndDecodeBytes`, `decodeBytes`, `decodeBytesMulti`, `detectAndDecodeBytesMulti` methods in Python (return `bytes`) and Java (return `byte[]`)
3. Allow Python bytes to std::string conversion in general and add `encode(byte[] encoded_info, Mat qrcode)` in Java
Python example with Kanji encoding:
```python
img = cv.imread("test.png")
detect = cv.QRCodeDetector()
data, points, straight_qrcode = detect.detectAndDecodeBytes(img)
print(data)
print(detect.getEncoding(), cv.QRCodeEncoder_ECI_SHIFT_JIS)
print(data.decode("shift-jis"))
```
```
b'\x82\xb1\x82\xf1\x82\xc9\x82\xbf\x82\xcd\x90\xa2\x8aE'
20 20
こんにちは世界
```
source: https://github.com/opencv/opencv/blob/ba4d6c859d21536f84e0328c16f4cc3e96bf3065/modules/objdetect/test/test_qrcode_encode.cpp#L332
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Explicitly specify enum type scopes to improve Java wrapper generation #27228
Changed DataLayout and ImagePaddingMode to dnn::DataLayout and dnn::ImagePaddingMode to explicitly specify their scopes. This allows gen_java.py to correctly register disc_type, preventing constructors and methods using these enum types from being skipped during Java wrapper generation.
Similarly updated QRCodeEncoder::CorrectionLevel and QRCodeEncoder::EncodeMode with explicit scope declarations.
Also added a new Java test class `DnnBlobFromImageWithParamsTest` based on: https://github.com/opencv/opencv/blob/4.x/modules/dnn/test/test_misc.cpp#L133-L243
Related issues
#23753
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
Fix QR code encoder with autoversion #27244
The autodetected version is not honored in the `QRCodeEncoderImpl::encode*` methods. This fixes#27183
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
Add test for ArucoDetector::detectMarkers #27079
### Pull Request Readiness Checklist
Related to #26968 and #26922
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Fix Aruco marker incorrect detection near image edge #26968
### Pull Request Readiness Checklist
Fix#26922
As I understood the algorithm, at the first stage we search for the contours of the marker several times (adaptive threshold with different windows sizes). Therefore, for the same marker, we get several contours (inner and outer with different sizes due to the different windows sizes). In the second stage, we group the contours for the same marker into one group, from which we take the largest contour as the best candidate (which should best match the border of the marker).
The problem is that using the `minDistanceToBorder` parameter, we discard contours at the first stage. Thus, we discard the best candidates most appropriate to the marker border, and inner contours may remain, representing a significantly smaller marker border (which we observe in the issue).
But if we use the `minDistanceToBorder` parameter to discard the best candidate of the group at the second stage, then there will be no such problems and we will completely discard markers located too close to the border of the image.
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Use map to manage unique marker size candidate trees.
Avoid code duplication.
Add a test to show double detection with overlapping dictionaries.
Generalize to marker sizes of not only predefined dictionaries.
Fix rotated aruco marker board generation #26753
### Issue : [25884](https://github.com/opencv/opencv/issues/25884)
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
Use size_t when calculating size of all_points #26650Closes: #26642
Asan log
```
=================================================================
==41401==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x7fc55a02a3fc at pc 0x7fc58e304131 bp 0x7ffd54787b00 sp 0x7ffd54787af8
WRITE of size 4 at 0x7fc55a02a3fc thread T0
#0 0x7fc58e304130 in cv::QRDetectMulti::checkSets(std::vector<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > >, std::allocator<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > > > >&, std::vector<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > >, std::allocator<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > > > >&, std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > >&) /home/fanta/source/opencv/modules/objdetect/src/qrcode.cpp:3726
#1 0x7fc58e3054b0 in cv::QRDetectMulti::localization() /home/fanta/source/opencv/modules/objdetect/src/qrcode.cpp:3829
#2 0x7fc58e308020 in cv::ImplContour::detectMulti(cv::_InputArray const&, cv::_OutputArray const&) const /home/fanta/source/opencv/modules/objdetect/src/qrcode.cpp:3987
#3 0x7fc58e30b5b1 in cv::ImplContour::detectAndDecodeMulti(cv::_InputArray const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&, cv::_OutputArray const&, cv::_OutputArray const&) const /home/fanta/source/opencv/modules/objdetect/src/qrcode.cpp:4176
#4 0x7fc58e28922f in cv::GraphicalCodeDetector::detectAndDecodeMulti(cv::_InputArray const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&, cv::_OutputArray const&, cv::_OutputArray const&) const /home/fanta/source/opencv/modules/objdetect/src/graphical_code_detector.cpp:42
#5 0x5954e8 in Body /home/fanta/source/opencv/modules/objdetect/test/test_qrcode.cpp:48
#6 0x594fc0 in TestBody /home/fanta/source/opencv/modules/objdetect/test/test_qrcode.cpp:42
#7 0x67ee6a in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:3919
#8 0x6734a4 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:3955
#9 0x641fe8 in testing::Test::Run() /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:3993
#10 0x6431ac in testing::TestInfo::Run() /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:4169
#11 0x643d15 in testing::TestCase::Run() /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:4287
#12 0x659ff3 in testing::internal::UnitTestImpl::RunAllTests() /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:6662
#13 0x681205 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:3919
#14 0x675127 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:3955
#15 0x65734c in testing::UnitTest::Run() /home/fanta/source/opencv/modules/ts/src/ts_gtest.cpp:6271
#16 0x5907f0 in RUN_ALL_TESTS() /home/fanta/source/opencv/modules/ts/include/opencv2/ts/ts_gtest.h:22240
#17 0x590cdd in main (/home/fanta/source/opencv-build-4.x-clang/bin/opencv_test_objdetect+0x590cdd) (BuildId: a9363fc788d57c48225fc0559ac9199d07d415db)
#18 0x7fc58ab242ad in __libc_start_call_main (/lib64/libc.so.6+0x2a2ad) (BuildId: 03f1631dc9760d3e30311fe62e15cc4baaa89db7)
#19 0x7fc58ab24378 in __libc_start_main@@GLIBC_2.34 (/lib64/libc.so.6+0x2a378) (BuildId: 03f1631dc9760d3e30311fe62e15cc4baaa89db7)
#20 0x417014 in _start ../sysdeps/x86_64/start.S:115
0x7fc55a02a3fc is located 0 bytes after 2938510332-byte region [0x7fc4aadc8800,0x7fc55a02a3fc)
allocated by thread T0 here:
#0 0x7fc58e590298 in operator new(unsigned long) (/lib64/libasan.so.8+0xfd298) (BuildId: da72ee674d801ced58193987786b90646d94ff8d)
#1 0x7fc58e34d010 in std::__new_allocator<cv::Vec<int, 3> >::allocate(unsigned long, void const*) /usr/include/c++/14/bits/new_allocator.h:151
SUMMARY: AddressSanitizer: heap-buffer-overflow /home/fanta/source/opencv/modules/objdetect/src/qrcode.cpp:3726 in cv::QRDetectMulti::checkSets(std::vector<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > >, std::allocator<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > > > >&, std::vector<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > >, std::allocator<std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > > > >&, std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > >&)
Shadow bytes around the buggy address:
0x7fc55a02a100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x7fc55a02a180: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x7fc55a02a200: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x7fc55a02a280: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x7fc55a02a300: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x7fc55a02a380: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00[04]
0x7fc55a02a400: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x7fc55a02a480: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x7fc55a02a500: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x7fc55a02a580: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x7fc55a02a600: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
==41401==ABORTING
```
`(true_points_group[i].size()` is 1794 and `(true_points_group[i].size() - 2 ) * (true_points_group[i].size() - 1) * true_points_group[i].size())` is 5764222464 which overflows `int`
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
- Implemented a new `create` method in `FaceRecognizerSF` to allow model and configuration loading from memory buffers (std::vector<uchar>), similar to the existing functionality in `FaceDetectorYN`.
- Updated `face_recognize.cpp` with a new constructor in `FaceRecognizerSFImpl` that supports buffer-based loading for both model weights and network configuration.
- Ensured compatibility with both file-based and buffer-based model loading by maintaining consistent backend and target settings across both constructors.
- This change improves flexibility, allowing FaceRecognizerSF to be instantiated from memory buffers, which is useful for dynamic model loading scenarios such as embedded systems or applications where models are loaded in-memory.
Split Javascript white-list to support contrib modules #25986
Single whitelist converted to several per-module json files. They are concatenated automatically and can be overriden by user config.
Related to https://github.com/opencv/opencv/pull/25656
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
Properly check markers when none are provided. #25938
CharucoDetectorImpl::detectBoard finds temporary markers when none are provided but those are discarded when
charucoDetectorImpl::checkBoard is called.
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Alexander Smorkalov
raimbekovm
Alexander Smorkalov
Adrian Kretz
Jonas Perolini
AdwaithBatchu
Dmitry Kurtaev
Maxim Smolskiy
Muhammed Jaseem Pallikkal
Marius Wachtler
Kumataro
kallaballa
Sergei Nikiforov
utibenkei
Benjamin Knecht
Skreg
shyama7004
FantasqueX
Maksim Shabunin
quantizs
Vincent Rabaud
Alexander Panov