Identify ArUco markers based on threshold to reduce false positives #28289
**Goal:** parametrize the current marker identification process (pixel-based majority count) to reduce the number of false positives while maintaining high recall. Useful in high risk scenarios in which false positives are not acceptable.
**Context:** This PR builds on top of https://github.com/opencv/opencv/pull/23190 in which we've introduced a pixel-based confidence in the marker detection.
**Solution:** Include a new parameter: `validBitIdThreshold` used to identify markers based on the pixel count of each cell. Set the parameter default either to 50% which is equivalent to the current majority count implementation or to 49% which already singnificantly reduces the number of false positives (see details below).
**Test coverage:**
- Unit tests: `CV_ArucoDetectionThreshold`, `CV_InvertedArucoDetectionThreshold`
- The impact of `validBitIdThreshold` on false positives was also tested using the benchmark dataset: `MIRFLICKR-25k` https://www.kaggle.com/datasets/skfrost19/mirflickr25k which contains random images without any markers. Every marker detection is a false positive.
Example of images in the dataset:
**Results:** A threshold of 49% already allows to significantly reduce the number of false positives for the dict `DICT_4X4_1000`:
- `5942` false positives for `validBitIdThreshold = 0.5`
- `629` false positives for `validBitIdThreshold = 0.49` and `0.46`
- number of false positives divided by `9.5` when compared to `validBitIdThreshold = 0.5`
- `139` false positives for `validBitIdThreshold = 0.43` and `0.4`
- number of false positives divided by `42` when compared to `validBitIdThreshold = 0.5`
Dicts with a higher number of cells are not as impacted since it's much harder to obtain false positives. However, the less cells in a marker the further away it can be reliably detected, so the dict `DICT_4X4_1000` is commonly used.
<img width="1280" height="800" alt="false_positive_image_rate" src="https://github.com/user-attachments/assets/1a0ee16a-221d-443e-835b-022ed6dea6b0" />
In the image attached, the values of `validBitIdThreshold` tested are: `0.10f, 0.20f, 0.30f, 0.40f, 0.43f, 0.46f, 0.49f, 0.50f, 0.53f, 0.56f, 0.60f, 0.70f, 0.80f, 0.90f`
Summary of the results: [summary.csv](https://github.com/user-attachments/files/24315662/summary.csv)
Note that we can also analyse the number of false positives per marker `id`. For example, here's the histogram for the dict `DICT_4X4_1000`. (The CSV attached contains all the results)
<img width="1440" height="640" alt="false_positive_ids_DICT_4X4_1000_thr0 50" src="https://github.com/user-attachments/assets/af4f3ff8-9b8f-4682-9d51-a090c2610d8c" />
For example, the marker id 17 is detected 252 times with `validBitIdThreshold = 0.5` and only 34 times with `validBitIdThreshold = 0.49`. Looking at marker 17 (see below), we understand that this simple pattern randomly occurs in images.
<img width="447" height="441" alt="Marker17" src="https://github.com/user-attachments/assets/f5d09227-b39b-4598-94f9-b529f8300703" />
Results for every dict and every `validBitIdThreshold` [per_id.csv](https://github.com/user-attachments/files/24315667/per_id.csv)
**Missing coverage:** there is no labeled dataset with images containing markers to analyse the impact of on the recall (i.e. look at the true positive rate). For my specific use case (drones) any threshold above `0.4` allows to maintain a high recall in all conditions.
### Pull Request Readiness Checklist
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [ ] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
Include pixel-based confidence in ArUco marker detection #23190
The aim of this pull request is to compute a **pixel-based confidence** of the marker detection. The confidence [0;1] is defined as the percentage of correctly detected pixels, with 1 describing a pixel perfect detection. Currently it is possible to get the normalized Hamming distance between the detected marker and the dictionary ground truth [Dictionary::getDistanceToId()](https://github.com/opencv/opencv/blob/4.x/modules/objdetect/src/aruco/aruco_dictionary.cpp#L114) However, this distance is based on the extracted bits and we lose information in the [majority count step](https://github.com/opencv/opencv/blob/4.x/modules/objdetect/src/aruco/aruco_detector.cpp#L487). For example, even if each cell has 49% incorrect pixels, we still obtain a perfect Hamming distance.
**Implementation tests**: Generate 36 synthetic images containing 4 markers each (with different ids) so a total of 144 markers. Invert a given percentage of pixels in each cell of the marker to simulate uncertain detection. Assuming a perfect detection, define the ground truth uncertainty as the percentage of inverted pixels. The test is passed if `abs(computedConfidece - groundTruthConfidence) < 0.05` where `0.05` accounts for minor detection inaccuracies.
- Performed for both regular and inverted markers
- Included perspective-distorted markers
- Markers in all 4 possible rotations [0, 90, 180, 270]
- Different set of detection params:
- `perspectiveRemovePixelPerCell`
- `perspectiveRemoveIgnoredMarginPerCell`
- `markerBorderBits`
The code properly builds locally and `opencv_test_objdetect` and `opencv_test_core` passed. Please let me know if there are any further modifications needed.
Thanks!
I've also pushed minor unrelated improvement (let me know if you want a separate PR) in the [bit extraction method](https://github.com/opencv/opencv/blob/4.x/modules/objdetect/src/aruco/aruco_detector.cpp#L435). `CV_Assert(perspectiveRemoveIgnoredMarginPerCell <=1)` should be `< 0.5`. Since there are margins on both sides of the cell, the margins must be smaller than half of the cell. When setting `perspectiveRemoveIgnoredMarginPerCell >= 0.5`, `opencv_test_objdetect` fails. Note: 0.499 is ok because `int()` will floor the result, thus `cellMarginPixels = int(cellMarginRate * cellSize)` will be smaller than `cellSize / 2`
### Pull Request Readiness Checklist
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [ ] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The PR is proposed to the proper branch
- [x] The feature is well documented and sample code can be built with the project CMake
Add test for ArucoDetector::detectMarkers #27079
### Pull Request Readiness Checklist
Related to #26968 and #26922
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Use map to manage unique marker size candidate trees.
Avoid code duplication.
Add a test to show double detection with overlapping dictionaries.
Generalize to marker sizes of not only predefined dictionaries.
Megre together with https://github.com/opencv/opencv_contrib/pull/3325
1. Move aruco_detector, aruco_board, aruco_dictionary, aruco_utils to objdetect
1.1 add virtual Board::draw(), virtual ~Board()
1.2 move `testCharucoCornersCollinear` to Board classes (and rename to `checkCharucoCornersCollinear`)
1.3 add wrappers to keep the old api working
3. Reduce inludes
4. Fix java tests (add objdetect import)
5. Refactoring
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
```
**WIP**
force_builders=linux,win64,docs,Linux x64 Debug,Custom
Xbuild_contrib:Docs=OFF
build_image:Custom=ubuntu:22.04
build_worker:Custom=linux-1
```
Jonas Perolini
Maxim Smolskiy
Benjamin Knecht
Alex
Stefan Becker