Identify ArUco markers based on threshold to reduce false positives #28289
**Goal:** parametrize the current marker identification process (pixel-based majority count) to reduce the number of false positives while maintaining high recall. Useful in high risk scenarios in which false positives are not acceptable.
**Context:** This PR builds on top of https://github.com/opencv/opencv/pull/23190 in which we've introduced a pixel-based confidence in the marker detection.
**Solution:** Include a new parameter: `validBitIdThreshold` used to identify markers based on the pixel count of each cell. Set the parameter default either to 50% which is equivalent to the current majority count implementation or to 49% which already singnificantly reduces the number of false positives (see details below).
**Test coverage:**
- Unit tests: `CV_ArucoDetectionThreshold`, `CV_InvertedArucoDetectionThreshold`
- The impact of `validBitIdThreshold` on false positives was also tested using the benchmark dataset: `MIRFLICKR-25k` https://www.kaggle.com/datasets/skfrost19/mirflickr25k which contains random images without any markers. Every marker detection is a false positive.
Example of images in the dataset:
**Results:** A threshold of 49% already allows to significantly reduce the number of false positives for the dict `DICT_4X4_1000`:
- `5942` false positives for `validBitIdThreshold = 0.5`
- `629` false positives for `validBitIdThreshold = 0.49` and `0.46`
- number of false positives divided by `9.5` when compared to `validBitIdThreshold = 0.5`
- `139` false positives for `validBitIdThreshold = 0.43` and `0.4`
- number of false positives divided by `42` when compared to `validBitIdThreshold = 0.5`
Dicts with a higher number of cells are not as impacted since it's much harder to obtain false positives. However, the less cells in a marker the further away it can be reliably detected, so the dict `DICT_4X4_1000` is commonly used.
<img width="1280" height="800" alt="false_positive_image_rate" src="https://github.com/user-attachments/assets/1a0ee16a-221d-443e-835b-022ed6dea6b0" />
In the image attached, the values of `validBitIdThreshold` tested are: `0.10f, 0.20f, 0.30f, 0.40f, 0.43f, 0.46f, 0.49f, 0.50f, 0.53f, 0.56f, 0.60f, 0.70f, 0.80f, 0.90f`
Summary of the results: [summary.csv](https://github.com/user-attachments/files/24315662/summary.csv)
Note that we can also analyse the number of false positives per marker `id`. For example, here's the histogram for the dict `DICT_4X4_1000`. (The CSV attached contains all the results)
<img width="1440" height="640" alt="false_positive_ids_DICT_4X4_1000_thr0 50" src="https://github.com/user-attachments/assets/af4f3ff8-9b8f-4682-9d51-a090c2610d8c" />
For example, the marker id 17 is detected 252 times with `validBitIdThreshold = 0.5` and only 34 times with `validBitIdThreshold = 0.49`. Looking at marker 17 (see below), we understand that this simple pattern randomly occurs in images.
<img width="447" height="441" alt="Marker17" src="https://github.com/user-attachments/assets/f5d09227-b39b-4598-94f9-b529f8300703" />
Results for every dict and every `validBitIdThreshold` [per_id.csv](https://github.com/user-attachments/files/24315667/per_id.csv)
**Missing coverage:** there is no labeled dataset with images containing markers to analyse the impact of on the recall (i.e. look at the true positive rate). For my specific use case (drones) any threshold above `0.4` allows to maintain a high recall in all conditions.
### Pull Request Readiness Checklist
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [ ] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
Fix ~0.5px systematic offset in CharucoDetector subpixel refinement #28380Resolves#25539
### Problem
`CharucoDetector::detectBoard` produces Charuco corner coordinates that are consistently offset by approximately **+0.5 pixels** compared to `findChessboardCorners + cornerSubPix`.
This offset is systematic (mean ≈ −0.5 px when comparing legacy − Charuco) and reproducible across images. Visual inspection also shows that the legacy chessboard detector aligns better with the actual corner locations.
### Root cause
In `charuco_detector.cpp`, the points passed to `cornerSubPix` are manually shifted by `-Point2f(0.5f, 0.5f)` before refinement and then shifted back by `+Point2f(0.5f, 0.5f)` after refinement.
However, `cornerSubPix` refines corners in **absolute image coordinates** and converges to the true saddle point based on image gradients. Shifting the initial guess does not affect the converged result as long as the true corner lies within the refinement window.
The additional `+0.5` shift applied after refinement therefore introduces a constant bias, resulting in:
```
P_out = P_true + 0.5
```
### Solution
Remove the manual `±0.5` coordinate shifts around the `cornerSubPix` call and let the refined result be returned directly.
### Test updates
Updated expected corner values in the following tests:
- `testBoardSubpixelCoords`
- `testSeveralBoardsWithCustomIds`
The previous expected values (e.g. `200`, `250`, `300`) were only correct due to the +0.5px bias introduced by the bug.
`generateImage` creates checkerboard squares of exactly **50 pixels**, which places true corner locations on **pixel boundaries** rather than pixel centers. As a result, the correct subpixel coordinates are:
```
199.5, 249.5, 299.5
```
instead of the previously expected integer values.
After updating the expected values, all **30 Charuco-related tests pass** with the fix applied.
### Verification
I verified the fix using a Python reproducer that compares `CharucoDetector` output against `findChessboardCorners + cornerSubPix`:
- **Before fix:** mean error ≈ −0.501 px
- **After fix:** mean error ≈ −0.001 px
After the change, the Charuco detector output aligns with the legacy chessboard detector both numerically and visually.
### Notes
This change only affects the post-refinement coordinate handling and does not alter detection logic, refinement parameters, or performance.
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Consider QRCode ECI encoding #24426
### Pull Request Readiness Checklist
related: https://github.com/opencv/opencv/pull/24350#pullrequestreview-1661658421
1. Add `getEncoding` method to obtain ECI number
2. Add `detectAndDecodeBytes`, `decodeBytes`, `decodeBytesMulti`, `detectAndDecodeBytesMulti` methods in Python (return `bytes`) and Java (return `byte[]`)
3. Allow Python bytes to std::string conversion in general and add `encode(byte[] encoded_info, Mat qrcode)` in Java
Python example with Kanji encoding:
```python
img = cv.imread("test.png")
detect = cv.QRCodeDetector()
data, points, straight_qrcode = detect.detectAndDecodeBytes(img)
print(data)
print(detect.getEncoding(), cv.QRCodeEncoder_ECI_SHIFT_JIS)
print(data.decode("shift-jis"))
```
```
b'\x82\xb1\x82\xf1\x82\xc9\x82\xbf\x82\xcd\x90\xa2\x8aE'
20 20
こんにちは世界
```
source: https://github.com/opencv/opencv/blob/ba4d6c859d21536f84e0328c16f4cc3e96bf3065/modules/objdetect/test/test_qrcode_encode.cpp#L332
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
* attempt to add 0d/1d mat support to OpenCV
* revised the patch; now 1D mat is treated as 1xN 2D mat rather than Nx1.
* a step towards 'green' tests
* another little step towards 'green' tests
* calib test failures seem to be fixed now
* more fixes _core & _dnn
* another step towards green ci; even 0D mat's (a.k.a. scalars) are now partly supported!
* * fixed strange bug in aruco/charuco detector, not sure why it did not work
* also fixed a few remaining failures (hopefully) in dnn & core
* disabled failing GAPI tests - too complex to dig into this compiler pipeline
* hopefully fixed java tests
* trying to fix some more tests
* quick followup fix
* continue to fix test failures and warnings
* quick followup fix
* trying to fix some more tests
* partly fixed support for 0D/scalar UMat's
* use updated parseReduce() from upstream
* trying to fix the remaining test failures
* fixed [ch]aruco tests in Python
* still trying to fix tests
* revert "fix" in dnn's CUDA tensor
* trying to fix dnn+CUDA test failures
* fixed 1D umat creation
* hopefully fixed remaining cuda test failures
* removed training whitespaces
Fix detect diamonds api #23848
`detectDiamonds` cannot be called from python, reproducer:
```
import numpy as np
import cv2 as cv
detector = cv.aruco.CharucoDetector(
cv.aruco.CharucoBoard(
(3, 3), 200.0, 100.0,
cv.aruco.getPredefinedDictionary(cv.aruco.DICT_4X4_250)
)
)
image = np.zeros((640, 480, 1), dtype=np.uint8)
res = detector.detectDiamonds(image)
print(res)
```
The error in `detectDiamonds` API fixed by replacing `InputOutputArrayOfArrays markerIds` with `InputOutputArray markerIds`.
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
Fix python bindings for setCharucoParameters #23436
setCharucoParameters fails in python
Fixes: https://github.com/opencv/opencv/issues/23440
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
merge with https://github.com/opencv/opencv_contrib/pull/3394
move Charuco API from contrib to main repo:
- add CharucoDetector:
```
CharucoDetector::detectBoard(InputArray image, InputOutputArrayOfArrays markerCorners, InputOutputArray markerIds,
OutputArray charucoCorners, OutputArray charucoIds) const // detect charucoCorners and/or markerCorners
CharucoDetector::detectDiamonds(InputArray image, InputOutputArrayOfArrays _markerCorners,
InputOutputArrayOfArrays _markerIds, OutputArrayOfArrays _diamondCorners,
OutputArray _diamondIds) const
```
- add `matchImagePoints()` for `CharucoBoard`
- remove contrib aruco dependencies from interactive-calibration tool
- move almost all aruco tests to objdetect
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Megre together with https://github.com/opencv/opencv_contrib/pull/3325
1. Move aruco_detector, aruco_board, aruco_dictionary, aruco_utils to objdetect
1.1 add virtual Board::draw(), virtual ~Board()
1.2 move `testCharucoCornersCollinear` to Board classes (and rename to `checkCharucoCornersCollinear`)
1.3 add wrappers to keep the old api working
3. Reduce inludes
4. Fix java tests (add objdetect import)
5. Refactoring
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
```
**WIP**
force_builders=linux,win64,docs,Linux x64 Debug,Custom
Xbuild_contrib:Docs=OFF
build_image:Custom=ubuntu:22.04
build_worker:Custom=linux-1
```
QR-Code detector : multiple detection
* change in qr-codes detection
* change in qr-codes detection
* change in test
* change in test
* add multiple detection
* multiple detection
* multiple detect
* add parallel implementation
* add functional for performance tests
* change in test
* add perftest
* returned implementation for 1 qr-code, added support for vector<Mat> and vector<vector<Point2f>> in MultipleDetectAndDecode
* deleted all lambda expressions
* changing in triangle sort
* fixed warnings
* fixed errors
* add java and python tests
* change in java tests
* change in java and python tests
* change in perf test
* change in qrcode.cpp
* add spaces
* change in qrcode.cpp
* change in qrcode.cpp
* change in qrcode.cpp
* change in java tests
* change in java tests
* solved problems
* solved problems
* change in java and python tests
* change in python tests
* change in python tests
* change in python tests
* change in methods name
* deleted sample qrcode_multi, change in qrcode.cpp
* change in perf tests
* change in objdetect.hpp
* deleted code duplication in sample qrcode.cpp
* returned spaces
* added spaces
* deleted draw function
* change in qrcode.cpp
* change in qrcode.cpp
* deleted all draw functions
* objdetect(QR): extractVerticalLines
* objdetect(QR): whitespaces
* objdetect(QR): simplify operations, avoid duplicated code
* change in interface, additional checks in java and python tests, added new key in sample for saving original image from camera
* fix warnings and errors in python test
* fix
* write in file with space key
* solved error with empty mat check in python test
* correct path to test image
* deleted spaces
* solved error with check empty mat in python tests
* added check of empty vector of points
* samples: rework qrcode.cpp
* objdetect(QR): fix API, input parameters must be first
* objdetect(QR): test/fix points layout
Alexander Smorkalov
Jonas Perolini
Siddharth Panditrao
Dmitry Kurtaev
Benjamin Knecht
WU Jia
Alexander Smorkalov
Vadim Pisarevsky
Alex
Maksim Shabunin
Alex
keith siilats
Alexander Alekhin
Suleyman TURKMEN
Polina Smolnikova
olramde
Alexander Nesterov