mirror of
https://github.com/opencv/opencv.git
synced 2026-07-29 23:33:05 +04:00
Merge pull request #28289 from JonasPerolini:pr-aruco-identification
Identify ArUco markers based on threshold to reduce false positives #28289 **Goal:** parametrize the current marker identification process (pixel-based majority count) to reduce the number of false positives while maintaining high recall. Useful in high risk scenarios in which false positives are not acceptable. **Context:** This PR builds on top of https://github.com/opencv/opencv/pull/23190 in which we've introduced a pixel-based confidence in the marker detection. **Solution:** Include a new parameter: `validBitIdThreshold` used to identify markers based on the pixel count of each cell. Set the parameter default either to 50% which is equivalent to the current majority count implementation or to 49% which already singnificantly reduces the number of false positives (see details below). **Test coverage:** - Unit tests: `CV_ArucoDetectionThreshold`, `CV_InvertedArucoDetectionThreshold` - The impact of `validBitIdThreshold` on false positives was also tested using the benchmark dataset: `MIRFLICKR-25k` https://www.kaggle.com/datasets/skfrost19/mirflickr25k which contains random images without any markers. Every marker detection is a false positive. Example of images in the dataset:   **Results:** A threshold of 49% already allows to significantly reduce the number of false positives for the dict `DICT_4X4_1000`: - `5942` false positives for `validBitIdThreshold = 0.5` - `629` false positives for `validBitIdThreshold = 0.49` and `0.46` - number of false positives divided by `9.5` when compared to `validBitIdThreshold = 0.5` - `139` false positives for `validBitIdThreshold = 0.43` and `0.4` - number of false positives divided by `42` when compared to `validBitIdThreshold = 0.5` Dicts with a higher number of cells are not as impacted since it's much harder to obtain false positives. However, the less cells in a marker the further away it can be reliably detected, so the dict `DICT_4X4_1000` is commonly used. <img width="1280" height="800" alt="false_positive_image_rate" src="https://github.com/user-attachments/assets/1a0ee16a-221d-443e-835b-022ed6dea6b0" /> In the image attached, the values of `validBitIdThreshold` tested are: `0.10f, 0.20f, 0.30f, 0.40f, 0.43f, 0.46f, 0.49f, 0.50f, 0.53f, 0.56f, 0.60f, 0.70f, 0.80f, 0.90f` Summary of the results: [summary.csv](https://github.com/user-attachments/files/24315662/summary.csv) Note that we can also analyse the number of false positives per marker `id`. For example, here's the histogram for the dict `DICT_4X4_1000`. (The CSV attached contains all the results) <img width="1440" height="640" alt="false_positive_ids_DICT_4X4_1000_thr0 50" src="https://github.com/user-attachments/assets/af4f3ff8-9b8f-4682-9d51-a090c2610d8c" /> For example, the marker id 17 is detected 252 times with `validBitIdThreshold = 0.5` and only 34 times with `validBitIdThreshold = 0.49`. Looking at marker 17 (see below), we understand that this simple pattern randomly occurs in images. <img width="447" height="441" alt="Marker17" src="https://github.com/user-attachments/assets/f5d09227-b39b-4598-94f9-b529f8300703" /> Results for every dict and every `validBitIdThreshold` [per_id.csv](https://github.com/user-attachments/files/24315667/per_id.csv) **Missing coverage:** there is no labeled dataset with images containing markers to analyse the impact of on the recall (i.e. look at the true positive rate). For my specific use case (drones) any threshold above `0.4` allows to maintain a high recall in all conditions. ### Pull Request Readiness Checklist - [x] I agree to contribute to the project under Apache 2 License. - [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV - [ ] The PR is proposed to the proper branch - [ ] There is a reference to the original bug report and related work - [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable Patch to opencv_extra has the same branch name. - [ ] The feature is well documented and sample code can be built with the project CMake
This commit is contained in:
Jonas Perolini
GitHub
@@ -473,7 +473,7 @@ markerDetectionGT applyTemperingToMarkerCells(cv::Mat &marker,
|
||||
++cellsTempered;
|
||||
|
||||
// cell too tempered, no detection expected
|
||||
if(cellTempConfig.cellRatioToTemper > 0.5f) {
|
||||
if(cellTempConfig.cellRatioToTemper > params.validBitIdThreshold) {
|
||||
if(isBorder){
|
||||
++borderErrors;
|
||||
} else {
|
||||
@@ -556,7 +556,7 @@ static void runArucoDetectionConfidence(ArucoAlgParams arucoAlgParam) {
|
||||
// make sure there are no bits have any detection errors
|
||||
params.maxErroneousBitsInBorderRate = 0.0;
|
||||
params.errorCorrectionRate = 0.0;
|
||||
params.perspectiveRemovePixelPerCell = 8; // esnsure that there is enough resolution to properly handle distortions
|
||||
params.perspectiveRemovePixelPerCell = 8; // ensure that there is enough resolution to properly handle distortions
|
||||
aruco::ArucoDetector detector(aruco::getPredefinedDictionary(aruco::DICT_6X6_250), params);
|
||||
|
||||
const bool detectInvertedMarker = (arucoAlgParam == ArucoAlgParams::DETECT_INVERTED_MARKER);
|
||||
@@ -674,6 +674,144 @@ static void runArucoDetectionConfidence(ArucoAlgParams arucoAlgParam) {
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
// Helper struc and functions for CV_ArucoDetectionUnc
|
||||
struct ArucoThresholdTestConfig {
|
||||
MarkerTemperingConfig markerTemperingConfig; // Configuration of cells to invert (percentage, number and markerRegionToTemper)
|
||||
float validBitIdThreshold; // range [0,1], define the acceptable threshold when comparing the detected marker to the dictionary during marker identification.
|
||||
float perspectiveRemoveIgnoredMarginPerCell; // Width of the margin of pixels on each cell not considered for the marker identification
|
||||
int markerBorderBits; // Number of bits of the marker border
|
||||
float distortionRatio; // Percentage of offset used for perspective distortion, bigger means more distorted
|
||||
};
|
||||
|
||||
/**
|
||||
* @brief Test the param validBitIdThreshold
|
||||
* Loops over a set of detector configurations (validBitIdThreshold, distortion, DetectorParameters such as markerBorderBits)
|
||||
* For each configuration, it creates a synthetic image containing four markers arranged in a 2x2 grid.
|
||||
* Each marker is generated with its own configuration (id, size, rotation).
|
||||
* Make sure that markers are detected or not based on validBitIdThreshold and percentage of tempering.
|
||||
* Finally, it runs the detector and checks that each marker is detected or not based on the threshold.
|
||||
*
|
||||
*/
|
||||
static void runArucoDetectionThreshold(ArucoAlgParams arucoAlgParam) {
|
||||
|
||||
aruco::DetectorParameters params;
|
||||
// make sure there are no bits have any detection errors
|
||||
params.perspectiveRemovePixelPerCell = 20; // ensure that there is enough resolution to properly handle distortions
|
||||
params.maxErroneousBitsInBorderRate = 0.f;
|
||||
params.errorCorrectionRate = 0.f;
|
||||
aruco::ArucoDetector detector(aruco::getPredefinedDictionary(aruco::DICT_5X5_250), params); // Max correction: 6bits
|
||||
|
||||
const bool detectInvertedMarker = (arucoAlgParam == ArucoAlgParams::DETECT_INVERTED_MARKER);
|
||||
|
||||
// define several detector configurations to test different settings
|
||||
// {{MarkerTemperingConfig}, validBitIdThreshold, perspectiveRemoveIgnoredMarginPerCell, markerBorderBits, distortionRatio}
|
||||
|
||||
vector<ArucoThresholdTestConfig> detectorConfigs = {
|
||||
// No tempering, expect detection for every threshold
|
||||
{{0.f, 0, MarkerRegionToTemper::ALL}, 0.3f, 0.f, 1, 0.f},
|
||||
{{0.f, 0, MarkerRegionToTemper::ALL}, 0.5f, 0.f, 1, 0.f},
|
||||
{{0.f, 0, MarkerRegionToTemper::ALL}, 0.9f, 0.f, 1, 0.f},
|
||||
// Include distortions
|
||||
{{0.f, 0, MarkerRegionToTemper::ALL}, 0.3f, 0.f, 1, 0.05f},
|
||||
{{0.f, 0, MarkerRegionToTemper::ALL}, 0.5f, 0.f, 1, 0.1f},
|
||||
{{0.f, 0, MarkerRegionToTemper::ALL}, 0.9f, 0.f, 1, 0.2f},
|
||||
|
||||
// 20% temper, expect detection with threshold above 0.2
|
||||
{{0.2f, 5, MarkerRegionToTemper::BORDER}, 0.30f, 0.f, 1, 0.f}, // Detection
|
||||
|
||||
{{0.2f, 1, MarkerRegionToTemper::BORDER}, 0.18f, 0.f, 1, 0.f}, // No detection
|
||||
{{0.2f, 1, MarkerRegionToTemper::BORDER}, 0.18f, 0.f, 1, 0.f}, // No detection
|
||||
{{0.2f, 10, MarkerRegionToTemper::INNER}, 0.22f, 0.f, 1, 0.f}, // Detection
|
||||
{{0.2f, 1, MarkerRegionToTemper::INNER}, 0.18f, 0.f, 1, 0.f} // No detection
|
||||
|
||||
// distortions
|
||||
};
|
||||
|
||||
// define marker configurations for the 4 markers in each image
|
||||
const int markerSidePixels = 700; // To simplify the cell division, markerSidePixels is a multiple of 7. (5x5 dict + 2 border bits)
|
||||
vector<MarkerCreationConfig> markerCreationConfig = {
|
||||
{0, markerSidePixels, markerRot::ROT_90}, // {id, markerSidePixels, rotation}
|
||||
{1, markerSidePixels, markerRot::ROT_270},
|
||||
{2, markerSidePixels, markerRot::NONE},
|
||||
{3, markerSidePixels, markerRot::ROT_180}
|
||||
};
|
||||
|
||||
// loop over each detector configuration
|
||||
for (size_t cfgIdx = 0; cfgIdx < detectorConfigs.size(); cfgIdx++) {
|
||||
ArucoThresholdTestConfig detCfg = detectorConfigs[cfgIdx];
|
||||
|
||||
// update detector parameters
|
||||
params.validBitIdThreshold =detCfg.validBitIdThreshold;
|
||||
params.perspectiveRemoveIgnoredMarginPerCell = detCfg.perspectiveRemoveIgnoredMarginPerCell;
|
||||
params.markerBorderBits = detCfg.markerBorderBits;
|
||||
params.detectInvertedMarker = detectInvertedMarker;
|
||||
detector.setDetectorParameters(params);
|
||||
|
||||
// create a blank image large enough to hold 4 markers in a 2x2 grid
|
||||
const int margin = markerSidePixels / 2;
|
||||
const int imageSize = (markerSidePixels * 2) + margin * 3;
|
||||
Mat img(imageSize, imageSize, CV_8UC1, Scalar(255));
|
||||
|
||||
vector<markerDetectionGT> groundTruths;
|
||||
const aruco::Dictionary &dictionary = detector.getDictionary();
|
||||
|
||||
// place each marker into the image
|
||||
for (int row = 0; row < 2; row++) {
|
||||
for (int col = 0; col < 2; col++) {
|
||||
int index = row * 2 + col;
|
||||
MarkerCreationConfig markerCfg = markerCreationConfig[index];
|
||||
// adjust marker id to be unique for each detector configuration
|
||||
markerCfg.id += static_cast<int>(cfgIdx * markerCreationConfig.size());
|
||||
|
||||
// generate img
|
||||
Mat markerImg;
|
||||
markerDetectionGT gt = generateTemperedMarkerImage(markerImg, markerCfg, detCfg.markerTemperingConfig, params, dictionary, detCfg.distortionRatio);
|
||||
|
||||
groundTruths.push_back(gt);
|
||||
|
||||
// place marker in the image
|
||||
Point2f topLeft(static_cast<float>(margin + col * (markerSidePixels + margin)),
|
||||
static_cast<float>(margin + row * (markerSidePixels + margin)));
|
||||
placeMarker(img, markerImg, topLeft);
|
||||
}
|
||||
}
|
||||
|
||||
// if testing inverted markers globally, invert the whole image
|
||||
if (detectInvertedMarker) {
|
||||
bitwise_not(img, img);
|
||||
}
|
||||
|
||||
// run detection.
|
||||
vector<vector<Point2f>> corners, rejected;
|
||||
vector<int> ids;
|
||||
vector<float> markerConfidence;
|
||||
detector.detectMarkersWithConfidence(img, corners, ids, markerConfidence, rejected);
|
||||
|
||||
ASSERT_EQ(ids.size(), corners.size());
|
||||
ASSERT_EQ(ids.size(), markerConfidence.size());
|
||||
|
||||
std::map<int, float> confidenceById;
|
||||
for (size_t i = 0; i < ids.size(); i++) {
|
||||
confidenceById[ids[i]] = markerConfidence[i];
|
||||
}
|
||||
|
||||
// verify that every marker is detected and its confidence is within tolerance
|
||||
for (const auto& currentGT : groundTruths) {
|
||||
const auto it = confidenceById.find(currentGT.id);
|
||||
const bool detected = it != confidenceById.end();
|
||||
EXPECT_EQ(currentGT.expectDetection, detected)
|
||||
<< "Marker id: " << currentGT.id << " (detector config " << cfgIdx << ")";
|
||||
|
||||
if (currentGT.expectDetection && detected) {
|
||||
EXPECT_NEAR(currentGT.confidence, it->second, 0.05)
|
||||
<< "Marker id: " << currentGT.id << " (detector config " << cfgIdx << ")";
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* @brief Check max and min size in marker detection parameters
|
||||
*/
|
||||
@@ -981,6 +1119,14 @@ TEST(CV_InvertedFlagArucoDetectionConfidence, algorithmic) {
|
||||
}
|
||||
}
|
||||
|
||||
TEST(CV_ArucoDetectionThreshold, algorithmic) {
|
||||
runArucoDetectionThreshold(ArucoAlgParams::USE_DEFAULT);
|
||||
}
|
||||
|
||||
TEST(CV_InvertedArucoDetectionThreshold, algorithmic) {
|
||||
runArucoDetectionThreshold(ArucoAlgParams::DETECT_INVERTED_MARKER);
|
||||
}
|
||||
|
||||
TEST(CV_ArucoDetectMarkers, regression_3192)
|
||||
{
|
||||
aruco::ArucoDetector detector(aruco::getPredefinedDictionary(aruco::DICT_4X4_50));
|
||||
@@ -1304,6 +1450,7 @@ TEST_P(ArucoThreading, number_of_threads_does_not_change_results)
|
||||
|
||||
aruco::DetectorParameters detectorParameters = detector.getDetectorParameters();
|
||||
detectorParameters.cornerRefinementMethod = (int)GetParam();
|
||||
detectorParameters.validBitIdThreshold = 0.5f;
|
||||
detector.setDetectorParameters(detectorParameters);
|
||||
|
||||
vector<vector<Point2f> > original_corners;
|
||||
|
||||
Reference in New Issue
Block a user