mirror of
https://github.com/opencv/opencv.git
synced 2026-07-29 23:33:05 +04:00
Add Java and Python code for AKAZE local features matching tutorial. Fix incorrect uses of Mat.mul() in Java code.
Uniform Lowe's ratio test in the code.
This commit is contained in:
catree
@@ -7,8 +7,7 @@ Introduction
|
||||
In this tutorial we will learn how to use AKAZE @cite ANB13 local features to detect and match keypoints on
|
||||
two images.
|
||||
We will find keypoints on a pair of images with given homography matrix, match them and count the
|
||||
|
||||
number of inliers (i. e. matches that fit in the given homography).
|
||||
number of inliers (i.e. matches that fit in the given homography).
|
||||
|
||||
You can find expanded version of this example here:
|
||||
<https://github.com/pablofdezalc/test_kaze_akaze_opencv>
|
||||
@@ -16,7 +15,7 @@ You can find expanded version of this example here:
|
||||
Data
|
||||
----
|
||||
|
||||
We are going to use images 1 and 3 from *Graffity* sequence of Oxford dataset.
|
||||
We are going to use images 1 and 3 from *Graffiti* sequence of [Oxford dataset](http://www.robots.ox.ac.uk/~vgg/data/data-aff.html).
|
||||
|
||||
|
||||
|
||||
@@ -27,107 +26,148 @@ Homography is given by a 3 by 3 matrix:
|
||||
3.4663091e-04 -1.4364524e-05 1.0000000e+00
|
||||
@endcode
|
||||
You can find the images (*graf1.png*, *graf3.png*) and homography (*H1to3p.xml*) in
|
||||
*opencv/samples/cpp*.
|
||||
*opencv/samples/data/*.
|
||||
|
||||
### Source Code
|
||||
|
||||
@include cpp/tutorial_code/features2D/AKAZE_match.cpp
|
||||
@add_toggle_cpp
|
||||
- **Downloadable code**: Click
|
||||
[here](https://raw.githubusercontent.com/opencv/opencv/3.4/samples/cpp/tutorial_code/features2D/AKAZE_match.cpp)
|
||||
|
||||
- **Code at glance:**
|
||||
@include samples/cpp/tutorial_code/features2D/AKAZE_match.cpp
|
||||
@end_toggle
|
||||
|
||||
@add_toggle_java
|
||||
- **Downloadable code**: Click
|
||||
[here](https://raw.githubusercontent.com/opencv/opencv/3.4/samples/java/tutorial_code/features2D/akaze_matching/AKAZEMatchDemo.java)
|
||||
|
||||
- **Code at glance:**
|
||||
@include samples/java/tutorial_code/features2D/akaze_matching/AKAZEMatchDemo.java
|
||||
@end_toggle
|
||||
|
||||
@add_toggle_python
|
||||
- **Downloadable code**: Click
|
||||
[here](https://raw.githubusercontent.com/opencv/opencv/3.4/samples/python/tutorial_code/features2D/akaze_matching/AKAZE_match.py)
|
||||
|
||||
- **Code at glance:**
|
||||
@include samples/python/tutorial_code/features2D/akaze_matching/AKAZE_match.py
|
||||
@end_toggle
|
||||
|
||||
### Explanation
|
||||
|
||||
-# **Load images and homography**
|
||||
@code{.cpp}
|
||||
Mat img1 = imread("graf1.png", IMREAD_GRAYSCALE);
|
||||
Mat img2 = imread("graf3.png", IMREAD_GRAYSCALE);
|
||||
- **Load images and homography**
|
||||
|
||||
Mat homography;
|
||||
FileStorage fs("H1to3p.xml", FileStorage::READ);
|
||||
fs.getFirstTopLevelNode() >> homography;
|
||||
@endcode
|
||||
We are loading grayscale images here. Homography is stored in the xml created with FileStorage.
|
||||
@add_toggle_cpp
|
||||
@snippet samples/cpp/tutorial_code/features2D/AKAZE_match.cpp load
|
||||
@end_toggle
|
||||
|
||||
-# **Detect keypoints and compute descriptors using AKAZE**
|
||||
@code{.cpp}
|
||||
vector<KeyPoint> kpts1, kpts2;
|
||||
Mat desc1, desc2;
|
||||
@add_toggle_java
|
||||
@snippet samples/java/tutorial_code/features2D/akaze_matching/AKAZEMatchDemo.java load
|
||||
@end_toggle
|
||||
|
||||
AKAZE akaze;
|
||||
akaze(img1, noArray(), kpts1, desc1);
|
||||
akaze(img2, noArray(), kpts2, desc2);
|
||||
@endcode
|
||||
We create AKAZE object and use it's *operator()* functionality. Since we don't need the *mask*
|
||||
parameter, *noArray()* is used.
|
||||
@add_toggle_python
|
||||
@snippet samples/python/tutorial_code/features2D/akaze_matching/AKAZE_match.py load
|
||||
@end_toggle
|
||||
|
||||
-# **Use brute-force matcher to find 2-nn matches**
|
||||
@code{.cpp}
|
||||
BFMatcher matcher(NORM_HAMMING);
|
||||
vector< vector<DMatch> > nn_matches;
|
||||
matcher.knnMatch(desc1, desc2, nn_matches, 2);
|
||||
@endcode
|
||||
We use Hamming distance, because AKAZE uses binary descriptor by default.
|
||||
We are loading grayscale images here. Homography is stored in the xml created with FileStorage.
|
||||
|
||||
-# **Use 2-nn matches to find correct keypoint matches**
|
||||
@code{.cpp}
|
||||
for(size_t i = 0; i < nn_matches.size(); i++) {
|
||||
DMatch first = nn_matches[i][0];
|
||||
float dist1 = nn_matches[i][0].distance;
|
||||
float dist2 = nn_matches[i][1].distance;
|
||||
- **Detect keypoints and compute descriptors using AKAZE**
|
||||
|
||||
if(dist1 < nn_match_ratio * dist2) {
|
||||
matched1.push_back(kpts1[first.queryIdx]);
|
||||
matched2.push_back(kpts2[first.trainIdx]);
|
||||
}
|
||||
}
|
||||
@endcode
|
||||
If the closest match is *ratio* closer than the second closest one, then the match is correct.
|
||||
@add_toggle_cpp
|
||||
@snippet samples/cpp/tutorial_code/features2D/AKAZE_match.cpp AKAZE
|
||||
@end_toggle
|
||||
|
||||
-# **Check if our matches fit in the homography model**
|
||||
@code{.cpp}
|
||||
for(int i = 0; i < matched1.size(); i++) {
|
||||
Mat col = Mat::ones(3, 1, CV_64F);
|
||||
col.at<double>(0) = matched1[i].pt.x;
|
||||
col.at<double>(1) = matched1[i].pt.y;
|
||||
@add_toggle_java
|
||||
@snippet samples/java/tutorial_code/features2D/akaze_matching/AKAZEMatchDemo.java AKAZE
|
||||
@end_toggle
|
||||
|
||||
col = homography * col;
|
||||
col /= col.at<double>(2);
|
||||
float dist = sqrt( pow(col.at<double>(0) - matched2[i].pt.x, 2) +
|
||||
pow(col.at<double>(1) - matched2[i].pt.y, 2));
|
||||
@add_toggle_python
|
||||
@snippet samples/python/tutorial_code/features2D/akaze_matching/AKAZE_match.py AKAZE
|
||||
@end_toggle
|
||||
|
||||
if(dist < inlier_threshold) {
|
||||
int new_i = inliers1.size();
|
||||
inliers1.push_back(matched1[i]);
|
||||
inliers2.push_back(matched2[i]);
|
||||
good_matches.push_back(DMatch(new_i, new_i, 0));
|
||||
}
|
||||
}
|
||||
@endcode
|
||||
If the distance from first keypoint's projection to the second keypoint is less than threshold,
|
||||
then it it fits in the homography.
|
||||
We create AKAZE and detect and compute AKAZE keypoints and descriptors. Since we don't need the *mask*
|
||||
parameter, *noArray()* is used.
|
||||
|
||||
We create a new set of matches for the inliers, because it is required by the drawing function.
|
||||
- **Use brute-force matcher to find 2-nn matches**
|
||||
|
||||
-# **Output results**
|
||||
@code{.cpp}
|
||||
Mat res;
|
||||
drawMatches(img1, inliers1, img2, inliers2, good_matches, res);
|
||||
imwrite("res.png", res);
|
||||
...
|
||||
@endcode
|
||||
Here we save the resulting image and print some statistics.
|
||||
@add_toggle_cpp
|
||||
@snippet samples/cpp/tutorial_code/features2D/AKAZE_match.cpp 2-nn matching
|
||||
@end_toggle
|
||||
|
||||
### Results
|
||||
@add_toggle_java
|
||||
@snippet samples/java/tutorial_code/features2D/akaze_matching/AKAZEMatchDemo.java 2-nn matching
|
||||
@end_toggle
|
||||
|
||||
Found matches
|
||||
-------------
|
||||
@add_toggle_python
|
||||
@snippet samples/python/tutorial_code/features2D/akaze_matching/AKAZE_match.py 2-nn matching
|
||||
@end_toggle
|
||||
|
||||
We use Hamming distance, because AKAZE uses binary descriptor by default.
|
||||
|
||||
- **Use 2-nn matches and ratio criterion to find correct keypoint matches**
|
||||
@add_toggle_cpp
|
||||
@snippet samples/cpp/tutorial_code/features2D/AKAZE_match.cpp ratio test filtering
|
||||
@end_toggle
|
||||
|
||||
@add_toggle_java
|
||||
@snippet samples/java/tutorial_code/features2D/akaze_matching/AKAZEMatchDemo.java ratio test filtering
|
||||
@end_toggle
|
||||
|
||||
@add_toggle_python
|
||||
@snippet samples/python/tutorial_code/features2D/akaze_matching/AKAZE_match.py ratio test filtering
|
||||
@end_toggle
|
||||
|
||||
If the closest match distance is significantly lower than the second closest one, then the match is correct (match is not ambiguous).
|
||||
|
||||
- **Check if our matches fit in the homography model**
|
||||
|
||||
@add_toggle_cpp
|
||||
@snippet samples/cpp/tutorial_code/features2D/AKAZE_match.cpp homography check
|
||||
@end_toggle
|
||||
|
||||
@add_toggle_java
|
||||
@snippet samples/java/tutorial_code/features2D/akaze_matching/AKAZEMatchDemo.java homography check
|
||||
@end_toggle
|
||||
|
||||
@add_toggle_python
|
||||
@snippet samples/python/tutorial_code/features2D/akaze_matching/AKAZE_match.py homography check
|
||||
@end_toggle
|
||||
|
||||
If the distance from first keypoint's projection to the second keypoint is less than threshold,
|
||||
then it fits the homography model.
|
||||
|
||||
We create a new set of matches for the inliers, because it is required by the drawing function.
|
||||
|
||||
- **Output results**
|
||||
|
||||
@add_toggle_cpp
|
||||
@snippet samples/cpp/tutorial_code/features2D/AKAZE_match.cpp draw final matches
|
||||
@end_toggle
|
||||
|
||||
@add_toggle_java
|
||||
@snippet samples/java/tutorial_code/features2D/akaze_matching/AKAZEMatchDemo.java draw final matches
|
||||
@end_toggle
|
||||
|
||||
@add_toggle_python
|
||||
@snippet samples/python/tutorial_code/features2D/akaze_matching/AKAZE_match.py draw final matches
|
||||
@end_toggle
|
||||
|
||||
Here we save the resulting image and print some statistics.
|
||||
|
||||
Results
|
||||
-------
|
||||
|
||||
### Found matches
|
||||
|
||||
|
||||
|
||||
A-KAZE Matching Results
|
||||
-----------------------
|
||||
Depending on your OpenCV version, you should get results coherent with:
|
||||
|
||||
@code{.none}
|
||||
Keypoints 1: 2943
|
||||
Keypoints 2: 3511
|
||||
Matches: 447
|
||||
Inliers: 308
|
||||
Inlier Ratio: 0.689038}
|
||||
Inlier Ratio: 0.689038
|
||||
@endcode
|
||||
|
||||
@@ -98,6 +98,8 @@ OpenCV.
|
||||
|
||||
- @subpage tutorial_akaze_matching
|
||||
|
||||
*Languages:* C++, Java, Python
|
||||
|
||||
*Compatibility:* \> OpenCV 3.0
|
||||
|
||||
*Author:* Fedor Morozov
|
||||
|
||||
Reference in New Issue
Block a user