[GSOC] feat: Add ALIKED feature extractor and LightGlue matcher with DNN integration #28986
## PR Description
### Summary
Integrate ALIKED and LightGlue into OpenCV's `features` module as native`Feature2D` and `DescriptorMatcher` implementations, enabling end-to-end neural feature matching within OpenCV's ecosystem.
---
### What's included
#### New classes
- **`cv::ALIKED`** extends `Feature2D`
- CNN-based keypoint detection
- 128-D descriptor extraction via ONNX Runtime
- **`cv::LightGlueMatcher`** extends `DescriptorMatcher`
- Deep feature matching with spatial context
- Uses keypoints and image sizes during matching
---
#### API design
- Standard OpenCV patterns:
- `detectAndCompute()`
- `match()`
- `knnMatch()`
- Multiple factory methods:
- ONNX model path
- In-memory model buffer
- Pre-loaded `dnn::Net`
- `Params` structs use `CV_EXPORTS_W_SIMPLE`
for Python/Java bindings support
- Optional DNN dependency:
- `HAVE_OPENCV_DNN` guards
- Stub implementations throw `StsNotImplemented`
---
### Files added
| File | Description |
|------|-------------|
| `src/feature2d_aliked.cpp` | ALIKED implementation |
| `src/matchers_lightglue.cpp` | LightGlueMatcher implementation |
| `src/aliked_context.hpp` | Shared internal context struct |
| `test/test_aliked_lightglue.cpp` | Unit tests (9 test cases) |
| `samples/cpp/example_features_aliked_lightglue.cpp` | Demo application |
---
### Files modified
- `CMakeLists.txt`
- Add `opencv_dnn` as optional dependency
- `features.hpp`
- Add ALIKED and LightGlueMatcher declarations
- `precomp.hpp`
- Add DNN include guard
---
### Usage
```cpp
// Feature extraction
Ptr<ALIKED> aliked =
ALIKED::create("aliked-n16rot-top1k-640.onnx");
vector<KeyPoint> kpts;
Mat descs;
aliked->detectAndCompute(image, Mat(), kpts, descs);
// Feature matching
Ptr<LightGlueMatcher> lg =
LightGlueMatcher::create("aliked_lightglue.onnx");
lg->setPairInfo(
kpts1Mat,
kpts2Mat,
img1.size(),
img2.size()
);
vector<DMatch> matches;
lg->match(descs1, descs2, matches);
````
please refer to samples/cpp/example_features_aliked_lightglue.cpp
---
### Test plan
* Build with `BUILD_LIST=features,dnn`
* Build without DNN:
* Verify stubs compile
* Verify `StsNotImplemented` is thrown
* Run:
* `ctest -R Features2d_ALIKED`
* `ctest -R Features2d_LightGlueMatcher`
* Run sample application with:
* Real images
* Real ONNX models
* Verify Python/Java bindings compile and work
---
### Related
Phase 1 of the
"End-to-End AI Feature Extraction and LightGlue Matching Pipeline"
GSoC project.
Designed to be extensible to:
* XFeat
* SuperPoint
* Other neural feature extractors
### test dependency
Depends on the opencv_extra PR adding ALIKED and LightGlue test models:
- [opencv_extra PR](https://github.com/opencv/opencv_extra/pull/1366)
This PR adds the following ONNX models to `download_models.py`:
- `aliked-n16rot-top1k-640.onnx`
- `aliked_lightglue.onnx`
These models are required for the `features2d` tests in the main OpenCV repository to validate the ALIKED and LightGlue feature extraction and matching pipeline.
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Inverted imgproc-geometry dependency and moved more functions to geometry #29230
Fixes: https://github.com/opencv/opencv/issues/20267
Continues: https://github.com/opencv/opencv/pull/29175
OpenCV Contrib: https://github.com/opencv/opencv_contrib/pull/4137
OpenCV Extra: https://github.com/opencv/opencv_extra/pull/1377
Summary:
- LSD returned back to imgproc
- drawing functions moved to imgproc
- undistort image and related perf-pixel functions moved to imgproc
- moments moved to geometry
- estimateXXXtransform moved to geometry
After the patch the geometry module depends on code and Flann and may be used everywhere without potential circular dependencies
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [ ] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
Replaces wasteful temporary object construction with in-place construction using emplace_back. Covers hot paths in objdetect, imgproc, and stitching modules.
stitching: pass warp params by value to avoid CUDA constant races #28118
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under the Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on code under GPL or another license that is incompatible with OpenCV.
- [x] The PR is proposed to the proper branch.
- [x] There is a reference to the original bug report and related work.
- [ ] There are accuracy tests, performance tests, and test data in the opencv_extra repository, if applicable.
The patch to opencv_extra uses the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake configuration.
Fixes#26870.
In `modules/stitching/src/cuda/build_warp_maps.cu`, the original implementation copied parameters into global GPU constant symbols:
```cpp
cudaSafeCall(cudaMemcpyToSymbol(build_warp_maps::ck_rinv, k_rinv, 9 * sizeof(float)));
cudaSafeCall(cudaMemcpyToSymbol(build_warp_maps::cr_kinv, r_kinv, 9 * sizeof(float)));
cudaSafeCall(cudaMemcpyToSymbol(build_warp_maps::cscale, &scale, sizeof(float)));
```
As discussed in the issue, this can cause race conditions when multiple warps are built concurrently. This patch removes the use of these global constant symbols and instead passes the required data as kernel parameters (a total of 11 floats encapsulated in `WarpParams`).
One potential concern is increased register pressure due to additional kernel arguments. However, based on experiments using the test case from issue #26870, there is no significant performance regression; in fact, a small speed‑up was observed.
Testing was performed on an NVIDIA GeForce RTX 4090 (single GPU).
Note: ./bin/opencv_perf_stitching did not run successfully on my system even with an unmodified git checkout, so performance was evaluated using the test case from issue #26870 instead.
Fixed multiband blender memory leak 27333 #28085Fixes#27333
This PR fixes a memory leak in MultiBandBlender where memory from pyramid vectors was not being released when prepare() was called multiple times or when the blender object was reused.
Problem:
MultiBandBlender retains hundreds of MB to several GB of memory even after the blender pointer is released. The issue occurs because:
1. The resize() function on std::vector does not release memory when the new size is less than or equal to the current size
2. It only adjusts the size marker while retaining the capacity and existing data
3. When prepare() is called, the pyramid vectors are resized but old data remains allocated
Example from the bug report: Blending 14 images (1920x1080) retained ~200MB after blender.release(). With larger images, several GB could be retained.
Root Cause:
GPU path (lines 254-256): Correctly calls clear() before operations
Non-GPU path (lines 285-298): Missing clear() calls, causing resize() to retain old data
Solution:
Added .clear() calls before .resize() in the non-GPU path to match the GPU path behavior. This ensures:
- Memory from previous blend operations is released in prepare()
- Reusing a blender object doesn't accumulate memory
- Behavior is consistent between GPU and non-GPU code paths
Changes:
modules/stitching/src/blenders.cpp: Added 2 clear() calls (dst_pyr_laplace_.clear() and dst_band_weights_.clear()) before resize() in the non-GPU path
Pull Request Readiness Checklist:
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch (4.x)
- [x] There is a reference to the original bug report and related work (issue #27333)
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
N/A - This is a memory management fix that doesn't affect algorithm behavior. Existing stitching tests verify correctness.
- [x] The feature is well documented and sample code can be built with the project CMake
The fix maintains existing API behavior, no documentation changes needed
Add direct pdf links in the bibliography #26754
Update and add pdf links in the bibliography.
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
Remove MSVS 2013 related #26651
OpenCV 5.x requires MSVC >= 2017 15.7 and MSVS 2013 is EOF currently.
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
Features2d cleanup: Move several feature detectors and descriptors to opencv_contrib #25292
features2d cleanup: #24999
The PR moves KAZE, AKAZE, AgastFeatureDetector, BRISK and BOW to opencv_contrib/xfeatures2d.
Related PR: opencv/opencv_contrib#3709
C-API cleanup: moved cvErrorStr to new interface, minor ts changes #26101
Merge with opencv/opencv_contrib#3786
**Note:** `toString` might be too generic name (even though it is in `cv::Error::` namespace), another variant is `codeToString` (we have `typeToString` and `depthToString` in check.hpp).
**Note:** _ts_ module seem to have no other C API usage except for `ArrayTest` class which requires refactoring.
Imgproc: use double to determine whether the corners points are within src #26022close#26016
Related https://github.com/opencv/opencv_contrib/pull/3778
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Removed obsolete python samples #25268
Clean Samples #25006
This PR removes 36 obsolete python samples from the project, as part of an effort to keep the codebase clean and focused on current best practices. Some of these samples will be updated with latest algorithms or will be combined with other existing samples.
Removed Samples:
> browse.py
camshift.py
coherence.py
color_histogram.py
contours.py
deconvolution.py
dft.py
dis_opt_flow.py
distrans.py
edge.py
feature_homography.py
find_obj.py
fitline.py
gabor_threads.py
hist.py
houghcircles.py
houghlines.py
inpaint.py
kalman.py
kmeans.py
laplace.py
lk_homography.py
lk_track.py
logpolar.py
mosse.py
mser.py
opt_flow.py
plane_ar.py
squares.py
stitching.py
text_skewness_correction.py
texture_flow.py
turing.py
video_threaded.py
video_v4l2.py
watershed.py
These changes aim to improve the repository's clarity and usability by removing examples that are no longer relevant or have been superseded by more up-to-date techniques.
Move API focused C++ samples to snippets #25252
Clean Samples #25006
This PR removes 39 outdated C++ samples from the project, as part of an effort to keep the codebase clean and focused on current best practices.
C-API cleanup: apps, imgproc_c and some constants #25075
Merge with https://github.com/opencv/opencv_contrib/pull/3642
* Removed obsolete apps - traincascade and createsamples (please use older OpenCV versions if you need them). These apps relied heavily on C-API
* removed all mentions of imgproc C-API headers (imgproc_c.h, types_c.h) - they were empty, included core C-API headers
* replaced usage of several C constants with C++ ones (error codes, norm modes, RNG modes, PCA modes, ...) - most part of this PR (split into two parts - all modules and calib+3d - for easier backporting)
* removed imgproc C-API headers (as separate commit, so that other changes could be backported to 4.x)
Most of these changes can be backported to 4.x.
Remove bypass for ABI check in warpPointBackward #24989
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] ~~There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.~~
- [x] The feature is well documented and sample code can be built with the project CMake
### What?
In my [PR](https://github.com/opencv/opencv/commit/d31b6c3480c601305fa1e153a819aad0bd247fe2) regarding `warpPointBackward`, I was asked by alalek to [add a bypass](https://github.com/opencv/opencv/pull/18607/files#r508423486) to python wrapper class, presumably to ship the changes in the newest patch release (4.5.1).
The bypass was not removed in 4.6.0 release, so please remove it in either 4.10.0 or 4.11.0. Thanks!
Co-authored-by: Martin Stefanak <martin.stefanak@codasip.com>
Co-authored-by: Alexander Smorkalov <alexander.smorkalov@xperience.ai>
Removed all pre-C++11 code, workarounds, and branches #23736
This removes a bunch of pre-C++11 workrarounds that are no longer necessary as C++11 is now required.
It is a nice clean up and simplification.
* No longer unconditionally #include <array> in cvdef.h, include explicitly where needed
* Removed deprecated CV_NODISCARD, already unused in the codebase
* Removed some pre-C++11 workarounds, and simplified some backwards compat defines
* Removed CV_CXX_STD_ARRAY
* Removed CV_CXX_MOVE_SEMANTICS and CV_CXX_MOVE
* Removed all tests of CV_CXX11, now assume it's always true. This allowed removing a lot of dead code.
* Updated some documentation consequently.
* Removed all tests of CV_CXX11, now assume it's always true
* Fixed links.
---------
Co-authored-by: Maksim Shabunin <maksim.shabunin@gmail.com>
Co-authored-by: Alexander Smorkalov <alexander.smorkalov@xperience.ai>
* started working on adding 32u, 64u, 64s, bool and 16bf types to OpenCV
* core & imgproc tests seem to pass
* fixed a few compile errors and test failures on macOS x86
* hopefully fixed some compile problems and test failures
* fixed some more warnings and test failures
* trying to fix small deviations in perf_core & perf_imgproc by revering randf_64f to exact version used before
* trying to fix behavior of the new OpenCV with old plugins; there is (quite strong) assumption that video capture would give us frames with depth == CV_8U (0) or CV_16U (2). If depth is > 7 then it means that the plugin is built with the old OpenCV. It needs to be recompiled, of course and then this hack can be removed.
* try to repair the case when target arch does not have FP64 SIMD
* 1. fixed bug in itoa() found by alalek
2. restored ==, !=, > and < univ. intrinsics on ARM32/ARM64.
Yang Guanyuhan
Alexander Smorkalov
Alexander Smorkalov
Vincent Rabaud
raimbekovm
AdwaithBatchu
sinkboy-chen
Ghazi-raad
cudawarped
Alexander Smorkalov
s-trinh
FantasqueX
WU Jia
Yuantao Feng
Maksim Shabunin
Kumataro
Gursimar Singh
Martin Štefaňák
Sean McBride
Vadim Pisarevsky
Avasam
Wang Kai
Jaakko Rantala
Lilit Grigoryan
Alexander Alekhin