potok/opencv
1
0
Fork 0
mirror of https://github.com/opencv/opencv.git synced 2026-07-31 08:13:04 +04:00
Code Packages Releases Wiki Activity

Merge pull request #28986 from YangGuanyuhan:ai-aliked-lightglue-pipeline

Browse Source 
[GSOC] feat: Add ALIKED feature extractor and LightGlue matcher with DNN integration #28986

## PR Description

### Summary

Integrate ALIKED and LightGlue into OpenCV's `features` module as native`Feature2D` and `DescriptorMatcher` implementations, enabling end-to-end neural feature matching within OpenCV's ecosystem.

---

### What's included

#### New classes

- **`cv::ALIKED`** extends `Feature2D`
  - CNN-based keypoint detection
  - 128-D descriptor extraction via ONNX Runtime

- **`cv::LightGlueMatcher`** extends `DescriptorMatcher`
  - Deep feature matching with spatial context
  - Uses keypoints and image sizes during matching

---

#### API design

- Standard OpenCV patterns:
  - `detectAndCompute()`
  - `match()`
  - `knnMatch()`

- Multiple factory methods:
  - ONNX model path
  - In-memory model buffer
  - Pre-loaded `dnn::Net`

- `Params` structs use `CV_EXPORTS_W_SIMPLE`
  for Python/Java bindings support

- Optional DNN dependency:
  - `HAVE_OPENCV_DNN` guards
  - Stub implementations throw `StsNotImplemented`

---

### Files added

| File | Description |
|------|-------------|
| `src/feature2d_aliked.cpp` | ALIKED implementation |
| `src/matchers_lightglue.cpp` | LightGlueMatcher implementation |
| `src/aliked_context.hpp` | Shared internal context struct |
| `test/test_aliked_lightglue.cpp` | Unit tests (9 test cases) |
| `samples/cpp/example_features_aliked_lightglue.cpp` | Demo application |

---

### Files modified

- `CMakeLists.txt`
  - Add `opencv_dnn` as optional dependency

- `features.hpp`
  - Add ALIKED and LightGlueMatcher declarations

- `precomp.hpp`
  - Add DNN include guard

---

### Usage

```cpp
// Feature extraction
Ptr<ALIKED> aliked =
    ALIKED::create("aliked-n16rot-top1k-640.onnx");

vector<KeyPoint> kpts;
Mat descs;

aliked->detectAndCompute(image, Mat(), kpts, descs);

// Feature matching
Ptr<LightGlueMatcher> lg =
    LightGlueMatcher::create("aliked_lightglue.onnx");

lg->setPairInfo(
    kpts1Mat,
    kpts2Mat,
    img1.size(),
    img2.size()
);

vector<DMatch> matches;
lg->match(descs1, descs2, matches);
````
please refer to samples/cpp/example_features_aliked_lightglue.cpp

---

### Test plan

* Build with `BUILD_LIST=features,dnn`
* Build without DNN:

  * Verify stubs compile
  * Verify `StsNotImplemented` is thrown
* Run:

  * `ctest -R Features2d_ALIKED`
  * `ctest -R Features2d_LightGlueMatcher`
* Run sample application with:

  * Real images
  * Real ONNX models
* Verify Python/Java bindings compile and work

---

### Related

Phase 1 of the
"End-to-End AI Feature Extraction and LightGlue Matching Pipeline"
GSoC project.

Designed to be extensible to:

* XFeat
* SuperPoint
* Other neural feature extractors

### test dependency

Depends on the opencv_extra PR adding ALIKED and LightGlue test models:

- [opencv_extra PR](https://github.com/opencv/opencv_extra/pull/1366)

This PR adds the following ONNX models to `download_models.py`:

- `aliked-n16rot-top1k-640.onnx`
- `aliked_lightglue.onnx`

These models are required for the `features2d` tests in the main OpenCV repository to validate the ALIKED and LightGlue feature extraction and matching pipeline.

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
This commit is contained in:
Yang Guanyuhan
2026-06-05 19:20:53 +08:00
committed by GitHub
parent 04aee009aa
commit 527f01449d
14 changed files with 1240 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files
modules/dnn/src/layers/cast2_layer.cpp
+38
View File
@@ -21,6 +21,40 @@ namespace cv { namespace dnn {
namespace
{
// ONNX Cast float->int truncates toward zero; Mat::convertTo rounds. Truncate to match the spec.
template<typename DT>
inline void truncateToIntImpl(const Mat& src, Mat& dst)
{
const int n = (int)src.total() * src.channels();
DT* d = dst.ptr<DT>();
if (src.depth() == CV_32F)
{
const float* s = src.ptr<float>();
for (int i = 0; i < n; ++i)
d[i] = saturate_cast<DT>(std::trunc(s[i]));
}
else
{
const double* s = src.ptr<double>();
for (int i = 0; i < n; ++i)
d[i] = saturate_cast<DT>(std::trunc(s[i]));
}
}
inline void truncateFloatToInt(const Mat& src, Mat& dst)
{
switch (dst.depth())
{
case CV_8U: truncateToIntImpl<uchar>(src, dst); break;
case CV_8S: truncateToIntImpl<schar>(src, dst); break;
case CV_16U: truncateToIntImpl<ushort>(src, dst); break;
case CV_16S: truncateToIntImpl<short>(src, dst); break;
case CV_32S: truncateToIntImpl<int>(src, dst); break;
case CV_64S: truncateToIntImpl<int64_t>(src, dst); break;
default: src.convertTo(dst, dst.depth()); break;
}
}
inline void castQuantized(const Mat& src, Mat& dst, int targetDepth)
{
if (targetDepth == CV_16F)
@@ -297,6 +331,10 @@ public:
else
src.convertTo(dst, ddepth);
}
else if ((sdepth == CV_32F || sdepth == CV_64F) && CV_IS_INT_TYPE(ddepth))
{
truncateFloatToInt(src, dst);
}
else
{
src.convertTo(dst, ddepth);
modules/dnn/src/layers/topk2_layer.cpp
-1
View File
@@ -50,7 +50,6 @@ public:
axis = params.get<int>("axis", -1);
largest = params.get<int>("largest", 1) == 1;
sorted = params.get<int>("sorted", 1) == 1;
CV_CheckTrue(sorted, "TopK2: sorted == false is not supported");
if (params.has("k")) {
K = params.get<int>("k");
CV_CheckGT(K, 0, "TopK2: K needs to be a positive integer");