potok/opencv
1
0
Fork 0
mirror of https://github.com/opencv/opencv.git synced 2026-07-30 07:43:03 +04:00
Code Packages Releases Wiki Activity

Merge pull request #29333 from abhishek-gola:attention_layer_extension

Browse Source 
Extended Attention layer support #29333

Implemented present/past KV support.

Merge with: https://github.com/opencv/opencv_extra/pull/1381

Co-authored by: @Akansha-977 

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
This commit is contained in:
Abhishek Gola
2026-07-01 18:11:44 +05:30
committed by GitHub
parent 5866bbd9ff
commit c7b8fb28b6
9 changed files with 504 additions and 201 deletions
Download Patch File Download Diff File Expand all files Collapse all files
modules/dnn/test/test_onnx_conformance.cpp
+18
View File
@@ -1903,6 +1903,15 @@ TEST_P(Test_ONNX_conformance, Layer_Test)
if (name == "test_nllloss_NCd1d2d3d4d5_mean_weight_expanded") {
default_l1 = 2e-5; // Expected: (normL1) <= (l1), actual: 1.06394e-05 vs 1e-05
}
// fp16 Attention models retain fp16 accumulation precision (~9e-5 L1, ~2.4e-4 Inf)
// even when executed on an fp32 target.
if (name == "test_attention_4d_fp16" ||
name == "test_attention_4d_fp16_expanded" ||
name == "test_attention_4d_gqa_with_past_and_present_fp16" ||
name == "test_attention_4d_gqa_with_past_and_present_fp16_expanded") {
default_l1 = std::max(default_l1, 2e-4);
default_lInf = std::max(default_lInf, 1e-3);
}
}
#ifdef HAVE_HALIDE
else if (backend == DNN_BACKEND_HALIDE)
@@ -1965,6 +1974,15 @@ TEST_P(Test_ONNX_conformance, Layer_Test)
if (name == "test_roialign_aligned_false" || name == "test_roialign_aligned_true") {
default_l1 = 3e-5;
}
// fp16 Attention models retain fp16 accumulation precision (~9e-5 L1, ~2.4e-4 Inf)
// even when executed on an fp32 target (the layer falls back to the CPU path).
if (name == "test_attention_4d_fp16" ||
name == "test_attention_4d_fp16_expanded" ||
name == "test_attention_4d_gqa_with_past_and_present_fp16" ||
name == "test_attention_4d_gqa_with_past_and_present_fp16_expanded") {
default_l1 = std::max(default_l1, 2e-4);
default_lInf = std::max(default_lInf, 1e-3);
}
}
#endif
else