Port of the 4.x patch to 5.x. Registers a new
ConsecutiveTransposePairsSubgraph in onnx_graph_simplifier.cpp's
simplifySubgraphs() so that consecutive Transpose nodes whose composed
permutation is identity are eliminated at the ONNX proto level. This
runs before either engine takes over, so both ENGINE_CLASSIC and
ENGINE_NEW benefit.
Equivalent optimizations exist in tf2onnx (TransposeOptimizer._transpose_handler)
and ONNX Runtime (HandleTransposeImpl, 'Permutations cancel' branch).
Fixed subgraph name scoping in new DNN engine #28971
closes: https://github.com/opencv/opencv/issues/23663, https://github.com/opencv/opencv/issues/19977
OpenCV extra: https://github.com/opencv/opencv_extra/pull/1362
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Added net profiling support #28752
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Added custom layer support in new DNN engine #28963
Closes: https://github.com/opencv/opencv/issues/26200
Merge with: https://github.com/opencv/opencv_extra/pull/1358
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Extend attention fusion for runtime QK scale and add MatMul to Gemm rewriter #28957
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
fixes a bug in ENGINE_NEW of incorrect fusion of conv+relu+batchnorm #28722
### Pull Request Readiness Checklist
OpenCV Extra: https://github.com/opencv/opencv_extra/pull/1325
Closes Issue https://github.com/opencv/opencv/issues/28689
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
fix getUnconnectedOutLayers() in new engine #28982
Solves https://github.com/opencv/opencv/issues/26491
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
update default ONNX Runtime version to 1.25.1 #28969
Bump ONNX Runtime default version to latest version: 1.25.1
Includes the following update:
https://github.com/microsoft/onnxruntime/pull/27164 : Adds LpNormalization opset-22 kernel support missing in 1.24.2.
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Fixed dequantizelinear slicing bug #28966
closes: https://github.com/opencv/opencv/issues/25999
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Added fully functional convTranspose layer to new DNN engine #27560
Closes https://github.com/opencv/opencv/issues/26307
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Extend several operations to support block layout #28839
Requires opencv_extra: https://github.com/opencv/opencv_extra/pull/1347
After this PR, we observed the following improvements in the listed models:
Model | Before (ENGINE_NEW) | After (ENGINE_NEW) | ENGINE_ORT | % Improvement (Before v/s After)
-- | -- | -- | -- | --
Face_Paint | 514.68ms | 384.40ms | 394.38ms | 25.31%
BlazeFace | 1.03ms | 0.83ms | 0.66ms | 19.42%
**Device details:**
Model name: Intel(R) Core(TM) i9-14900KS, x86_64, 32 Cores, Ubuntu 22.04.5 LTS
Operations covered:
- [x] Pad
- [x] Resize
- [x] Reshape
- [x] Shape
- [x] InstanceNorm
- [x] GroupNorm
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Added Attention fusion and thin gemm support #28859
Merge with: https://github.com/opencv/opencv_extra/pull/1350
Performance numbers after these optimizations:
For Device: Intel(R) Core(TM) i9-14900KS, x86, 32 Cores, ubuntu 24.04,
| Model | `ENGINE_NEW (Before)` | `ENGINE_NEW (After)` | `ENGINE_ORT` |
| :--- | :--- | :--- | :--- |
| **BERT** |26.3 ms| 9.15 ms| 9.13 ms|
| **ViT** | 79.65 ms| 63.23 ms| 32.3 ms|
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Add Gemma3 tokenizer support for dnn #28837
- Adds Gemma3 tokenizer support
- Implements character-level BPE
- Adds 6 tests covering English, phrase, mixed case, numbers, special tokens, and encode/decode
- add gemma3_inference.py
Merge with:
- **Companion PR** : https://github.com/opencv/opencv_extra/pull/1346
- forward pass bug in gemma3_inference.py : https://github.com/opencv/opencv/pull/28836
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Added QLinear layer support #28811
closes: https://github.com/opencv/opencv/issues/26310
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Add Prior Box Layer to support the WeChat QR model #28855
OpenCV extra: https://github.com/opencv/opencv_extra/pull/1349
### Pull Request Readiness Checklist
This PR is part 1 of the split from the original PR: WeChatQR-fix conversion Caffe to ONNX #28746.
It focuses on adding the PriorBox layer and its related unit tests. The WeChatQR specific changes will be submitted in a separate PR.
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Parallelize DNN layers using chunking #28821
At a high level, I replaced tensor-level parallelism with chunk-level parallelism. Previously, parallel_for_ was dispatched over the number of input or output tensors i.e. one thread handled one whole tensor's copy.
The new approach precomputes each tensor's destination offset and per-slice size upfront, then slices the total byte work into fixed 64 KB chunks. The full chunk count is handed to parallel_for_ as a single flat range, and each worker decodes its chunk index back into (tensor, slice, byte_offset) using a prefix-sum table before running a plain memcpy on its piece. A small-size threshold falls back to the sequential path so we don't get threading overhead on small tensors.
Performance numbers after these optimizations:
For Device: Intel(R) Core(TM) i9-14900KS, x86, 32 Cores, ubuntu 24.04,
| Model | `ENGINE_NEW` | `ENGINE_ORT` |
| :--- | :--- | :--- |
| **YOLOv8n** |10.9 ms| 12.15 ms|
| **YOLOv5n** | 8.36 ms| 9.23 ms|
| **YOLOX-S** | 23.46 ms| 25.16 ms|
For Device: Macbook M1 Air
| Model | `ENGINE_NEW` | `ENGINE_ORT` |
| :--- | :--- | :--- |
| **YOLOv8n** |34.45 ms| 42.52 ms|
| **YOLOv5n** | 31.62 ms| 25.52 ms|
| **YOLOX-S** | 88.9 ms| 116.7 ms|
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Fix dnn NaN bugs in GELU SIMD and softmax for large inputs #28836
### Bug Description
- **GELU SIMD bug:** `exp(2*inner)` overflows to `inf` for large inputs (e.g. `x=10.6` → `inner≈51`), causing `inf/inf = NaN`; fixed by clamping `inner` to [-9, 9] as `tanh` already saturates to ±1.0 beyond this range.
- **Softmax bug:** All `-inf` inputs (masked attention rows) produce `sum=0`, then `1/0 = inf` and `0*inf = NaN`;
fixed by outputting zeros when `sum == 0`.
- Add regression tests for both fixes
Depends on : https://github.com/opencv/opencv/pull/28837
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Fix flatten onnx axis==rank case #28812
fix ONNX Flatten layer incorrect output when axis equals input rank edge case
Resolves : https://github.com/opencv/opencv/pull/28781#discussion_r3060088247
OpenCV Extra: https://github.com/opencv/opencv_extra/pull/1342
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Add Qwen2.5 tokenizer support for dnn #28781
OpenCV extra: https://github.com/opencv/opencv_extra/pull/1334
Extended the dnn tokenizer support to qwen2.5 tokenization.
- Add QWEN2_5 pre-tokenizer regex pattern to utils.hpp
- Generalised buildTokenizerGPT to buildTokenizerFromJson to handle gpt2/gpt4/ and qwen2.5
- Add qwen2/qwen2.5 model type support with special token handling
- Add Qwen2.5 tests
- Add end-to-end qwen_inference script for Qwen2.5 ONNX model
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Extended fusion for activation functions in new DNN engine #28750
After this fusion, we see following improvements in YOLO models:
| Model | Before (`ENGINE_NEW`) | After (`ENGINE_NEW`) | `ENGINE_ORT` | % Improvement (Before v/s After) |
| :--- | :--- | :--- | :--- | :--- |
| **YOLOv8n** | 18.89 ms| 12.06 ms| 12.15 ms| 36.16% |
| **YOLOv5n** | 17.12 ms| 9.29 ms| 9.23 ms| 45.73% |
| **YOLOX-S** | 38.78 ms| 25.56 ms| 25.16 ms| 34.09% |
Device details:
- Model name: Intel(R) Core(TM) i9-14900KS, x86, 32 Cores, ubuntu 24.04,
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Int8 block layout support #28741
After this patch we got the following speed ups on **resnet50-qdq.onnx** model.
- Inference time now: **_~6.6ms_** (inference time using onnxruntime is ~5.7ms).
- Inference time before: _**~11.5ms**_ [after QDQ PR #28595]
- Speed up: _**~42.6% or 1.74x**_
- Device details:
- Model name: Intel(R) Core(TM) i9-14900KS, x86, 32 Cores, ubuntu 24.04,
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Added OnnxRuntime GPU wrapper #28588
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Alexander Smorkalov
Kevin Lin
Abhishek Gola
omrope79
Prasad Ayush Kumar
Alexander Smorkalov
Varun Jaiswal
Teddy-Yangjiale
Dmitry Kurtaev
Kumataro
Prasad Ayush Kumar
vrooomy
SheliaJimenez