potok/opencv
1
0
Fork 0
mirror of https://github.com/opencv/opencv.git synced 2026-07-29 15:23:05 +04:00
Code Packages Releases Wiki Activity

Merge pull request #28821 from abhishek-gola:optimized_layers

Browse Source 
Parallelize DNN layers using chunking #28821

At a high level, I replaced tensor-level parallelism with chunk-level parallelism. Previously, parallel_for_ was dispatched over the number of input or output tensors i.e. one thread handled one whole tensor's copy.

The new approach precomputes each tensor's destination offset and per-slice size upfront, then slices the total byte work into fixed 64 KB chunks. The full chunk count is handed to parallel_for_ as a single flat range, and each worker decodes its chunk index back into (tensor, slice, byte_offset) using a prefix-sum table before running a plain memcpy on its piece. A small-size threshold falls back to the sequential path so we don't get threading overhead on small tensors.

Performance numbers after these optimizations:

For Device:  Intel(R) Core(TM) i9-14900KS, x86, 32 Cores, ubuntu 24.04,
| Model | `ENGINE_NEW` | `ENGINE_ORT` |
| :--- | :--- | :--- | 
| **YOLOv8n** |10.9 ms| 12.15 ms|
| **YOLOv5n** | 8.36 ms| 9.23 ms|
| **YOLOX-S** | 23.46 ms| 25.16 ms|

For Device: Macbook M1 Air
| Model | `ENGINE_NEW` | `ENGINE_ORT` |
| :--- | :--- | :--- | 
| **YOLOv8n** |34.45 ms| 42.52 ms|
| **YOLOv5n** | 31.62 ms| 25.52 ms|
| **YOLOX-S** | 88.9 ms| 116.7 ms|

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
This commit is contained in:
Abhishek Gola
2026-04-22 17:23:12 +05:30
committed by GitHub
parent 0102afa86f
commit d95badefa4
7 changed files with 384 additions and 104 deletions
Download Patch File Download Diff File Expand all files Collapse all files
modules/dnn/perf/perf_net.cpp
+14
View File
@@ -124,6 +124,11 @@ PERF_TEST_P_(DNNTestNetwork, MobileNetv2_ONNX)
processNet("dnn/onnx/models/mobilenetv2.onnx", "", cv::Size(224, 224));
}
PERF_TEST_P_(DNNTestNetwork, ResNet50_QDQ_ONNX)
{
processNet("dnn/onnx/models/resnet50-v1-12-qdq.onnx", "", cv::Size(224, 224));
}
PERF_TEST_P_(DNNTestNetwork, SqueezeNet_v1_1)
{
processNet("dnn/squeezenet_v1.1.caffemodel", "dnn/squeezenet_v1.1.prototxt", cv::Size(227, 227));
@@ -157,6 +162,15 @@ PERF_TEST_P_(DNNTestNetwork, MobileNet_SSD_v2_TensorFlow)
processNet("dnn/ssd_mobilenet_v2_coco_2018_03_29.pb", "ssd_mobilenet_v2_coco_2018_03_29.pbtxt", cv::Size(300, 300));
}
PERF_TEST_P_(DNNTestNetwork, MobileNet_SSD_v1_ONNX)
{
Mat image(cv::Size(300, 300), CV_8UC3);
randu(image, 0, 255);
int imsize[] = {1, image.rows, image.cols, 3};
Mat input(4, imsize, CV_8U, image.data);
processNet("dnn/onnx/models/ssd_mobilenet_v1_12.onnx", "", input);
}
PERF_TEST_P_(DNNTestNetwork, DenseNet_121)
{
processNet("dnn/DenseNet_121.caffemodel", "dnn/DenseNet_121.prototxt", cv::Size(224, 224));