potok/opencv
1
0
Fork 0
mirror of https://github.com/opencv/opencv.git synced 2026-07-29 15:23:05 +04:00
Code Packages Releases Wiki Activity

Merge pull request #28741 from abhishek-gola:int8_block_layout

Browse Source 
Int8 block layout support #28741

After this patch we got the following speed ups on **resnet50-qdq.onnx** model.

- Inference time now: **_~6.6ms_** (inference time using onnxruntime is ~5.7ms).
- Inference time before: _**~11.5ms**_ [after QDQ PR #28595]
- Speed up: _**~42.6% or 1.74x**_
- Device details:
- Model name: Intel(R) Core(TM) i9-14900KS, x86, 32 Cores, ubuntu 24.04, 

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
This commit is contained in:
Abhishek Gola
2026-04-12 22:10:03 +05:30
committed by GitHub
parent 1a6f669763
commit 53d9a67cf3
11 changed files with 3092 additions and 406 deletions
Download Patch File Download Diff File Expand all files Collapse all files
modules/dnn/include/opencv2/dnn/all_layers.hpp
+49
View File
@@ -382,6 +382,21 @@ CV__DNN_INLINE_NS_BEGIN
bool ceil_mode;
};
class CV_EXPORTS Conv2Int8Layer : public Layer
{
public:
static Ptr<Conv2Int8Layer> create(const LayerParams& params);
int input_zp, output_zp;
float input_sc, output_sc;
bool per_channel;
std::vector<int> strides, dilations, pads;
int ngroups;
AutoPadding auto_pad;
bool ceil_mode;
};
class CV_EXPORTS LRNLayer : public Layer
{
public:
@@ -461,6 +476,9 @@ CV__DNN_INLINE_NS_BEGIN
static Ptr<PoolingLayer> create(const LayerParams& params);
};
// Old-engine int8 pooling. Created directly by the ONNX importer for
// QLinearAveragePool / QLinearGlobalAveragePool / int8 MaxPool ops.
// Inherits PoolingLayer so it can delegate to TIMVX / NGRAPH backends.
class CV_EXPORTS PoolingLayerInt8 : public PoolingLayer
{
public:
@@ -469,6 +487,25 @@ CV__DNN_INLINE_NS_BEGIN
static Ptr<PoolingLayerInt8> create(const LayerParams& params);
};
// New-engine int8 pooling with block memory layout (DATA_LAYOUT_BLOCK).
// Created by the QDQ graph fusion pass (graph_fusion_qdq.cpp) when it
// detects a DequantizeLinear -> Pooling -> QuantizeLinear pattern.
// Uses optimised SIMD kernels; OPENCV (CPU) backend only.
class CV_EXPORTS Pool2Int8Layer : public Layer
{
public:
static Ptr<Pool2Int8Layer> create(const LayerParams& params);
int input_zp, output_zp;
float input_sc, output_sc;
std::vector<int> kernel_shape, strides, dilations, pads;
AutoPadding auto_pad;
bool ceil_mode;
bool is_global_pooling;
bool is_max_pool;
};
class CV_EXPORTS AveragePoolLayer : public Layer
{
public:
@@ -559,6 +596,7 @@ CV__DNN_INLINE_NS_BEGIN
public:
int input_zp, output_zp;
float input_sc, output_sc;
int output_type; // CV_8S or CV_8U
// quantization type flag. The perChannel default is true, that means it contains the parameters
// of per-Channel quantization. Otherwise, that means this layer contains per-Tensor quantized parameters.
@@ -1214,6 +1252,17 @@ CV__DNN_INLINE_NS_BEGIN
static Ptr<EltwiseLayerInt8> create(const LayerParams &params);
};
class CV_EXPORTS Eltwise2Int8Layer : public Layer
{
public:
static Ptr<Eltwise2Int8Layer> create(const LayerParams& params);
std::vector<float> scales;
std::vector<int> zeropoints;
float output_sc;
int output_zp;
};
class CV_EXPORTS NaryEltwiseLayer : public Layer
{
public: