potok/opencv
1
0
Fork 0
mirror of https://github.com/opencv/opencv.git synced 2026-07-29 15:23:05 +04:00
Code Packages Releases Wiki Activity

Merge pull request #25810 from fengyuentau:python/fix_parsing_3d_mat_in_dnn

Browse Source 
python: attempts to fix 3d mat parsing problem for dnn #25810

Fixes https://github.com/opencv/opencv/issues/25762 https://github.com/opencv/opencv/issues/23242
Relates https://github.com/opencv/opencv/issues/25763 https://github.com/opencv/opencv/issues/19091

Although `cv.Mat` has already been introduced to workaround this problem, people do not know it and it kind of leads to confusion with `numpy.array`. This patch adds a "switch" to turn off the auto multichannel feature when the API is from cv::dnn::Net (more specifically, `setInput`) and the parameter is of type `Mat`. This patch only leads to changes of three places in `pyopencv_generated_types_content.h`:

```.diff
static PyObject* pyopencv_cv_dnn_dnn_Net_setInput(PyObject* self, PyObject* py_args, PyObject* kw)
{
...
- pyopencv_to_safe(pyobj_blob, blob, ArgInfo("blob", 0)) &&
+ pyopencv_to_safe(pyobj_blob, blob, ArgInfo("blob", 8)) &&
...
}

// I guess we also need to change this as one-channel blob is expected for param
static PyObject* pyopencv_cv_dnn_dnn_Net_setParam(PyObject* self, PyObject* py_args, PyObject* kw)
{
...
- pyopencv_to_safe(pyobj_blob, blob, ArgInfo("blob", 0)) )
+ pyopencv_to_safe(pyobj_blob, blob, ArgInfo("blob", 8)) )
...
- pyopencv_to_safe(pyobj_blob, blob, ArgInfo("blob", 0)) )
+ pyopencv_to_safe(pyobj_blob, blob, ArgInfo("blob", 8)) )
...
}
```

Others are unchanged, e.g. `dnn_SegmentationModel` and stuff like that.

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
This commit is contained in:
Yuantao Feng
2024-07-04 13:33:20 +08:00
committed by GitHub
parent e644ab0b40
commit 5510718381
8 changed files with 102 additions and 24 deletions
Download Patch File Download Diff File Expand all files Collapse all files
modules/dnn/misc/python/test/test_dnn.py
+55 -6
View File
@@ -455,10 +455,6 @@ class dnn_test(NewOpenCVTests):
"Verify OPENCV_DNN_TEST_DATA_PATH configuration parameter.")
input = np.load(input_file)
# we have to expand the shape of input tensor because Python bindings cut 3D tensors to 2D
# it should be fixed in future. see : https://github.com/opencv/opencv/issues/19091
# please remove `expand_dims` after that
input = np.expand_dims(input, axis=3)
gold_output = np.load(output_file)
for backend, target in self.dnnBackendsAndTargets:
@@ -469,10 +465,63 @@ class dnn_test(NewOpenCVTests):
net.setPreferableBackend(backend)
net.setPreferableTarget(target)
# Check whether 3d shape is parsed correctly for setInput
net.setInput(input)
real_output = net.forward()
normAssert(self, real_output, gold_output, "", getDefaultThreshold(target))
# Case 0: test API `forward(const String& outputName = String()`
real_output = net.forward() # Retval is a np.array of shape [2, 5, 3]
normAssert(self, real_output, gold_output, "Case 1", getDefaultThreshold(target))
'''
Pre-allocate output memory with correct shape.
Normally Python users do not use in this way,
but we have to test it since we design API in this way
'''
# Case 1: a np.array with a string of output name.
# It tests API `forward(OutputArrayOfArrays outputBlobs, const String& outputName = String()`
# when outputBlobs is a np.array and we expect it to be the only output.
real_output = np.empty([2, 5, 3], dtype=np.float32)
real_output = net.forward(real_output, "237") # Retval is a tuple with a np.array of shape [2, 5, 3]
normAssert(self, real_output, gold_output, "Case 1", getDefaultThreshold(target))
# Case 2: a tuple of np.array with a string of output name.
# It tests API `forward(OutputArrayOfArrays outputBlobs, const String& outputName = String()`
# when outputBlobs is a container of several np.array and we expect to save all outputs accordingly.
real_output = tuple(np.empty([2, 5, 3], dtype=np.float32))
real_output = net.forward(real_output, "237") # Retval is a tuple with a np.array of shape [2, 5, 3]
normAssert(self, real_output, gold_output, "Case 2", getDefaultThreshold(target))
# Case 3: a tuple of np.array with a string of output name.
# It tests API `forward(OutputArrayOfArrays outputBlobs, const std::vector<String>& outBlobNames)`
real_output = tuple(np.empty([2, 5, 3], dtype=np.float32))
# Note that it does not support parsing a list , e.g. ["237"]
real_output = net.forward(real_output, ("237")) # Retval is a tuple with a np.array of shape [2, 5, 3]
normAssert(self, real_output, gold_output, "Case 3", getDefaultThreshold(target))
def test_set_param_3d(self):
model_path = self.find_dnn_file('dnn/onnx/models/matmul_3d_init.onnx')
input_file = self.find_dnn_file('dnn/onnx/data/input_matmul_3d_init.npy')
output_file = self.find_dnn_file('dnn/onnx/data/output_matmul_3d_init.npy')
input = np.load(input_file)
output = np.load(output_file)
for backend, target in self.dnnBackendsAndTargets:
printParams(backend, target)
net = cv.dnn.readNet(model_path)
node_name = net.getLayerNames()[0]
w = net.getParam(node_name, 0) # returns the original tensor of three-dimensional shape
net.setParam(node_name, 0, w) # set param once again to see whether tensor is converted with correct shape
net.setPreferableBackend(backend)
net.setPreferableTarget(target)
net.setInput(input)
res_output = net.forward()
normAssert(self, output, res_output, "", getDefaultThreshold(target))
def test_scalefactor_assign(self):
params = cv.dnn.Image2BlobParams()