potok/opencv
1
0
Fork 0
mirror of https://github.com/opencv/opencv.git synced 2026-07-29 15:23:05 +04:00
Code Packages Releases Wiki Activity

Merge pull request #23843 from TolyaTalamanov:at/fix-missing-opaque-kind-for-kernel

Browse Source 
G-API: Fix incorrect OpaqueKind for Kernel outputs #23843

### Pull Request Readiness Checklist

#### Overview
The PR is going to fix several problems:
1. Major: `GKernel` doesn't hold `kind` for its outputs. Since `GModelBuilder` traverse graph from outputs to inputs once it reaches any output of the operation it will use its `kind` to create  `Data` meta for all operation outputs. Since it essential for `python` to know `GTypeInfo` (which is `shape` and `kind`) it will be confused.

Consider this operation:
```
 @cv.gapi.op('custom.square_mean', in_types=[cv.GArray.Int], out_types=[cv.GOpaque.Float, cv.GArray.Int])
    class GSquareMean:
        @staticmethod
        def outMeta(desc):
            return cv.empty_gopaque_desc(), cv.empty_array_desc()
```
Even though `GOpaque` is `Float`, corresponding metadata might have `Int` kind because it might be taken from `cv.GArray.Int`
so it will be a problem if one of the outputs of these operation is graph output because python will cast it to the wrong type based on `Data` meta.

2. Minor: Some of the OpenVINO `IR`'s doesn't any layout information for input. It's usually true only for `IRv10` but since `OpenVINO 2.0` need this information to correctly configure resize we need to put default layout if there no such assigned in `ov::Model`. 

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [ ] I agree to contribute to the project under Apache 2 License.
- [ ] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [ ] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
This commit is contained in:
Anatoliy Talamanov
2023-06-22 10:46:25 +01:00
committed by GitHub
parent 61d48dd0f8
commit 60848519b5
9 changed files with 95 additions and 23 deletions
Download Patch File Download Diff File Expand all files Collapse all files
modules/gapi/src/backends/ov/govbackend.cpp
+13 -1
View File
@@ -738,6 +738,15 @@ public:
const auto explicit_in_model_layout = lookUp(m_input_model_layout, input_name);
if (explicit_in_model_layout) {
input_info.model().set_layout(::ov::Layout(*explicit_in_model_layout));
} else if (m_model->input(input_name).get_shape().size() == 4u) {
// NB: Back compatibility with IR's without any layout information.
// Note that default is only applicable for 4D inputs in order to
// support auto resize for image use cases.
GAPI_LOG_WARNING(NULL, "Failed to find layout for input layer \""
<< input_name << "\" - NCHW is set by default");
const std::string default_layout = "NCHW";
input_info.model().set_layout(::ov::Layout(default_layout));
m_input_model_layout.emplace(input_name, default_layout);
}
const auto explicit_in_tensor_layout = lookUp(m_input_tensor_layout, input_name);
if (explicit_in_tensor_layout) {
@@ -765,6 +774,7 @@ public:
const auto &matdesc = cv::util::get<cv::GMatDesc>(input_meta);
const auto explicit_in_tensor_layout = lookUp(m_input_tensor_layout, input_name);
const auto explicit_in_model_layout = lookUp(m_input_model_layout, input_name);
const auto explicit_resize = lookUp(m_interpolation, input_name);
if (disable_img_resize && explicit_resize.has_value()) {
@@ -810,7 +820,9 @@ public:
if (matdesc.isND()) {
// NB: ND case - need to obtain "H" and "W" positions
// in order to configure resize.
const auto model_layout = ::ov::layout::get_layout(m_model->input(input_name));
const auto model_layout = explicit_in_model_layout
? ::ov::Layout(*explicit_in_model_layout)
: ::ov::layout::get_layout(m_model->input(input_name));
if (!explicit_in_tensor_layout && model_layout.empty()) {
std::stringstream ss;
ss << "Resize for input layer: " << input_name
modules/gapi/src/compiler/gmodelbuilder.cpp
+9 -9
View File
@@ -59,7 +59,6 @@ private:
} // namespace
cv::gimpl::Unrolled cv::gimpl::unrollExpr(const GProtoArgs &ins,
const GProtoArgs &outs)
{
@@ -135,18 +134,19 @@ cv::gimpl::Unrolled cv::gimpl::unrollExpr(const GProtoArgs &ins,
// Put the outputs object description of the node
// so that they are not lost if they are not consumed by other operations
GAPI_Assert(call_p.m_k.outCtors.size() == call_p.m_k.outShapes.size());
for (const auto it : ade::util::indexed(call_p.m_k.outShapes))
for (const auto it : ade::util::indexed(ade::util::zip(call_p.m_k.outShapes,
call_p.m_k.outCtors,
call_p.m_k.outKinds)))
{
std::size_t port = ade::util::index(it);
GShape shape = ade::util::value(it);
// FIXME: then use ZIP
HostCtor ctor = call_p.m_k.outCtors[port];
auto port = ade::util::index(it);
auto &val = ade::util::value(it);
auto shape = std::get<0>(val);
auto ctor = std::get<1>(val);
auto kind = std::get<2>(val);
// NB: Probably this fixes all other "missing host ctor"
// problems.
// TODO: Clean-up the old workarounds if it really is.
GOrigin org {shape, node, port, std::move(ctor), origin.kind};
GOrigin org {shape, node, port, std::move(ctor), kind};
origins.insert(org);
}