Merge pull request #26056 from vpisarev:new_dnn_engine

New dnn engine #26056

This is the 1st PR with the new engine; CI is green and PR is ready to be merged, I think.
Merge together with https://github.com/opencv/opencv_contrib/pull/3794

---

**Known limitations:**
* [solved] OpenVINO is temporarily disabled, but is probably easy to restore (it's not a deal breaker to merge this PR, I guess)
* The new engine does not support any backends nor any targets except for the default CPU implementation. But it's possible to choose the old engine when loading a model, then all the functionality is available.
* [Caffe patch is here: #26208] The new engine only supports ONNX. When a model is constructed manually or is loaded from a file of different format (.tf, .tflite, .caffe, .darknet), the old engine is used.
* Even in the case of ONNX some layers are not supported by the new engine, such as all quantized layers (including DequantizeLinear, QuantizeLinear, QLinearConv etc.), LSTM, GRU, .... It's planned, of course, to have full support for ONNX by OpenCV 5.0 gold release. When a loaded model contains unsupported layers, we switch to the old engine automatically  (at ONNX parsing time, not at `forward()` time).
* Some layers , e.g. Expat, are only partially supported by the new engine. In the case of unsupported flavours it switches to the old engine automatically (at ONNX parsing time, not at `forward()` time).
* 'Concat' graph optimization is disabled. The optimization eliminates Concat layer and instead makes the layers that generate tensors to be concatenated to write the outputs to the final destination. Of course, it's only possible when `axis=0` or `axis=N=1`. The optimization is not compatible with dynamic shapes since we need to know in advance where to store the tensors. Because some of the layer implementations have been modified to become more compatible with the new engine, the feature appears to be broken even when the old engine is used.
* Some `dnn::Net` API is not available with the new engine. Also, shape inference may return false if some of the output or intermediate tensors' shapes cannot be inferred without running the model. Probably this can be fixed by a dummy run of the model with zero inputs.
* Some overloads of `dnn::Net::getFLOPs()` and `dnn::Net::getMemoryConsumption()` are not exposed any longer in wrapper generators; but the most useful overloads are exposed (and checked by Java tests).
* [in progress] A few Einsum tests related to empty shapes have been disabled due to crashes in the tests and in Einsum implementations. The code and the tests need to be repaired.
* OpenCL implementation of Deconvolution is disabled. It's very bad and very slow anyway; need to be completely revised.
* Deconvolution3D test is now skipped, because it was only supported by CUDA and OpenVINO backends, both of which are not supported by the new engine.
* Some tests, such as FastNeuralStyle, checked that the in the case of CUDA backend there is no fallback to CPU. Currently all layers in the new engine are processed on CPU, so there are many fallbacks. The checks, therefore, have been temporarily disabled.

---

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake

modules/core/src/matrix.cpp

@@ -7,6 +7,409 @@
 
 namespace cv {
 
+std::string layoutToString(DataLayout layout)
+{
+    return
+        layout == DATA_LAYOUT_ND ? "ND" :
+        layout == DATA_LAYOUT_NCHW ? "NCHW" :
+        layout == DATA_LAYOUT_NHWC ? "NHWC" :
+        layout == DATA_LAYOUT_BLOCK ? "NC1HWC0" :
+        layout == DATA_LAYOUT_NCDHW ? "NCDHW" :
+        layout == DATA_LAYOUT_NDHWC ? "NDHWC" :
+        layout == DATA_LAYOUT_PLANAR ? "PLANAR" :
+        layout == DATA_LAYOUT_UNKNOWN ? "Unknown" : "???";
+}
+
+bool operator == (const MatShape& size1, const MatShape& size2)
+{
+    if (size1.dims != size2.dims)
+        return false;
+    if (size1.layout != size2.layout &&
+        size1.layout != DATA_LAYOUT_UNKNOWN &&
+        size2.layout != DATA_LAYOUT_UNKNOWN)
+        return false;
+    if (size1.layout == DATA_LAYOUT_BLOCK &&
+        size2.layout == DATA_LAYOUT_BLOCK &&
+        size1.C != size2.C)
+        return false;
+    for (int i = 0; i < size1.dims; i++) {
+        if (size1.p[i] != size2.p[i])
+            return false;
+    }
+    return true;
+}
+
+bool operator != (const MatShape& size1, const MatShape& size2)
+{
+    return !(size1 == size2);
+}
+
+/////////////////////////// MatShape ////////////////////////////////
+
+MatShape MatShape::scalar()
+{
+    return MatShape(0);
+}
+
+void MatShape::clear()
+{
+    dims = -1;
+    layout = DATA_LAYOUT_UNKNOWN;
+    C = 0;
+    for (int i = 0; i < MAX_DIMS; i++)
+        p[i] = 0;
+}
+
+void MatShape::resize(size_t newSize, int value)
+{
+    CV_Assert(newSize < (size_t)MAX_DIMS);
+    int old_dims = std::max(dims, 0);
+    dims = (int)newSize;
+    for (int i = old_dims; i < dims; i++)
+        p[i] = value;
+}
+
+void MatShape::reserve(size_t)
+{
+    // no op; maybe need to add a check for overflow, but we check it anyway in other operations
+}
+
+void MatShape::assign(size_t newSize, int value)
+{
+    CV_Assert(newSize < (size_t)MAX_DIMS);
+    dims = (int)newSize;
+    for (int i = 0; i < dims; i++)
+        p[i] = value;
+}
+
+void MatShape::assign(int newSize, int value)
+{
+    assign((size_t)newSize, value);
+}
+
+void MatShape::assign(const int* begin, const int* end)
+{
+    assign_(begin, end);
+}
+
+void MatShape::assign_(const int* begin, const int* end)
+{
+    ptrdiff_t newSize = end - begin;
+    CV_Assert(0 <= newSize && newSize < (ptrdiff_t)MAX_DIMS);
+    dims = (int)newSize;
+    for (int i = 0; i < dims; i++)
+        p[i] = begin[i];
+}
+
+int* MatShape::begin() { return p; }
+const int* MatShape::begin() const { return p; }
+int* MatShape::end() { return p + std::max(dims, 0); }
+const int* MatShape::end() const { return p + std::max(dims, 0); }
+int& MatShape::back() { return p[std::max(dims-1, 0)]; }
+const int& MatShape::back() const { return p[std::max(dims-1, 0)]; }
+
+void MatShape::push_back(int value)
+{
+    CV_Assert(dims+1 < MAX_DIMS);
+    dims = std::max(dims+1, 1);
+    p[dims-1] = value;
+}
+
+void MatShape::emplace_back(int value)
+{
+    push_back(value);
+}
+
+void MatShape::insert(int* where, int value)
+{
+    int old_dims = std::max(dims, 0);
+    CV_Assert(old_dims+1 < MAX_DIMS);
+    ptrdiff_t ofs = where - p;
+    CV_Assert(0 <= ofs && ofs <= old_dims);
+    dims = old_dims+1;
+    for (int i = old_dims-1; i >= (int)ofs; i--)
+        p[i+1] = p[i];
+    p[ofs] = value;
+}
+
+void MatShape::insert(int* where, size_t count, int value)
+{
+    int old_dims = std::max(dims, 0);
+    CV_Assert((size_t)(old_dims+count) < (size_t)MAX_DIMS);
+    ptrdiff_t ofs = where - p;
+    CV_Assert(0 <= ofs && ofs <= old_dims);
+    dims = (int)(old_dims+count);
+    for (int i = old_dims-1; i >= (int)ofs; i--)
+        p[i+count] = p[i];
+    for (int i = 0; i < (int)count; i++)
+        p[i+ofs] = value;
+}
+
+void MatShape::insert(int* where, int count, int value)
+{
+    insert(where, (size_t)count, value);
+}
+
+void MatShape::insert(int* where, const int* begin, const int* end)
+{
+    insert_(where, begin, end);
+}
+
+void MatShape::insert_(int* where, const int* begin, const int* end)
+{
+    int old_dims = std::max(dims, 0);
+    ptrdiff_t delta = end - begin;
+    CV_Assert(0 <= delta && old_dims+delta < MAX_DIMS);
+    ptrdiff_t ofs = where - p;
+    CV_Assert(0 <= ofs && ofs <= old_dims);
+    dims = (int)(old_dims+delta);
+    for (int i = old_dims-1; i >= (int)ofs; i--)
+        p[i+delta] = p[i];
+    for (int i = 0; i < (int)delta; i++)
+        p[i+ofs] = begin[i];
+}
+
+void MatShape::erase(int* where)
+{
+    CV_Assert(dims > 0);
+    ptrdiff_t ofs = where - p;
+    CV_Assert(0 <= ofs && ofs <= dims);
+    if (ofs == dims)
+        return;
+    dims--;
+    for (int i = (int)ofs+1; i <= dims; i++)
+        p[i-1] = p[i];
+}
+
+size_t MatShape::total() const
+{
+    size_t result = 1;
+    if (dims < 0)
+        return 0;
+    for (int i = 0; i < dims; i++)
+        result *= p[i];
+    return result;
+}
+
+std::string MatShape::str() const
+{
+    std::stringstream sstrm;
+    if (empty()) {
+        sstrm << "<empty>";
+    } else if (dims == 0) {
+        sstrm << "<scalar>";
+    } else {
+        sstrm << "[";
+        for (int i = 0; i < dims; i++) {
+            sstrm << (i > 0 ? " x " : "") << p[i];
+        }
+        sstrm << "]";
+    }
+    return sstrm.str();
+}
+
+static void finalizeBlockLayout(MatShape& size, int C=0)
+{
+    if (size.layout == DATA_LAYOUT_BLOCK) {
+        CV_Assert(size.dims >= 4);
+        int C0 = size.p[size.dims-1];
+        CV_Assert(C0 > 1 && (C0 & (C0-1)) == 0);
+        size.C = C > 0 ? C : size.p[1]*size.p[size.dims-1];
+    } else {
+        size.C = 0;
+    }
+    for (int i = std::max(size.dims, 0); i < MatShape::MAX_DIMS; i++)
+        size.p[i] = 0;
+    if (size.dims == 0)
+        size.p[0] = 1;
+}
+
+MatShape::MatShape()
+{
+    clear();
+}
+
+MatShape::MatShape(size_t dims_, const int* size_, DataLayout layout_, int C_)
+{
+    layout = layout_;
+    CV_Assert(dims_ <= (size_t)MAX_DIMS);
+    dims = (int)dims_;
+    for (int i = 0; i < dims; i++) {
+        p[i] = size_ ? size_[i] : 0;
+    }
+    finalizeBlockLayout(*this, C_);
+}
+
+MatShape::MatShape(size_t dims_, int value, DataLayout layout_)
+{
+    layout = layout_;
+    CV_Assert(dims_ <= (size_t)MAX_DIMS);
+    dims = (int)dims_;
+    for (int i = 0; i < dims; i++) {
+        p[i] = value;
+    }
+    finalizeBlockLayout(*this, 0);
+}
+
+MatShape::MatShape(std::initializer_list<int> shape)
+{
+    layout = DATA_LAYOUT_UNKNOWN;
+    CV_Assert(shape.size() <= (size_t)MAX_DIMS);
+    dims = (int)shape.size();
+    auto it = shape.begin();
+    for (int i = 0; i < dims; i++, ++it) {
+        p[i] = *it;
+    }
+    finalizeBlockLayout(*this, 0);
+}
+
+MatShape::MatShape(int dims_, int value, DataLayout layout_)
+{
+    layout = layout_;
+    CV_Assert(dims_ <= MAX_DIMS);
+    dims = dims_;
+    for (int i = 0; i < dims; i++) {
+        p[i] = value;
+    }
+    finalizeBlockLayout(*this, 0);
+}
+
+MatShape::MatShape(const std::vector<int>& shape_, DataLayout layout_, int C_)
+{
+    layout = layout_;
+    size_t shape_size = shape_.size();
+    CV_Assert(shape_size < (size_t)MAX_DIMS);
+    dims = (int)shape_size;
+    for (int i = 0; i < dims; i++) {
+        p[i] = shape_[i];
+    }
+    finalizeBlockLayout(*this, C_);
+}
+
+MatShape::MatShape(const int* begin, const int* end, DataLayout layout_, int C_)
+{
+    layout = layout_;
+    ptrdiff_t shape_size = end - begin;
+    CV_Assert(0 <= shape_size && shape_size < MAX_DIMS);
+    dims = (int)shape_size;
+    for (int i = 0; i < dims; i++) {
+        p[i] = begin[i];
+    }
+    finalizeBlockLayout(*this, C_);
+}
+
+MatShape::MatShape(const MatShape& shape)
+{
+    dims = shape.dims;
+    layout = shape.layout;
+    C = shape.C;
+    for (int i = 0; i < MAX_DIMS; i++)
+        p[i] = shape.p[i];
+}
+
+MatShape& MatShape::operator = (const MatShape& shape)
+{
+    if (this != &shape) {
+        dims = shape.dims;
+        layout = shape.layout;
+        C = shape.C;
+        for (int i = 0; i < MAX_DIMS; i++)
+            p[i] = shape.p[i];
+    }
+    return *this;
+}
+
+bool MatShape::hasSymbols() const
+{
+    for (int i = 0; i < dims; i++) {
+        if (p[i] < 0)
+            return true;
+    }
+    return false;
+}
+
+MatShape MatShape::toBlock(int C0) const
+{
+    CV_Assert(dims >= 3);
+    // C0 should be > 1 and be a power-of-2: 2, 4, 8, ...
+    CV_Assert(C0 > 1 && (C0 & (C0-1)) == 0);
+    CV_Assert(layout == DATA_LAYOUT_NCHW || layout == DATA_LAYOUT_NHWC);
+    int c_idx = layout == DATA_LAYOUT_NCHW ? 1 : dims-1;
+
+    MatShape newsize = *this;
+    newsize.layout = DATA_LAYOUT_BLOCK;
+    newsize.C = p[c_idx];
+    newsize.p[newsize.dims++] = C0;
+    newsize.p[c_idx] = (p[c_idx] + C0 - 1)/C0;
+
+    return newsize;
+}
+
+MatShape MatShape::fromBlock(DataLayout newLayout) const
+{
+    CV_Assert(dims >= 4);
+    CV_Assert(layout == DATA_LAYOUT_BLOCK);
+    // C0 should be > 1 and be a power-of-2: 2, 4, 8, ...
+    int C0 = p[dims-1];
+    CV_Assert(C0 > 1 && (C0 & (C0-1)) == 0);
+    CV_Assert(p[1] == (C + C0-1)/C0);
+    CV_Assert(newLayout == DATA_LAYOUT_NCHW || newLayout == DATA_LAYOUT_NHWC);
+    int c_idx = newLayout == DATA_LAYOUT_NCHW ? 1 : dims-2;
+
+    MatShape newsize = *this;
+    newsize.layout = newLayout;
+    newsize.C = 0;
+    newsize.p[c_idx] = C;
+    newsize.dims--;
+
+    return newsize;
+}
+
+MatShape MatShape::expand(const MatShape& another) const
+{
+    if (dims == 0)
+        return another;
+    if (another.dims == 0)
+        return *this;
+
+    if ((layout == DATA_LAYOUT_NCHW || layout == DATA_LAYOUT_NHWC) &&
+        (another.layout == DATA_LAYOUT_NCHW || another.layout == DATA_LAYOUT_NHWC)) {
+        CV_Assert(layout == another.layout);
+    }
+    // [TODO] support block layout
+    CV_Assert(layout != DATA_LAYOUT_BLOCK && another.layout != DATA_LAYOUT_BLOCK);
+
+    MatShape result;
+
+    if (dims < 0 || another.dims < 0)
+        return result;
+
+    result = *this;
+    result.dims = std::max(dims, another.dims);
+    result.layout = layout == DATA_LAYOUT_UNKNOWN ? another.layout :
+        layout == DATA_LAYOUT_ND && (another.layout == DATA_LAYOUT_NCHW ||
+        another.layout == DATA_LAYOUT_NHWC) ? another.layout : layout;
+    for (int i = result.dims-1; i >= 0; i--) {
+        int i1 = i - (result.dims - dims);
+        int i2 = i - (result.dims - another.dims);
+        int sz1 = i1 < 0 ? 1 : p[i1];
+        int sz2 = i2 < 0 ? 1 : another.p[i2];
+        CV_Assert(sz1 == sz2 || sz1 == 1 || sz2 == 1);
+        // [TODO] handle symbolic shapes
+        result.p[i] = std::max(sz1, sz2);
+    }
+    return result;
+}
+
+MatShape::operator std::vector<int>() const
+{
+    if (dims < 0)
+        return std::vector<int>(1, 0);
+    return std::vector<int>(p, p + dims);
+}
+
+/////////////////////////// MatAllocator ////////////////////////////
+
 void MatAllocator::map(UMatData*, AccessFlag) const
 {
 }
@@ -403,6 +806,36 @@ Mat::Mat(const std::vector<int>& _sz, int _type, const Scalar& _s)
     *this = _s;
 }
 
+Mat::Mat(const MatShape& _shape, int _type)
+    : flags(MAGIC_VAL), dims(0), rows(0), cols(0), data(0), datastart(0), dataend(0),
+      datalimit(0), allocator(0), u(0), size(&rows), step(0)
+{
+    create(_shape, _type);
+}
+
+Mat::Mat(std::initializer_list<int> _shape, int _type)
+    : flags(MAGIC_VAL), dims(0), rows(0), cols(0), data(0), datastart(0), dataend(0),
+      datalimit(0), allocator(0), u(0), size(&rows), step(0)
+{
+    create(_shape, _type);
+}
+
+Mat::Mat(const MatShape& _shape, int _type, const Scalar& _s)
+    : flags(MAGIC_VAL), dims(0), rows(0), cols(0), data(0), datastart(0), dataend(0),
+      datalimit(0), allocator(0), u(0), size(&rows), step(0)
+{
+    create(_shape, _type);
+    *this = _s;
+}
+
+Mat::Mat(std::initializer_list<int> _shape, int _type, const Scalar& _s)
+    : flags(MAGIC_VAL), dims(0), rows(0), cols(0), data(0), datastart(0), dataend(0),
+      datalimit(0), allocator(0), u(0), size(&rows), step(0)
+{
+    create(_shape, _type);
+    *this = _s;
+}
+
 Mat::Mat(const Mat& m)
     : flags(m.flags), dims(m.dims), rows(m.rows), cols(m.cols), data(m.data),
       datastart(m.datastart), dataend(m.dataend), datalimit(m.datalimit), allocator(m.allocator),
@@ -557,6 +990,65 @@ void Mat::createSameSize(InputArray m, int type)
     _OutputArray(*this).createSameSize(m, type);
 }
 
+void Mat::fit(int _dims, const int* _sizes, int _type)
+{
+    size_t oldTotalBytes = u ? u->size : 0;
+    size_t esz = CV_ELEM_SIZE(_type), newTotal = _dims >= 0;
+    for (int i = 0; i < _dims; i++)
+        newTotal *= _sizes[i];
+    size_t newTotalBytes = newTotal*esz;
+    if (newTotalBytes > 0 && (!isContinuous() ||
+                              newTotalBytes > oldTotalBytes ||
+                              data != datastart)) {
+        create(_dims, _sizes, _type);
+    } else {
+        flags = (flags & ~Mat::TYPE_MASK) | CV_MAT_TYPE(_type);
+        int _dummy_size = 0;
+        setSize(*this, (_dims >= 0 ? _dims : 1), (_dims >= 0 ? _sizes : &_dummy_size), nullptr, true);
+        finalizeHdr(*this);
+    }
+}
+
+void Mat::fit(const std::vector<int>& _shape, int _type)
+{
+    fit((int)_shape.size(), _shape.data(), _type);
+}
+
+void Mat::fit(const MatShape& _shape, int _type)
+{
+    fit(_shape.dims, _shape.p, _type);
+}
+
+void Mat::fit(std::initializer_list<int> _shape, int _type)
+{
+    int new_shape[MatShape::MAX_DIMS];
+    int new_ndims = (int)_shape.size();
+    CV_Assert(new_ndims <= MatShape::MAX_DIMS);
+    auto it = _shape.begin();
+    for (int i = 0; i < new_ndims; i++, ++it)
+        new_shape[i] = *it;
+    fit(new_ndims, new_shape, _type);
+}
+
+void Mat::fit(int _rows, int _cols, int _type)
+{
+    _type &= TYPE_MASK;
+    int sz[] = {_rows, _cols};
+    fit(2, sz, _type);
+}
+
+void Mat::fit(Size _sz, int _type)
+{
+    fit(_sz.height, _sz.width, _type);
+}
+
+void Mat::fitSameSize(InputArray m, int _type)
+{
+    int _sizes[CV_MAX_DIM];
+    int _dims = m.sizend(_sizes);
+    fit(_dims, _sizes, _type);
+}
+
 void Mat::addref()
 {
     if( u )
@@ -613,6 +1105,10 @@ size_t Mat::total(int startDim, int endDim) const
     return p;
 }
 
+MatShape Mat::shape() const
+{
+    return dims == 0 && data == 0 ? MatShape() : MatShape(dims, size.p);
+}
 
 Mat::Mat(Mat&& m) CV_NOEXCEPT
     : flags(m.flags), dims(m.dims), rows(m.rows), cols(m.cols), data(m.data),
@@ -747,6 +1243,27 @@ void Mat::create(const std::vector<int>& _sizes, int _type)
     create((int)_sizes.size(), _sizes.data(), _type);
 }
 
+void Mat::create(const MatShape& _shape, int _type)
+{
+    if (_shape.dims < 0) {
+        release();
+        return;
+    }
+    create(_shape.dims, _shape.p, _type);
+}
+
+void Mat::create(std::initializer_list<int> _shape, int _type)
+{
+    int new_shape[MatShape::MAX_DIMS];
+    int new_ndims = (int)_shape.size();
+    CV_Assert(new_ndims <= MatShape::MAX_DIMS);
+    auto it = _shape.begin();
+    for (int i = 0; i < new_ndims; i++, ++it)
+        new_shape[i] = *it;
+
+    create(new_ndims, new_shape, _type);
+}
+
 void Mat::copySize(const Mat& m)
 {
     setSize(*this, m.dims, 0, 0);
@@ -870,6 +1387,37 @@ Mat::Mat(const std::vector<int>& _sizes, int _type, void* _data, const size_t* _
     finalizeHdr(*this);
 }
 
+Mat::Mat(const MatShape& _shape, int _type, void* _data, const size_t* _steps)
+    : flags(MAGIC_VAL), dims(0), rows(0), cols(0), data(0), datastart(0), dataend(0),
+      datalimit(0), allocator(0), u(0), size(&rows)
+{
+    flags |= CV_MAT_TYPE(_type);
+    datastart = data = (uchar*)_data;
+    if (_shape.dims >= 0) {
+        setSize(*this, _shape.dims, _shape.p, _steps, true);
+    }
+    else {
+        CV_Assert(!data);
+    }
+    finalizeHdr(*this);
+}
+
+Mat::Mat(std::initializer_list<int> _shape, int _type, void* _data, const size_t* _steps)
+    : flags(MAGIC_VAL), dims(0), rows(0), cols(0), data(0), datastart(0), dataend(0),
+      datalimit(0), allocator(0), u(0), size(&rows)
+{
+    int new_shape[MatShape::MAX_DIMS];
+    int new_ndims = (int)_shape.size();
+    CV_Assert(new_ndims <= MatShape::MAX_DIMS);
+    auto it = _shape.begin();
+    for (int i = 0; i < new_ndims; i++, ++it)
+        new_shape[i] = *it;
+
+    flags |= CV_MAT_TYPE(_type);
+    datastart = data = (uchar*)_data;
+    setSize(*this, new_ndims, new_shape, _steps, true);
+    finalizeHdr(*this);
+}
 
 Mat::Mat(const Mat& m, const Range* ranges)
     : flags(MAGIC_VAL), dims(0), rows(0), cols(0), data(0), datastart(0), dataend(0),
@@ -1297,6 +1845,26 @@ Mat Mat::reshape(int _cn, const std::vector<int>& _newshape) const
     return reshape(_cn, newdims, newdims > 0 ? &_newshape[0] : 0);
 }
 
+Mat Mat::reshape(int _cn, const MatShape& _newshape) const
+{
+    if (_newshape.dims < 0) {
+        int newshape[] = {0};
+        return reshape(_cn, 1, newshape);
+    }
+    return reshape(_cn, _newshape.dims, _newshape.p);
+}
+
+Mat Mat::reshape(int _cn, std::initializer_list<int> newshape_) const
+{
+    int newshape[MatShape::MAX_DIMS];
+    size_t i, newshape_dims = newshape_.size();
+    CV_Assert(newshape_dims <= (size_t)MatShape::MAX_DIMS);
+    auto it = newshape_.begin();
+    for (i = 0; i < newshape_dims; i++, ++it)
+        newshape[i] = *it;
+    return reshape(_cn, (int)newshape_dims, newshape);
+}
+
 Mat Mat::diag(const Mat& d)
 {
     CV_Assert( d.cols == 1 || d.rows == 1 );

modules/core/src/matrix_transform.cpp

@@ -1081,6 +1081,16 @@ void broadcast(InputArray _src, InputArray _shape, OutputArray _dst) {
     }
 }
 
+void broadcast(InputArray _src, const MatShape& _shape, OutputArray _dst)
+{
+    if (_shape.dims < 0) {
+        _dst.release();
+    } else {
+        Mat shape(1, _shape.dims, CV_32S, (int*)_shape.p);
+        broadcast(_src, shape, _dst);
+    }
+}
+
 static void rotateImpl(InputArray _src, OutputArray _dst, int rotateMode)
 {
     switch (rotateMode)

modules/core/src/matrix_wrap.cpp

@@ -593,6 +593,41 @@ int _InputArray::sizend(int* arrsz, int i) const
     return d;
 }
 
+bool _InputArray::empty(int i) const
+{
+    _InputArray::KindFlag k = kind();
+    if (i >= 0) {
+        if (k == STD_VECTOR_MAT) {
+            auto mv = reinterpret_cast<const std::vector<Mat>*>(obj);
+            CV_Assert((size_t)i < mv->size());
+            return mv->at(i).empty();
+        }
+        else if (k == STD_VECTOR_MAT) {
+            auto umv = reinterpret_cast<const std::vector<UMat>*>(obj);
+            CV_Assert((size_t)i < umv->size());
+            return umv->at(i).empty();
+        }
+        else if (k == STD_VECTOR_VECTOR) {
+            auto vv = reinterpret_cast<const std::vector<std::vector<int> >*>(obj);
+            CV_Assert((size_t)i < vv->size());
+            return vv->at(i).empty();
+        } else {
+            CV_Error(Error::StsNotImplemented, "");
+        }
+    }
+    return empty();
+}
+
+MatShape _InputArray::shape(int i) const
+{
+    int sizes[CV_MAX_DIM];
+    int dims = sizend(sizes, i);
+
+    if (dims == 0 && empty(i))
+        return MatShape();
+    return MatShape(dims, sizes);
+}
+
 bool _InputArray::sameSize(const _InputArray& arr) const
 {
     _InputArray::KindFlag k1 = kind(), k2 = arr.kind();
@@ -1673,12 +1708,104 @@ void _OutputArray::create(int d, const int* sizes, int mtype, int i,
     CV_Error(Error::StsNotImplemented, "Unknown/unsupported array type");
 }
 
+void _OutputArray::create(const MatShape& shape, int mtype, int i,
+                          bool allowTransposed, _OutputArray::DepthMask fixedDepthMask) const
+{
+    if (shape.dims < 0) {
+        release();
+    } else {
+        create(shape.dims, shape.p, mtype, i, allowTransposed, fixedDepthMask);
+    }
+}
+
 void _OutputArray::createSameSize(const _InputArray& arr, int mtype) const
 {
     int arrsz[CV_MAX_DIM], d = arr.sizend(arrsz);
     create(d, arrsz, mtype);
 }
 
+void _OutputArray::fit(int d, const int* sizes, int mtype, int i,
+                       bool allowTransposed, _OutputArray::DepthMask fixedDepthMask) const
+{
+    int size0 = d > 0 ? sizes[0] : 1, size1 = d > 1 ? sizes[1] : 1;
+    _InputArray::KindFlag k = kind();
+    mtype = CV_MAT_TYPE(mtype);
+
+    if( (k == MAT && i < 0) || (k == STD_VECTOR_MAT && i >= 0) )
+    {
+        Mat* m;
+        if (k == MAT)
+            m = (Mat*)obj;
+        else {
+            std::vector<Mat>& v = *(std::vector<Mat>*)obj;
+            CV_Assert((size_t)i < v.size());
+            m = &v[i];
+        }
+        CV_Assert(!(m->empty() && fixedType() && fixedSize()) && "Can't reallocate empty Mat with locked layout (probably due to misused 'const' modifier)");
+        if (!m->empty() && d <= 2 && m->dims <= 2 &&
+            m->type() == mtype &&
+            ((m->rows == size0 && m->cols == size1) ||
+             (allowTransposed && m->rows == size1 && m->cols == size0 && m->isContinuous())))
+        {
+            return;
+        }
+
+        if(fixedType())
+        {
+            if(CV_MAT_CN(mtype) == m->channels() && ((1 << CV_MAT_DEPTH(flags)) & fixedDepthMask) != 0 )
+                mtype = m->type();
+            else
+                CV_CheckTypeEQ(m->type(), CV_MAT_TYPE(mtype), "Can't reallocate Mat with locked type (probably due to misused 'const' modifier)");
+        }
+        if(fixedSize())
+        {
+            CV_CheckEQ(m->dims, d, "Can't reallocate Mat with locked size (probably due to misused 'const' modifier)");
+            for(int j = 0; j < d; ++j)
+                CV_CheckEQ(m->size[j], sizes[j], "Can't reallocate Mat with locked size (probably due to misused 'const' modifier)");
+        }
+        m->fit(d, sizes, mtype);
+        return;
+    }
+
+    if( (k == UMAT && i < 0) || (k == STD_VECTOR_UMAT && i >= 0) )
+    {
+        UMat* m;
+        if (k == UMAT)
+            m = (UMat*)obj;
+        else {
+            std::vector<UMat>& v = *(std::vector<UMat>*)obj;
+            CV_Assert((size_t)i < v.size());
+            m = &v[i];
+        }
+        CV_Assert(!(m->empty() && fixedType() && fixedSize()) && "Can't reallocate empty Mat with locked layout (probably due to misused 'const' modifier)");
+        if (!m->empty() && d <= 2 && m->dims <= 2 &&
+            m->type() == mtype &&
+            ((m->rows == size0 && m->cols == size1) ||
+             (allowTransposed && m->rows == size1 && m->cols == size0 && m->isContinuous())))
+        {
+            return;
+        }
+
+        if(fixedType())
+        {
+            if(CV_MAT_CN(mtype) == m->channels() && ((1 << CV_MAT_DEPTH(flags)) & fixedDepthMask) != 0 )
+                mtype = m->type();
+            else
+                CV_CheckTypeEQ(m->type(), CV_MAT_TYPE(mtype), "Can't reallocate Mat with locked type (probably due to misused 'const' modifier)");
+        }
+        if(fixedSize())
+        {
+            CV_CheckEQ(m->dims, d, "Can't reallocate Mat with locked size (probably due to misused 'const' modifier)");
+            for(int j = 0; j < d; ++j)
+                CV_CheckEQ(m->size[j], sizes[j], "Can't reallocate Mat with locked size (probably due to misused 'const' modifier)");
+        }
+        m->fit(d, sizes, mtype);
+        return;
+    }
+
+    create(d, sizes, mtype, i, allowTransposed, fixedDepthMask);
+}
+
 void _OutputArray::release() const
 {
     CV_Assert(!fixedSize());

modules/core/src/out.cpp

@@ -403,4 +403,186 @@ namespace cv
         }
         return makePtr<DefaultFormatter>();
     }
+
+    template<typename _Tp> struct Fmt
+    {
+        typedef int temp_type;
+        static const char* fmt() { return "%d"; }
+    };
+
+    template<> struct Fmt<uint32_t>
+    {
+        typedef unsigned temp_type;
+        static const char* fmt() { return "%u"; }
+    };
+
+    template<> struct Fmt<int64_t>
+    {
+        typedef long long temp_type;
+        static const char* fmt() { return "%lld"; }
+    };
+
+    template<> struct Fmt<uint64_t>
+    {
+        typedef unsigned long long temp_type;
+        static const char* fmt() { return "%llu"; }
+    };
+
+    template<> struct Fmt<float>
+    {
+        typedef float temp_type;
+        static const char* fmt() { return "%.5g"; }
+    };
+
+    template<> struct Fmt<double>
+    {
+        typedef double temp_type;
+        static const char* fmt() { return "%.5g"; }
+    };
+
+    template<> struct Fmt<hfloat>
+    {
+        typedef float temp_type;
+        static const char* fmt() { return "%.5g"; }
+    };
+
+    template<> struct Fmt<bfloat>
+    {
+        typedef float temp_type;
+        static const char* fmt() { return "%.4g"; }
+    };
+
+    template <typename _Tp>
+    static void pprintRow(std::ostream& strm, const _Tp* ptr, int n, size_t ofs, int edge)
+    {
+        char buf[128];
+        const char* fmt = Fmt<_Tp>::fmt();
+        int i, ndump = edge > 0 ? std::min(n, edge*2+1) : n;
+        if (edge == 0)
+            edge = ndump;
+        for (i = 0; i < ndump; i++) {
+            int j = n == ndump || i < edge ? i : i == edge ? -1 : n-edge*2-1+i;
+            if (i > 0)
+                strm << ", ";
+            if (j >= 0) {
+                snprintf(buf, sizeof(buf), fmt, (typename Fmt<_Tp>::temp_type)ptr[ofs + j]);
+                strm << buf;
+            } else
+                strm << "... ";
+        }
+    }
+
+    static void pprintSlice(std::ostream& strm, const Mat& tensor,
+                            const size_t* step, int d,
+                            size_t ofs, int edge)
+    {
+        MatShape shape = tensor.shape();
+        int ndims = shape.dims;
+        int n = d >= ndims ? 1 : shape[d];
+        if (d >= ndims - 1) {
+            int typ = tensor.depth();
+            void* data = tensor.data;
+            CV_Assert(data);
+            n *= tensor.channels();
+            if (typ == CV_8U)
+                pprintRow(strm, (const uint8_t*)data, n, ofs, edge);
+            else if (typ == CV_8S)
+                pprintRow(strm, (const int8_t*)data, n, ofs, edge);
+            else if (typ == CV_16U)
+                pprintRow(strm, (const uint16_t*)data, n, ofs, edge);
+            else if (typ == CV_16S)
+                pprintRow(strm, (const int16_t*)data, n, ofs, edge);
+            else if (typ == CV_32U)
+                pprintRow(strm, (const unsigned*)data, n, ofs, edge);
+            else if (typ == CV_32S)
+                pprintRow(strm, (const int*)data, n, ofs, edge);
+            else if (typ == CV_64U)
+                pprintRow(strm, (const uint64_t*)data, n, ofs, edge);
+            else if (typ == CV_64S)
+                pprintRow(strm, (const int64_t*)data, n, ofs, edge);
+            else if (typ == CV_32F)
+                pprintRow(strm, (const float*)data, n, ofs, edge);
+            else if (typ == CV_64F)
+                pprintRow(strm, (const double*)data, n, ofs, edge);
+            else if (typ == CV_16F)
+                pprintRow(strm, (const hfloat*)data, n, ofs, edge);
+            else if (typ == CV_16BF)
+                pprintRow(strm, (const bfloat*)data, n, ofs, edge);
+            else if (typ == CV_Bool)
+                pprintRow(strm, (const bool*)data, n, ofs, edge);
+            else {
+                CV_Error(Error::StsNotImplemented, "unsupported type");
+            }
+        } else {
+            int i, ndump = edge > 0 ? std::min(n, edge*2+1) : n;
+            bool dots = false;
+            for (i = 0; i < ndump; i++) {
+                if (i > 0 && !dots) {
+                    int nempty_lines = ndims - 2 - d;
+                    for (int k = 0; k < nempty_lines; k++)
+                        strm << "\n";
+                }
+                if (i > 0)
+                    strm << "\n";
+                int j = n == ndump || i < edge ? i :
+                        i == edge ? -1 :
+                        n - edge*2 - 1 + i;
+                dots = j < 0;
+                if (!dots)
+                    pprintSlice(strm, tensor, step, d+1, ofs + j*step[d], edge);
+                else
+                    strm << "...";
+            }
+        }
+    }
+
+    std::ostream& pprint(std::ostream& strm, InputArray array,
+                         int /*indent*/, int edge_,
+                         int wholeTensorThreshold,
+                         char parens)
+    {
+        char oparen = parens;
+        char cparen = parens == '(' ? ')' :
+                      parens == '[' ? ']' :
+                      parens == '{' ? '}' :
+                      parens == '<' ? '>' :
+                      parens;
+        int edge = edge_ > 0 ? edge_ : 3;
+        wholeTensorThreshold = wholeTensorThreshold > 0 ? wholeTensorThreshold : 100;
+
+        Mat tensor = array.getMat();
+        if (!tensor.isContinuous()) {
+            // [TODO] print non-continous arrays without copy
+            Mat temp;
+            tensor.copyTo(temp);
+            tensor = temp;
+        }
+
+        MatShape shape = tensor.shape();
+        size_t sz_all = tensor.total();
+
+        if (parens)
+            strm << oparen;
+        if (sz_all == 0) {
+            if (!parens)
+                strm << "<empty>";
+        } else {
+            if (sz_all <= (size_t)wholeTensorThreshold)
+                edge = 0;
+
+            int ndims = shape.dims;
+            int cn = tensor.channels();
+            size_t step[MatShape::MAX_DIMS];
+            step[std::max(ndims-1, 0)] = 1;
+            for (int i = ndims-2; i >= 0; i--) {
+                step[i] = step[i+1]*shape[i+1]*cn;
+                cn = 1;
+            }
+            pprintSlice(strm, tensor, step, 0, 0, edge);
+        }
+        if (parens)
+            strm << cparen;
+        return strm;
+    }
+
 } // cv

modules/core/src/umatrix.cpp

@@ -412,6 +412,10 @@ size_t UMat::total() const
     return p;
 }
 
+MatShape UMat::shape() const
+{
+    return dims == 0 && u == 0 ? MatShape() : MatShape(dims, size.p);
+}
 
 UMat::UMat(UMat&& m)
 : flags(m.flags), dims(m.dims), rows(m.rows), cols(m.cols), allocator(m.allocator),
@@ -751,6 +755,67 @@ void UMat::create(const std::vector<int>& _sizes, int _type, UMatUsageFlags _usa
     create((int)_sizes.size(), _sizes.data(), _type, _usageFlags);
 }
 
+void UMat::create(const MatShape& _shape, int _type, UMatUsageFlags _usageFlags)
+{
+    if (_shape.dims < 0) {
+        release();
+    } else {
+        create(_shape.dims, _shape.p, _type, _usageFlags);
+    }
+}
+
+void UMat::fit(int _dims, const int* _sizes, int _type, UMatUsageFlags _usageFlags)
+{
+    if (_usageFlags == cv::USAGE_DEFAULT)
+        _usageFlags = usageFlags;
+    size_t oldTotalBytes = u ? u->size : 0;
+    size_t esz = CV_ELEM_SIZE(_type), newTotal = _dims >= 0;
+    for (int i = 0; i < _dims; i++)
+        newTotal *= _sizes[i];
+    size_t newTotalBytes = newTotal*esz;
+    if (newTotalBytes > 0 &&
+        (!isContinuous() ||
+         newTotalBytes > oldTotalBytes ||
+         offset != 0 ||
+         _usageFlags != usageFlags)) {
+        create(_dims, _sizes, _type, _usageFlags);
+    } else {
+        flags = (flags & ~Mat::TYPE_MASK) | CV_MAT_TYPE(_type);
+        int _dummy_size = 0;
+        setSize(*this, (_dims >= 0 ? _dims : 1), (_dims >= 0 ? _sizes : &_dummy_size), nullptr, true);
+        finalizeHdr(*this);
+    }
+}
+
+void UMat::fit(const std::vector<int>& _shape, int _type, UMatUsageFlags _usageFlags)
+{
+    fit((int)_shape.size(), _shape.data(), _type, _usageFlags);
+}
+
+void UMat::fit(const MatShape& _shape, int _type, UMatUsageFlags _usageFlags)
+{
+    fit(_shape.dims, _shape.p, _type, _usageFlags);
+}
+
+void UMat::fit(int _rows, int _cols, int _type, UMatUsageFlags _usageFlags)
+{
+    _type &= TYPE_MASK;
+    int sz[] = {_rows, _cols};
+    fit(2, sz, _type, _usageFlags);
+}
+
+void UMat::fit(Size _sz, int _type, UMatUsageFlags _usageFlags)
+{
+    fit(_sz.height, _sz.width, _type, _usageFlags);
+}
+
+void UMat::fitSameSize(InputArray m, int _type, UMatUsageFlags _usageFlags)
+{
+    int _sizes[CV_MAX_DIM];
+    int _dims = m.sizend(_sizes);
+    fit(_dims, _sizes, _type, _usageFlags);
+}
+
 void UMat::copySize(const UMat& m)
 {
     setSize(*this, m.dims, 0, 0);
@@ -1101,6 +1166,15 @@ UMat UMat::reshape(int _cn, int _newndims, const int* _newsz) const
     CV_Error(cv::Error::StsNotImplemented, "Reshaping of n-dimensional non-continuous matrices is not supported yet");
 }
 
+UMat UMat::reshape(int _cn, const MatShape& _newshape) const
+{
+    if (_newshape.dims < 0) {
+        int newshape[] = {0};
+        return reshape(_cn, 1, newshape);
+    }
+    return reshape(_cn, _newshape.dims, _newshape.p);
+}
+
 Mat UMat::getMat(AccessFlag accessFlags) const
 {
     if(!u)