Use MatShape instead of MatSize inside cv::Mat/cv::UMat #27757
**Merge together with https://github.com/opencv/opencv_contrib/pull/3996**
---
This PR continues cv::Mat/cv::UMat refactoring. See #26056, where `MatShape` was introduced. Now it's put inside cv::Mat/cv::UMat instead of a weird `MatSize`. MatSize is now an alias for MatShape:
**before:**
```
struct MatShape { ... };
struct MatSize { ... };
struct Mat {
...
int dims;
int rows;
int cols;
...
MatShape shape() const { ... /* constructs MatShape out of MatSize and returns it;
layout is always 'unknown', because we don't store it */ }
MatSize size; // size is not valid without the parent cv::Mat,
// because size.p may point to Mat::rows or to Mat::cols,
// depending on the dimensionality, and dims() returns Mat::dims.
MatStep step; // may allocate memory, depending on the dimensionality.
...
};
```
**after:**
```
struct MatShape { ... };
typedef MatShape MatSize; // they are now synonyms
struct Mat {
...
int dims;
int rows;
int cols;
...
MatShape shape() const { return size; } // just return the embedded shape (including the proper layout information)
MatSize size; // size is self-contained data structure that can be used without the parent cv::Mat.
// size.dims is now a copy of dims; size.p[*] contains copies of Mat::rows and Mat::cols when dims <= 2.
MatStep step; // does not allocate extra memory buffers.
...
};
```
There are several reasons to do that:
1. the main reason is to be able to store data layout (MatShape::layout) inside each cv::Mat/cv::UMat. This is necessary for the proper shape inference in DNN module. In particular, it's necessary for the next step of DNN inference optimization where we introduce block-layout-optimized convolution and other operations. Later on, we can use layout information to support non-interleaved images (e.g. RRR...GGG...BBB...) or even batches of such images in core/imgproc modules.
2. the other reason is to represent 3D/4D/5D etc. tensors as cv::Mat/cv::UMat instances more conveniently, without extra dynamic memory allocation. Before this patch we allocated some memory buffers dynamically to store shape & steps for more than 2D arrays. Now the whole cv::Mat/cv::UMat header can be stored completely on stack/in a container. Creating another copy of Mat/UMat header is now done more efficiently.
3. the third reason is to introduce the new coding pattern: `dst.create(src.size, <dst_type>);`. The pattern is suitable for most of element-wise (including cloning) and filtering operations. This pattern does not only look crisp and self-documenting, it will also automatically copy shape (including layout) from the source tensor into the destination matrix/tensor.
4. in the future we might add `colorspace` member to MatShape that will allow to distinguish RGB from BGR or NV12. `dst.create(src.size, <dst_type>);` will then copy the colorspace information as well.
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Added nonzero layer to new DNN engine #27701
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Added Pow layer to new DNN engine #27710
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Added gridsample layer to new DNN engine #27700
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Optimize FFmpeg VideoCapture with swscale threads option #27755
### Pull Request Readiness Checklist
resolves https://github.com/opencv/opencv/issues/21969
* Switch to `sws_scale_from` for `libswscale >= 6.4.100` (FFmpeg >= 5.0)
* Use new context init API with threads option (`libswscale >= 8.12.100`: https://github.com/FFmpeg/FFmpeg/commit/2a091d4f2ee1e367d05a6bbbe96b204257cbda87)
* Replicate `sws_getCachedContext` with threads option for `libswscale < 8.12.100`
1 hour mp4 video every frame reading
| HW | sws_scale | sws_scale_frame + 16 threads | sws_scale_frame + 24 threads (#cpus) |
|---|---|---|---|
| Intel Core i9-12900 CPU | 45.1 sec | 25.4 sec (x1.77) | 30 sec (x1.50) |
| NVIDIA GPU 4090 | 232 sec | 89.4 sec (x2.59) | 77 sec (x3.01) |
```
import time
import numpy as np
import os
import cv2 as cv
# os.environ["OPENCV_FFMPEG_CAPTURE_OPTIONS"] = "hwaccel;cuvid|video_codec;h264_cuvid|vsync;0"
start = time.time()
video = "test.mp4"
cap = cv.VideoCapture(video, cv.CAP_FFMPEG)
while True:
has_frame, frame = cap.read()
if not has_frame:
break
print(time.time() - start)
```
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
imgcodecs: bmp: support to write 32bpp BMP with BI_BITFIELDS #27559
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
Set limitation to IPP Bilateral Filter tiles number to avoid too small tiles #27720
### Pull Request Readiness Checklist
This PR fixes the following issue in Bilateral Filter tiling in IPP integration: image ROI can't be closer to the image border than the filter window radius. This issue shows itself during separation of the image to tiles for multithreaded processing. If the tile size small enough, the second tile is closer to the upper image border than the bilateral filter radius, which leads to the incorrect result. To fix this, we need a limitation to the tile size - done in this PR.
_Note: red build status looks like unrelated to the current change_
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
[videoio][VideoWriter] Fix return code from CvVideoWriter_FFMPEG::writeFrame() when encapsulating encoded video #27737
Currently the return code from `CvVideoWriter_FFMPEG::writeFrame()` when `encode_video==true` (encapsulating raw encoded video) is wrong and results in the following warning implying it has been unsuccessful
> [ WARN:0@15.551] global cap_ffmpeg.cpp:198 write FFmpeg: Failed to write frame
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [ ] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
doc: fix doxygen warnings for imgcodecs, flann and objdetect #27730
Close https://github.com/opencv/opencv/issues/27729
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
libpng upgrade to 1.6.45 and cICP metadata support for PNG imwrite #27741
### Pull Request Readiness Checklist
resolves#24185
libpng docs: https://www.w3.org/TR/png-3/#cICP-chunk
similar code from ffmpeg: https://github.com/FFmpeg/FFmpeg/blob/a700f0f72d1f073e5adcfbb16f4633850b0ef51c/libavcodec/pngenc.c#L452-L456
So issue #24185 can be solved by replacing `cv.imwrite` in user's code to `cv.imwriteWithMetadata`:
```python
cv.imwriteWithMetadata("frame.png", frame, [cv.IMAGE_METADATA_CICP], np.array([[9, 18, 0, 1]], np.uint8))
```
```
$ exiftool /home/d.kurtaev/opencv_build/frames_pr/image_38.png
ExifTool Version Number : 12.76
File Name : image_38.png
Directory : /home/d.kurtaev/opencv_build/frames_pr
File Size : 3.8 MB
File Modification Date/Time : 2025:09:02 20:48:22+03:00
File Access Date/Time : 2025:09:02 20:48:22+03:00
File Inode Change Date/Time : 2025:09:02 20:48:22+03:00
File Permissions : -rw-r--r--
File Type : PNG
File Type Extension : png
MIME Type : image/png
Image Width : 1080
Image Height : 1920
Bit Depth : 8
Color Type : RGB
Compression : Deflate/Inflate
Filter : Adaptive
Interlace : Noninterlaced
Color Primaries : BT.2020, BT.2100
Transfer Characteristics : BT.2100 HLG, ARIB STD-B67
Matrix Coefficients : Identity matrix
Video Full Range Flag : 1
Image Size : 1080x1920
Megapixels : 2.1
```
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
fix: FFmpeg 8.0 support #27746
### Pull Request Readiness Checklist
related comment: https://github.com/opencv/opencv/pull/27691#discussion_r2322695640
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Improve fitEllipseDirect tests #27717
### Pull Request Readiness Checklist
Previous `fit_and_check_ellipse` implementation was very weak - it only checks that points center lies inside ellipse.
Current implementation `fit_and_check_ellipse` checks that points RMS (Root Mean Square) algebraic distance is quite small. It means that on average points are near boundary of ellipse. Because for points on ellipse algebraic distance is equal to `0` and for points that are close to boundary of ellipse is quite small
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
Fix checking that point lies inside ellipse #27704
### Pull Request Readiness Checklist
Previous `check_pt_in_ellipse` implementation was incorrect. For points on ellipse `cv::norm(to_pt)` should be equal to `el_dist`.
I tested current implementation with following Python script:
```
import cv2
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.patches import Ellipse
def check_pt_in_ellipse(pt, el):
center, axes, angle = ellipse
to_pt = pt - center
el_angle = angle * np.pi / 180
to_pt_r_x = to_pt[0] * np.cos(-el_angle) - to_pt[1] * np.sin(-el_angle)
to_pt_r_y = to_pt[0] * np.sin(-el_angle) + to_pt[1] * np.cos(-el_angle)
pt_angle = np.arctan2(to_pt_r_y / axes[1], to_pt_r_x / axes[0])
x_dist = 0.5 * axes[0] * np.cos(pt_angle)
y_dist = 0.5 * axes[1] * np.sin(pt_angle)
el_dist = np.sqrt(x_dist * x_dist + y_dist * y_dist)
assert abs(np.linalg.norm(to_pt) - el_dist) < 1e-10
# TEST(Imgproc_FitEllipse_Issue_4515, accuracy) {
points = np.array([
[327, 317],
[328, 316],
[329, 315],
[330, 314],
[331, 314],
[332, 314],
[333, 315],
[333, 316],
[333, 317],
[333, 318],
[333, 319],
[333, 320],
])
ellipse = cv2.fitEllipseDirect(points)
center, axes, angle = ellipse
angle_rad = np.deg2rad(angle)
points_on_ellipse = []
for point_angle_deg in range(0, 360, 10):
point_angle = np.deg2rad(point_angle_deg)
point = np.array([0., 0.])
point_x = axes[0] * 0.5 * np.cos(point_angle)
point_y = axes[1] * 0.5 * np.sin(point_angle)
point[0] = point_x * np.cos(angle_rad) - point_y * np.sin(angle_rad)
point[1] = point_x * np.sin(angle_rad) + point_y * np.cos(angle_rad)
point[0] += center[0]
point[1] += center[1]
points_on_ellipse.append(point)
points_on_ellipse = np.array(points_on_ellipse)
for point in points_on_ellipse:
check_pt_in_ellipse(point, ellipse)
plt.figure(figsize=(8, 8))
plt.scatter(points[:, 0], points[:, 1], c='red', label='points')
plt.scatter(points_on_ellipse[:, 0], points_on_ellipse[:, 1], c='yellow', label='ellipse')
ellipse = Ellipse(xy=center, width=axes[0], height=axes[1],
angle=angle, facecolor='none', edgecolor='b')
plt.gca().add_patch(ellipse)
plt.gca().set_aspect('equal')
plt.legend()
plt.show()
```
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
- Added vector_MatShape and vector_vector_MatShape to gen_dict.json
- Implemented MatShape_to_vector_MatShape, vector_MatShape_to_MatShape, MatShape_to_vector_vector_MatShape, and vector_vector_MatShape_to_MatShape conversion functions in dnn_converters.h/cpp and Converters.java
- Added testGetLayersShapes test to verify List<List<MatShape>> conversion
- Added vector_vector_Mat to gen_dict.json
- Implemented Mat_to_vector_vector_Mat and vector_vector_Mat_to_Mat conversion functions in converters.h/cpp and Converters.java
- Added DnnForwardAndRetrieve.java test to verify List<List<Mat>> conversion : Reference: C++ test in modules/dnn/test/test_misc.cpp - TEST(Net, forwardAndRetrieve)
Enable Java wrapper generation for Vec4i #27567
Fixes an issue where Java wrapper generation skips methods using Vec4i.
Related PR in opencv_contrib: https://github.com/opencv/opencv_contrib/pull/3988
The root cause was the absence of Vec4i in gen_java.json, which led to important methods such as aruco.drawCharucoDiamond() and ximgproc.HoughPoint2Line() being omitted from the Java bindings.
This PR includes the following changes:
- Added Vec4i definition to gen_java.json
- Updated gen_java.py to handle jintArray-based types properly
- ~~Also adjusted jn_args and jni_var for Vec2d and Vec3d to ensure correct JNI behavior~~
The modified Java wrapper generator successfully builds and includes the expected methods using Vec4i.
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
### Pull Request Readiness Checklist
resolves#16295
```
docker run --gpus 0 -v ~/opencv:/opencv -v ~/opencv_contrib:/opencv_contrib -it nvidia/cuda:12.8.1-cudnn-devel-ubuntu22.04
apt-get update && apt-get install -y cmake python3-dev python3-pip python3-venv &&
python3 -m venv .venv &&
source .venv/bin/activate &&
pip install -U pip &&
pip install -U numpy &&
pip install torch --index-url https://download.pytorch.org/whl/cu128 &&
cmake \
-DWITH_OPENCL=OFF \
-DCMAKE_BUILD_TYPE=Release \
-DBUILD_DOCS=OFF \
-DWITH_CUDA=ON \
-DOPENCV_DNN_CUDA=ON \
-DOPENCV_EXTRA_MODULES_PATH=/opencv_contrib/modules \
-DBUILD_LIST=ts,cudev,python3 \
-S /opencv -B /opencv_build &&
cmake --build /opencv_build -j16
export PYTHONPATH=/opencv_build/lib/python3/:$PYTHONPATH
```
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Vadim Pisarevsky
Alexander Smorkalov
Abhishek Gola
Alexander Smorkalov
cudawarped
Kavyansh Tyagi
Dmitry Kurtaev
Kumataro
Nadezhda Mizonova
Alexander Smorkalov
nina16448
Arsenii Rzhevskii
MaximSmolskiy
eplankin
utibenkei
kallaballa
utibenkei
onikolskyy