1
0
mirror of https://github.com/opencv/opencv.git synced 2026-07-30 15:53:03 +04:00

Compare commits

...

41 Commits

Author SHA1 Message Date
Abhishek Gola e35ad60e4e Merge pull request #29544 from xi-guo-0/core-optimize-extract-channel-8uc2
core: optimize 8UC2 channel extraction
2026-07-30 16:49:19 +05:30
Abhishek Gola f42fa7b214 Merge pull request #29553 from mvanhorn/fix/26746-warn-external-grfmt-fallback
cmake: warn when external image-codec library is requested but not found 🤖🤖🤖
2026-07-30 16:38:23 +05:30
Abhishek Gola 2fb7adbd53 Merge pull request #29552 from sergiud/opencl-remap-artifacts
imgproc: fix OpenCL remap negative border rounding
2026-07-30 15:46:30 +05:30
Abhishek Gola 2b6c3df954 Merge pull request #29546 from uwezkhan/dtree-split-index-bound
bound categorical split indices in dtrees readSplit
2026-07-29 13:31:38 +05:30
Abhishek Gola a906dfbed2 Merge pull request #29626 from ArneshBanerjee/fix-29615-icvcvt-pointer-overflow-4x
imgcodecs: fix pointer overflow in 16u icvCvt helpers (4.x)
2026-07-28 19:52:22 +05:30
Arnesh Banerjee b7860407f0 imgcodecs: fix pointer overflow in 16u icvCvt helpers
The 16-bit icvCvt_* helpers in utils.cpp move the row pointer with
step/sizeof(x[0]) - size.width*N. sizeof is unsigned, so this is done in
unsigned math. When the caller passes step == 0, it underflows to a huge
value and the pointer goes out of range, which is undefined behavior.

The TIFF decoder hits this: it walks rows itself and calls these helpers
with step = 0, so reading a 16-bit 4-channel TIFF triggers the bug.

Cast sizeof(...) to int so the math is signed. Same result for normal
calls, no overflow when the step is 0.

Added Imgcodecs_Tiff.regression_29615_16UC4 which round-trips a CV_16UC4
TIFF in memory.

This is the 4.x version of the fix, requested in
https://github.com/opencv/opencv/pull/29620 (issue #29615).
2026-07-28 16:19:17 +05:30
Uwez Khan dee2bb288b bound categorical split indices in dtrees readSplit 2026-07-27 10:54:57 +05:30
Sergiu Deitsch 096759c0b0 imgproc: fix OpenCL remap negative rounding
Truncation toward zero produces incorrect interpolation indices for
negative border coordinates. Use nearest-even rounding so OpenCL remap
matches the CPU behavior.
2026-07-27 10:46:53 +05:30
Sergiu Deitsch 73c9537f3b imgproc: fix NVIDIA OpenCL remap kernel types
Match interpolation coefficient types to the working type so the double
kernel compiles on NVIDIA devices.
2026-07-27 10:46:53 +05:30
Alexander Smorkalov 70fc346580 Merge pull request #29549 from hanu-14:fix-https-readme-md
fix: upgrade http:// to https:// in README.md
2026-07-26 15:02:04 +03:00
Andrei Fedorov 664f59e750 Merge pull request #29542 from intel-staging:ipp/enable_instrumentation_fix
Implemented a check to enable instrumentation for IPP HAL #29542

To enable instrumentation for IPP HAL correctly it needed to generalize a part of `private.hpp` and put it to the separate file that can be included in `private.hpp` and `ipp_utils.hpp` both to avoid duplication.

Built with `-DENABLE_INSTRUMENTATION=ON` and ran `opencv_perf_imgproc --perf_instrument=1`:

  | Test | IPP weight before | IPP weight after |
  |---|---|---|
  | `distanceTransform` (8U->32F) | 0.0% | ~76% |
  | `ippSobel` | 0.0% | ~98% |
  | `scharrViaSobelFilter` | 0.0% | ~90% |

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
2026-07-26 15:01:24 +03:00
Alexander Smorkalov 19ef7e6e16 Merge pull request #29583 from uditjainstjis:fix-23580-hog-buffer-overflow
objdetect: fix out-of-bounds write in HOG gaussian weights #23580 🤖🤖🤖
2026-07-25 10:01:30 +03:00
Udit Jain c8446af29a objdetect: fix out-of-bounds write in HOG gaussian weights (blockSize.height vs width)
In HOGCache::init the block gaussian weights are computed into two buffers:
  AutoBuffer<float> di(blockSize.height), dj(blockSize.width);
The vectorized (CV_SIMD128) loop that fills dj was bounded by
`blockSize.height` instead of `blockSize.width`. When the block is taller than
it is wide, `v_store(_dj + j, ...)` writes past the end of the width-sized dj
buffer, causing a heap buffer overflow / access violation (e.g. HOGDescriptor
with a 198x281 window, or the 2399x2400 case from the report).

Bound the dj loop by `blockSize.width` to match the buffer size and the scalar
tail loop. This keeps the SIMD path (unlike the workaround in the report, which
disabled it).

Added a regression test that computes a HOGDescriptor with a tall block; without
the fix it triggers a heap-buffer-overflow (confirmed with AddressSanitizer at
hog.cpp:731 inside HOGCache::init).

Fixes #23580
2026-07-24 04:51:45 +05:30
Yuantao Feng 0e09f1a238 Merge pull request #29557 from fengyuentau:4x/hal_rvv/split_opt
hal_rvv core: Cover more data types for split with native RVV intrinsics #29557

Covering 16-bit, 32-bit and 64-bit split.

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2026-07-21 10:16:38 +03:00
Alexander Smorkalov ba53d1d6b7 Merge pull request #29566 from fengyuentau:4x/hal_rvv/copy_mask_opt
hal_rvv core: Optimize RVV HAL masked copy
2026-07-21 10:15:07 +03:00
FENG Yuantao 1f36f15a49 Optimize RVV HAL masked copy 2026-07-20 23:43:41 +08:00
Alexander Smorkalov 2b6ea5f344 Merge pull request #29556 from fengyuentau:4x/hal_rvv/merge_opt
hal_rvv core: Optimize 32-bit and 64-bit merge
2026-07-20 12:26:12 +03:00
Xi Guo 1bf1e70c0f core: optimize 8UC2 channel extraction 2026-07-20 10:21:00 +08:00
FENG Yuantao 898f1e9691 Optimize RVV HAL 32-bit and 64-bit merge 2026-07-20 00:22:38 +08:00
Matt Van Horn a859f3feff cmake: warn when external grfmt library requested but not found
When a user disables a bundled image-codec dependency (e.g.
BUILD_ZLIB=OFF) to supply an external build but find_package()
cannot locate it, OpenCVFindLibsGrfmt.cmake silently falls back to
compiling the bundled 3rdparty version. As reported in the issue,
this is misleading: the editable ZLIB_LIBRARY_DEBUG/RELEASE cache
entries have no effect after the fallback and nothing points the
user at the real knob (ZLIB_ROOT / ZLIB_DIR).

Emit a message(WARNING) in each fallback branch, guarded by
NOT BUILD_<lib>, so the built-in fallback is no longer silent when
the user explicitly opted out of the bundled build. The warning
names the correct search variables for that library and points to
the CMake find_package / Find<lib> documentation. Applied
consistently to the zlib, libjpeg, libtiff, libpng and libwebp
fallback branches.

The existing fallback behavior is intentionally left unchanged, so
configurations that rely on the built-in fallback continue to build.

Fixes #26746
2026-07-18 10:02:13 -07:00
Alexander Smorkalov 6f402272e7 Merge branch 'as/release_4.14.0' into 4.x 2026-07-18 09:55:21 +03:00
MOHAMMED HANAN M T P 5ab3e2e580 fix: upgrade http:// to https:// in README.md 2026-07-18 11:13:02 +05:30
Alexander Smorkalov 0654a42e19 Release: OpenCV 4.14.0 2026-07-17 15:55:59 +03:00
Alexander Smorkalov f7dd7df170 Merge pull request #29531 from asmorkalov:as/openvino_ci_fail
Disable test test that sporadically fails with OpenVINO on CI #29531

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [ ] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
2026-07-17 13:32:48 +03:00
Alexander Smorkalov 7aa163b83f Merge pull request #29539 from uwezkhan:aruco-readdict-marker-bound
bound marker string length in aruco readDictionary
2026-07-17 11:42:51 +03:00
Alexander Smorkalov c47541acbd Merge pull request #29525 from asmorkalov:as/ffmpeg_update_4.14
FFmpeg wrapper update for 4.14.0 release.
2026-07-17 10:37:20 +03:00
Uwez Khan b5fe8dfef3 bound marker string length in aruco readDictionary 2026-07-16 19:44:15 +05:30
Alexander Smorkalov 4d1e206f5e FFmpeg wrapper update for 4.14.0 release. 2026-07-16 15:12:50 +03:00
Yuantao Feng 2e778c52c1 Merge pull request #27510 from fengyuentau:4x/core/reduce_simd
core: vectorize cv::reduce #27510

- [ ] ~reduceR_~ Dropped due to performance
- [x] reduceC_

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
2026-07-15 22:36:29 +03:00
Madan mohan Manokar 6b640b424c Merge pull request #28650 from amd:fast_pyrDown
imgproc: Optimized OpticalFlowPyrLK (PyrDownH) #28650

- Optimized Uchar PyrDown Horizontal processing intermediate storage.
- Improved horizontal processing with AVX512 vbmi ISA.
- Todo: further refinement for other archs.

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2026-07-15 12:31:28 +03:00
Alexander Smorkalov 06574736b3 Merge pull request #29503 from uwezkhan:flann-kdtree-load-bounds
bound kd-tree node indices when loading a saved flann index
2026-07-14 15:36:16 +03:00
Madan mohan Manokar 4fe51e51e0 Merge pull request #29507 from amd:fast_remap_ext
imgproc: Optimized remap interpolation #29507

- Add a SIMD dispatch file for remap and vectorize the single-channel (C1) in-bounds paths of bilinear, bicubic and lanczos4 interpolation (32F / 16U / 16S) using width-agnostic gather
- Dispatch the bilinear C1 path and drop the per-pixel weight-table gathers
- Widen the fixed-point coordinate map conversion
- 32F lanczos4 is kept on the scalar path: its vectorized 64-tap accumulation deviates beyond the set float accuracy tolerance
- Vectorize the bilinear inlier/outlier run detection so any-channel linear remap skips constant-status runs with SIMD instead of a per-pixel bounds test

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2026-07-14 15:17:22 +03:00
Alexander Smorkalov 335abd236f Merge pull request #29523 from asmorkalov:as/relax_gapi_gstreamer_test
Relaxed G-API tests for GStreamer to support Ubuntu 26.04.
2026-07-14 12:09:48 +03:00
Alexander Smorkalov 46d1b6c99d Relaxed G-API tests for GStreamer to support Ubuntu 26.04. 2026-07-14 08:47:33 +03:00
Alexander Smorkalov e6b1ef272e Merge pull request #29511 from Akansha-977:histogram_IPP_4.x
Extracted IPP to HAL for calcHist function in 4.x
2026-07-14 08:42:06 +03:00
Alexander Smorkalov 1bea199e00 Merge pull request #29437 from intel-staging:dev/tizmajlo/ippicv_20260630
Update IPP ICV integration to IPP 2026.0.0 (20260630)
2026-07-14 08:25:34 +03:00
Alexander Smorkalov 42cd88d737 Merge pull request #29509 from asmorkalov:as/ubuntu_26.04_warn_fix
Samples build warning fix on Ubuntu 26.04
2026-07-14 08:19:57 +03:00
Alexander Smorkalov 4c3895e96c Samples build warning fix on Ubuntu 26.04 2026-07-13 15:22:08 +03:00
Akansha Mallick 2c14cc1897 Extracted IPP to HAL for calcHist function 2026-07-13 14:12:52 +05:30
Uwez Khan 13fb140932 bound kd-tree node indices when loading a saved flann index 2026-07-12 21:36:42 +05:30
Timur Izmajlov 9d66a589b4 Update IPP ICV integration to IPP 2026.0.0 (20260630) 2026-07-03 17:25:24 +02:00
56 changed files with 6461 additions and 554 deletions
+6 -6
View File
@@ -1,9 +1,9 @@
# Binaries branch name: ffmpeg/4.x_20251226 # Binaries branch name: ffmpeg/4.x_20260715
# Binaries were created for OpenCV: cff7581175d2abfc6aef2e4f04f482e258b5c864 # Binaries were created for OpenCV: 6b640b424c516d27217700392a7ed6a362e040a5
ocv_update(FFMPEG_BINARIES_COMMIT "d82ad9a54a7b42a1648a9cae8fed5c2f20ea396c") ocv_update(FFMPEG_BINARIES_COMMIT "bd9418020a5c342be979c56d6e6434261959d3af")
ocv_update(FFMPEG_FILE_HASH_BIN32 "47730de2286110b0d1250ff9cf50ce56") ocv_update(FFMPEG_FILE_HASH_BIN32 "31968b434799d3dd56969b15aac3efa8")
ocv_update(FFMPEG_FILE_HASH_BIN64 "3248b4663ffef770cdb54ec8b9d16a28") ocv_update(FFMPEG_FILE_HASH_BIN64 "84757ed0f16ddedce99227529d3165e9")
ocv_update(FFMPEG_FILE_HASH_CMAKE "8862c87496e2e8c375965e1277dee1c7") ocv_update(FFMPEG_FILE_HASH_CMAKE "e09efc33312d1173be8a9446f3b088fe")
function(download_win_ffmpeg script_var) function(download_win_ffmpeg script_var)
set(${script_var} "" PARENT_SCOPE) set(${script_var} "" PARENT_SCOPE)
+5 -5
View File
@@ -2,7 +2,7 @@ function(download_ippicv root_var)
set(${root_var} "" PARENT_SCOPE) set(${root_var} "" PARENT_SCOPE)
# Commit SHA in the opencv_3rdparty repo # Commit SHA in the opencv_3rdparty repo
set(IPPICV_COMMIT "406d398c436d0465c8e53dd432d9ecd9301d5f4a") set(IPPICV_COMMIT "8338862a733cb3980d8b51d8e14917fe0e695f71")
# Define actual ICV versions # Define actual ICV versions
if(APPLE) if(APPLE)
set(IPPICV_COMMIT "0cc4aa06bf2bef4b05d237c69a5a96b9cd0cb85a") set(IPPICV_COMMIT "0cc4aa06bf2bef4b05d237c69a5a96b9cd0cb85a")
@@ -14,8 +14,8 @@ function(download_ippicv root_var)
set(OPENCV_ICV_PLATFORM "linux") set(OPENCV_ICV_PLATFORM "linux")
set(OPENCV_ICV_PACKAGE_SUBDIR "ippicv_lnx") set(OPENCV_ICV_PACKAGE_SUBDIR "ippicv_lnx")
if(X86_64) if(X86_64)
set(OPENCV_ICV_NAME "ippicv_2026.0.0_lnx_intel64_20260327_general.tgz") set(OPENCV_ICV_NAME "ippicv_2026.0.0_lnx_intel64_20260630_general.tgz")
set(OPENCV_ICV_HASH "9a3ee0c5c3c02102faa422d60bfd1f4a") set(OPENCV_ICV_HASH "a77e60db544e07a126ae98f5ffe83be1")
else() else()
if(ANDROID) if(ANDROID)
set(IPPICV_COMMIT "c7c6d527dde5fee7cb914ee9e4e20f7436aab3a1") set(IPPICV_COMMIT "c7c6d527dde5fee7cb914ee9e4e20f7436aab3a1")
@@ -31,8 +31,8 @@ function(download_ippicv root_var)
set(OPENCV_ICV_PLATFORM "windows") set(OPENCV_ICV_PLATFORM "windows")
set(OPENCV_ICV_PACKAGE_SUBDIR "ippicv_win") set(OPENCV_ICV_PACKAGE_SUBDIR "ippicv_win")
if(X86_64) if(X86_64)
set(OPENCV_ICV_NAME "ippicv_2026.0.0_win_intel64_20260327_general.zip") set(OPENCV_ICV_NAME "ippicv_2026.0.0_win_intel64_20260630_general.zip")
set(OPENCV_ICV_HASH "73bc67cd5e4c8da706fa88fe84630231") set(OPENCV_ICV_HASH "d81c8b7d40da2867df82f0077a40afa1")
else() else()
set(IPPICV_COMMIT "7f55c0c26be418d494615afca15218566775c725") set(IPPICV_COMMIT "7f55c0c26be418d494615afca15218566775c725")
set(OPENCV_ICV_NAME "ippicv_2021.12.0_win_ia32_20240425_general.zip") set(OPENCV_ICV_NAME "ippicv_2021.12.0_win_ia32_20240425_general.zip")
+4 -4
View File
@@ -7,7 +7,7 @@
* Courses: <https://opencv.org/courses> * Courses: <https://opencv.org/courses>
* Docs: <https://docs.opencv.org/4.x/> * Docs: <https://docs.opencv.org/4.x/>
* Q&A forum: <https://forum.opencv.org> * Q&A forum: <https://forum.opencv.org>
* previous forum (read only): <http://answers.opencv.org> * previous forum (read only): <https://answers.opencv.org>
* Issue tracking: <https://github.com/opencv/opencv/issues> * Issue tracking: <https://github.com/opencv/opencv/issues>
* Additional OpenCV functionality: <https://github.com/opencv/opencv_contrib> * Additional OpenCV functionality: <https://github.com/opencv/opencv_contrib>
* Donate to OpenCV: <https://opencv.org/support/> * Donate to OpenCV: <https://opencv.org/support/>
@@ -28,9 +28,9 @@ Please read the [contribution guidelines](https://github.com/opencv/opencv/wiki/
### Additional Resources ### Additional Resources
* [Submit your OpenCV-based project](https://form.jotform.com/233105358823151) for inclusion in Community Friday on opencv.org * [Submit your OpenCV-based project](https://form.jotform.com/233105358823151) for inclusion in Community Friday on opencv.org
* [Subscribe to the OpenCV YouTube Channel](http://youtube.com/@opencvofficial) featuring OpenCV Live, an hour-long streaming show * [Subscribe to the OpenCV YouTube Channel](https://youtube.com/@opencvofficial) featuring OpenCV Live, an hour-long streaming show
* [Follow OpenCV on LinkedIn](http://linkedin.com/company/opencv/) for daily posts showing the state-of-the-art in computer vision & AI * [Follow OpenCV on LinkedIn](https://linkedin.com/company/opencv/) for daily posts showing the state-of-the-art in computer vision & AI
* [Apply to be an OpenCV Volunteer](https://form.jotform.com/232745316792159) to help organize events and online campaigns as well as amplify them * [Apply to be an OpenCV Volunteer](https://form.jotform.com/232745316792159) to help organize events and online campaigns as well as amplify them
* [Follow OpenCV on Mastodon](http://mastodon.social/@opencv) in the Fediverse * [Follow OpenCV on Mastodon](https://mastodon.social/@opencv) in the Fediverse
* [Follow OpenCV on Twitter](https://twitter.com/opencvlive) * [Follow OpenCV on Twitter](https://twitter.com/opencvlive)
* [OpenCV.ai](https://opencv.ai): Computer Vision and AI development services from the OpenCV team. * [OpenCV.ai](https://opencv.ai): Computer Vision and AI development services from the OpenCV team.
+27
View File
@@ -45,6 +45,11 @@ else()
endif() endif()
if(NOT ZLIB_FOUND) if(NOT ZLIB_FOUND)
if(NOT BUILD_ZLIB)
message(WARNING "ZLIB library has not been found, falling back to built-in version. "
"Set ZLIB_ROOT or ZLIB_DIR variable so that find_package(ZLIB) would be able to find it. "
"See find_package and FindZLIB cmake documentation for details.")
endif()
ocv_clear_vars(ZLIB_LIBRARY ZLIB_LIBRARIES ZLIB_INCLUDE_DIR) ocv_clear_vars(ZLIB_LIBRARY ZLIB_LIBRARIES ZLIB_INCLUDE_DIR)
set(ZLIB_LIBRARY zlib CACHE INTERNAL "") set(ZLIB_LIBRARY zlib CACHE INTERNAL "")
@@ -83,6 +88,12 @@ if(WITH_JPEG)
endif() endif()
if(NOT JPEG_FOUND) if(NOT JPEG_FOUND)
if(NOT BUILD_JPEG)
message(WARNING "JPEG library has not been found, falling back to built-in version. "
"Add the external installation prefix to CMAKE_PREFIX_PATH, or set JPEG_INCLUDE_DIR "
"and JPEG_LIBRARY, so that FindJPEG can locate it. "
"See FindJPEG cmake documentation for details.")
endif()
ocv_clear_vars(JPEG_LIBRARY JPEG_INCLUDE_DIR) ocv_clear_vars(JPEG_LIBRARY JPEG_INCLUDE_DIR)
if(NOT BUILD_JPEG_TURBO_DISABLE) if(NOT BUILD_JPEG_TURBO_DISABLE)
@@ -153,6 +164,12 @@ if(WITH_TIFF)
endif() endif()
if(NOT TIFF_FOUND) if(NOT TIFF_FOUND)
if(NOT BUILD_TIFF)
message(WARNING "TIFF library has not been found, falling back to built-in version. "
"Add the external installation prefix to CMAKE_PREFIX_PATH, or set TIFF_INCLUDE_DIR "
"and TIFF_LIBRARY, so that FindTIFF can locate it. "
"See FindTIFF cmake documentation for details.")
endif()
ocv_clear_vars(TIFF_LIBRARY TIFF_LIBRARIES TIFF_INCLUDE_DIR) ocv_clear_vars(TIFF_LIBRARY TIFF_LIBRARIES TIFF_INCLUDE_DIR)
set(TIFF_LIBRARY libtiff CACHE INTERNAL "") set(TIFF_LIBRARY libtiff CACHE INTERNAL "")
@@ -201,6 +218,11 @@ endif()
if(WITH_WEBP AND NOT WEBP_FOUND if(WITH_WEBP AND NOT WEBP_FOUND
AND (NOT ANDROID OR HAVE_CPUFEATURES) AND (NOT ANDROID OR HAVE_CPUFEATURES)
) )
if(NOT BUILD_WEBP)
message(WARNING "WebP library has not been found, falling back to built-in version. "
"Set WEBP_INCLUDE_DIR and WEBP_LIBRARY, or add the external installation prefix to "
"CMAKE_PREFIX_PATH, so that the external WebP can be located.")
endif()
ocv_clear_vars(WEBP_LIBRARY WEBP_INCLUDE_DIR) ocv_clear_vars(WEBP_LIBRARY WEBP_INCLUDE_DIR)
set(WEBP_LIBRARY libwebp CACHE INTERNAL "") set(WEBP_LIBRARY libwebp CACHE INTERNAL "")
set(WEBP_LIBRARIES ${WEBP_LIBRARY}) set(WEBP_LIBRARIES ${WEBP_LIBRARY})
@@ -353,6 +375,11 @@ if(NOT HAVE_SPNG AND WITH_PNG)
endif() endif()
if(NOT PNG_FOUND) if(NOT PNG_FOUND)
if(NOT BUILD_PNG)
message(WARNING "PNG library has not been found, falling back to built-in version. "
"Set PNG_ROOT or PNG_DIR variable so that find_package(PNG) would be able to find it. "
"See find_package and FindPNG cmake documentation for details.")
endif()
ocv_clear_vars(PNG_LIBRARY PNG_LIBRARIES PNG_INCLUDE_DIR PNG_DEFINITIONS) ocv_clear_vars(PNG_LIBRARY PNG_LIBRARIES PNG_INCLUDE_DIR PNG_DEFINITIONS)
set(PNG_LIBRARY libpng CACHE INTERNAL "") set(PNG_LIBRARY libpng CACHE INTERNAL "")
+8
View File
@@ -26,6 +26,7 @@ add_library(ipphal STATIC
"${CMAKE_CURRENT_SOURCE_DIR}/src/canny_ipp.cpp" "${CMAKE_CURRENT_SOURCE_DIR}/src/canny_ipp.cpp"
"${CMAKE_CURRENT_SOURCE_DIR}/src/threshold_ipp.cpp" "${CMAKE_CURRENT_SOURCE_DIR}/src/threshold_ipp.cpp"
"${CMAKE_CURRENT_SOURCE_DIR}/src/distancetransform_ipp.cpp" "${CMAKE_CURRENT_SOURCE_DIR}/src/distancetransform_ipp.cpp"
"${CMAKE_CURRENT_SOURCE_DIR}/src/histogram_ipp.cpp"
) )
#TODO: HAVE_IPP_ICV and HAVE_IPP_IW added as private macro till OpenCV itself is #TODO: HAVE_IPP_ICV and HAVE_IPP_IW added as private macro till OpenCV itself is
@@ -45,6 +46,13 @@ if(WITH_IPP_CALLS_ENFORCED)
message("WITH_IPP_CALLS_ENFORCED=${WITH_IPP_CALLS_ENFORCED}: enforced IPP calls are enabled in IPP HAL") message("WITH_IPP_CALLS_ENFORCED=${WITH_IPP_CALLS_ENFORCED}: enforced IPP calls are enabled in IPP HAL")
endif() endif()
# Enable cv::instr tracing of IPP calls inside the HAL so they show up in the
# instrumentation trace / IPP weight. Reuses the engine exported from
# opencv_core (symbols resolved at link time as ipphal is archived into core).
if(ENABLE_INSTRUMENTATION)
target_compile_definitions(ipphal PRIVATE ENABLE_INSTRUMENTATION)
endif()
target_include_directories(ipphal PRIVATE "${CMAKE_CURRENT_SOURCE_DIR}/include") target_include_directories(ipphal PRIVATE "${CMAKE_CURRENT_SOURCE_DIR}/include")
ocv_warnings_disable(CMAKE_CXX_FLAGS -Wno-suggest-override) ocv_warnings_disable(CMAKE_CXX_FLAGS -Wno-suggest-override)
+5
View File
@@ -154,6 +154,11 @@ int ipp_hal_distanceTransform(const uchar* src_data, size_t src_step, uchar* dst
#undef cv_hal_distanceTransform #undef cv_hal_distanceTransform
#define cv_hal_distanceTransform ipp_hal_distanceTransform #define cv_hal_distanceTransform ipp_hal_distanceTransform
int ipp_hal_calcHist(const uchar* src_data, size_t src_step, int src_type, int src_width, int src_height,
float* hist_data, int hist_size, const float** ranges, bool uniform, bool accumulate);
#undef cv_hal_calcHist
#define cv_hal_calcHist ipp_hal_calcHist
#endif // IPP_VERSION_X100 >= 700 #endif // IPP_VERSION_X100 >= 700
#define IPP_DISABLE_PERF_CANNY_MT 1 // cv::Canny OpenCV MT performance is better #define IPP_DISABLE_PERF_CANNY_MT 1 // cv::Canny OpenCV MT performance is better
+16
View File
@@ -1,6 +1,7 @@
// This file is part of OpenCV project. // This file is part of OpenCV project.
// It is subject to the license terms in the LICENSE file found in the top-level directory // It is subject to the license terms in the LICENSE file found in the top-level directory
// of this distribution and at http://opencv.org/license.html // of this distribution and at http://opencv.org/license.html
// Copyright (C) 2026, Intel Corporation, all rights reserved.
#ifndef __IPP_HAL_UTILS_HPP__ #ifndef __IPP_HAL_UTILS_HPP__
#define __IPP_HAL_UTILS_HPP__ #define __IPP_HAL_UTILS_HPP__
@@ -23,7 +24,22 @@
# include "ipp.h" # include "ipp.h"
#endif #endif
// IPP call instrumentation for the standalone IPP HAL.
//
// The HAL is built as a separate static library that does not include the
// internal core/private.hpp, so by default its IPP calls never appear in the
// cv::instr trace (reported IPP weight = 0%). When the parent build enables
// ENABLE_INSTRUMENTATION, the shared instrumentation.private.hpp provides the
// exact same CV_INSTRUMENT_FUN_IPP macro used by core (backed by symbols exported
// from libopencv_core, resolved at link time), so HAL IPP calls are traced
// identically to in-module ones. Otherwise the no-op passthrough is used.
#if !defined(CV_INSTRUMENT_FUN_IPP)
#if defined(ENABLE_INSTRUMENTATION)
#include "opencv2/core/utils/instrumentation.private.hpp"
#else
#define CV_INSTRUMENT_FUN_IPP(FUN, ...) ((FUN)(__VA_ARGS__)) #define CV_INSTRUMENT_FUN_IPP(FUN, ...) ((FUN)(__VA_ARGS__))
#endif // defined(ENABLE_INSTRUMENTATION)
#endif // !defined(CV_INSTRUMENT_FUN_IPP)
#define CV_HAL_CHECK_USE_IPP() if(!cv::ipp::useIPP()) return CV_HAL_ERROR_NOT_IMPLEMENTED; #define CV_HAL_CHECK_USE_IPP() if(!cv::ipp::useIPP()) return CV_HAL_ERROR_NOT_IMPLEMENTED;
+223
View File
@@ -0,0 +1,223 @@
// This file is part of OpenCV project.
// It is subject to the license terms in the LICENSE file found in the top-level directory
// of this distribution and at http://opencv.org/license.html.
// Copyright (C) 2026, BigVision LLC, all rights reserved.
// Third party copyrights are property of their respective owners.
#include "ipp_hal_imgproc.hpp"
#include <opencv2/core.hpp>
#include <opencv2/core/utils/tls.hpp>
#include "precomp_ipp.hpp"
#if IPP_VERSION_X100 >= 700
#define IPP_HISTOGRAM_PARALLEL 1
namespace cv { namespace ipp { unsigned long long getIppTopFeatures(); } }
using namespace cv;
namespace {
typedef IppStatus(CV_STDCALL * IppiHistogram_C1)(const void* pSrc, int srcStep,
IppiSize roiSize, Ipp32u* pHist, const IppiHistogramSpec* pSpec, Ipp8u* pBuffer);
static IppiHistogram_C1 getIppiHistogramFunction_C1(int type)
{
IppiHistogram_C1 ippFunction =
(type == CV_8UC1) ? (IppiHistogram_C1)ippiHistogram_8u_C1R :
(type == CV_16UC1) ? (IppiHistogram_C1)ippiHistogram_16u_C1R :
(type == CV_32FC1) ? (IppiHistogram_C1)ippiHistogram_32f_C1R :
NULL;
return ippFunction;
}
class ipp_calcHistParallelTLS
{
public:
ipp_calcHistParallelTLS() {}
IppAutoBuffer<IppiHistogramSpec> spec;
IppAutoBuffer<Ipp8u> buffer;
IppAutoBuffer<Ipp32u> thist;
};
class ipp_calcHistParallel: public ParallelLoopBody
{
public:
ipp_calcHistParallel(const Mat &src, Mat &hist, Ipp32s histSize, const float *ranges, bool uniform, bool &ok):
ParallelLoopBody(), m_src(src), m_hist(hist), m_ok(ok)
{
ok = true;
m_uniform = uniform;
m_ranges = ranges;
m_histSize = histSize;
m_type = ippiGetDataType(src.type());
m_levelsNum = histSize+1;
ippiHistogram_C1 = getIppiHistogramFunction_C1(src.type());
m_fullRoi = ippiSize(src.size());
m_bufferSize = 0;
m_specSize = 0;
if(!ippiHistogram_C1)
{
ok = false;
return;
}
if(ippiHistogramGetBufferSize(m_type, m_fullRoi, &m_levelsNum, 1, 1, &m_specSize, &m_bufferSize) < 0)
{
ok = false;
return;
}
hist.setTo(0);
}
virtual void operator() (const Range & range) const CV_OVERRIDE
{
if(!m_ok)
return;
ipp_calcHistParallelTLS *pTls = m_tls.get();
IppiSize roi = {m_src.cols, range.end - range.start };
bool mtLoop = false;
if(m_fullRoi.height != roi.height)
mtLoop = true;
if(!pTls->spec)
{
pTls->spec.allocate(m_specSize);
if(!pTls->spec.get())
{
m_ok = false;
return;
}
pTls->buffer.allocate(m_bufferSize);
if(!pTls->buffer.get() && m_bufferSize)
{
m_ok = false;
return;
}
if(m_uniform)
{
if(ippiHistogramUniformInit(m_type, (Ipp32f*)&m_ranges[0], (Ipp32f*)&m_ranges[1], (Ipp32s*)&m_levelsNum, 1, pTls->spec) < 0)
{
m_ok = false;
return;
}
}
else
{
if(ippiHistogramInit(m_type, (const Ipp32f**)&m_ranges, (Ipp32s*)&m_levelsNum, 1, pTls->spec) < 0)
{
m_ok = false;
return;
}
}
pTls->thist.allocate(m_histSize*sizeof(Ipp32u));
}
if(CV_INSTRUMENT_FUN_IPP(ippiHistogram_C1, m_src.ptr(range.start), (int)m_src.step, roi, pTls->thist, pTls->spec, pTls->buffer) < 0)
{
m_ok = false;
return;
}
if(mtLoop)
{
for(int i = 0; i < m_histSize; i++)
CV_XADD((int*)(m_hist.ptr(i)), *(int*)((Ipp32u*)pTls->thist + i));
}
else
ippiCopy_32s_C1R((Ipp32s*)pTls->thist.get(), sizeof(Ipp32u), (Ipp32s*)m_hist.ptr(), (int)m_hist.step, ippiSize(1, m_histSize));
}
private:
const Mat &m_src;
Mat &m_hist;
Ipp32s m_histSize;
const float *m_ranges;
bool m_uniform;
IppiHistogram_C1 ippiHistogram_C1;
IppiSize m_fullRoi;
IppDataType m_type;
Ipp32s m_levelsNum;
int m_bufferSize;
int m_specSize;
mutable Mutex m_syncMutex;
TLSData<ipp_calcHistParallelTLS> m_tls;
volatile bool &m_ok;
const ipp_calcHistParallel & operator = (const ipp_calcHistParallel & );
};
static bool ipp_calchist(const Mat &image, Mat &hist, int histSize, const float** ranges, bool uniform, bool accumulate)
{
#if IPP_VERSION_X100 < 201801
// No SSE42 optimization for uniform 32f
if(uniform && image.depth() == CV_32F && cv::ipp::getIppTopFeatures() == ippCPUID_SSE42)
return false;
#endif
// IPP_DISABLE_HISTOGRAM - https://github.com/opencv/opencv/issues/11544
// and https://github.com/opencv/opencv/issues/21595
if ((uniform && (ranges[0][1] - ranges[0][0]) != histSize) || abs(ranges[0][0]) != cvFloor(ranges[0][0]))
return false;
Mat ihist = hist;
if(accumulate)
ihist.create(1, &histSize, CV_32S);
bool ok = true;
int threads = ippiSuggestThreadsNum(image.cols, image.rows, image.elemSize(), (1+((double)ihist.total()/image.total()))*2);
Range range(0, image.rows);
ipp_calcHistParallel invoker(image, ihist, histSize, ranges[0], uniform, ok);
if(!ok)
return false;
if(IPP_HISTOGRAM_PARALLEL && threads > 1)
parallel_for_(range, invoker, threads*2);
else
invoker(range);
if(ok)
{
if(accumulate)
{
IppiSize histRoi = ippiSize(1, histSize);
IppAutoBuffer<Ipp32f> fhist(histSize*sizeof(Ipp32f));
CV_INSTRUMENT_FUN_IPP(ippiConvert_32s32f_C1R, (Ipp32s*)ihist.ptr(), (int)ihist.step, (Ipp32f*)fhist, sizeof(Ipp32f), histRoi);
CV_INSTRUMENT_FUN_IPP(ippiAdd_32f_C1IR, (Ipp32f*)fhist, sizeof(Ipp32f), (Ipp32f*)hist.ptr(), (int)hist.step, histRoi);
}
else
CV_INSTRUMENT_FUN_IPP(ippiConvert_32s32f_C1R, (Ipp32s*)ihist.ptr(), (int)ihist.step, (Ipp32f*)hist.ptr(), (int)hist.step, ippiSize(1, histSize));
}
return ok;
}
} // namespace
int ipp_hal_calcHist(const uchar* src_data, size_t src_step, int src_type, int src_width, int src_height,
float* hist_data, int hist_size, const float** ranges, bool uniform, bool accumulate)
{
CV_HAL_CHECK_USE_IPP();
Mat image(src_height, src_width, src_type, (void*)src_data, src_step);
Mat hist(1, &hist_size, CV_32F, (void*)hist_data);
if(ipp_calchist(image, hist, hist_size, ranges, uniform, accumulate))
return CV_HAL_ERROR_OK;
return CV_HAL_ERROR_NOT_IMPLEMENTED;
}
#endif
+26
View File
@@ -120,4 +120,30 @@ static inline int ippiSuggestRowThreadsNum(const ::ipp::IwiImage &image, size_t
} }
#endif #endif
#if IPP_VERSION_X100 >= 201700
#define CV_IPP_MALLOC(SIZE) ippMalloc_L(SIZE)
#else
#define CV_IPP_MALLOC(SIZE) ippMalloc((int)SIZE)
#endif
template<typename T>
class IppAutoBuffer
{
public:
IppAutoBuffer() { m_size = 0; m_pBuffer = NULL; }
explicit IppAutoBuffer(size_t size) { m_size = 0; m_pBuffer = NULL; allocate(size); }
~IppAutoBuffer() { deallocate(); }
T* allocate(size_t size) { if(m_size < size) { deallocate(); m_pBuffer = (T*)CV_IPP_MALLOC(size); m_size = size; } return m_pBuffer; }
void deallocate() { if(m_pBuffer) { ippFree(m_pBuffer); m_pBuffer = NULL; } m_size = 0; }
inline T* get() { return (T*)m_pBuffer;}
inline operator T* () { return (T*)m_pBuffer;}
inline operator const T* () const { return (const T*)m_pBuffer;}
private:
IppAutoBuffer(IppAutoBuffer &) {}
IppAutoBuffer& operator =(const IppAutoBuffer &) {return *this;}
size_t m_size;
T* m_pBuffer;
};
#endif //__PRECOMP_IPP_HPP__ #endif //__PRECOMP_IPP_HPP__
+9
View File
@@ -99,9 +99,18 @@ int fast_atan_64(const double* y, const double* x, double* dst, size_t n, bool a
/* ############ split ############ */ /* ############ split ############ */
int split8u(const uchar* src, uchar** dst, int len, int cn); int split8u(const uchar* src, uchar** dst, int len, int cn);
int split16u(const ushort* src, ushort** dst, int len, int cn);
int split32s(const int* src, int** dst, int len, int cn);
int split64s(const int64* src, int64** dst, int len, int cn);
#undef cv_hal_split8u #undef cv_hal_split8u
#define cv_hal_split8u cv::rvv_hal::core::split8u #define cv_hal_split8u cv::rvv_hal::core::split8u
#undef cv_hal_split16u
#define cv_hal_split16u cv::rvv_hal::core::split16u
#undef cv_hal_split32s
#define cv_hal_split32s cv::rvv_hal::core::split32s
#undef cv_hal_split64s
#define cv_hal_split64s cv::rvv_hal::core::split64s
/* ############ sqrt ############ */ /* ############ sqrt ############ */
+21 -2
View File
@@ -30,8 +30,27 @@ static int copyToMasked_e##X##c1(const uchar *src_data, size_t src_step, const u
return CV_HAL_ERROR_OK; \ return CV_HAL_ERROR_OK; \
} }
CV_HAL_RVV_COPY_MASK_eXc1(8, 8) #define CV_HAL_RVV_COPY_MASK_MERGE_eXc1(X, data_lmul, mask_lmul) \
CV_HAL_RVV_COPY_MASK_eXc1(16, 4) static int copyToMasked_e##X##c1(const uchar *src_data, size_t src_step, const uchar *mask_data, size_t mask_step, \
uchar *dst_data, size_t dst_step, int width, int height) { \
for (; height--; mask_data += mask_step, src_data += src_step, dst_data += dst_step) { \
const uint##X##_t *src = (const uint##X##_t*)src_data; \
uint##X##_t *dst = (uint##X##_t*)dst_data; \
int vl; \
for (int i = 0; i < width; i += vl) { \
vl = __riscv_vsetvl_e##X##m##data_lmul(width - i); \
auto m = __riscv_vmsne(__riscv_vle8_v_u8m##mask_lmul(mask_data + i, vl), 0, vl); \
auto src_v = __riscv_vle##X##_v_u##X##m##data_lmul(src + i, vl); \
auto dst_v = __riscv_vle##X##_v_u##X##m##data_lmul(dst + i, vl); \
__riscv_vse##X##_v_u##X##m##data_lmul(dst + i, __riscv_vmerge(dst_v, src_v, m, vl), vl); \
} \
} \
return CV_HAL_ERROR_OK; \
}
CV_HAL_RVV_COPY_MASK_MERGE_eXc1(8, 8, 8)
CV_HAL_RVV_COPY_MASK_MERGE_eXc1(16, 8, 4)
CV_HAL_RVV_COPY_MASK_eXc1(32, 2) CV_HAL_RVV_COPY_MASK_eXc1(32, 2)
CV_HAL_RVV_COPY_MASK_eXc1(64, 1) CV_HAL_RVV_COPY_MASK_eXc1(64, 1)
+135 -98
View File
@@ -28,6 +28,12 @@
#define __riscv_vcreate_v_u16m4x2(v0, v1) OPENCV_HAL_IMPL_RVV_VCREATE_x2(u16, 4, v0, v1) #define __riscv_vcreate_v_u16m4x2(v0, v1) OPENCV_HAL_IMPL_RVV_VCREATE_x2(u16, 4, v0, v1)
#define __riscv_vcreate_v_u16m2x3(v0, v1, v2) OPENCV_HAL_IMPL_RVV_VCREATE_x3(u16, 2, v0, v1, v2) #define __riscv_vcreate_v_u16m2x3(v0, v1, v2) OPENCV_HAL_IMPL_RVV_VCREATE_x3(u16, 2, v0, v1, v2)
#define __riscv_vcreate_v_u16m2x4(v0, v1, v2, v3) OPENCV_HAL_IMPL_RVV_VCREATE_x4(u16, 2, v0, v1, v2, v3) #define __riscv_vcreate_v_u16m2x4(v0, v1, v2, v3) OPENCV_HAL_IMPL_RVV_VCREATE_x4(u16, 2, v0, v1, v2, v3)
#define __riscv_vcreate_v_i32m4x2(v0, v1) OPENCV_HAL_IMPL_RVV_VCREATE_x2(i32, 4, v0, v1)
#define __riscv_vcreate_v_i32m2x3(v0, v1, v2) OPENCV_HAL_IMPL_RVV_VCREATE_x3(i32, 2, v0, v1, v2)
#define __riscv_vcreate_v_i32m2x4(v0, v1, v2, v3) OPENCV_HAL_IMPL_RVV_VCREATE_x4(i32, 2, v0, v1, v2, v3)
#define __riscv_vcreate_v_i64m4x2(v0, v1) OPENCV_HAL_IMPL_RVV_VCREATE_x2(i64, 4, v0, v1)
#define __riscv_vcreate_v_i64m2x3(v0, v1, v2) OPENCV_HAL_IMPL_RVV_VCREATE_x3(i64, 2, v0, v1, v2)
#define __riscv_vcreate_v_i64m2x4(v0, v1, v2, v3) OPENCV_HAL_IMPL_RVV_VCREATE_x4(i64, 2, v0, v1, v2, v3)
#endif // clang < 18 #endif // clang < 18
namespace cv { namespace rvv_hal { namespace core { namespace cv { namespace rvv_hal { namespace core {
@@ -204,146 +210,177 @@ int merge16u(const ushort** src, ushort* dst, int len, int cn ) {
return CV_HAL_ERROR_OK; return CV_HAL_ERROR_OK;
} }
#if defined __GNUC__ && !defined(__clang__)
__attribute__((optimize("no-tree-vectorize")))
#endif
int merge32s(const int** src, int* dst, int len, int cn ) { int merge32s(const int** src, int* dst, int len, int cn ) {
int k = cn % 4 ? cn % 4 : 4; int vl = 0;
int i, j; if (cn == 1)
if( k == 1 )
{ {
const int* src0 = src[0]; const int* src0 = src[0];
#if defined(__clang__) for (int i = 0; i < len; i += vl)
#pragma clang loop vectorize(disable)
#endif
for( i = j = 0; i < len; i++, j += cn )
dst[j] = src0[i];
}
else if( k == 2 )
{
const int *src0 = src[0], *src1 = src[1];
i = j = 0;
#if defined(__clang__)
#pragma clang loop vectorize(disable)
#endif
for( ; i < len; i++, j += cn )
{ {
dst[j] = src0[i]; vl = __riscv_vsetvl_e32m8(len - i);
dst[j+1] = src1[i]; __riscv_vse32_v_i32m8(dst + i, __riscv_vle32_v_i32m8(src0 + i, vl), vl);
} }
} }
else if( k == 3 ) else if (cn == 2)
{
const int *src0 = src[0], *src1 = src[1];
for (int i = 0; i < len; i += vl)
{
vl = __riscv_vsetvl_e32m4(len - i);
vint32m4x2_t seg = __riscv_vcreate_v_i32m4x2(
__riscv_vle32_v_i32m4(src0 + i, vl),
__riscv_vle32_v_i32m4(src1 + i, vl)
);
__riscv_vsseg2e32_v_i32m4x2(dst + i * cn, seg, vl);
}
}
else if (cn == 3)
{ {
const int *src0 = src[0], *src1 = src[1], *src2 = src[2]; const int *src0 = src[0], *src1 = src[1], *src2 = src[2];
i = j = 0; for (int i = 0; i < len; i += vl)
#if defined(__clang__)
#pragma clang loop vectorize(disable)
#endif
for( ; i < len; i++, j += cn )
{ {
dst[j] = src0[i]; vl = __riscv_vsetvl_e32m2(len - i);
dst[j+1] = src1[i]; vint32m2x3_t seg = __riscv_vcreate_v_i32m2x3(
dst[j+2] = src2[i]; __riscv_vle32_v_i32m2(src0 + i, vl),
__riscv_vle32_v_i32m2(src1 + i, vl),
__riscv_vle32_v_i32m2(src2 + i, vl)
);
__riscv_vsseg3e32_v_i32m2x3(dst + i * cn, seg, vl);
}
}
else if (cn == 4)
{
const int *src0 = src[0], *src1 = src[1], *src2 = src[2], *src3 = src[3];
for (int i = 0; i < len; i += vl)
{
vl = __riscv_vsetvl_e32m2(len - i);
vint32m2x4_t seg = __riscv_vcreate_v_i32m2x4(
__riscv_vle32_v_i32m2(src0 + i, vl),
__riscv_vle32_v_i32m2(src1 + i, vl),
__riscv_vle32_v_i32m2(src2 + i, vl),
__riscv_vle32_v_i32m2(src3 + i, vl)
);
__riscv_vsseg4e32_v_i32m2x4(dst + i * cn, seg, vl);
} }
} }
else else
{ {
const int *src0 = src[0], *src1 = src[1], *src2 = src[2], *src3 = src[3]; int k = 0;
i = j = 0; for (; k <= cn - 4; k += 4)
#if defined(__clang__)
#pragma clang loop vectorize(disable)
#endif
for( ; i < len; i++, j += cn )
{ {
dst[j] = src0[i]; dst[j+1] = src1[i]; const int *src0 = src[k], *src1 = src[k + 1], *src2 = src[k + 2], *src3 = src[k + 3];
dst[j+2] = src2[i]; dst[j+3] = src3[i]; for (int i = 0; i < len; i += vl)
{
vl = __riscv_vsetvl_e32m2(len - i);
vint32m2x4_t seg = __riscv_vcreate_v_i32m2x4(
__riscv_vle32_v_i32m2(src0 + i, vl),
__riscv_vle32_v_i32m2(src1 + i, vl),
__riscv_vle32_v_i32m2(src2 + i, vl),
__riscv_vle32_v_i32m2(src3 + i, vl)
);
__riscv_vssseg4e32_v_i32m2x4(dst + k + i * cn, cn * sizeof(int), seg, vl);
}
} }
} for (; k < cn; ++k)
#if defined(__clang__)
#pragma clang loop vectorize(disable)
#endif
for( ; k < cn; k += 4 )
{
const int *src0 = src[k], *src1 = src[k+1], *src2 = src[k+2], *src3 = src[k+3];
for( i = 0, j = k; i < len; i++, j += cn )
{ {
dst[j] = src0[i]; dst[j+1] = src1[i]; const int* srcK = src[k];
dst[j+2] = src2[i]; dst[j+3] = src3[i]; for (int i = 0; i < len; i += vl)
{
vl = __riscv_vsetvl_e32m2(len - i);
vint32m2_t seg = __riscv_vle32_v_i32m2(srcK + i, vl);
__riscv_vsse32_v_i32m2(dst + k + i * cn, cn * sizeof(int), seg, vl);
}
} }
} }
return CV_HAL_ERROR_OK; return CV_HAL_ERROR_OK;
} }
#if defined __GNUC__ && !defined(__clang__)
__attribute__((optimize("no-tree-vectorize")))
#endif
int merge64s(const int64** src, int64* dst, int len, int cn ) { int merge64s(const int64** src, int64* dst, int len, int cn ) {
int k = cn % 4 ? cn % 4 : 4; int vl = 0;
int i, j; if (cn == 1)
if( k == 1 )
{ {
const int64* src0 = src[0]; const int64* src0 = src[0];
#if defined(__clang__) for (int i = 0; i < len; i += vl)
#pragma clang loop vectorize(disable)
#endif
for( i = j = 0; i < len; i++, j += cn )
dst[j] = src0[i];
}
else if( k == 2 )
{
const int64 *src0 = src[0], *src1 = src[1];
i = j = 0;
#if defined(__clang__)
#pragma clang loop vectorize(disable)
#endif
for( ; i < len; i++, j += cn )
{ {
dst[j] = src0[i]; vl = __riscv_vsetvl_e64m8(len - i);
dst[j+1] = src1[i]; __riscv_vse64_v_i64m8(dst + i, __riscv_vle64_v_i64m8(src0 + i, vl), vl);
} }
} }
else if( k == 3 ) else if (cn == 2)
{
const int64 *src0 = src[0], *src1 = src[1];
for (int i = 0; i < len; i += vl)
{
vl = __riscv_vsetvl_e64m4(len - i);
vint64m4x2_t seg = __riscv_vcreate_v_i64m4x2(
__riscv_vle64_v_i64m4(src0 + i, vl),
__riscv_vle64_v_i64m4(src1 + i, vl)
);
__riscv_vsseg2e64_v_i64m4x2(dst + i * cn, seg, vl);
}
}
else if (cn == 3)
{ {
const int64 *src0 = src[0], *src1 = src[1], *src2 = src[2]; const int64 *src0 = src[0], *src1 = src[1], *src2 = src[2];
i = j = 0; for (int i = 0; i < len; i += vl)
#if defined(__clang__)
#pragma clang loop vectorize(disable)
#endif
for( ; i < len; i++, j += cn )
{ {
dst[j] = src0[i]; vl = __riscv_vsetvl_e64m2(len - i);
dst[j+1] = src1[i]; vint64m2x3_t seg = __riscv_vcreate_v_i64m2x3(
dst[j+2] = src2[i]; __riscv_vle64_v_i64m2(src0 + i, vl),
__riscv_vle64_v_i64m2(src1 + i, vl),
__riscv_vle64_v_i64m2(src2 + i, vl)
);
__riscv_vsseg3e64_v_i64m2x3(dst + i * cn, seg, vl);
}
}
else if (cn == 4)
{
const int64 *src0 = src[0], *src1 = src[1], *src2 = src[2], *src3 = src[3];
for (int i = 0; i < len; i += vl)
{
vl = __riscv_vsetvl_e64m2(len - i);
vint64m2x4_t seg = __riscv_vcreate_v_i64m2x4(
__riscv_vle64_v_i64m2(src0 + i, vl),
__riscv_vle64_v_i64m2(src1 + i, vl),
__riscv_vle64_v_i64m2(src2 + i, vl),
__riscv_vle64_v_i64m2(src3 + i, vl)
);
__riscv_vsseg4e64_v_i64m2x4(dst + i * cn, seg, vl);
} }
} }
else else
{ {
const int64 *src0 = src[0], *src1 = src[1], *src2 = src[2], *src3 = src[3]; int k = 0;
i = j = 0; for (; k <= cn - 4; k += 4)
#if defined(__clang__)
#pragma clang loop vectorize(disable)
#endif
for( ; i < len; i++, j += cn )
{ {
dst[j] = src0[i]; dst[j+1] = src1[i]; const int64 *src0 = src[k], *src1 = src[k + 1], *src2 = src[k + 2], *src3 = src[k + 3];
dst[j+2] = src2[i]; dst[j+3] = src3[i]; for (int i = 0; i < len; i += vl)
{
vl = __riscv_vsetvl_e64m2(len - i);
vint64m2x4_t seg = __riscv_vcreate_v_i64m2x4(
__riscv_vle64_v_i64m2(src0 + i, vl),
__riscv_vle64_v_i64m2(src1 + i, vl),
__riscv_vle64_v_i64m2(src2 + i, vl),
__riscv_vle64_v_i64m2(src3 + i, vl)
);
__riscv_vssseg4e64_v_i64m2x4(dst + k + i * cn, cn * sizeof(int64), seg, vl);
}
} }
} for (; k < cn; ++k)
#if defined(__clang__)
#pragma clang loop vectorize(disable)
#endif
for( ; k < cn; k += 4 )
{
const int64 *src0 = src[k], *src1 = src[k+1], *src2 = src[k+2], *src3 = src[k+3];
for( i = 0, j = k; i < len; i++, j += cn )
{ {
dst[j] = src0[i]; dst[j+1] = src1[i]; const int64* srcK = src[k];
dst[j+2] = src2[i]; dst[j+3] = src3[i]; for (int i = 0; i < len; i += vl)
{
vl = __riscv_vsetvl_e64m2(len - i);
vint64m2_t seg = __riscv_vle64_v_i64m2(srcK + i, vl);
__riscv_vsse64_v_i64m2(dst + k + i * cn, cn * sizeof(int64), seg, vl);
}
} }
} }
return CV_HAL_ERROR_OK; return CV_HAL_ERROR_OK;
} }
#endif // CV_HAL_RVV_1P0_ENABLED #endif // CV_HAL_RVV_1P0_ENABLED
}}} // cv::rvv_hal::core }}} // cv::rvv_hal::core
+84 -76
View File
@@ -8,84 +8,92 @@ namespace cv { namespace rvv_hal { namespace core {
#if CV_HAL_RVV_1P0_ENABLED #if CV_HAL_RVV_1P0_ENABLED
int split8u(const uchar* src, uchar** dst, int len, int cn) #define OPENCV_HAL_IMPL_RVV_SPLIT(func, T, vtype, suffix, width) \
{ int func(const T* src, T** dst, int len, int cn) \
int vl = 0; { \
if (cn == 1) int vl = 0; \
{ if (cn == 1) \
uchar* dst0 = dst[0]; { \
for (int i = 0; i < len; i += vl) T* dst0 = dst[0]; \
{ for (int i = 0; i < len; i += vl) \
vl = __riscv_vsetvl_e8m8(len - i); { \
__riscv_vse8_v_u8m8(dst0 + i, __riscv_vle8_v_u8m8(src + i, vl), vl); vl = __riscv_vsetvl_e##width##m8(len - i); \
} __riscv_vse##width##_v_##suffix##m8(dst0 + i, __riscv_vle##width##_v_##suffix##m8(src + i, vl), vl); \
} } \
else if (cn == 2) } \
{ else if (cn == 2) \
uchar *dst0 = dst[0], *dst1 = dst[1]; { \
for (int i = 0; i < len; i += vl) T *dst0 = dst[0], *dst1 = dst[1]; \
{ for (int i = 0; i < len; i += vl) \
vl = __riscv_vsetvl_e8m4(len - i); { \
vuint8m4x2_t seg = __riscv_vlseg2e8_v_u8m4x2(src + i * cn, vl); vl = __riscv_vsetvl_e##width##m4(len - i); \
__riscv_vse8_v_u8m4(dst0 + i, __riscv_vget_v_u8m4x2_u8m4(seg, 0), vl); v##vtype##m4x2_t seg = __riscv_vlseg2e##width##_v_##suffix##m4x2(src + i * cn, vl); \
__riscv_vse8_v_u8m4(dst1 + i, __riscv_vget_v_u8m4x2_u8m4(seg, 1), vl); __riscv_vse##width##_v_##suffix##m4(dst0 + i, __riscv_vget_v_##suffix##m4x2_##suffix##m4(seg, 0), vl); \
} __riscv_vse##width##_v_##suffix##m4(dst1 + i, __riscv_vget_v_##suffix##m4x2_##suffix##m4(seg, 1), vl); \
} } \
else if (cn == 3) } \
{ else if (cn == 3) \
uchar *dst0 = dst[0], *dst1 = dst[1], *dst2 = dst[2]; { \
for (int i = 0; i < len; i += vl) T *dst0 = dst[0], *dst1 = dst[1], *dst2 = dst[2]; \
{ for (int i = 0; i < len; i += vl) \
vl = __riscv_vsetvl_e8m2(len - i); { \
vuint8m2x3_t seg = __riscv_vlseg3e8_v_u8m2x3(src + i * cn, vl); vl = __riscv_vsetvl_e##width##m2(len - i); \
__riscv_vse8_v_u8m2(dst0 + i, __riscv_vget_v_u8m2x3_u8m2(seg, 0), vl); v##vtype##m2x3_t seg = __riscv_vlseg3e##width##_v_##suffix##m2x3(src + i * cn, vl); \
__riscv_vse8_v_u8m2(dst1 + i, __riscv_vget_v_u8m2x3_u8m2(seg, 1), vl); __riscv_vse##width##_v_##suffix##m2(dst0 + i, __riscv_vget_v_##suffix##m2x3_##suffix##m2(seg, 0), vl); \
__riscv_vse8_v_u8m2(dst2 + i, __riscv_vget_v_u8m2x3_u8m2(seg, 2), vl); __riscv_vse##width##_v_##suffix##m2(dst1 + i, __riscv_vget_v_##suffix##m2x3_##suffix##m2(seg, 1), vl); \
} __riscv_vse##width##_v_##suffix##m2(dst2 + i, __riscv_vget_v_##suffix##m2x3_##suffix##m2(seg, 2), vl); \
} } \
else if (cn == 4) } \
{ else if (cn == 4) \
uchar *dst0 = dst[0], *dst1 = dst[1], *dst2 = dst[2], *dst3 = dst[3]; { \
for (int i = 0; i < len; i += vl) T *dst0 = dst[0], *dst1 = dst[1], *dst2 = dst[2], *dst3 = dst[3]; \
{ for (int i = 0; i < len; i += vl) \
vl = __riscv_vsetvl_e8m2(len - i); { \
vuint8m2x4_t seg = __riscv_vlseg4e8_v_u8m2x4(src + i * cn, vl); vl = __riscv_vsetvl_e##width##m2(len - i); \
__riscv_vse8_v_u8m2(dst0 + i, __riscv_vget_v_u8m2x4_u8m2(seg, 0), vl); v##vtype##m2x4_t seg = __riscv_vlseg4e##width##_v_##suffix##m2x4(src + i * cn, vl); \
__riscv_vse8_v_u8m2(dst1 + i, __riscv_vget_v_u8m2x4_u8m2(seg, 1), vl); __riscv_vse##width##_v_##suffix##m2(dst0 + i, __riscv_vget_v_##suffix##m2x4_##suffix##m2(seg, 0), vl); \
__riscv_vse8_v_u8m2(dst2 + i, __riscv_vget_v_u8m2x4_u8m2(seg, 2), vl); __riscv_vse##width##_v_##suffix##m2(dst1 + i, __riscv_vget_v_##suffix##m2x4_##suffix##m2(seg, 1), vl); \
__riscv_vse8_v_u8m2(dst3 + i, __riscv_vget_v_u8m2x4_u8m2(seg, 3), vl); __riscv_vse##width##_v_##suffix##m2(dst2 + i, __riscv_vget_v_##suffix##m2x4_##suffix##m2(seg, 2), vl); \
} __riscv_vse##width##_v_##suffix##m2(dst3 + i, __riscv_vget_v_##suffix##m2x4_##suffix##m2(seg, 3), vl); \
} } \
else } \
{ else \
int k = 0; { \
for (; k <= cn - 4; k += 4) int k = 0; \
{ for (; k <= cn - 4; k += 4) \
uchar *dst0 = dst[k], *dst1 = dst[k + 1], *dst2 = dst[k + 2], *dst3 = dst[k + 3]; { \
for (int i = 0; i < len; i += vl) T *dst0 = dst[k], *dst1 = dst[k + 1], *dst2 = dst[k + 2], *dst3 = dst[k + 3]; \
{ for (int i = 0; i < len; i += vl) \
vl = __riscv_vsetvl_e8m2(len - i); { \
vuint8m2x4_t seg = __riscv_vlsseg4e8_v_u8m2x4(src + k + i * cn, cn, vl); vl = __riscv_vsetvl_e##width##m2(len - i); \
__riscv_vse8_v_u8m2(dst0 + i, __riscv_vget_v_u8m2x4_u8m2(seg, 0), vl); v##vtype##m2x4_t seg = __riscv_vlsseg4e##width##_v_##suffix##m2x4(src + k + i * cn, cn * sizeof(T), vl); \
__riscv_vse8_v_u8m2(dst1 + i, __riscv_vget_v_u8m2x4_u8m2(seg, 1), vl); __riscv_vse##width##_v_##suffix##m2(dst0 + i, __riscv_vget_v_##suffix##m2x4_##suffix##m2(seg, 0), vl); \
__riscv_vse8_v_u8m2(dst2 + i, __riscv_vget_v_u8m2x4_u8m2(seg, 2), vl); __riscv_vse##width##_v_##suffix##m2(dst1 + i, __riscv_vget_v_##suffix##m2x4_##suffix##m2(seg, 1), vl); \
__riscv_vse8_v_u8m2(dst3 + i, __riscv_vget_v_u8m2x4_u8m2(seg, 3), vl); __riscv_vse##width##_v_##suffix##m2(dst2 + i, __riscv_vget_v_##suffix##m2x4_##suffix##m2(seg, 2), vl); \
} __riscv_vse##width##_v_##suffix##m2(dst3 + i, __riscv_vget_v_##suffix##m2x4_##suffix##m2(seg, 3), vl); \
} } \
for (; k < cn; ++k) } \
{ for (; k < cn; ++k) \
uchar* dstK = dst[k]; { \
for (int i = 0; i < len; i += vl) T* dstK = dst[k]; \
{ for (int i = 0; i < len; i += vl) \
vl = __riscv_vsetvl_e8m2(len - i); { \
vuint8m2_t seg = __riscv_vlse8_v_u8m2(src + k + i * cn, cn, vl); vl = __riscv_vsetvl_e##width##m2(len - i); \
__riscv_vse8_v_u8m2(dstK + i, seg, vl); v##vtype##m2_t seg = __riscv_vlse##width##_v_##suffix##m2(src + k + i * cn, cn * sizeof(T), vl); \
} __riscv_vse##width##_v_##suffix##m2(dstK + i, seg, vl); \
} } \
} } \
return CV_HAL_ERROR_OK; } \
return CV_HAL_ERROR_OK; \
} }
OPENCV_HAL_IMPL_RVV_SPLIT(split8u, uchar, uint8, u8, 8)
OPENCV_HAL_IMPL_RVV_SPLIT(split16u, ushort, uint16, u16, 16)
OPENCV_HAL_IMPL_RVV_SPLIT(split32s, int, int32, i32, 32)
OPENCV_HAL_IMPL_RVV_SPLIT(split64s, int64, int64, i64, 64)
#undef OPENCV_HAL_IMPL_RVV_SPLIT
#endif // CV_HAL_RVV_1P0_ENABLED #endif // CV_HAL_RVV_1P0_ENABLED
}}} // cv::rvv_hal::core }}} // cv::rvv_hal::core
@@ -939,7 +939,7 @@ inline scalartype v_reduce_sum(const _Tpvec& a) \
} }
OPENCV_HAL_IMPL_RVV_REDUCE_SUM_FP(v_float32, v_float32, vfloat32m1_t, float, f32, VTraits<v_float32>::vlanes()) OPENCV_HAL_IMPL_RVV_REDUCE_SUM_FP(v_float32, v_float32, vfloat32m1_t, float, f32, VTraits<v_float32>::vlanes())
#if CV_SIMD_SCALABLE_64F #if CV_SIMD_SCALABLE_64F
OPENCV_HAL_IMPL_RVV_REDUCE_SUM_FP(v_float64, v_float64, vfloat64m1_t, float, f64, VTraits<v_float64>::vlanes()) OPENCV_HAL_IMPL_RVV_REDUCE_SUM_FP(v_float64, v_float64, vfloat64m1_t, double, f64, VTraits<v_float64>::vlanes())
#endif #endif
#define OPENCV_HAL_IMPL_RVV_REDUCE(_Tpvec, _nTpvec, func, scalartype, suffix, vl, red) \ #define OPENCV_HAL_IMPL_RVV_REDUCE(_Tpvec, _nTpvec, func, scalartype, suffix, vl, red) \
+2 -55
View File
@@ -690,23 +690,13 @@ public:
TLSData<InstrTLSStruct> tlsStruct; TLSData<InstrTLSStruct> tlsStruct;
}; };
class CV_EXPORTS IntrumentationRegion
{
public:
IntrumentationRegion(const char* funName, const char* fileName, int lineNum, void *retAddress, bool alwaysExpand, TYPE instrType = TYPE_GENERAL, IMPL implType = IMPL_PLAIN);
~IntrumentationRegion();
private:
bool m_disabled; // region status
uint64 m_regionTicks;
};
CV_EXPORTS InstrStruct& getInstrumentStruct(); CV_EXPORTS InstrStruct& getInstrumentStruct();
InstrTLSStruct& getInstrumentTLSStruct(); InstrTLSStruct& getInstrumentTLSStruct();
CV_EXPORTS InstrNode* getCurrentNode();
} }
} }
#include "opencv2/core/utils/instrumentation.private.hpp"
#ifdef _WIN32 #ifdef _WIN32
#define CV_INSTRUMENT_GET_RETURN_ADDRESS _ReturnAddress() #define CV_INSTRUMENT_GET_RETURN_ADDRESS _ReturnAddress()
#else #else
@@ -718,45 +708,6 @@ CV_EXPORTS InstrNode* getCurrentNode();
#define CV_INSTRUMENT_REGION_CUSTOM_META(NAME, ALWAYS_EXPAND, TYPE, IMPL)\ #define CV_INSTRUMENT_REGION_CUSTOM_META(NAME, ALWAYS_EXPAND, TYPE, IMPL)\
void *CVAUX_CONCAT(__curr_address__, __LINE__) = [&]() {return CV_INSTRUMENT_GET_RETURN_ADDRESS;}();\ void *CVAUX_CONCAT(__curr_address__, __LINE__) = [&]() {return CV_INSTRUMENT_GET_RETURN_ADDRESS;}();\
::cv::instr::IntrumentationRegion CVAUX_CONCAT(__instr_region__, __LINE__) (NAME, __FILE__, __LINE__, CVAUX_CONCAT(__curr_address__, __LINE__), false, ::cv::instr::TYPE_GENERAL, ::cv::instr::IMPL_PLAIN); ::cv::instr::IntrumentationRegion CVAUX_CONCAT(__instr_region__, __LINE__) (NAME, __FILE__, __LINE__, CVAUX_CONCAT(__curr_address__, __LINE__), false, ::cv::instr::TYPE_GENERAL, ::cv::instr::IMPL_PLAIN);
// Instrument functions with non-void return type
#define CV_INSTRUMENT_FUN_RT_META(TYPE, IMPL, ERROR_COND, FUN, ...) ([&]()\
{\
if(::cv::instr::useInstrumentation()){\
::cv::instr::IntrumentationRegion __instr__(#FUN, __FILE__, __LINE__, NULL, false, TYPE, IMPL);\
try{\
auto instrStatus = ((FUN)(__VA_ARGS__));\
if(ERROR_COND){\
::cv::instr::getCurrentNode()->m_payload.m_funError = true;\
CV_INSTRUMENT_MARK_META(IMPL, #FUN " - BadExit");\
}\
return instrStatus;\
}catch(...){\
::cv::instr::getCurrentNode()->m_payload.m_funError = true;\
CV_INSTRUMENT_MARK_META(IMPL, #FUN " - BadExit");\
throw;\
}\
}else{\
return ((FUN)(__VA_ARGS__));\
}\
}())
// Instrument functions with void return type
#define CV_INSTRUMENT_FUN_RV_META(TYPE, IMPL, FUN, ...) ([&]()\
{\
if(::cv::instr::useInstrumentation()){\
::cv::instr::IntrumentationRegion __instr__(#FUN, __FILE__, __LINE__, NULL, false, TYPE, IMPL);\
try{\
(FUN)(__VA_ARGS__);\
}catch(...){\
::cv::instr::getCurrentNode()->m_payload.m_funError = true;\
CV_INSTRUMENT_MARK_META(IMPL, #FUN "- BadExit");\
throw;\
}\
}else{\
(FUN)(__VA_ARGS__);\
}\
}())
// Instrumentation information marker
#define CV_INSTRUMENT_MARK_META(IMPL, NAME, ...) {::cv::instr::IntrumentationRegion __instr_mark__(NAME, __FILE__, __LINE__, NULL, false, ::cv::instr::TYPE_MARKER, IMPL);}
///// General instrumentation ///// General instrumentation
// General OpenCV region instrumentation macro // General OpenCV region instrumentation macro
@@ -769,10 +720,6 @@ CV_EXPORTS InstrNode* getCurrentNode();
///// IPP instrumentation ///// IPP instrumentation
// Wrapper region instrumentation macro // Wrapper region instrumentation macro
#define CV_INSTRUMENT_REGION_IPP(); CV_INSTRUMENT_REGION_META(__FUNCTION__, false, ::cv::instr::TYPE_WRAPPER, ::cv::instr::IMPL_IPP) #define CV_INSTRUMENT_REGION_IPP(); CV_INSTRUMENT_REGION_META(__FUNCTION__, false, ::cv::instr::TYPE_WRAPPER, ::cv::instr::IMPL_IPP)
// Function instrumentation macro
#define CV_INSTRUMENT_FUN_IPP(FUN, ...) CV_INSTRUMENT_FUN_RT_META(::cv::instr::TYPE_FUN, ::cv::instr::IMPL_IPP, instrStatus < 0, FUN, __VA_ARGS__)
// Diagnostic markers
#define CV_INSTRUMENT_MARK_IPP(NAME) CV_INSTRUMENT_MARK_META(::cv::instr::IMPL_IPP, NAME)
///// OpenCL instrumentation ///// OpenCL instrumentation
// Wrapper region instrumentation macro // Wrapper region instrumentation macro
@@ -0,0 +1,88 @@
// This file is part of OpenCV project.
// It is subject to the license terms in the LICENSE file found in the top-level directory
// of this distribution and at http://opencv.org/license.html.
// Copyright (C) 2026, Intel Corporation, all rights reserved.
#ifndef OPENCV_CORE_UTILS_INSTRUMENTATION_PRIVATE_HPP
#define OPENCV_CORE_UTILS_INSTRUMENTATION_PRIVATE_HPP
#include "opencv2/core/utils/instrumentation.hpp"
// Region-based function-instrumentation primitives.
//
// Only meaningful when the build was configured with ENABLE_INSTRUMENTATION;
// otherwise this header contributes nothing and each includer provides its own
// no-op fallbacks. The IntrumentationRegion / getCurrentNode() definitions are
// compiled into and exported (CV_EXPORTS) from the core library, so the HAL only
// needs these declarations and resolves the symbols at link time.
#ifdef ENABLE_INSTRUMENTATION
namespace cv { namespace instr {
// Scoped region: records one instrumentation node on construction and closes it
// on destruction. Defined in modules/core/src/system.cpp.
class CV_EXPORTS IntrumentationRegion
{
public:
IntrumentationRegion(const char* funName, const char* fileName, int lineNum, void *retAddress,
bool alwaysExpand, TYPE instrType = TYPE_GENERAL, IMPL implType = IMPL_PLAIN);
~IntrumentationRegion();
private:
bool m_disabled; // region status
uint64 m_regionTicks;
};
CV_EXPORTS InstrNode* getCurrentNode();
}} // namespace cv::instr
// Instrumentation information marker
#define CV_INSTRUMENT_MARK_META(IMPL, NAME, ...) {::cv::instr::IntrumentationRegion __instr_mark__(NAME, __FILE__, __LINE__, NULL, false, ::cv::instr::TYPE_MARKER, IMPL);}
// Instrument functions with non-void return type
#define CV_INSTRUMENT_FUN_RT_META(TYPE, IMPL, ERROR_COND, FUN, ...) ([&]() \
{ \
if(::cv::instr::useInstrumentation()){ \
::cv::instr::IntrumentationRegion __instr__(#FUN, __FILE__, __LINE__, NULL, false, TYPE, IMPL); \
try{ \
auto instrStatus = ((FUN)(__VA_ARGS__)); \
if(ERROR_COND){ \
::cv::instr::getCurrentNode()->m_payload.m_funError = true; \
CV_INSTRUMENT_MARK_META(IMPL, #FUN " - BadExit"); \
} \
return instrStatus; \
}catch(...){ \
::cv::instr::getCurrentNode()->m_payload.m_funError = true; \
CV_INSTRUMENT_MARK_META(IMPL, #FUN " - BadExit"); \
throw; \
} \
}else{ \
return ((FUN)(__VA_ARGS__)); \
} \
}())
// Instrument functions with void return type
#define CV_INSTRUMENT_FUN_RV_META(TYPE, IMPL, FUN, ...) ([&]() \
{ \
if(::cv::instr::useInstrumentation()){ \
::cv::instr::IntrumentationRegion __instr__(#FUN, __FILE__, __LINE__, NULL, false, TYPE, IMPL); \
try{ \
(FUN)(__VA_ARGS__); \
}catch(...){ \
::cv::instr::getCurrentNode()->m_payload.m_funError = true; \
CV_INSTRUMENT_MARK_META(IMPL, #FUN " - BadExit"); \
throw; \
} \
}else{ \
(FUN)(__VA_ARGS__); \
} \
}())
// IPP function instrumentation macros
#define CV_INSTRUMENT_FUN_IPP(FUN, ...) CV_INSTRUMENT_FUN_RT_META(::cv::instr::TYPE_FUN, ::cv::instr::IMPL_IPP, instrStatus < 0, FUN, __VA_ARGS__)
#define CV_INSTRUMENT_MARK_IPP(NAME) CV_INSTRUMENT_MARK_META(::cv::instr::IMPL_IPP, NAME)
#endif // ENABLE_INSTRUMENTATION
#endif // OPENCV_CORE_UTILS_INSTRUMENTATION_PRIVATE_HPP
@@ -6,9 +6,9 @@
#define OPENCV_VERSION_HPP #define OPENCV_VERSION_HPP
#define CV_VERSION_MAJOR 4 #define CV_VERSION_MAJOR 4
#define CV_VERSION_MINOR 14 #define CV_VERSION_MINOR 15
#define CV_VERSION_REVISION 0 #define CV_VERSION_REVISION 0
#define CV_VERSION_STATUS "-pre" #define CV_VERSION_STATUS "-dev"
#define CVAUX_STR_EXP(__A) #__A #define CVAUX_STR_EXP(__A) #__A
#define CVAUX_STR(__A) CVAUX_STR_EXP(__A) #define CVAUX_STR(__A) CVAUX_STR_EXP(__A)
+3
View File
@@ -1202,6 +1202,9 @@ typedef TestBaseWithParam<ReduceMinMaxParams> ReduceMinMaxFixture;
OCL_PERF_TEST_P(ReduceMinMaxFixture, Reduce, OCL_PERF_TEST_P(ReduceMinMaxFixture, Reduce,
::testing::Combine(OCL_TEST_SIZES, ::testing::Combine(OCL_TEST_SIZES,
OCL_PERF_ENUM(std::make_pair<MatType, MatType>(CV_8UC1, CV_8UC1), OCL_PERF_ENUM(std::make_pair<MatType, MatType>(CV_8UC1, CV_8UC1),
std::make_pair<MatType, MatType>(CV_8UC4, CV_8UC4),
std::make_pair<MatType, MatType>(CV_32FC1, CV_32FC1),
std::make_pair<MatType, MatType>(CV_32FC3, CV_32FC3),
std::make_pair<MatType, MatType>(CV_32FC4, CV_32FC4)), std::make_pair<MatType, MatType>(CV_32FC4, CV_32FC4)),
OCL_PERF_ENUM(0, 1), OCL_PERF_ENUM(0, 1),
ReduceMinMaxOp::all())) ReduceMinMaxOp::all()))
+24
View File
@@ -30,4 +30,28 @@ PERF_TEST_P( Size_Depth_Channels, split,
SANITY_CHECK(mv, 2e-5); SANITY_CHECK(mv, 2e-5);
} }
typedef tuple<Size, int> Size_Channel_t;
typedef perf::TestBaseWithParam<Size_Channel_t> Size_Channel;
PERF_TEST_P( Size_Channel, extractChannel,
testing::Combine
(
testing::Values(TYPICAL_MAT_SIZES),
testing::Values(0, 1)
)
)
{
Size sz = get<0>(GetParam());
int channel = get<1>(GetParam());
Mat src(sz, CV_8UC2);
Mat dst(sz, CV_8UC1);
randu(src, 0, 255);
int runs = (sz.width <= 640) ? 8 : 1;
TEST_CYCLE_MULTIRUN(runs) extractChannel(src, dst, channel);
SANITY_CHECK(dst);
}
} // namespace } // namespace
+38
View File
@@ -418,6 +418,35 @@ static bool ipp_insertChannel(const Mat &src, Mat &dst, int channel)
} }
#endif #endif
namespace cv
{
template<int channel> static void extractChannel8uC2_(const Mat& src, Mat& dst)
{
for (int row = 0; row < src.rows; ++row)
{
const uchar* source = src.ptr<uchar>(row);
uchar* destination = dst.ptr<uchar>(row);
int column = 0;
#if (CV_SIMD || CV_SIMD_SCALABLE)
const int lanes = VTraits<v_uint8>::vlanes();
for (; column <= src.cols - lanes; column += lanes)
{
v_uint8 channel0, channel1;
v_load_deinterleave(source + column * 2, channel0, channel1);
v_store(destination + column, channel == 0 ? channel0 : channel1);
}
#endif
for (; column < src.cols; ++column)
destination[column] = source[column * 2 + channel];
}
#if (CV_SIMD || CV_SIMD_SCALABLE)
vx_cleanup();
#endif
}
}
void cv::extractChannel(InputArray _src, OutputArray _dst, int coi) void cv::extractChannel(InputArray _src, OutputArray _dst, int coi)
{ {
CV_INSTRUMENT_REGION(); CV_INSTRUMENT_REGION();
@@ -443,6 +472,15 @@ void cv::extractChannel(InputArray _src, OutputArray _dst, int coi)
CV_IPP_RUN_FAST(ipp_extractChannel(src, dst, coi)) CV_IPP_RUN_FAST(ipp_extractChannel(src, dst, coi))
if (src.dims <= 2 && src.type() == CV_8UC2)
{
if (coi == 0)
extractChannel8uC2_<0>(src, dst);
else
extractChannel8uC2_<1>(src, dst);
return;
}
mixChannels(&src, 1, &dst, 1, ch, 1); mixChannels(&src, 1, &dst, 1, ch, 1);
} }
+25 -14
View File
@@ -338,9 +338,13 @@ cv::Mat cv::Mat::cross(InputArray _m) const
namespace cv namespace cv
{ {
typedef void (*ReduceSumFunc)(const Mat& src, Mat& dst); typedef void (*ReduceFunc)( const Mat& src, Mat& dst );
ReduceSumFunc getReduceCSumFunc(int sdepth, int ddepth); ReduceFunc getReduceCSumFunc(int sdepth, int ddepth);
ReduceSumFunc getReduceRSumFunc(int sdepth, int ddepth); ReduceFunc getReduceCAvgFunc(int sdepth, int ddepth);
ReduceFunc getReduceCMaxFunc(int sdepth, int ddepth);
ReduceFunc getReduceCMinFunc(int sdepth, int ddepth);
ReduceFunc getReduceCSum2Func(int sdepth, int ddepth);
ReduceFunc getReduceRSumFunc(int sdepth, int ddepth);
template <typename T, typename WT, typename Op> template <typename T, typename WT, typename Op>
struct ReduceR_SIMD struct ReduceR_SIMD
@@ -351,7 +355,6 @@ struct ReduceR_SIMD
} }
}; };
template<typename T, typename ST, typename WT, class Op, class OpInit> template<typename T, typename ST, typename WT, class Op, class OpInit>
class ReduceR_Invoker : public ParallelLoopBody class ReduceR_Invoker : public ParallelLoopBody
{ {
@@ -471,8 +474,6 @@ reduceC_( const Mat& srcmat, Mat& dstmat)
parallel_for_(Range(0, srcmat.size().height), body); parallel_for_(Range(0, srcmat.size().height), body);
} }
typedef void (*ReduceFunc)( const Mat& src, Mat& dst );
} }
#define reduceSumR8u32s reduceR_<uchar, int, OpAdd<int>, OpNop<int> > #define reduceSumR8u32s reduceR_<uchar, int, OpAdd<int>, OpNop<int> >
@@ -818,9 +819,9 @@ void cv::reduce(InputArray _src, OutputArray _dst, int dim, int op, int dtype)
{ {
if( op == REDUCE_SUM ) if( op == REDUCE_SUM )
{ {
ReduceSumFunc simd_func = getReduceRSumFunc(sdepth, ddepth); ReduceFunc simd_func = getReduceRSumFunc(sdepth, ddepth);
if(simd_func) if(simd_func)
func = (ReduceFunc)simd_func; func = simd_func;
else if(sdepth == CV_8U && ddepth == CV_32S) else if(sdepth == CV_8U && ddepth == CV_32S)
func = reduceSumR8u32s; func = reduceSumR8u32s;
else if(sdepth == CV_8U && ddepth == CV_32F) else if(sdepth == CV_8U && ddepth == CV_32F)
@@ -896,9 +897,11 @@ void cv::reduce(InputArray _src, OutputArray _dst, int dim, int op, int dtype)
{ {
if(op == REDUCE_SUM) if(op == REDUCE_SUM)
{ {
ReduceSumFunc simd_func = getReduceCSumFunc(sdepth, ddepth); ReduceFunc simd_func = op0 == REDUCE_AVG
? getReduceCAvgFunc(sdepth, ddepth)
: getReduceCSumFunc(sdepth, ddepth);
if(simd_func) if(simd_func)
func = (ReduceFunc)simd_func; func = simd_func;
else if(sdepth == CV_8U && ddepth == CV_32S) else if(sdepth == CV_8U && ddepth == CV_32S)
func = reduceSumC8u32s; func = reduceSumC8u32s;
else if(sdepth == CV_8U && ddepth == CV_32F) else if(sdepth == CV_8U && ddepth == CV_32F)
@@ -922,7 +925,10 @@ void cv::reduce(InputArray _src, OutputArray _dst, int dim, int op, int dtype)
} }
else if(op == REDUCE_MAX) else if(op == REDUCE_MAX)
{ {
if(sdepth == CV_8U && ddepth == CV_8U) ReduceFunc simd_func = getReduceCMaxFunc(sdepth, ddepth);
if(simd_func)
func = simd_func;
else if(sdepth == CV_8U && ddepth == CV_8U)
func = reduceMaxC8u; func = reduceMaxC8u;
else if(sdepth == CV_16U && ddepth == CV_16U) else if(sdepth == CV_16U && ddepth == CV_16U)
func = reduceMaxC16u; func = reduceMaxC16u;
@@ -935,7 +941,10 @@ void cv::reduce(InputArray _src, OutputArray _dst, int dim, int op, int dtype)
} }
else if(op == REDUCE_MIN) else if(op == REDUCE_MIN)
{ {
if(sdepth == CV_8U && ddepth == CV_8U) ReduceFunc simd_func = getReduceCMinFunc(sdepth, ddepth);
if(simd_func)
func = simd_func;
else if(sdepth == CV_8U && ddepth == CV_8U)
func = reduceMinC8u; func = reduceMinC8u;
else if(sdepth == CV_16U && ddepth == CV_16U) else if(sdepth == CV_16U && ddepth == CV_16U)
func = reduceMinC16u; func = reduceMinC16u;
@@ -948,7 +957,10 @@ void cv::reduce(InputArray _src, OutputArray _dst, int dim, int op, int dtype)
} }
else if(op == REDUCE_SUM2) else if(op == REDUCE_SUM2)
{ {
if(sdepth == CV_8U && ddepth == CV_32S) ReduceFunc simd_func = getReduceCSum2Func(sdepth, ddepth);
if(simd_func)
func = simd_func;
else if(sdepth == CV_8U && ddepth == CV_32S)
func = reduceSum2C8u32s; func = reduceSum2C8u32s;
else if(sdepth == CV_8U && ddepth == CV_32F) else if(sdepth == CV_8U && ddepth == CV_32F)
func = reduceSum2C8u32f; func = reduceSum2C8u32f;
@@ -1038,7 +1050,6 @@ template<typename T> static void sort_( const Mat& src, Mat& dst, int flags )
}); });
} }
#if defined(HAVE_IPP) && !IPP_DISABLE_SORT #if defined(HAVE_IPP) && !IPP_DISABLE_SORT
typedef IppStatus (CV_STDCALL *IppSortFunc)(void *pSrcDst, int len, Ipp8u *pBuffer); typedef IppStatus (CV_STDCALL *IppSortFunc)(void *pSrcDst, int len, Ipp8u *pBuffer);
+37 -5
View File
@@ -9,18 +9,50 @@
namespace cv { namespace cv {
typedef void (*ReduceSumFunc)(const Mat& src, Mat& dst); typedef void (*ReduceFunc)( const Mat& src, Mat& dst );
ReduceSumFunc getReduceCSumFunc(int sdepth, int ddepth); ReduceFunc getReduceCSumFunc(int sdepth, int ddepth);
ReduceSumFunc getReduceRSumFunc(int sdepth, int ddepth); ReduceFunc getReduceCAvgFunc(int sdepth, int ddepth);
ReduceFunc getReduceCMaxFunc(int sdepth, int ddepth);
ReduceFunc getReduceCMinFunc(int sdepth, int ddepth);
ReduceFunc getReduceCSum2Func(int sdepth, int ddepth);
ReduceFunc getReduceRSumFunc(int sdepth, int ddepth);
ReduceSumFunc getReduceCSumFunc(int sdepth, int ddepth) ReduceFunc getReduceCSumFunc(int sdepth, int ddepth)
{ {
CV_INSTRUMENT_REGION(); CV_INSTRUMENT_REGION();
CV_CPU_DISPATCH(getReduceCSumFunc, (sdepth, ddepth), CV_CPU_DISPATCH(getReduceCSumFunc, (sdepth, ddepth),
CV_CPU_DISPATCH_MODES_ALL); CV_CPU_DISPATCH_MODES_ALL);
} }
ReduceSumFunc getReduceRSumFunc(int sdepth, int ddepth) ReduceFunc getReduceCAvgFunc(int sdepth, int ddepth)
{
CV_INSTRUMENT_REGION();
CV_CPU_DISPATCH(getReduceCAvgFunc, (sdepth, ddepth),
CV_CPU_DISPATCH_MODES_ALL);
}
ReduceFunc getReduceCMaxFunc(int sdepth, int ddepth)
{
CV_INSTRUMENT_REGION();
CV_CPU_DISPATCH(getReduceCMaxFunc, (sdepth, ddepth),
CV_CPU_DISPATCH_MODES_ALL);
}
ReduceFunc getReduceCMinFunc(int sdepth, int ddepth)
{
CV_INSTRUMENT_REGION();
CV_CPU_DISPATCH(getReduceCMinFunc, (sdepth, ddepth),
CV_CPU_DISPATCH_MODES_ALL);
}
ReduceFunc getReduceCSum2Func(int sdepth, int ddepth)
{
CV_INSTRUMENT_REGION();
CV_CPU_DISPATCH(getReduceCSum2Func, (sdepth, ddepth),
CV_CPU_DISPATCH_MODES_ALL);
}
ReduceFunc getReduceRSumFunc(int sdepth, int ddepth)
{ {
CV_INSTRUMENT_REGION(); CV_INSTRUMENT_REGION();
CV_CPU_DISPATCH(getReduceRSumFunc, (sdepth, ddepth), CV_CPU_DISPATCH(getReduceRSumFunc, (sdepth, ddepth),
+74 -5
View File
@@ -7,12 +7,20 @@
namespace cv { namespace cv {
CV_CPU_OPTIMIZATION_NAMESPACE_BEGIN CV_CPU_OPTIMIZATION_NAMESPACE_BEGIN
typedef void (*ReduceSumFunc)(const Mat& src, Mat& dst); typedef void (*ReduceFunc)( const Mat& src, Mat& dst );
ReduceSumFunc getReduceCSumFunc(int sdepth, int ddepth); ReduceFunc getReduceCSumFunc(int sdepth, int ddepth);
ReduceSumFunc getReduceRSumFunc(int sdepth, int ddepth); ReduceFunc getReduceCAvgFunc(int sdepth, int ddepth);
ReduceFunc getReduceCMaxFunc(int sdepth, int ddepth);
ReduceFunc getReduceCMinFunc(int sdepth, int ddepth);
ReduceFunc getReduceCSum2Func(int sdepth, int ddepth);
ReduceFunc getReduceRSumFunc(int sdepth, int ddepth);
#ifndef CV_CPU_OPTIMIZATION_DECLARATIONS_ONLY #ifndef CV_CPU_OPTIMIZATION_DECLARATIONS_ONLY
#if (CV_SIMD || CV_SIMD_SCALABLE)
#include "reduce_c.simd.hpp"
#endif
// ===================================================================== // =====================================================================
// Col reduce SUM (dim=1): sum each row into cn output values // Col reduce SUM (dim=1): sum each row into cn output values
// ===================================================================== // =====================================================================
@@ -1089,7 +1097,7 @@ static void reduceRowSum_64f64f(const Mat& srcmat, Mat& dstmat)
// Dispatchers // Dispatchers
// ===================================================================== // =====================================================================
ReduceSumFunc getReduceCSumFunc(int sdepth, int ddepth) ReduceFunc getReduceCSumFunc(int sdepth, int ddepth)
{ {
#if (CV_SIMD || CV_SIMD_SCALABLE) #if (CV_SIMD || CV_SIMD_SCALABLE)
if (sdepth == CV_8U && ddepth == CV_32S) return reduceColSum_8u32s; if (sdepth == CV_8U && ddepth == CV_32S) return reduceColSum_8u32s;
@@ -1108,7 +1116,68 @@ ReduceSumFunc getReduceCSumFunc(int sdepth, int ddepth)
return nullptr; return nullptr;
} }
ReduceSumFunc getReduceRSumFunc(int sdepth, int ddepth) ReduceFunc getReduceCAvgFunc(int sdepth, int ddepth)
{
return getReduceCSumFunc(sdepth, ddepth);
}
ReduceFunc getReduceCMaxFunc(int sdepth, int ddepth)
{
#if (CV_SIMD || CV_SIMD_SCALABLE)
if (sdepth == CV_8U && ddepth == CV_8U) return reduceColMax_8u;
if (sdepth == CV_16U && ddepth == CV_16U) return reduceColMax_16u;
if (sdepth == CV_16S && ddepth == CV_16S) return reduceColMax_16s;
if (sdepth == CV_32F && ddepth == CV_32F) return reduceColMax_32f;
#if (CV_SIMD_64F || CV_SIMD_SCALABLE_64F)
if (sdepth == CV_64F && ddepth == CV_64F) return reduceColMax_64f;
#endif
#else
CV_UNUSED(sdepth);
CV_UNUSED(ddepth);
#endif
return nullptr;
}
ReduceFunc getReduceCMinFunc(int sdepth, int ddepth)
{
#if (CV_SIMD || CV_SIMD_SCALABLE)
if (sdepth == CV_8U && ddepth == CV_8U) return reduceColMin_8u;
if (sdepth == CV_16U && ddepth == CV_16U) return reduceColMin_16u;
if (sdepth == CV_16S && ddepth == CV_16S) return reduceColMin_16s;
if (sdepth == CV_32F && ddepth == CV_32F) return reduceColMin_32f;
#if (CV_SIMD_64F || CV_SIMD_SCALABLE_64F)
if (sdepth == CV_64F && ddepth == CV_64F) return reduceColMin_64f;
#endif
#else
CV_UNUSED(sdepth);
CV_UNUSED(ddepth);
#endif
return nullptr;
}
ReduceFunc getReduceCSum2Func(int sdepth, int ddepth)
{
#if (CV_SIMD || CV_SIMD_SCALABLE)
if (sdepth == CV_8U && ddepth == CV_32S) return reduceColSum2_8u32s;
if (sdepth == CV_8U && ddepth == CV_32F) return reduceColSum2_8u32f;
if (sdepth == CV_8U && ddepth == CV_64F) return reduceColSum2_8u64f;
if (sdepth == CV_16U && ddepth == CV_32F) return reduceColSum2_16u32f;
if (sdepth == CV_16S && ddepth == CV_32F) return reduceColSum2_16s32f;
if (sdepth == CV_32F && ddepth == CV_32F) return reduceColSum2_32f32f;
#if (CV_SIMD_64F || CV_SIMD_SCALABLE_64F)
if (sdepth == CV_16U && ddepth == CV_64F) return reduceColSum2_16u64f;
if (sdepth == CV_16S && ddepth == CV_64F) return reduceColSum2_16s64f;
if (sdepth == CV_32F && ddepth == CV_64F) return reduceColSum2_32f64f;
if (sdepth == CV_64F && ddepth == CV_64F) return reduceColSum2_64f64f;
#endif
#else
CV_UNUSED(sdepth);
CV_UNUSED(ddepth);
#endif
return nullptr;
}
ReduceFunc getReduceRSumFunc(int sdepth, int ddepth)
{ {
#if (CV_SIMD || CV_SIMD_SCALABLE) #if (CV_SIMD || CV_SIMD_SCALABLE)
if (sdepth == CV_8U && ddepth == CV_32S) return reduceRowSum_8u32s; if (sdepth == CV_8U && ddepth == CV_32S) return reduceRowSum_8u32s;
+539
View File
@@ -0,0 +1,539 @@
// This file is part of OpenCV project.
// It is subject to the license terms in the LICENSE file found in the top-level directory
// of this distribution and at http://opencv.org/license.html
#include "reduce_c_generic.hpp"
#if CV_RVV
#include "reduce_c_rvv.hpp"
#endif
#if CV_NEON
#include "reduce_c_neon.hpp"
#endif
#if CV_AVX2
#include "reduce_c_avx2.hpp"
#endif
template<bool isMax>
static void reduceColMinMax_8uC1(const Mat& srcmat, Mat& dstmat)
{
#if CV_RVV
reduce_c_rvv::minMax8uC1<isMax>(srcmat, dstmat);
#elif CV_NEON
reduce_c_neon::minMax8uC1<isMax>(srcmat, dstmat);
#elif CV_AVX2
reduce_c_avx2::minMax8uC1<isMax>(srcmat, dstmat);
#else
reduceColMinMax_8uFallback<isMax>(srcmat, dstmat);
#endif
}
template<bool isMax>
static void reduceColMinMax_8uC3(const Mat& srcmat, Mat& dstmat)
{
#if CV_RVV
reduce_c_rvv::minMax8uC3<isMax>(srcmat, dstmat);
#elif CV_NEON
reduce_c_neon::minMax8uC3<isMax>(srcmat, dstmat);
#elif CV_AVX2
reduce_c_avx2::minMax8uC3<isMax>(srcmat, dstmat);
#else
reduceColMinMax_8uFallback<isMax>(srcmat, dstmat);
#endif
}
template<bool isMax>
static void reduceColMinMax_8uC4(const Mat& srcmat, Mat& dstmat)
{
#if CV_RVV
reduce_c_rvv::minMax8uC4<isMax>(srcmat, dstmat);
#elif CV_NEON
reduce_c_neon::minMax8uC4<isMax>(srcmat, dstmat);
#elif CV_AVX2
reduce_c_avx2::minMax8uC4<isMax>(srcmat, dstmat);
#else
reduceColMinMax_8uFallback<isMax>(srcmat, dstmat);
#endif
}
template<bool isMax>
static void reduceColMinMax_8u(const Mat& srcmat, Mat& dstmat)
{
const int cn = srcmat.channels();
if (cn == 1)
reduceColMinMax_8uC1<isMax>(srcmat, dstmat);
else if (cn == 3)
reduceColMinMax_8uC3<isMax>(srcmat, dstmat);
else if (cn == 4)
reduceColMinMax_8uC4<isMax>(srcmat, dstmat);
else
reduceColMinMax_8uFallback<isMax>(srcmat, dstmat);
}
static void reduceColMax_8u(const Mat& srcmat, Mat& dstmat)
{
reduceColMinMax_8u<true>(srcmat, dstmat);
}
static void reduceColMin_8u(const Mat& srcmat, Mat& dstmat)
{
reduceColMinMax_8u<false>(srcmat, dstmat);
}
template<bool isMax>
static void reduceColMinMax_16uC1(const Mat& srcmat, Mat& dstmat)
{
#if CV_RVV
reduce_c_rvv::minMax16uC1<isMax>(srcmat, dstmat);
#elif CV_NEON
reduce_c_neon::minMax16uC1<isMax>(srcmat, dstmat);
#elif CV_AVX2
reduce_c_avx2::minMax16uC1<isMax>(srcmat, dstmat);
#else
reduceColMinMax_16uFallback<isMax>(srcmat, dstmat);
#endif
}
template<bool isMax>
static void reduceColMinMax_16uC4(const Mat& srcmat, Mat& dstmat)
{
#if CV_RVV
reduce_c_rvv::minMax16uC4<isMax>(srcmat, dstmat);
#elif CV_NEON
reduce_c_neon::minMax16uC4<isMax>(srcmat, dstmat);
#elif CV_AVX2
reduce_c_avx2::minMax16uC4<isMax>(srcmat, dstmat);
#else
reduceColMinMax_16uFallback<isMax>(srcmat, dstmat);
#endif
}
template<bool isMax>
static void reduceColMinMax_16uC3(const Mat& srcmat, Mat& dstmat)
{
#if CV_RVV
reduce_c_rvv::minMax16uC3<isMax>(srcmat, dstmat);
#elif CV_NEON
reduce_c_neon::minMax16uC3<isMax>(srcmat, dstmat);
#elif CV_AVX2
reduce_c_avx2::minMax16uC3<isMax>(srcmat, dstmat);
#else
reduceColMinMax_16uFallback<isMax>(srcmat, dstmat);
#endif
}
template<bool isMax>
static void reduceColMinMax_16u(const Mat& srcmat, Mat& dstmat)
{
if (srcmat.channels() == 1)
reduceColMinMax_16uC1<isMax>(srcmat, dstmat);
else if (srcmat.channels() == 3)
reduceColMinMax_16uC3<isMax>(srcmat, dstmat);
else if (srcmat.channels() == 4)
reduceColMinMax_16uC4<isMax>(srcmat, dstmat);
else
reduceColMinMax_16uFallback<isMax>(srcmat, dstmat);
}
static void reduceColMax_16u(const Mat& srcmat, Mat& dstmat)
{
reduceColMinMax_16u<true>(srcmat, dstmat);
}
static void reduceColMin_16u(const Mat& srcmat, Mat& dstmat)
{
reduceColMinMax_16u<false>(srcmat, dstmat);
}
template<bool isMax>
static void reduceColMinMax_16sC1(const Mat& srcmat, Mat& dstmat)
{
#if CV_RVV
reduce_c_rvv::minMax16sC1<isMax>(srcmat, dstmat);
#elif CV_NEON
reduce_c_neon::minMax16sC1<isMax>(srcmat, dstmat);
#elif CV_AVX2
reduce_c_avx2::minMax16sC1<isMax>(srcmat, dstmat);
#else
reduceColMinMax_16sFallback<isMax>(srcmat, dstmat);
#endif
}
template<bool isMax>
static void reduceColMinMax_16sC4(const Mat& srcmat, Mat& dstmat)
{
#if CV_RVV
reduce_c_rvv::minMax16sC4<isMax>(srcmat, dstmat);
#elif CV_NEON
reduce_c_neon::minMax16sC4<isMax>(srcmat, dstmat);
#elif CV_AVX2
reduce_c_avx2::minMax16sC4<isMax>(srcmat, dstmat);
#else
reduceColMinMax_16sFallback<isMax>(srcmat, dstmat);
#endif
}
template<bool isMax>
static void reduceColMinMax_16sC3(const Mat& srcmat, Mat& dstmat)
{
#if CV_RVV
reduce_c_rvv::minMax16sC3<isMax>(srcmat, dstmat);
#elif CV_NEON
reduce_c_neon::minMax16sC3<isMax>(srcmat, dstmat);
#elif CV_AVX2
reduce_c_avx2::minMax16sC3<isMax>(srcmat, dstmat);
#else
reduceColMinMax_16sFallback<isMax>(srcmat, dstmat);
#endif
}
template<bool isMax>
static void reduceColMinMax_16s(const Mat& srcmat, Mat& dstmat)
{
if (srcmat.channels() == 1)
reduceColMinMax_16sC1<isMax>(srcmat, dstmat);
else if (srcmat.channels() == 3)
reduceColMinMax_16sC3<isMax>(srcmat, dstmat);
else if (srcmat.channels() == 4)
reduceColMinMax_16sC4<isMax>(srcmat, dstmat);
else
reduceColMinMax_16sFallback<isMax>(srcmat, dstmat);
}
static void reduceColMax_16s(const Mat& srcmat, Mat& dstmat)
{
reduceColMinMax_16s<true>(srcmat, dstmat);
}
static void reduceColMin_16s(const Mat& srcmat, Mat& dstmat)
{
reduceColMinMax_16s<false>(srcmat, dstmat);
}
template<bool isMax>
static void reduceColMinMax_32fC1(const Mat& srcmat, Mat& dstmat)
{
#if CV_RVV
reduce_c_rvv::minMax32fC1<isMax>(srcmat, dstmat);
#elif CV_NEON
reduce_c_neon::minMax32fC1<isMax>(srcmat, dstmat);
#elif CV_AVX2
reduce_c_avx2::minMax32fC1<isMax>(srcmat, dstmat);
#else
reduceColMinMax_32fFallback<isMax>(srcmat, dstmat);
#endif
}
template<bool isMax>
static void reduceColMinMax_32fC3(const Mat& srcmat, Mat& dstmat)
{
#if CV_RVV
reduce_c_rvv::minMax32fC3<isMax>(srcmat, dstmat);
#else
reduceColMinMax_32fFallback<isMax>(srcmat, dstmat);
#endif
}
template<bool isMax>
static void reduceColMinMax_32fC4(const Mat& srcmat, Mat& dstmat)
{
#if CV_RVV
reduce_c_rvv::minMax32fC4<isMax>(srcmat, dstmat);
#elif CV_NEON
reduce_c_neon::minMax32fC4<isMax>(srcmat, dstmat);
#elif CV_AVX2
reduce_c_avx2::minMax32fC4<isMax>(srcmat, dstmat);
#else
reduceColMinMax_32fFallback<isMax>(srcmat, dstmat);
#endif
}
template<bool isMax>
static void reduceColMinMax_32f(const Mat& srcmat, Mat& dstmat)
{
const int cn = srcmat.channels();
if (cn == 1)
reduceColMinMax_32fC1<isMax>(srcmat, dstmat);
else if (cn == 3)
reduceColMinMax_32fC3<isMax>(srcmat, dstmat);
else if (cn == 4)
reduceColMinMax_32fC4<isMax>(srcmat, dstmat);
else
reduceColMinMax_32fFallback<isMax>(srcmat, dstmat);
}
static void reduceColMax_32f(const Mat& srcmat, Mat& dstmat)
{
reduceColMinMax_32f<true>(srcmat, dstmat);
}
static void reduceColMin_32f(const Mat& srcmat, Mat& dstmat)
{
reduceColMinMax_32f<false>(srcmat, dstmat);
}
#if (CV_SIMD_64F || CV_SIMD_SCALABLE_64F)
template<bool isMax>
static void reduceColMinMax_64fC1(const Mat& srcmat, Mat& dstmat)
{
#if CV_RVV && CV_SIMD_SCALABLE_64F
reduce_c_rvv::minMax64fC1<isMax>(srcmat, dstmat);
#elif CV_NEON && (defined(__aarch64__) || defined(_M_ARM64))
reduce_c_neon::minMax64fC1<isMax>(srcmat, dstmat);
#elif CV_AVX2
reduce_c_avx2::minMax64fC1<isMax>(srcmat, dstmat);
#else
reduceColMinMax_64fFallback<isMax>(srcmat, dstmat);
#endif
}
template<bool isMax>
static void reduceColMinMax_64f(const Mat& srcmat, Mat& dstmat)
{
if (srcmat.channels() == 1)
reduceColMinMax_64fC1<isMax>(srcmat, dstmat);
else
reduceColMinMax_64fFallback<isMax>(srcmat, dstmat);
}
static void reduceColMax_64f(const Mat& srcmat, Mat& dstmat)
{
reduceColMinMax_64f<true>(srcmat, dstmat);
}
static void reduceColMin_64f(const Mat& srcmat, Mat& dstmat)
{
reduceColMinMax_64f<false>(srcmat, dstmat);
}
#endif
template<typename DT>
static void reduceColSum2_8uC1(const Mat& srcmat, Mat& dstmat)
{
#if CV_RVV
reduce_c_rvv::sum2_8uC1<DT>(srcmat, dstmat);
#elif CV_NEON && (defined(__aarch64__) || defined(_M_ARM64))
reduce_c_neon::sum2_8uC1<DT>(srcmat, dstmat);
#elif CV_AVX2
reduce_c_avx2::sum2_8uC1<DT>(srcmat, dstmat);
#else
reduceColSum2_8uFallback<DT>(srcmat, dstmat);
#endif
}
template<typename DT>
static void reduceColSum2_8uC3(const Mat& srcmat, Mat& dstmat)
{
#if CV_RVV
reduce_c_rvv::sum2_8uC3<DT>(srcmat, dstmat);
#elif CV_NEON && (defined(__aarch64__) || defined(_M_ARM64))
reduce_c_neon::sum2_8uC3<DT>(srcmat, dstmat);
#elif CV_AVX2
reduce_c_avx2::sum2_8uC3<DT>(srcmat, dstmat);
#else
reduceColSum2_8uFallback<DT>(srcmat, dstmat);
#endif
}
template<typename DT>
static void reduceColSum2_8uC4(const Mat& srcmat, Mat& dstmat)
{
#if CV_RVV
reduce_c_rvv::sum2_8uC4<DT>(srcmat, dstmat);
#elif CV_NEON && (defined(__aarch64__) || defined(_M_ARM64))
reduce_c_neon::sum2_8uC4<DT>(srcmat, dstmat);
#elif CV_AVX2
reduce_c_avx2::sum2_8uC4<DT>(srcmat, dstmat);
#else
reduceColSum2_8uFallback<DT>(srcmat, dstmat);
#endif
}
template<typename DT>
static void reduceColSum2_8u(const Mat& srcmat, Mat& dstmat)
{
const int cn = srcmat.channels();
if (cn == 1)
reduceColSum2_8uC1<DT>(srcmat, dstmat);
else if (cn == 3)
reduceColSum2_8uC3<DT>(srcmat, dstmat);
else if (cn == 4)
reduceColSum2_8uC4<DT>(srcmat, dstmat);
else
reduceColSum2_8uFallback<DT>(srcmat, dstmat);
}
static void reduceColSum2_8u32s(const Mat& srcmat, Mat& dstmat)
{
reduceColSum2_8u<int>(srcmat, dstmat);
}
static void reduceColSum2_8u32f(const Mat& srcmat, Mat& dstmat)
{
reduceColSum2_8u<float>(srcmat, dstmat);
}
static void reduceColSum2_8u64f(const Mat& srcmat, Mat& dstmat)
{
reduceColSum2_8u<double>(srcmat, dstmat);
}
static void reduceColSum2_16u32f(const Mat& srcmat, Mat& dstmat)
{
if (srcmat.channels() != 1)
{
reduceColSum2_16u32fFallback(srcmat, dstmat);
return;
}
#if CV_RVV
reduce_c_rvv::sum2_16u32fC1(srcmat, dstmat);
#elif CV_NEON && (defined(__aarch64__) || defined(_M_ARM64))
reduce_c_neon::sum2_16u32fC1(srcmat, dstmat);
#elif CV_AVX2
reduce_c_avx2::sum2_16u32fC1(srcmat, dstmat);
#else
reduceColSum2_16u32fFallback(srcmat, dstmat);
#endif
}
static void reduceColSum2_16s32f(const Mat& srcmat, Mat& dstmat)
{
if (srcmat.channels() != 1)
{
reduceColSum2_16s32fFallback(srcmat, dstmat);
return;
}
#if CV_RVV
reduce_c_rvv::sum2_16s32fC1(srcmat, dstmat);
#elif CV_NEON && (defined(__aarch64__) || defined(_M_ARM64))
reduce_c_neon::sum2_16s32fC1(srcmat, dstmat);
#elif CV_AVX2
reduce_c_avx2::sum2_16s32fC1(srcmat, dstmat);
#else
reduceColSum2_16s32fFallback(srcmat, dstmat);
#endif
}
static void reduceColSum2_32f32fC1(const Mat& srcmat, Mat& dstmat)
{
#if CV_RVV
reduce_c_rvv::sum2_32fC1(srcmat, dstmat);
#elif CV_NEON
reduce_c_neon::sum2_32fC1(srcmat, dstmat);
#elif CV_AVX2
reduce_c_avx2::sum2_32fC1(srcmat, dstmat);
#else
reduceColSum2_32f32fFallback(srcmat, dstmat);
#endif
}
static void reduceColSum2_32f32fC3(const Mat& srcmat, Mat& dstmat)
{
#if CV_RVV
reduce_c_rvv::sum2_32fC3(srcmat, dstmat);
#else
reduceColSum2_32f32fFallback(srcmat, dstmat);
#endif
}
static void reduceColSum2_32f32fC4(const Mat& srcmat, Mat& dstmat)
{
#if CV_RVV
reduce_c_rvv::sum2_32fC4(srcmat, dstmat);
#elif CV_NEON
reduce_c_neon::sum2_32fC4(srcmat, dstmat);
#elif CV_AVX2
reduce_c_avx2::sum2_32fC4(srcmat, dstmat);
#else
reduceColSum2_32f32fFallback(srcmat, dstmat);
#endif
}
static void reduceColSum2_32f32f(const Mat& srcmat, Mat& dstmat)
{
const int cn = srcmat.channels();
if (cn == 1)
reduceColSum2_32f32fC1(srcmat, dstmat);
else if (cn == 3)
reduceColSum2_32f32fC3(srcmat, dstmat);
else if (cn == 4)
reduceColSum2_32f32fC4(srcmat, dstmat);
else
reduceColSum2_32f32fFallback(srcmat, dstmat);
}
#if (CV_SIMD_64F || CV_SIMD_SCALABLE_64F)
static void reduceColSum2_16u64f(const Mat& srcmat, Mat& dstmat)
{
if (srcmat.channels() != 1)
{
reduceColSum2_16u64fFallback(srcmat, dstmat);
return;
}
#if CV_RVV && CV_SIMD_SCALABLE_64F
reduce_c_rvv::sum2_16u64fC1(srcmat, dstmat);
#elif CV_NEON && (defined(__aarch64__) || defined(_M_ARM64))
reduce_c_neon::sum2_16u64fC1(srcmat, dstmat);
#elif CV_AVX2
reduce_c_avx2::sum2_16u64fC1(srcmat, dstmat);
#else
reduceColSum2_16u64fFallback(srcmat, dstmat);
#endif
}
static void reduceColSum2_16s64f(const Mat& srcmat, Mat& dstmat)
{
if (srcmat.channels() != 1)
{
reduceColSum2_16s64fFallback(srcmat, dstmat);
return;
}
#if CV_RVV && CV_SIMD_SCALABLE_64F
reduce_c_rvv::sum2_16s64fC1(srcmat, dstmat);
#elif CV_NEON && (defined(__aarch64__) || defined(_M_ARM64))
reduce_c_neon::sum2_16s64fC1(srcmat, dstmat);
#elif CV_AVX2
reduce_c_avx2::sum2_16s64fC1(srcmat, dstmat);
#else
reduceColSum2_16s64fFallback(srcmat, dstmat);
#endif
}
static void reduceColSum2_32f64f(const Mat& srcmat, Mat& dstmat)
{
if (srcmat.channels() != 1)
{
reduceColSum2_32f64fFallback(srcmat, dstmat);
return;
}
#if CV_RVV && CV_SIMD_SCALABLE_64F
reduce_c_rvv::sum2_32f64fC1(srcmat, dstmat);
#elif CV_NEON && (defined(__aarch64__) || defined(_M_ARM64))
reduce_c_neon::sum2_32f64fC1(srcmat, dstmat);
#elif CV_AVX2
reduce_c_avx2::sum2_32f64fC1(srcmat, dstmat);
#else
reduceColSum2_32f64fFallback(srcmat, dstmat);
#endif
}
static void reduceColSum2_64f64f(const Mat& srcmat, Mat& dstmat)
{
if (srcmat.channels() != 1)
{
reduceColSum2_64f64fFallback(srcmat, dstmat);
return;
}
#if CV_RVV && CV_SIMD_SCALABLE_64F
reduce_c_rvv::sum2_64f64fC1(srcmat, dstmat);
#elif CV_NEON && (defined(__aarch64__) || defined(_M_ARM64))
reduce_c_neon::sum2_64f64fC1(srcmat, dstmat);
#elif CV_AVX2
reduce_c_avx2::sum2_64f64fC1(srcmat, dstmat);
#else
reduceColSum2_64f64fFallback(srcmat, dstmat);
#endif
}
#endif
+989
View File
@@ -0,0 +1,989 @@
// This file is part of OpenCV project.
// It is subject to the license terms in the LICENSE file found in the top-level directory
// of this distribution and at http://opencv.org/license.html
namespace reduce_c_avx2
{
// Optimized ReduceC support in this backend:
//
// | Input -> output type/channel | SUM | AVG | MIN | MAX | SUM2 |
// |------------------------------------|:---:|:---:|:---:|:---:|:----:|
// | 8UC1/C3/C4 -> 8UC1/C3/C4 | - | - | x | x | - |
// | 8UC1/C3/C4 -> 32SC1/C3/C4 | x | x | - | - | x |
// | 8UC1/C3/C4 -> 32FC1/C3/C4 | x | x | - | - | x |
// | 8UC1/C3/C4 -> 64FC1/C3/C4 | - | - | - | - | x |
// | 16UC1/C3/C4 -> 16UC1/C3/C4 | - | - | x | x | - |
// | 16UC1 -> 32FC1 | x | x | - | - | x |
// | 16UC3/C4 -> 32FC3/C4 | x | x | - | - | - |
// | 16UC1 -> 64FC1 | - | - | - | - | x |
// | 16UC3/C4 -> 64FC3/C4 | - | - | - | - | - |
// | 16SC1/C3/C4 -> 16SC1/C3/C4 | - | - | x | x | - |
// | 16SC1 -> 32FC1 | x | x | - | - | x |
// | 16SC3/C4 -> 32FC3/C4 | x | x | - | - | - |
// | 16SC1 -> 64FC1 | - | - | - | - | x |
// | 16SC3/C4 -> 64FC3/C4 | - | - | - | - | - |
// | 32FC1 -> 32FC1 | x | x | x | x | x |
// | 32FC3 -> 32FC3 | x | x | - | - | - |
// | 32FC4 -> 32FC4 | x | x | x | x | x |
// | 32FC1 -> 64FC1 | x | x | - | - | x |
// | 32FC3/C4 -> 64FC3/C4 | x | x | - | - | - |
// | 64FC1 -> 64FC1 | x | x | x | x | x |
// | 64FC3/C4 -> 64FC3/C4 | x | x | - | - | - |
//
// 'x' in SUM/AVG denotes the existing shared universal-intrinsics kernel; 'x'
// in MIN/MAX/SUM2 denotes a native AVX2 kernel. For legal MIN/MAX/SUM2
// combinations marked '-', and for other channel counts, dispatch uses the
// shared generic fallback.
template<bool isMax>
static void minMax8uC1(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const uchar* src = srcmat.ptr<uchar>(y);
uchar* dst = dstmat.ptr<uchar>(y);
uchar result = isMax ? 0 : UCHAR_MAX;
int x = 0;
__m256i acc = _mm256_set1_epi8((char)result);
for (; x <= cols - 32; x += 32)
{
__m256i v = _mm256_loadu_si256((const __m256i*)(src + x));
acc = isMax ? _mm256_max_epu8(acc, v) : _mm256_min_epu8(acc, v);
}
uchar lanes[32];
_mm256_storeu_si256((__m256i*)lanes, acc);
for (int i = 0; i < 32; i++)
result = reduceScalarMinMax<isMax>(result, lanes[i]);
for (; x < cols; x++)
result = reduceScalarMinMax<isMax>(result, src[x]);
dst[0] = result;
}
});
v_cleanup();
}
template<bool isMax>
static void minMax16uC1(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const ushort* src = srcmat.ptr<ushort>(y);
ushort result = isMax ? 0 : USHRT_MAX;
int x = 0;
__m256i acc = _mm256_set1_epi16((short)result);
for (; x <= cols - 16; x += 16)
{
__m256i v = _mm256_loadu_si256((const __m256i*)(src + x));
acc = isMax ? _mm256_max_epu16(acc, v) : _mm256_min_epu16(acc, v);
}
ushort lanes[16];
_mm256_storeu_si256((__m256i*)lanes, acc);
for (int i = 0; i < 16; i++)
result = isMax ? std::max(result, lanes[i]) : std::min(result, lanes[i]);
for (; x < cols; x++)
result = isMax ? std::max(result, src[x]) : std::min(result, src[x]);
dstmat.ptr<ushort>(y)[0] = result;
}
});
v_cleanup();
}
template<bool isMax>
static void minMax16sC1(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const short* src = srcmat.ptr<short>(y);
short result = isMax ? SHRT_MIN : SHRT_MAX;
int x = 0;
__m256i acc = _mm256_set1_epi16(result);
for (; x <= cols - 16; x += 16)
{
__m256i v = _mm256_loadu_si256((const __m256i*)(src + x));
acc = isMax ? _mm256_max_epi16(acc, v) : _mm256_min_epi16(acc, v);
}
short lanes[16];
_mm256_storeu_si256((__m256i*)lanes, acc);
for (int i = 0; i < 16; i++)
result = isMax ? std::max(result, lanes[i]) : std::min(result, lanes[i]);
for (; x < cols; x++)
result = isMax ? std::max(result, src[x]) : std::min(result, src[x]);
dstmat.ptr<short>(y)[0] = result;
}
});
v_cleanup();
}
template<bool isMax>
static void minMax16uC4(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
const ushort initial = isMax ? 0 : USHRT_MAX;
const __m256i accInit = _mm256_set1_epi16((short)initial);
const __m256i validMask = _mm256_broadcastsi128_si256(
_mm_setr_epi8(-1, -1, -1, -1, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0));
const __m256i masks[4] = {
_mm256_broadcastsi128_si256(_mm_setr_epi8(0, 1, 8, 9, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1)),
_mm256_broadcastsi128_si256(_mm_setr_epi8(2, 3, 10, 11, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1)),
_mm256_broadcastsi128_si256(_mm_setr_epi8(4, 5, 12, 13, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1)),
_mm256_broadcastsi128_si256(_mm_setr_epi8(6, 7, 14, 15, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1))
};
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const ushort* src = srcmat.ptr<ushort>(y);
ushort* dst = dstmat.ptr<ushort>(y);
__m256i acc[4] = {accInit, accInit, accInit, accInit};
int x = 0;
for (; x <= cols - 4; x += 4)
{
__m256i v = _mm256_loadu_si256((const __m256i*)(src + x * 4));
for (int c = 0; c < 4; c++)
{
__m256i vc = _mm256_shuffle_epi8(v, masks[c]);
vc = _mm256_blendv_epi8(accInit, vc, validMask);
acc[c] = isMax ? _mm256_max_epu16(acc[c], vc)
: _mm256_min_epu16(acc[c], vc);
}
}
for (int c = 0; c < 4; c++)
{
ushort lanes[16];
_mm256_storeu_si256((__m256i*)lanes, acc[c]);
ushort result = initial;
for (int i = 0; i < 16; i++)
result = isMax ? std::max(result, lanes[i]) : std::min(result, lanes[i]);
for (int i = x; i < cols; i++)
result = isMax ? std::max(result, src[i * 4 + c])
: std::min(result, src[i * 4 + c]);
dst[c] = result;
}
}
});
v_cleanup();
}
template<bool isMax>
static void minMax16sC4(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
const short initial = isMax ? SHRT_MIN : SHRT_MAX;
const __m256i accInit = _mm256_set1_epi16(initial);
const __m256i validMask = _mm256_broadcastsi128_si256(
_mm_setr_epi8(-1, -1, -1, -1, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0));
const __m256i masks[4] = {
_mm256_broadcastsi128_si256(_mm_setr_epi8(0, 1, 8, 9, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1)),
_mm256_broadcastsi128_si256(_mm_setr_epi8(2, 3, 10, 11, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1)),
_mm256_broadcastsi128_si256(_mm_setr_epi8(4, 5, 12, 13, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1)),
_mm256_broadcastsi128_si256(_mm_setr_epi8(6, 7, 14, 15, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1))
};
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const short* src = srcmat.ptr<short>(y);
short* dst = dstmat.ptr<short>(y);
__m256i acc[4] = {accInit, accInit, accInit, accInit};
int x = 0;
for (; x <= cols - 4; x += 4)
{
__m256i v = _mm256_loadu_si256((const __m256i*)(src + x * 4));
for (int c = 0; c < 4; c++)
{
__m256i vc = _mm256_shuffle_epi8(v, masks[c]);
vc = _mm256_blendv_epi8(accInit, vc, validMask);
acc[c] = isMax ? _mm256_max_epi16(acc[c], vc)
: _mm256_min_epi16(acc[c], vc);
}
}
for (int c = 0; c < 4; c++)
{
short lanes[16];
_mm256_storeu_si256((__m256i*)lanes, acc[c]);
short result = initial;
for (int i = 0; i < 16; i++)
result = isMax ? std::max(result, lanes[i]) : std::min(result, lanes[i]);
for (int i = x; i < cols; i++)
result = isMax ? std::max(result, src[i * 4 + c])
: std::min(result, src[i * 4 + c]);
dst[c] = result;
}
}
});
v_cleanup();
}
template<typename T, bool isMax, bool isSigned>
static void minMax16C3(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
const T initial = isMax ? std::numeric_limits<T>::lowest() : std::numeric_limits<T>::max();
const __m256i accInit = _mm256_set1_epi16((short)initial);
const __m256i masks[3] = {
_mm256_setr_epi8(0, 1, 6, 7, 12, 13, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
2, 3, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1),
_mm256_setr_epi8(2, 3, 8, 9, 14, 15, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
4, 5, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1),
_mm256_setr_epi8(4, 5, 10, 11, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
0, 1, 6, 7, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1)
};
const __m256i validMasks[3] = {
_mm256_setr_epi8(-1, -1, -1, -1, -1, -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-1, -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0),
_mm256_setr_epi8(-1, -1, -1, -1, -1, -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-1, -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0),
_mm256_setr_epi8(-1, -1, -1, -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-1, -1, -1, -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0)
};
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const T* src = srcmat.ptr<T>(y);
T* dst = dstmat.ptr<T>(y);
__m256i acc[3] = {accInit, accInit, accInit};
int x = 0;
for (; x <= cols - 6; x += 4)
{
__m256i v = _mm256_loadu_si256((const __m256i*)(src + x * 3));
for (int c = 0; c < 3; c++)
{
__m256i vc = _mm256_shuffle_epi8(v, masks[c]);
vc = _mm256_blendv_epi8(accInit, vc, validMasks[c]);
if (isSigned)
acc[c] = isMax ? _mm256_max_epi16(acc[c], vc) : _mm256_min_epi16(acc[c], vc);
else
acc[c] = isMax ? _mm256_max_epu16(acc[c], vc) : _mm256_min_epu16(acc[c], vc);
}
}
for (int c = 0; c < 3; c++)
{
T lanes[16];
_mm256_storeu_si256((__m256i*)lanes, acc[c]);
T result = initial;
for (int i = 0; i < 16; i++)
result = isMax ? std::max(result, lanes[i]) : std::min(result, lanes[i]);
for (int i = x; i < cols; i++)
result = isMax ? std::max(result, src[i * 3 + c])
: std::min(result, src[i * 3 + c]);
dst[c] = result;
}
}
});
v_cleanup();
}
template<bool isMax>
static void minMax16uC3(const Mat& srcmat, Mat& dstmat)
{
minMax16C3<ushort, isMax, false>(srcmat, dstmat);
}
template<bool isMax>
static void minMax16sC3(const Mat& srcmat, Mat& dstmat)
{
minMax16C3<short, isMax, true>(srcmat, dstmat);
}
template<bool isMax>
static void minMax8uC3(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
const uchar initial = isMax ? 0 : UCHAR_MAX;
const __m256i accInit = _mm256_set1_epi8((char)initial);
const __m256i mask0 = _mm256_setr_epi8(
0, 3, 6, 9, 12, 15, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
2, 5, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1);
const __m256i mask1 = _mm256_setr_epi8(
1, 4, 7, 10, 13, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
0, 3, 6, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1);
const __m256i mask2 = _mm256_setr_epi8(
2, 5, 8, 11, 14, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
1, 4, 7, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1);
const __m256i invalid0 = _mm256_setr_epi8(
0, 0, 0, 0, 0, 0, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
0, 0, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1);
const __m256i invalid12 = _mm256_setr_epi8(
0, 0, 0, 0, 0, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
0, 0, 0, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1);
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const uchar* src = srcmat.ptr<uchar>(y);
uchar* dst = dstmat.ptr<uchar>(y);
__m256i acc0 = accInit, acc1 = accInit, acc2 = accInit;
int x = 0;
for (; x <= cols - 11; x += 8)
{
__m256i v = _mm256_loadu_si256((const __m256i*)(src + x * 3));
__m256i v0 = _mm256_shuffle_epi8(v, mask0);
__m256i v1 = _mm256_shuffle_epi8(v, mask1);
__m256i v2 = _mm256_shuffle_epi8(v, mask2);
if (!isMax)
{
v0 = _mm256_or_si256(v0, invalid0);
v1 = _mm256_or_si256(v1, invalid12);
v2 = _mm256_or_si256(v2, invalid12);
}
acc0 = isMax ? _mm256_max_epu8(acc0, v0) : _mm256_min_epu8(acc0, v0);
acc1 = isMax ? _mm256_max_epu8(acc1, v1) : _mm256_min_epu8(acc1, v1);
acc2 = isMax ? _mm256_max_epu8(acc2, v2) : _mm256_min_epu8(acc2, v2);
}
__m256i accs[3] = {acc0, acc1, acc2};
for (int c = 0; c < 3; c++)
{
uchar lanes[32];
_mm256_storeu_si256((__m256i*)lanes, accs[c]);
uchar result = initial;
for (int i = 0; i < 32; i++)
result = reduceScalarMinMax<isMax>(result, lanes[i]);
for (int i = x; i < cols; i++)
result = reduceScalarMinMax<isMax>(result, src[i * 3 + c]);
dst[c] = result;
}
}
});
v_cleanup();
}
template<bool isMax>
static void minMax8uC4(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
const uchar initial = isMax ? 0 : UCHAR_MAX;
const __m256i accInit = _mm256_set1_epi8((char)initial);
const __m256i maskInvalid = _mm256_broadcastsi128_si256(
_mm_setr_epi8(0, 0, 0, 0, -1, -1, -1, -1,
-1, -1, -1, -1, -1, -1, -1, -1));
const __m256i mask0 = _mm256_broadcastsi128_si256(
_mm_setr_epi8(0, 4, 8, 12, -1, -1, -1, -1,
-1, -1, -1, -1, -1, -1, -1, -1));
const __m256i mask1 = _mm256_broadcastsi128_si256(
_mm_setr_epi8(1, 5, 9, 13, -1, -1, -1, -1,
-1, -1, -1, -1, -1, -1, -1, -1));
const __m256i mask2 = _mm256_broadcastsi128_si256(
_mm_setr_epi8(2, 6, 10, 14, -1, -1, -1, -1,
-1, -1, -1, -1, -1, -1, -1, -1));
const __m256i mask3 = _mm256_broadcastsi128_si256(
_mm_setr_epi8(3, 7, 11, 15, -1, -1, -1, -1,
-1, -1, -1, -1, -1, -1, -1, -1));
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const uchar* src = srcmat.ptr<uchar>(y);
uchar* dst = dstmat.ptr<uchar>(y);
__m256i acc0 = accInit, acc1 = accInit, acc2 = accInit, acc3 = accInit;
int x = 0;
for (; x <= cols - 8; x += 8)
{
__m256i v = _mm256_loadu_si256((const __m256i*)(src + x * 4));
__m256i v0 = _mm256_shuffle_epi8(v, mask0);
__m256i v1 = _mm256_shuffle_epi8(v, mask1);
__m256i v2 = _mm256_shuffle_epi8(v, mask2);
__m256i v3 = _mm256_shuffle_epi8(v, mask3);
if (!isMax)
{
v0 = _mm256_or_si256(v0, maskInvalid);
v1 = _mm256_or_si256(v1, maskInvalid);
v2 = _mm256_or_si256(v2, maskInvalid);
v3 = _mm256_or_si256(v3, maskInvalid);
}
acc0 = isMax ? _mm256_max_epu8(acc0, v0) : _mm256_min_epu8(acc0, v0);
acc1 = isMax ? _mm256_max_epu8(acc1, v1) : _mm256_min_epu8(acc1, v1);
acc2 = isMax ? _mm256_max_epu8(acc2, v2) : _mm256_min_epu8(acc2, v2);
acc3 = isMax ? _mm256_max_epu8(acc3, v3) : _mm256_min_epu8(acc3, v3);
}
__m256i accs[4] = {acc0, acc1, acc2, acc3};
for (int c = 0; c < 4; c++)
{
uchar lanes[32];
_mm256_storeu_si256((__m256i*)lanes, accs[c]);
uchar result = initial;
for (int i = 0; i < 32; i++)
result = reduceScalarMinMax<isMax>(result, lanes[i]);
for (int i = x; i < cols; i++)
result = reduceScalarMinMax<isMax>(result, src[i * 4 + c]);
dst[c] = result;
}
}
});
v_cleanup();
}
template<bool isMax>
static void minMax32fC1(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const float* src = srcmat.ptr<float>(y);
float result = src[0];
int x = 0;
__m256 acc = _mm256_set1_ps(result);
for (; x <= cols - 8; x += 8)
{
__m256 v = _mm256_loadu_ps(src + x);
acc = isMax ? _mm256_max_ps(acc, v) : _mm256_min_ps(acc, v);
}
float lanes[8];
_mm256_storeu_ps(lanes, acc);
for (int i = 0; i < 8; i++)
result = reduceScalarMinMax<isMax>(result, lanes[i]);
for (; x < cols; x++)
result = reduceScalarMinMax<isMax>(result, src[x]);
dstmat.ptr<float>(y)[0] = result;
}
});
v_cleanup();
}
template<bool isMax>
static void minMax32fC4(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
const float initial = isMax ? std::numeric_limits<float>::lowest() : std::numeric_limits<float>::max();
const __m256 accInit = _mm256_set1_ps(initial);
const __m256 validMask = _mm256_castsi256_ps(_mm256_setr_epi32(-1, -1, 0, 0, 0, 0, 0, 0));
const __m256i idx0 = _mm256_setr_epi32(0, 4, 0, 0, 0, 0, 0, 0);
const __m256i idx1 = _mm256_setr_epi32(1, 5, 0, 0, 0, 0, 0, 0);
const __m256i idx2 = _mm256_setr_epi32(2, 6, 0, 0, 0, 0, 0, 0);
const __m256i idx3 = _mm256_setr_epi32(3, 7, 0, 0, 0, 0, 0, 0);
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const float* src = srcmat.ptr<float>(y);
float* dst = dstmat.ptr<float>(y);
__m256 acc0 = accInit, acc1 = accInit, acc2 = accInit, acc3 = accInit;
int x = 0;
for (; x <= cols - 2; x += 2)
{
__m256 v = _mm256_loadu_ps(src + x * 4);
__m256 v0 = _mm256_blendv_ps(accInit, _mm256_permutevar8x32_ps(v, idx0), validMask);
__m256 v1 = _mm256_blendv_ps(accInit, _mm256_permutevar8x32_ps(v, idx1), validMask);
__m256 v2 = _mm256_blendv_ps(accInit, _mm256_permutevar8x32_ps(v, idx2), validMask);
__m256 v3 = _mm256_blendv_ps(accInit, _mm256_permutevar8x32_ps(v, idx3), validMask);
acc0 = isMax ? _mm256_max_ps(acc0, v0) : _mm256_min_ps(acc0, v0);
acc1 = isMax ? _mm256_max_ps(acc1, v1) : _mm256_min_ps(acc1, v1);
acc2 = isMax ? _mm256_max_ps(acc2, v2) : _mm256_min_ps(acc2, v2);
acc3 = isMax ? _mm256_max_ps(acc3, v3) : _mm256_min_ps(acc3, v3);
}
__m256 accs[4] = {acc0, acc1, acc2, acc3};
for (int c = 0; c < 4; c++)
{
float lanes[8];
_mm256_storeu_ps(lanes, accs[c]);
float result = initial;
for (int i = 0; i < 8; i++)
result = reduceScalarMinMax<isMax>(result, lanes[i]);
for (int i = x; i < cols; i++)
result = reduceScalarMinMax<isMax>(result, src[i * 4 + c]);
dst[c] = result;
}
}
});
v_cleanup();
}
template<bool isMax>
static void minMax64fC1(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
const double initial = isMax ? std::numeric_limits<double>::lowest()
: std::numeric_limits<double>::max();
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const double* src = srcmat.ptr<double>(y);
double result = initial;
int x = 0;
__m256d acc0 = _mm256_set1_pd(initial);
__m256d acc1 = acc0, acc2 = acc0, acc3 = acc0;
for (; x <= cols - 16; x += 16)
{
__m256d v0 = _mm256_loadu_pd(src + x);
__m256d v1 = _mm256_loadu_pd(src + x + 4);
__m256d v2 = _mm256_loadu_pd(src + x + 8);
__m256d v3 = _mm256_loadu_pd(src + x + 12);
acc0 = isMax ? _mm256_max_pd(acc0, v0) : _mm256_min_pd(acc0, v0);
acc1 = isMax ? _mm256_max_pd(acc1, v1) : _mm256_min_pd(acc1, v1);
acc2 = isMax ? _mm256_max_pd(acc2, v2) : _mm256_min_pd(acc2, v2);
acc3 = isMax ? _mm256_max_pd(acc3, v3) : _mm256_min_pd(acc3, v3);
}
acc0 = isMax ? _mm256_max_pd(acc0, acc1) : _mm256_min_pd(acc0, acc1);
acc2 = isMax ? _mm256_max_pd(acc2, acc3) : _mm256_min_pd(acc2, acc3);
acc0 = isMax ? _mm256_max_pd(acc0, acc2) : _mm256_min_pd(acc0, acc2);
double lanes[4];
_mm256_storeu_pd(lanes, acc0);
for (int i = 0; i < 4; i++)
result = isMax ? std::max(result, lanes[i]) : std::min(result, lanes[i]);
for (; x < cols; x++)
result = isMax ? std::max(result, src[x]) : std::min(result, src[x]);
dstmat.ptr<double>(y)[0] = result;
}
});
v_cleanup();
}
template<typename DT>
static void sum2_8uC1(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const uchar* src = srcmat.ptr<uchar>(y);
DT* dst = dstmat.ptr<DT>(y);
uint32_t result = 0;
int x = 0;
__m256i acc = _mm256_setzero_si256();
for (; x <= cols - 32; x += 32)
{
__m256i bytes = _mm256_loadu_si256((const __m256i*)(src + x));
__m128i lo = _mm256_castsi256_si128(bytes);
__m128i hi = _mm256_extracti128_si256(bytes, 1);
__m256i lo16 = _mm256_cvtepu8_epi16(lo);
__m256i hi16 = _mm256_cvtepu8_epi16(hi);
acc = _mm256_add_epi32(acc, _mm256_madd_epi16(lo16, lo16));
acc = _mm256_add_epi32(acc, _mm256_madd_epi16(hi16, hi16));
}
uint32_t lanes[8];
_mm256_storeu_si256((__m256i*)lanes, acc);
for (int i = 0; i < 8; i++)
result += lanes[i];
for (; x < cols; x++)
result += (uint32_t)src[x] * src[x];
dst[0] = (DT)(int32_t)result;
}
});
v_cleanup();
}
static void sum2_16u32fC1(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const ushort* src = srcmat.ptr<ushort>(y);
__m256 acc0 = _mm256_setzero_ps(), acc1 = acc0;
int x = 0;
for (; x <= cols - 16; x += 16)
{
__m256i v = _mm256_loadu_si256((const __m256i*)(src + x));
__m256 v0 = _mm256_cvtepi32_ps(_mm256_cvtepu16_epi32(_mm256_castsi256_si128(v)));
__m256 v1 = _mm256_cvtepi32_ps(_mm256_cvtepu16_epi32(_mm256_extracti128_si256(v, 1)));
acc0 = _mm256_add_ps(acc0, _mm256_mul_ps(v0, v0));
acc1 = _mm256_add_ps(acc1, _mm256_mul_ps(v1, v1));
}
acc0 = _mm256_add_ps(acc0, acc1);
float lanes[8];
_mm256_storeu_ps(lanes, acc0);
float result = 0;
for (int i = 0; i < 8; i++)
result += lanes[i];
for (; x < cols; x++)
{
float value = (float)src[x];
result += value * value;
}
dstmat.ptr<float>(y)[0] = result;
}
});
v_cleanup();
}
static void sum2_16s32fC1(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const short* src = srcmat.ptr<short>(y);
__m256 acc0 = _mm256_setzero_ps(), acc1 = acc0;
int x = 0;
for (; x <= cols - 16; x += 16)
{
__m256i v = _mm256_loadu_si256((const __m256i*)(src + x));
__m256 v0 = _mm256_cvtepi32_ps(_mm256_cvtepi16_epi32(_mm256_castsi256_si128(v)));
__m256 v1 = _mm256_cvtepi32_ps(_mm256_cvtepi16_epi32(_mm256_extracti128_si256(v, 1)));
acc0 = _mm256_add_ps(acc0, _mm256_mul_ps(v0, v0));
acc1 = _mm256_add_ps(acc1, _mm256_mul_ps(v1, v1));
}
acc0 = _mm256_add_ps(acc0, acc1);
float lanes[8];
_mm256_storeu_ps(lanes, acc0);
float result = 0;
for (int i = 0; i < 8; i++)
result += lanes[i];
for (; x < cols; x++)
{
float value = (float)src[x];
result += value * value;
}
dstmat.ptr<float>(y)[0] = result;
}
});
v_cleanup();
}
static void sum2_16u64fC1(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const ushort* src = srcmat.ptr<ushort>(y);
__m256d acc0 = _mm256_setzero_pd(), acc1 = acc0, acc2 = acc0, acc3 = acc0;
int x = 0;
for (; x <= cols - 16; x += 16)
{
__m256i v = _mm256_loadu_si256((const __m256i*)(src + x));
__m256i v0 = _mm256_cvtepu16_epi32(_mm256_castsi256_si128(v));
__m256i v1 = _mm256_cvtepu16_epi32(_mm256_extracti128_si256(v, 1));
__m256d d0 = _mm256_cvtepi32_pd(_mm256_castsi256_si128(v0));
__m256d d1 = _mm256_cvtepi32_pd(_mm256_extracti128_si256(v0, 1));
__m256d d2 = _mm256_cvtepi32_pd(_mm256_castsi256_si128(v1));
__m256d d3 = _mm256_cvtepi32_pd(_mm256_extracti128_si256(v1, 1));
acc0 = _mm256_add_pd(acc0, _mm256_mul_pd(d0, d0));
acc1 = _mm256_add_pd(acc1, _mm256_mul_pd(d1, d1));
acc2 = _mm256_add_pd(acc2, _mm256_mul_pd(d2, d2));
acc3 = _mm256_add_pd(acc3, _mm256_mul_pd(d3, d3));
}
acc0 = _mm256_add_pd(_mm256_add_pd(acc0, acc1), _mm256_add_pd(acc2, acc3));
double lanes[4];
_mm256_storeu_pd(lanes, acc0);
double result = lanes[0] + lanes[1] + lanes[2] + lanes[3];
for (; x < cols; x++)
{
double value = (double)src[x];
result += value * value;
}
dstmat.ptr<double>(y)[0] = result;
}
});
v_cleanup();
}
static void sum2_16s64fC1(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const short* src = srcmat.ptr<short>(y);
__m256d acc0 = _mm256_setzero_pd(), acc1 = acc0, acc2 = acc0, acc3 = acc0;
int x = 0;
for (; x <= cols - 16; x += 16)
{
__m256i v = _mm256_loadu_si256((const __m256i*)(src + x));
__m256i v0 = _mm256_cvtepi16_epi32(_mm256_castsi256_si128(v));
__m256i v1 = _mm256_cvtepi16_epi32(_mm256_extracti128_si256(v, 1));
__m256d d0 = _mm256_cvtepi32_pd(_mm256_castsi256_si128(v0));
__m256d d1 = _mm256_cvtepi32_pd(_mm256_extracti128_si256(v0, 1));
__m256d d2 = _mm256_cvtepi32_pd(_mm256_castsi256_si128(v1));
__m256d d3 = _mm256_cvtepi32_pd(_mm256_extracti128_si256(v1, 1));
acc0 = _mm256_add_pd(acc0, _mm256_mul_pd(d0, d0));
acc1 = _mm256_add_pd(acc1, _mm256_mul_pd(d1, d1));
acc2 = _mm256_add_pd(acc2, _mm256_mul_pd(d2, d2));
acc3 = _mm256_add_pd(acc3, _mm256_mul_pd(d3, d3));
}
acc0 = _mm256_add_pd(_mm256_add_pd(acc0, acc1), _mm256_add_pd(acc2, acc3));
double lanes[4];
_mm256_storeu_pd(lanes, acc0);
double result = lanes[0] + lanes[1] + lanes[2] + lanes[3];
for (; x < cols; x++)
{
double value = (double)src[x];
result += value * value;
}
dstmat.ptr<double>(y)[0] = result;
}
});
v_cleanup();
}
static void sum2_32f64fC1(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const float* src = srcmat.ptr<float>(y);
__m256d acc0 = _mm256_setzero_pd(), acc1 = acc0;
int x = 0;
for (; x <= cols - 8; x += 8)
{
__m256 v = _mm256_loadu_ps(src + x);
__m256d v0 = _mm256_cvtps_pd(_mm256_castps256_ps128(v));
__m256d v1 = _mm256_cvtps_pd(_mm256_extractf128_ps(v, 1));
acc0 = _mm256_add_pd(acc0, _mm256_mul_pd(v0, v0));
acc1 = _mm256_add_pd(acc1, _mm256_mul_pd(v1, v1));
}
acc0 = _mm256_add_pd(acc0, acc1);
double lanes[4];
_mm256_storeu_pd(lanes, acc0);
double result = lanes[0] + lanes[1] + lanes[2] + lanes[3];
for (; x < cols; x++)
{
double value = (double)src[x];
result += value * value;
}
dstmat.ptr<double>(y)[0] = result;
}
});
v_cleanup();
}
static void sum2_64f64fC1(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const double* src = srcmat.ptr<double>(y);
__m256d acc0 = _mm256_setzero_pd(), acc1 = acc0, acc2 = acc0, acc3 = acc0;
int x = 0;
for (; x <= cols - 16; x += 16)
{
__m256d v0 = _mm256_loadu_pd(src + x);
__m256d v1 = _mm256_loadu_pd(src + x + 4);
__m256d v2 = _mm256_loadu_pd(src + x + 8);
__m256d v3 = _mm256_loadu_pd(src + x + 12);
acc0 = _mm256_add_pd(acc0, _mm256_mul_pd(v0, v0));
acc1 = _mm256_add_pd(acc1, _mm256_mul_pd(v1, v1));
acc2 = _mm256_add_pd(acc2, _mm256_mul_pd(v2, v2));
acc3 = _mm256_add_pd(acc3, _mm256_mul_pd(v3, v3));
}
acc0 = _mm256_add_pd(_mm256_add_pd(acc0, acc1), _mm256_add_pd(acc2, acc3));
double lanes[4];
_mm256_storeu_pd(lanes, acc0);
double result = lanes[0] + lanes[1] + lanes[2] + lanes[3];
for (; x < cols; x++)
result += src[x] * src[x];
dstmat.ptr<double>(y)[0] = result;
}
});
v_cleanup();
}
template<typename DT>
static void sum2_8uC3(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
const __m256i mask0 = _mm256_setr_epi8(
0, 3, 6, 9, 12, 15, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
2, 5, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1);
const __m256i mask1 = _mm256_setr_epi8(
1, 4, 7, 10, 13, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
0, 3, 6, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1);
const __m256i mask2 = _mm256_setr_epi8(
2, 5, 8, 11, 14, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
1, 4, 7, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1);
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const uchar* src = srcmat.ptr<uchar>(y);
DT* dst = dstmat.ptr<DT>(y);
__m256i acc0 = _mm256_setzero_si256();
__m256i acc1 = _mm256_setzero_si256();
__m256i acc2 = _mm256_setzero_si256();
int x = 0;
for (; x <= cols - 11; x += 8)
{
__m256i v = _mm256_loadu_si256((const __m256i*)(src + x * 3));
__m256i channels[3] = {
_mm256_shuffle_epi8(v, mask0),
_mm256_shuffle_epi8(v, mask1),
_mm256_shuffle_epi8(v, mask2)
};
__m256i* accs[3] = {&acc0, &acc1, &acc2};
for (int c = 0; c < 3; c++)
{
__m128i lo = _mm256_castsi256_si128(channels[c]);
__m128i hi = _mm256_extracti128_si256(channels[c], 1);
__m256i lo16 = _mm256_cvtepu8_epi16(lo);
__m256i hi16 = _mm256_cvtepu8_epi16(hi);
*accs[c] = _mm256_add_epi32(*accs[c], _mm256_madd_epi16(lo16, lo16));
*accs[c] = _mm256_add_epi32(*accs[c], _mm256_madd_epi16(hi16, hi16));
}
}
__m256i accs[3] = {acc0, acc1, acc2};
for (int c = 0; c < 3; c++)
{
uint32_t lanes[8];
uint32_t result = 0;
_mm256_storeu_si256((__m256i*)lanes, accs[c]);
for (int i = 0; i < 8; i++)
result += lanes[i];
for (int i = x; i < cols; i++)
{
uint32_t value = src[i * 3 + c];
result += value * value;
}
dst[c] = (DT)(int32_t)result;
}
}
});
v_cleanup();
}
template<typename DT>
static void sum2_8uC4(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
const __m256i mask0 = _mm256_broadcastsi128_si256(
_mm_setr_epi8(0, 4, 8, 12, -1, -1, -1, -1,
-1, -1, -1, -1, -1, -1, -1, -1));
const __m256i mask1 = _mm256_broadcastsi128_si256(
_mm_setr_epi8(1, 5, 9, 13, -1, -1, -1, -1,
-1, -1, -1, -1, -1, -1, -1, -1));
const __m256i mask2 = _mm256_broadcastsi128_si256(
_mm_setr_epi8(2, 6, 10, 14, -1, -1, -1, -1,
-1, -1, -1, -1, -1, -1, -1, -1));
const __m256i mask3 = _mm256_broadcastsi128_si256(
_mm_setr_epi8(3, 7, 11, 15, -1, -1, -1, -1,
-1, -1, -1, -1, -1, -1, -1, -1));
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const uchar* src = srcmat.ptr<uchar>(y);
DT* dst = dstmat.ptr<DT>(y);
__m256i acc0 = _mm256_setzero_si256();
__m256i acc1 = _mm256_setzero_si256();
__m256i acc2 = _mm256_setzero_si256();
__m256i acc3 = _mm256_setzero_si256();
int x = 0;
for (; x <= cols - 8; x += 8)
{
__m256i v = _mm256_loadu_si256((const __m256i*)(src + x * 4));
__m256i channels[4] = {
_mm256_shuffle_epi8(v, mask0),
_mm256_shuffle_epi8(v, mask1),
_mm256_shuffle_epi8(v, mask2),
_mm256_shuffle_epi8(v, mask3)
};
__m256i* accs[4] = {&acc0, &acc1, &acc2, &acc3};
for (int c = 0; c < 4; c++)
{
__m128i lo = _mm256_castsi256_si128(channels[c]);
__m128i hi = _mm256_extracti128_si256(channels[c], 1);
__m256i lo16 = _mm256_cvtepu8_epi16(lo);
__m256i hi16 = _mm256_cvtepu8_epi16(hi);
*accs[c] = _mm256_add_epi32(*accs[c], _mm256_madd_epi16(lo16, lo16));
*accs[c] = _mm256_add_epi32(*accs[c], _mm256_madd_epi16(hi16, hi16));
}
}
__m256i accs[4] = {acc0, acc1, acc2, acc3};
for (int c = 0; c < 4; c++)
{
uint32_t lanes[8];
uint32_t result = 0;
_mm256_storeu_si256((__m256i*)lanes, accs[c]);
for (int i = 0; i < 8; i++)
result += lanes[i];
for (int i = x; i < cols; i++)
{
uint32_t value = src[i * 4 + c];
result += value * value;
}
dst[c] = (DT)(int32_t)result;
}
}
});
v_cleanup();
}
static void sum2_32fC1(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const float* src = srcmat.ptr<float>(y);
float result = 0;
int x = 0;
__m256 acc = _mm256_setzero_ps();
for (; x <= cols - 8; x += 8)
{
__m256 v = _mm256_loadu_ps(src + x);
acc = _mm256_add_ps(acc, _mm256_mul_ps(v, v));
}
float lanes[8];
_mm256_storeu_ps(lanes, acc);
for (int i = 0; i < 8; i++)
result += lanes[i];
for (; x < cols; x++)
result += src[x] * src[x];
dstmat.ptr<float>(y)[0] = result;
}
});
v_cleanup();
}
static void sum2_32fC4(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
const __m256 zero = _mm256_setzero_ps();
const __m256 validMask = _mm256_castsi256_ps(_mm256_setr_epi32(-1, -1, 0, 0, 0, 0, 0, 0));
const __m256i idx0 = _mm256_setr_epi32(0, 4, 0, 0, 0, 0, 0, 0);
const __m256i idx1 = _mm256_setr_epi32(1, 5, 0, 0, 0, 0, 0, 0);
const __m256i idx2 = _mm256_setr_epi32(2, 6, 0, 0, 0, 0, 0, 0);
const __m256i idx3 = _mm256_setr_epi32(3, 7, 0, 0, 0, 0, 0, 0);
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const float* src = srcmat.ptr<float>(y);
float* dst = dstmat.ptr<float>(y);
__m256 acc0 = zero, acc1 = zero, acc2 = zero, acc3 = zero;
int x = 0;
for (; x <= cols - 2; x += 2)
{
__m256 v = _mm256_loadu_ps(src + x * 4);
__m256 v0 = _mm256_blendv_ps(zero, _mm256_permutevar8x32_ps(v, idx0), validMask);
__m256 v1 = _mm256_blendv_ps(zero, _mm256_permutevar8x32_ps(v, idx1), validMask);
__m256 v2 = _mm256_blendv_ps(zero, _mm256_permutevar8x32_ps(v, idx2), validMask);
__m256 v3 = _mm256_blendv_ps(zero, _mm256_permutevar8x32_ps(v, idx3), validMask);
acc0 = _mm256_add_ps(acc0, _mm256_mul_ps(v0, v0));
acc1 = _mm256_add_ps(acc1, _mm256_mul_ps(v1, v1));
acc2 = _mm256_add_ps(acc2, _mm256_mul_ps(v2, v2));
acc3 = _mm256_add_ps(acc3, _mm256_mul_ps(v3, v3));
}
__m256 accs[4] = {acc0, acc1, acc2, acc3};
for (int c = 0; c < 4; c++)
{
float lanes[8];
float result = 0;
_mm256_storeu_ps(lanes, accs[c]);
for (int i = 0; i < 8; i++)
result += lanes[i];
for (int i = x; i < cols; i++)
result += src[i * 4 + c] * src[i * 4 + c];
dst[c] = result;
}
}
});
v_cleanup();
}
} // namespace reduce_c_avx2
+701
View File
@@ -0,0 +1,701 @@
// This file is part of OpenCV project.
// It is subject to the license terms in the LICENSE file found in the top-level directory
// of this distribution and at http://opencv.org/license.html
#if (CV_SIMD || CV_SIMD_SCALABLE)
template<typename T, typename VT>
static inline VT vx_load_strided(const T *ptr, size_t step)
{
constexpr int nlanes = VTraits<VT>::max_nlanes;
T buf[nlanes];
for (int i = 0; i < VTraits<VT>::vlanes(); i++)
{
buf[i] = *ptr;
ptr += step;
}
return vx_load(buf);
}
#if CV_RVV
template<>
inline v_uint8 vx_load_strided(const uchar *ptr, size_t step)
{
return __riscv_vlse8_v_u8m2(ptr, step * sizeof(uchar), __riscv_vsetvlmax_e8m2());
}
template<>
inline v_uint16 vx_load_strided(const ushort *ptr, size_t step)
{
return __riscv_vlse16_v_u16m2(ptr, step * sizeof(ushort), __riscv_vsetvlmax_e16m2());
}
template<>
inline v_int16 vx_load_strided(const short *ptr, size_t step)
{
return __riscv_vlse16_v_i16m2(ptr, step * sizeof(short), __riscv_vsetvlmax_e16m2());
}
template<>
inline v_int32 vx_load_strided(const int *ptr, size_t step)
{
return __riscv_vlse32_v_i32m2(ptr, step * sizeof(int), __riscv_vsetvlmax_e32m2());
}
template<>
inline v_float32 vx_load_strided(const float *ptr, size_t step)
{
return __riscv_vlse32_v_f32m2(ptr, step * sizeof(float), __riscv_vsetvlmax_e32m2());
}
template<>
inline v_float64 vx_load_strided(const double *ptr, size_t step)
{
return __riscv_vlse64_v_f64m2(ptr, step * sizeof(double), __riscv_vsetvlmax_e64m2());
}
#endif
#endif
template<typename stype, typename itype>
struct ReduceOpAddSqr
{
using v_stype = stype;
using v_itype = itype;
static const int vlanes;
static inline stype load(const stype *ptr, size_t step) { (void)step; return *ptr; }
static inline itype init() { return (itype)0; }
static inline itype reduce(const itype &val) { return val; }
inline itype operator()(const itype &a, const stype &b) const { return a + (itype)b * (itype)b; }
};
template<typename stype, typename itype>
const int ReduceOpAddSqr<stype, itype>::vlanes = 1;
#if (CV_SIMD || CV_SIMD_SCALABLE)
template<typename stype, typename itype>
struct ReduceVecOpAddSqr;
template<>
struct ReduceVecOpAddSqr<uchar, int>
{
using stype = uchar;
using itype = int;
using v_stype = v_uint8;
using v_itype = v_int32;
static const int vlanes;
static inline v_stype load(const stype *ptr, size_t step) { return vx_load_strided<stype, v_stype>(ptr, step); }
static inline v_itype init() { return vx_setzero_s32(); }
static inline itype reduce(const v_itype &val) { return v_reduce_sum(val); }
inline v_itype operator()(const v_itype &a, const v_stype &b) const
{
v_uint16 b0, b1;
v_mul_expand(b, b, b0, b1);
v_uint32 s00, s01;
v_expand(b0, s00, s01);
s00 = v_add(s00, s01);
v_uint32 s10, s11;
v_expand(b1, s10, s11);
s10 = v_add(s10, s11);
return v_add(a, v_reinterpret_as_s32(v_add(s00, s10)));
}
};
const int ReduceVecOpAddSqr<uchar, int>::vlanes = VTraits<v_stype>::vlanes();
template<>
struct ReduceVecOpAddSqr<ushort, float>
{
using stype = ushort;
using itype = float;
using v_stype = v_uint16;
using v_itype = v_float32;
static const int vlanes;
static inline v_stype load(const stype *ptr, size_t step) { return vx_load_strided<stype, v_stype>(ptr, step); }
static inline v_itype init() { return vx_setzero_f32(); }
static inline itype reduce(const v_itype &val) { return v_reduce_sum(val); }
inline v_itype operator()(const v_itype &a, const v_stype &b) const
{
v_uint32 b0, b1;
v_mul_expand(b, b, b0, b1);
v_int32 sb = v_reinterpret_as_s32(v_add(b0, b1));
return v_add(a, v_cvt_f32(sb));
}
};
const int ReduceVecOpAddSqr<ushort, float>::vlanes = VTraits<v_stype>::vlanes();
template<>
struct ReduceVecOpAddSqr<short, float>
{
using stype = short;
using itype = float;
using v_stype = v_int16;
using v_itype = v_float32;
static const int vlanes;
static inline v_stype load(const stype *ptr, size_t step) { return vx_load_strided<stype, v_stype>(ptr, step); }
static inline v_itype init() { return vx_setzero_f32(); }
static inline itype reduce(const v_itype &val) { return v_reduce_sum(val); }
inline v_itype operator()(const v_itype &a, const v_stype &b) const
{
v_int32 b0, b1;
v_mul_expand(b, b, b0, b1);
v_int32 sb = v_add(b0, b1);
return v_add(a, v_cvt_f32(sb));
}
};
const int ReduceVecOpAddSqr<short, float>::vlanes = VTraits<v_stype>::vlanes();
template<>
struct ReduceVecOpAddSqr<float, float>
{
using stype = float;
using itype = float;
using v_stype = v_float32;
using v_itype = v_float32;
static const int vlanes;
static inline v_stype load(const stype *ptr, size_t step) { return vx_load_strided<stype, v_stype>(ptr, step); }
static inline v_itype init() { return vx_setzero_f32(); }
static inline itype reduce(const v_itype &val) { return v_reduce_sum(val); }
inline v_itype operator()(const v_itype &a, const v_stype &b) const { return v_add(a, v_mul(b, b)); }
};
const int ReduceVecOpAddSqr<float, float>::vlanes = VTraits<v_stype>::vlanes();
#endif
#if (CV_SIMD_64F || CV_SIMD_SCALABLE_64F)
template<>
struct ReduceVecOpAddSqr<short, double>
{
using stype = short;
using itype = double;
using v_stype = v_int16;
using v_itype = v_float64;
static const int vlanes;
static inline v_stype load(const stype *ptr, size_t step) { return vx_load_strided<stype, v_stype>(ptr, step); }
static inline v_itype init() { return vx_setzero_f64(); }
static inline itype reduce(const v_itype &val) { return v_reduce_sum(val); }
inline v_itype operator()(const v_itype &a, const v_stype &b) const
{
v_int32 b0, b1;
v_mul_expand(b, b, b0, b1);
v_int32 sb = v_add(b0, b1);
return v_add(a, v_add(v_cvt_f64(sb), v_cvt_f64_high(sb)));
}
};
const int ReduceVecOpAddSqr<short, double>::vlanes = VTraits<v_stype>::vlanes();
template<>
struct ReduceVecOpAddSqr<ushort, double>
{
using stype = ushort;
using itype = double;
using v_stype = v_uint16;
using v_itype = v_float64;
static const int vlanes;
static inline v_stype load(const stype *ptr, size_t step) { return vx_load_strided<stype, v_stype>(ptr, step); }
static inline v_itype init() { return vx_setzero_f64(); }
static inline itype reduce(const v_itype &val) { return v_reduce_sum(val); }
inline v_itype operator()(const v_itype &a, const v_stype &b) const
{
v_uint32 b0, b1;
v_mul_expand(b, b, b0, b1);
v_int32 sb = v_reinterpret_as_s32(v_add(b0, b1));
return v_add(a, v_add(v_cvt_f64(sb), v_cvt_f64_high(sb)));
}
};
const int ReduceVecOpAddSqr<ushort, double>::vlanes = VTraits<v_stype>::vlanes();
template<>
struct ReduceVecOpAddSqr<float, double>
{
using stype = float;
using itype = double;
using v_stype = v_float32;
using v_itype = v_float64;
static const int vlanes;
static inline v_stype load(const stype *ptr, size_t step) { return vx_load_strided<stype, v_stype>(ptr, step); }
static inline v_itype init() { return vx_setzero_f64(); }
static inline itype reduce(const v_itype &val) { return v_reduce_sum(val); }
inline v_itype operator()(const v_itype &a, const v_stype &b) const
{
v_itype b0 = v_cvt_f64(b), b1 = v_cvt_f64_high(b);
return v_add(a, v_add(v_mul(b0, b0), v_mul(b1, b1)));
}
};
const int ReduceVecOpAddSqr<float, double>::vlanes = VTraits<v_stype>::vlanes();
template<>
struct ReduceVecOpAddSqr<double, double>
{
using stype = double;
using itype = double;
using v_stype = v_float64;
using v_itype = v_float64;
static const int vlanes;
static inline v_stype load(const stype *ptr, size_t step) { return vx_load_strided<stype, v_stype>(ptr, step); }
static inline v_itype init() { return vx_setzero_f64(); }
static inline itype reduce(const v_itype &val) { return v_reduce_sum(val); }
inline v_itype operator()(const v_itype &a, const v_stype &b) const { return v_add(a, v_mul(b, b)); }
};
const int ReduceVecOpAddSqr<double, double>::vlanes = VTraits<v_stype>::vlanes();
#endif
template<typename stype>
struct ReduceOpMax
{
using v_stype = stype;
using v_itype = stype;
static const int vlanes;
static inline stype load(const stype *ptr, size_t step) { (void)step; return *ptr; }
static inline stype init() { return std::numeric_limits<stype>::lowest(); }
static inline stype reduce(const stype &val) { return val; }
inline stype operator()(const stype &a, const stype &b) const { return std::max(a, b); }
};
template<typename stype>
const int ReduceOpMax<stype>::vlanes = 1;
#if (CV_SIMD || CV_SIMD_SCALABLE)
template<typename stype>
struct ReduceVecOpMax;
template<>
struct ReduceVecOpMax<uchar>
{
using stype = uchar;
using itype = uchar;
using v_stype = v_uint8;
using v_itype = v_uint8;
static const int vlanes;
static inline v_stype load(const stype *ptr, size_t step) { return vx_load_strided<stype, v_stype>(ptr, step); }
static inline v_itype init() { return vx_setall_u8(std::numeric_limits<itype>::lowest()); }
static inline itype reduce(const v_itype &val) { return v_reduce_max(val); }
inline v_itype operator()(const v_itype &a, const v_stype &b) const { return v_max(a, b); }
};
const int ReduceVecOpMax<uchar>::vlanes = VTraits<v_stype>::vlanes();
template<>
struct ReduceVecOpMax<ushort>
{
using stype = ushort;
using itype = ushort;
using v_stype = v_uint16;
using v_itype = v_uint16;
static const int vlanes;
static inline v_stype load(const stype *ptr, size_t step) { return vx_load_strided<stype, v_stype>(ptr, step); }
static inline v_itype init() { return vx_setall_u16(std::numeric_limits<itype>::lowest()); }
static inline itype reduce(const v_itype &val) { return v_reduce_max(val); }
inline v_itype operator()(const v_itype &a, const v_stype &b) const { return v_max(a, b); }
};
const int ReduceVecOpMax<ushort>::vlanes = VTraits<v_stype>::vlanes();
template<>
struct ReduceVecOpMax<short>
{
using stype = short;
using itype = short;
using v_stype = v_int16;
using v_itype = v_int16;
static const int vlanes;
static inline v_stype load(const stype *ptr, size_t step) { return vx_load_strided<stype, v_stype>(ptr, step); }
static inline v_itype init() { return vx_setall_s16(std::numeric_limits<itype>::lowest()); }
static inline itype reduce(const v_itype &val) { return v_reduce_max(val); }
inline v_itype operator()(const v_itype &a, const v_stype &b) const { return v_max(a, b); }
};
const int ReduceVecOpMax<short>::vlanes = VTraits<v_stype>::vlanes();
template<>
struct ReduceVecOpMax<float>
{
using stype = float;
using itype = float;
using v_stype = v_float32;
using v_itype = v_float32;
static const int vlanes;
static inline v_stype load(const stype *ptr, size_t step) { return vx_load_strided<stype, v_stype>(ptr, step); }
static inline v_itype init() { return vx_setall_f32(std::numeric_limits<itype>::lowest()); }
static inline itype reduce(const v_itype &val) { return v_reduce_max(val); }
inline v_itype operator()(const v_itype &a, const v_stype &b) const { return v_max(a, b); }
};
const int ReduceVecOpMax<float>::vlanes = VTraits<v_stype>::vlanes();
#endif
#if (CV_SIMD_64F || CV_SIMD_SCALABLE_64F)
template<>
struct ReduceVecOpMax<double>
{
using stype = double;
using itype = double;
using v_stype = v_float64;
using v_itype = v_float64;
static const int vlanes;
static inline v_stype load(const stype *ptr, size_t step) { return vx_load_strided<stype, v_stype>(ptr, step); }
static inline v_itype init() { return vx_setall_f64(std::numeric_limits<itype>::lowest()); }
static inline itype reduce(const v_itype &val)
{
constexpr int nlanes = VTraits<v_itype>::max_nlanes;
itype buf[nlanes];
vx_store(buf, val);
itype m = buf[0];
for (int i = 1; i < VTraits<v_itype>::vlanes(); i++)
{
if (m < buf[i]) m = buf[i];
}
return m;
}
inline v_itype operator()(const v_itype &a, const v_stype &b) const { return v_max(a, b); }
};
const int ReduceVecOpMax<double>::vlanes = VTraits<v_stype>::vlanes();
#endif
template<typename stype>
struct ReduceOpMin
{
using v_stype = stype;
using v_itype = stype;
static const int vlanes;
static inline stype load(const stype *ptr, size_t step) { (void)step; return *ptr; }
static inline stype init() { return std::numeric_limits<stype>::max(); }
static inline stype reduce(const stype &val) { return (stype)val; }
inline stype operator()(const stype &a, const stype &b) const { return std::min(a, b); }
};
template<typename stype>
const int ReduceOpMin<stype>::vlanes = 1;
#if (CV_SIMD || CV_SIMD_SCALABLE)
template<typename stype>
struct ReduceVecOpMin;
template<>
struct ReduceVecOpMin<uchar>
{
using stype = uchar;
using itype = uchar;
using v_stype = v_uint8;
using v_itype = v_uint8;
static const int vlanes;
static inline v_stype load(const stype *ptr, size_t step) { return vx_load_strided<stype, v_stype>(ptr, step); }
static inline v_itype init() { return vx_setall_u8(std::numeric_limits<itype>::max()); }
static inline itype reduce(const v_itype &val) { return v_reduce_min(val); }
inline v_itype operator()(const v_itype &a, const v_stype &b) const { return v_min(a, b); }
};
const int ReduceVecOpMin<uchar>::vlanes = VTraits<v_stype>::vlanes();
template<>
struct ReduceVecOpMin<ushort>
{
using stype = ushort;
using itype = ushort;
using v_stype = v_uint16;
using v_itype = v_uint16;
static const int vlanes;
static inline v_stype load(const stype *ptr, size_t step) { return vx_load_strided<stype, v_stype>(ptr, step); }
static inline v_itype init() { return vx_setall_u16(std::numeric_limits<itype>::max()); }
static inline itype reduce(const v_itype &val) { return v_reduce_min(val); }
inline v_itype operator()(const v_itype &a, const v_stype &b) const { return v_min(a, b); }
};
const int ReduceVecOpMin<ushort>::vlanes = VTraits<v_stype>::vlanes();
template<>
struct ReduceVecOpMin<short>
{
using stype = short;
using itype = short;
using v_stype = v_int16;
using v_itype = v_int16;
static const int vlanes;
static inline v_stype load(const stype *ptr, size_t step) { return vx_load_strided<stype, v_stype>(ptr, step); }
static inline v_itype init() { return vx_setall_s16(std::numeric_limits<itype>::max()); }
static inline itype reduce(const v_itype &val) { return v_reduce_min(val); }
inline v_itype operator()(const v_itype &a, const v_stype &b) const { return v_min(a, b); }
};
const int ReduceVecOpMin<short>::vlanes = VTraits<v_stype>::vlanes();
template<>
struct ReduceVecOpMin<float>
{
using stype = float;
using itype = float;
using v_stype = v_float32;
using v_itype = v_float32;
static const int vlanes;
static inline v_stype load(const stype *ptr, size_t step) { return vx_load_strided<stype, v_stype>(ptr, step); }
static inline v_itype init() { return vx_setall_f32(std::numeric_limits<itype>::max()); }
static inline itype reduce(const v_itype &val) { return v_reduce_min(val); }
inline v_itype operator()(const v_itype &a, const v_stype &b) const { return v_min(a, b); }
};
const int ReduceVecOpMin<float>::vlanes = VTraits<v_stype>::vlanes();
#endif
#if (CV_SIMD_64F || CV_SIMD_SCALABLE_64F)
template<>
struct ReduceVecOpMin<double>
{
using stype = double;
using itype = double;
using v_stype = v_float64;
using v_itype = v_float64;
static const int vlanes;
static inline v_stype load(const stype *ptr, size_t step) { return vx_load_strided<stype, v_stype>(ptr, step); }
static inline v_itype init() { return vx_setall_f64(std::numeric_limits<itype>::max()); }
static inline itype reduce(const v_itype &val)
{
constexpr int nlanes = VTraits<v_itype>::max_nlanes;
itype buf[nlanes];
vx_store(buf, val);
itype m = buf[0];
for (int i = 1; i < VTraits<v_itype>::vlanes(); i++)
{
if (m > buf[i]) m = buf[i];
}
return m;
}
inline v_itype operator()(const v_itype &a, const v_stype &b) const { return v_min(a, b); }
};
const int ReduceVecOpMin<double>::vlanes = VTraits<v_stype>::vlanes();
#endif
using ReduceOpAddSqr_8U32S = ReduceOpAddSqr<uchar, int>;
using ReduceOpAddSqr_8U32F = ReduceOpAddSqr<uchar, int>;
using ReduceOpAddSqr_8U64F = ReduceOpAddSqr<uchar, int>;
using ReduceOpAddSqr_16U32F = ReduceOpAddSqr<ushort, float>;
using ReduceOpAddSqr_16U64F = ReduceOpAddSqr<ushort, double>;
using ReduceOpAddSqr_16S32F = ReduceOpAddSqr<short, float>;
using ReduceOpAddSqr_16S64F = ReduceOpAddSqr<short, double>;
using ReduceOpAddSqr_32F32F = ReduceOpAddSqr<float, float>;
using ReduceOpAddSqr_32F64F = ReduceOpAddSqr<float, double>;
using ReduceOpAddSqr_64F64F = ReduceOpAddSqr<double, double>;
using ReduceOpMax_8U = ReduceOpMax<uchar>;
using ReduceOpMax_16U = ReduceOpMax<ushort>;
using ReduceOpMax_16S = ReduceOpMax<short>;
using ReduceOpMax_32F = ReduceOpMax<float>;
using ReduceOpMax_64F = ReduceOpMax<double>;
using ReduceOpMin_8U = ReduceOpMin<uchar>;
using ReduceOpMin_16U = ReduceOpMin<ushort>;
using ReduceOpMin_16S = ReduceOpMin<short>;
using ReduceOpMin_32F = ReduceOpMin<float>;
using ReduceOpMin_64F = ReduceOpMin<double>;
#if (CV_SIMD || CV_SIMD_SCALABLE)
using ReduceVecOpAddSqr_8U32S = ReduceVecOpAddSqr<uchar, int>;
using ReduceVecOpAddSqr_8U32F = ReduceVecOpAddSqr<uchar, int>;
using ReduceVecOpAddSqr_8U64F = ReduceVecOpAddSqr<uchar, int>;
using ReduceVecOpAddSqr_16U32F = ReduceVecOpAddSqr<ushort, float>;
using ReduceVecOpAddSqr_16S32F = ReduceVecOpAddSqr<short, float>;
using ReduceVecOpAddSqr_32F32F = ReduceVecOpAddSqr<float, float>;
using ReduceVecOpMax_8U = ReduceVecOpMax<uchar>;
using ReduceVecOpMax_16U = ReduceVecOpMax<ushort>;
using ReduceVecOpMax_16S = ReduceVecOpMax<short>;
using ReduceVecOpMax_32F = ReduceVecOpMax<float>;
using ReduceVecOpMin_8U = ReduceVecOpMin<uchar>;
using ReduceVecOpMin_16U = ReduceVecOpMin<ushort>;
using ReduceVecOpMin_16S = ReduceVecOpMin<short>;
using ReduceVecOpMin_32F = ReduceVecOpMin<float>;
#else
using ReduceVecOpAddSqr_8U32S = ReduceOpAddSqr<uchar, int>;
using ReduceVecOpAddSqr_8U32F = ReduceOpAddSqr<uchar, int>;
using ReduceVecOpAddSqr_8U64F = ReduceOpAddSqr<uchar, int>;
using ReduceVecOpAddSqr_16U32F = ReduceOpAddSqr<ushort, float>;
using ReduceVecOpAddSqr_16S32F = ReduceOpAddSqr<short, float>;
using ReduceVecOpAddSqr_32F32F = ReduceOpAddSqr<float, float>;
using ReduceVecOpMax_8U = ReduceOpMax<uchar>;
using ReduceVecOpMax_16U = ReduceOpMax<ushort>;
using ReduceVecOpMax_16S = ReduceOpMax<short>;
using ReduceVecOpMax_32F = ReduceOpMax<float>;
using ReduceVecOpMin_8U = ReduceOpMin<uchar>;
using ReduceVecOpMin_16U = ReduceOpMin<ushort>;
using ReduceVecOpMin_16S = ReduceOpMin<short>;
using ReduceVecOpMin_32F = ReduceOpMin<float>;
#endif
#if (CV_SIMD_64F || CV_SIMD_SCALABLE_64F)
using ReduceVecOpAddSqr_16U64F = ReduceVecOpAddSqr<ushort, double>;
using ReduceVecOpAddSqr_16S64F = ReduceVecOpAddSqr<short, double>;
using ReduceVecOpAddSqr_32F64F = ReduceVecOpAddSqr<float, double>;
using ReduceVecOpAddSqr_64F64F = ReduceVecOpAddSqr<double, double>;
using ReduceVecOpMax_64F = ReduceVecOpMax<double>;
using ReduceVecOpMin_64F = ReduceVecOpMin<double>;
#else
using ReduceVecOpAddSqr_16U64F = ReduceOpAddSqr<ushort, double>;
using ReduceVecOpAddSqr_16S64F = ReduceOpAddSqr<short, double>;
using ReduceVecOpAddSqr_32F64F = ReduceOpAddSqr<float, double>;
using ReduceVecOpAddSqr_64F64F = ReduceOpAddSqr<double, double>;
using ReduceVecOpMax_64F = ReduceOpMax<double>;
using ReduceVecOpMin_64F = ReduceOpMin<double>;
#endif
template<typename T, typename ST, class Op, class VecOp>
class ReduceC_Invoker : public ParallelLoopBody
{
using WT = typename Op::v_itype;
using VT = typename VecOp::v_itype;
public:
ReduceC_Invoker(const Mat& aSrcmat, Mat& aDstmat, Op& aOp, VecOp& aVop)
:srcmat(aSrcmat),dstmat(aDstmat),op(aOp),vop(aVop)
{
}
void operator()(const Range& range) const CV_OVERRIDE
{
int channels = srcmat.channels();
int width = srcmat.cols;
const int nlanes = VecOp::vlanes;
for (int h = range.start; h < range.end; h++)
{
const T *srcrow = srcmat.ptr<T>(h);
ST *dst = dstmat.ptr<ST>(h);
for (int cn = 0; cn < channels; cn++)
{
const T *src = srcrow + cn;
VT vbuf = vop.init();
int w = 0;
for (; w <= width - nlanes; w += nlanes)
{
vbuf = vop(vbuf, vop.load(src+w*channels, channels));
}
WT wbuf = vop.reduce(vbuf);
for (; w < width; w++)
{
wbuf = op(wbuf, op.load(src+w*channels, channels));
}
dst[cn] = (ST)op.reduce(wbuf);
}
}
}
private:
const Mat& srcmat;
Mat& dstmat;
Op& op;
VecOp& vop;
};
template<typename T, typename ST, class Op, class VecOp> static void
reduceColGeneric(const Mat& srcmat, Mat& dstmat)
{
Op op;
VecOp vop;
ReduceC_Invoker<T, ST, Op, VecOp> body(srcmat, dstmat, op, vop);
parallel_for_(Range(0, srcmat.size().height), body);
}
template<bool isMax>
static inline uchar reduceScalarMinMax(uchar a, uchar b)
{
return isMax ? std::max(a, b) : std::min(a, b);
}
template<bool isMax>
static void reduceColMinMax_8uFallback(const Mat& srcmat, Mat& dstmat)
{
if (isMax)
reduceColGeneric<uchar, uchar, ReduceOpMax_8U, ReduceVecOpMax_8U>(srcmat, dstmat);
else
reduceColGeneric<uchar, uchar, ReduceOpMin_8U, ReduceVecOpMin_8U>(srcmat, dstmat);
}
template<bool isMax>
static void reduceColMinMax_16uFallback(const Mat& srcmat, Mat& dstmat)
{
if (isMax)
reduceColGeneric<ushort, ushort, ReduceOpMax_16U, ReduceVecOpMax_16U>(srcmat, dstmat);
else
reduceColGeneric<ushort, ushort, ReduceOpMin_16U, ReduceVecOpMin_16U>(srcmat, dstmat);
}
template<bool isMax>
static void reduceColMinMax_16sFallback(const Mat& srcmat, Mat& dstmat)
{
if (isMax)
reduceColGeneric<short, short, ReduceOpMax_16S, ReduceVecOpMax_16S>(srcmat, dstmat);
else
reduceColGeneric<short, short, ReduceOpMin_16S, ReduceVecOpMin_16S>(srcmat, dstmat);
}
template<bool isMax>
static inline float reduceScalarMinMax(float a, float b)
{
return isMax ? std::max(a, b) : std::min(a, b);
}
template<bool isMax>
static void reduceColMinMax_32fFallback(const Mat& srcmat, Mat& dstmat)
{
if (isMax)
reduceColGeneric<float, float, ReduceOpMax_32F, ReduceVecOpMax_32F>(srcmat, dstmat);
else
reduceColGeneric<float, float, ReduceOpMin_32F, ReduceVecOpMin_32F>(srcmat, dstmat);
}
#if (CV_SIMD_64F || CV_SIMD_SCALABLE_64F)
template<bool isMax>
static void reduceColMinMax_64fFallback(const Mat& srcmat, Mat& dstmat)
{
if (isMax)
reduceColGeneric<double, double, ReduceOpMax_64F, ReduceVecOpMax_64F>(srcmat, dstmat);
else
reduceColGeneric<double, double, ReduceOpMin_64F, ReduceVecOpMin_64F>(srcmat, dstmat);
}
#endif
template<typename DT>
static void reduceColSum2_8uFallback(const Mat& srcmat, Mat& dstmat)
{
if (std::is_same<DT, int>::value)
reduceColGeneric<uchar, int, ReduceOpAddSqr_8U32S, ReduceVecOpAddSqr_8U32S>(srcmat, dstmat);
else if (std::is_same<DT, float>::value)
reduceColGeneric<uchar, float, ReduceOpAddSqr_8U32F, ReduceVecOpAddSqr_8U32F>(srcmat, dstmat);
else
reduceColGeneric<uchar, double, ReduceOpAddSqr_8U64F, ReduceVecOpAddSqr_8U64F>(srcmat, dstmat);
}
static void reduceColSum2_16u32fFallback(const Mat& srcmat, Mat& dstmat)
{
reduceColGeneric<ushort, float, ReduceOpAddSqr_16U32F, ReduceVecOpAddSqr_16U32F>(srcmat, dstmat);
}
static void reduceColSum2_16s32fFallback(const Mat& srcmat, Mat& dstmat)
{
reduceColGeneric<short, float, ReduceOpAddSqr_16S32F, ReduceVecOpAddSqr_16S32F>(srcmat, dstmat);
}
static void reduceColSum2_32f32fFallback(const Mat& srcmat, Mat& dstmat)
{
reduceColGeneric<float, float, ReduceOpAddSqr_32F32F, ReduceVecOpAddSqr_32F32F>(srcmat, dstmat);
}
#if (CV_SIMD_64F || CV_SIMD_SCALABLE_64F)
static void reduceColSum2_16u64fFallback(const Mat& srcmat, Mat& dstmat)
{
reduceColGeneric<ushort, double, ReduceOpAddSqr_16U64F, ReduceVecOpAddSqr_16U64F>(srcmat, dstmat);
}
static void reduceColSum2_16s64fFallback(const Mat& srcmat, Mat& dstmat)
{
reduceColGeneric<short, double, ReduceOpAddSqr_16S64F, ReduceVecOpAddSqr_16S64F>(srcmat, dstmat);
}
static void reduceColSum2_32f64fFallback(const Mat& srcmat, Mat& dstmat)
{
reduceColGeneric<float, double, ReduceOpAddSqr_32F64F, ReduceVecOpAddSqr_32F64F>(srcmat, dstmat);
}
static void reduceColSum2_64f64fFallback(const Mat& srcmat, Mat& dstmat)
{
reduceColGeneric<double, double, ReduceOpAddSqr_64F64F, ReduceVecOpAddSqr_64F64F>(srcmat, dstmat);
}
#endif
+827
View File
@@ -0,0 +1,827 @@
// This file is part of OpenCV project.
// It is subject to the license terms in the LICENSE file found in the top-level directory
// of this distribution and at http://opencv.org/license.html
namespace reduce_c_neon
{
// Optimized ReduceC support in this backend:
//
// | Input -> output type/channel | SUM | AVG | MIN | MAX | SUM2 |
// |------------------------------------|:---:|:---:|:---:|:---:|:----:|
// | 8UC1/C3/C4 -> 8UC1/C3/C4 | - | - | x | x | - |
// | 8UC1/C3/C4 -> 32SC1/C3/C4 | x | x | - | - | x* |
// | 8UC1/C3/C4 -> 32FC1/C3/C4 | x | x | - | - | x* |
// | 8UC1/C3/C4 -> 64FC1/C3/C4 | - | - | - | - | x* |
// | 16UC1/C3/C4 -> 16UC1/C3/C4 | - | - | x | x | - |
// | 16UC1 -> 32FC1 | x | x | - | - | x* |
// | 16UC3/C4 -> 32FC3/C4 | x | x | - | - | - |
// | 16UC1 -> 64FC1 | - | - | - | - | x* |
// | 16UC3/C4 -> 64FC3/C4 | - | - | - | - | - |
// | 16SC1/C3/C4 -> 16SC1/C3/C4 | - | - | x | x | - |
// | 16SC1 -> 32FC1 | x | x | - | - | x* |
// | 16SC3/C4 -> 32FC3/C4 | x | x | - | - | - |
// | 16SC1 -> 64FC1 | - | - | - | - | x* |
// | 16SC3/C4 -> 64FC3/C4 | - | - | - | - | - |
// | 32FC1 -> 32FC1 | x | x | x | x | x |
// | 32FC3 -> 32FC3 | x | x | - | - | - |
// | 32FC4 -> 32FC4 | x | x | x | x | x |
// | 32FC1 -> 64FC1 | x* | x* | - | - | x* |
// | 32FC3/C4 -> 64FC3/C4 | x* | x* | - | - | - |
// | 64FC1 -> 64FC1 | x* | x* | x* | x* | x* |
// | 64FC3/C4 -> 64FC3/C4 | x* | x* | - | - | - |
//
// 'x' in SUM/AVG denotes the existing shared universal-intrinsics kernel; 'x'
// in MIN/MAX/SUM2 denotes a native NEON kernel. '*' requires AArch64. For legal
// MIN/MAX/SUM2 combinations marked '-', and for other channel counts, dispatch
// uses the shared generic fallback.
template<bool isMax>
static void minMax8uC1(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const uchar* src = srcmat.ptr<uchar>(y);
uchar* dst = dstmat.ptr<uchar>(y);
uchar result = isMax ? 0 : UCHAR_MAX;
int x = 0;
uint8x16_t acc = vdupq_n_u8(result);
for (; x <= cols - 16; x += 16)
{
uint8x16_t v = vld1q_u8(src + x);
acc = isMax ? vmaxq_u8(acc, v) : vminq_u8(acc, v);
}
uchar lanes[16];
vst1q_u8(lanes, acc);
for (int i = 0; i < 16; i++)
result = reduceScalarMinMax<isMax>(result, lanes[i]);
for (; x < cols; x++)
result = reduceScalarMinMax<isMax>(result, src[x]);
dst[0] = result;
}
});
v_cleanup();
}
template<bool isMax>
static void minMax16uC1(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const ushort* src = srcmat.ptr<ushort>(y);
ushort result = isMax ? 0 : USHRT_MAX;
int x = 0;
uint16x8_t acc = vdupq_n_u16(result);
for (; x <= cols - 8; x += 8)
{
uint16x8_t v = vld1q_u16(src + x);
acc = isMax ? vmaxq_u16(acc, v) : vminq_u16(acc, v);
}
ushort lanes[8];
vst1q_u16(lanes, acc);
for (int i = 0; i < 8; i++)
result = isMax ? std::max(result, lanes[i]) : std::min(result, lanes[i]);
for (; x < cols; x++)
result = isMax ? std::max(result, src[x]) : std::min(result, src[x]);
dstmat.ptr<ushort>(y)[0] = result;
}
});
v_cleanup();
}
template<bool isMax>
static void minMax16sC1(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const short* src = srcmat.ptr<short>(y);
short result = isMax ? SHRT_MIN : SHRT_MAX;
int x = 0;
int16x8_t acc = vdupq_n_s16(result);
for (; x <= cols - 8; x += 8)
{
int16x8_t v = vld1q_s16(src + x);
acc = isMax ? vmaxq_s16(acc, v) : vminq_s16(acc, v);
}
short lanes[8];
vst1q_s16(lanes, acc);
for (int i = 0; i < 8; i++)
result = isMax ? std::max(result, lanes[i]) : std::min(result, lanes[i]);
for (; x < cols; x++)
result = isMax ? std::max(result, src[x]) : std::min(result, src[x]);
dstmat.ptr<short>(y)[0] = result;
}
});
v_cleanup();
}
template<bool isMax>
static void minMax16uC4(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
const ushort initial = isMax ? 0 : USHRT_MAX;
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const ushort* src = srcmat.ptr<ushort>(y);
ushort* dst = dstmat.ptr<ushort>(y);
uint16x8x4_t acc = {{
vdupq_n_u16(initial), vdupq_n_u16(initial),
vdupq_n_u16(initial), vdupq_n_u16(initial)
}};
int x = 0;
for (; x <= cols - 8; x += 8)
{
uint16x8x4_t v = vld4q_u16(src + x * 4);
for (int c = 0; c < 4; c++)
acc.val[c] = isMax ? vmaxq_u16(acc.val[c], v.val[c])
: vminq_u16(acc.val[c], v.val[c]);
}
for (int c = 0; c < 4; c++)
{
ushort lanes[8];
vst1q_u16(lanes, acc.val[c]);
ushort result = initial;
for (int i = 0; i < 8; i++)
result = isMax ? std::max(result, lanes[i]) : std::min(result, lanes[i]);
for (int i = x; i < cols; i++)
result = isMax ? std::max(result, src[i * 4 + c])
: std::min(result, src[i * 4 + c]);
dst[c] = result;
}
}
});
v_cleanup();
}
template<bool isMax>
static void minMax16sC4(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
const short initial = isMax ? SHRT_MIN : SHRT_MAX;
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const short* src = srcmat.ptr<short>(y);
short* dst = dstmat.ptr<short>(y);
int16x8x4_t acc = {{
vdupq_n_s16(initial), vdupq_n_s16(initial),
vdupq_n_s16(initial), vdupq_n_s16(initial)
}};
int x = 0;
for (; x <= cols - 8; x += 8)
{
int16x8x4_t v = vld4q_s16(src + x * 4);
for (int c = 0; c < 4; c++)
acc.val[c] = isMax ? vmaxq_s16(acc.val[c], v.val[c])
: vminq_s16(acc.val[c], v.val[c]);
}
for (int c = 0; c < 4; c++)
{
short lanes[8];
vst1q_s16(lanes, acc.val[c]);
short result = initial;
for (int i = 0; i < 8; i++)
result = isMax ? std::max(result, lanes[i]) : std::min(result, lanes[i]);
for (int i = x; i < cols; i++)
result = isMax ? std::max(result, src[i * 4 + c])
: std::min(result, src[i * 4 + c]);
dst[c] = result;
}
}
});
v_cleanup();
}
template<bool isMax>
static void minMax16uC3(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
const ushort initial = isMax ? 0 : USHRT_MAX;
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const ushort* src = srcmat.ptr<ushort>(y);
ushort* dst = dstmat.ptr<ushort>(y);
uint16x8x3_t acc = {{vdupq_n_u16(initial), vdupq_n_u16(initial), vdupq_n_u16(initial)}};
int x = 0;
for (; x <= cols - 8; x += 8)
{
uint16x8x3_t v = vld3q_u16(src + x * 3);
for (int c = 0; c < 3; c++)
acc.val[c] = isMax ? vmaxq_u16(acc.val[c], v.val[c])
: vminq_u16(acc.val[c], v.val[c]);
}
for (int c = 0; c < 3; c++)
{
ushort lanes[8];
vst1q_u16(lanes, acc.val[c]);
ushort result = initial;
for (int i = 0; i < 8; i++)
result = isMax ? std::max(result, lanes[i]) : std::min(result, lanes[i]);
for (int i = x; i < cols; i++)
result = isMax ? std::max(result, src[i * 3 + c])
: std::min(result, src[i * 3 + c]);
dst[c] = result;
}
}
});
v_cleanup();
}
template<bool isMax>
static void minMax16sC3(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
const short initial = isMax ? SHRT_MIN : SHRT_MAX;
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const short* src = srcmat.ptr<short>(y);
short* dst = dstmat.ptr<short>(y);
int16x8x3_t acc = {{vdupq_n_s16(initial), vdupq_n_s16(initial), vdupq_n_s16(initial)}};
int x = 0;
for (; x <= cols - 8; x += 8)
{
int16x8x3_t v = vld3q_s16(src + x * 3);
for (int c = 0; c < 3; c++)
acc.val[c] = isMax ? vmaxq_s16(acc.val[c], v.val[c])
: vminq_s16(acc.val[c], v.val[c]);
}
for (int c = 0; c < 3; c++)
{
short lanes[8];
vst1q_s16(lanes, acc.val[c]);
short result = initial;
for (int i = 0; i < 8; i++)
result = isMax ? std::max(result, lanes[i]) : std::min(result, lanes[i]);
for (int i = x; i < cols; i++)
result = isMax ? std::max(result, src[i * 3 + c])
: std::min(result, src[i * 3 + c]);
dst[c] = result;
}
}
});
v_cleanup();
}
template<bool isMax>
static void minMax8uC3(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const uchar* src = srcmat.ptr<uchar>(y);
uchar* dst = dstmat.ptr<uchar>(y);
const uchar initial = isMax ? 0 : UCHAR_MAX;
uint8x16x3_t acc = {{
vdupq_n_u8(initial), vdupq_n_u8(initial), vdupq_n_u8(initial)
}};
int x = 0;
for (; x <= cols - 16; x += 16)
{
uint8x16x3_t v = vld3q_u8(src + x * 3);
for (int c = 0; c < 3; c++)
acc.val[c] = isMax ? vmaxq_u8(acc.val[c], v.val[c])
: vminq_u8(acc.val[c], v.val[c]);
}
for (int c = 0; c < 3; c++)
{
uchar lanes[16];
vst1q_u8(lanes, acc.val[c]);
uchar result = initial;
for (int i = 0; i < 16; i++)
result = reduceScalarMinMax<isMax>(result, lanes[i]);
for (int i = x; i < cols; i++)
result = reduceScalarMinMax<isMax>(result, src[i * 3 + c]);
dst[c] = result;
}
}
});
v_cleanup();
}
template<bool isMax>
static void minMax8uC4(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const uchar* src = srcmat.ptr<uchar>(y);
uchar* dst = dstmat.ptr<uchar>(y);
const uchar initial = isMax ? 0 : UCHAR_MAX;
uint8x16x4_t acc = {{
vdupq_n_u8(initial), vdupq_n_u8(initial),
vdupq_n_u8(initial), vdupq_n_u8(initial)
}};
int x = 0;
for (; x <= cols - 16; x += 16)
{
uint8x16x4_t v = vld4q_u8(src + x * 4);
for (int c = 0; c < 4; c++)
acc.val[c] = isMax ? vmaxq_u8(acc.val[c], v.val[c])
: vminq_u8(acc.val[c], v.val[c]);
}
for (int c = 0; c < 4; c++)
{
uchar lanes[16];
vst1q_u8(lanes, acc.val[c]);
uchar result = initial;
for (int i = 0; i < 16; i++)
result = reduceScalarMinMax<isMax>(result, lanes[i]);
for (int i = x; i < cols; i++)
result = reduceScalarMinMax<isMax>(result, src[i * 4 + c]);
dst[c] = result;
}
}
});
v_cleanup();
}
template<bool isMax>
static void minMax32fC1(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const float* src = srcmat.ptr<float>(y);
float result = src[0];
int x = 0;
float32x4_t acc = vdupq_n_f32(result);
for (; x <= cols - 4; x += 4)
{
float32x4_t v = vld1q_f32(src + x);
acc = isMax ? vmaxq_f32(acc, v) : vminq_f32(acc, v);
}
float lanes[4];
vst1q_f32(lanes, acc);
for (int i = 0; i < 4; i++)
result = reduceScalarMinMax<isMax>(result, lanes[i]);
for (; x < cols; x++)
result = reduceScalarMinMax<isMax>(result, src[x]);
dstmat.ptr<float>(y)[0] = result;
}
});
v_cleanup();
}
template<bool isMax>
static void minMax32fC4(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
const float initial = isMax ? std::numeric_limits<float>::lowest() : std::numeric_limits<float>::max();
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const float* src = srcmat.ptr<float>(y);
float* dst = dstmat.ptr<float>(y);
float32x4x4_t acc = {{
vdupq_n_f32(initial), vdupq_n_f32(initial),
vdupq_n_f32(initial), vdupq_n_f32(initial)
}};
int x = 0;
for (; x <= cols - 4; x += 4)
{
float32x4x4_t v = vld4q_f32(src + x * 4);
for (int c = 0; c < 4; c++)
acc.val[c] = isMax ? vmaxq_f32(acc.val[c], v.val[c])
: vminq_f32(acc.val[c], v.val[c]);
}
for (int c = 0; c < 4; c++)
{
float lanes[4];
vst1q_f32(lanes, acc.val[c]);
float result = initial;
for (int i = 0; i < 4; i++)
result = reduceScalarMinMax<isMax>(result, lanes[i]);
for (int i = x; i < cols; i++)
result = reduceScalarMinMax<isMax>(result, src[i * 4 + c]);
dst[c] = result;
}
}
});
v_cleanup();
}
#if defined(__aarch64__) || defined(_M_ARM64)
template<bool isMax>
static void minMax64fC1(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
const double initial = isMax ? std::numeric_limits<double>::lowest()
: std::numeric_limits<double>::max();
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const double* src = srcmat.ptr<double>(y);
double result = initial;
int x = 0;
float64x2_t acc0 = vdupq_n_f64(initial);
float64x2_t acc1 = acc0, acc2 = acc0, acc3 = acc0;
for (; x <= cols - 8; x += 8)
{
float64x2_t v0 = vld1q_f64(src + x);
float64x2_t v1 = vld1q_f64(src + x + 2);
float64x2_t v2 = vld1q_f64(src + x + 4);
float64x2_t v3 = vld1q_f64(src + x + 6);
acc0 = isMax ? vmaxq_f64(acc0, v0) : vminq_f64(acc0, v0);
acc1 = isMax ? vmaxq_f64(acc1, v1) : vminq_f64(acc1, v1);
acc2 = isMax ? vmaxq_f64(acc2, v2) : vminq_f64(acc2, v2);
acc3 = isMax ? vmaxq_f64(acc3, v3) : vminq_f64(acc3, v3);
}
acc0 = isMax ? vmaxq_f64(acc0, acc1) : vminq_f64(acc0, acc1);
acc2 = isMax ? vmaxq_f64(acc2, acc3) : vminq_f64(acc2, acc3);
acc0 = isMax ? vmaxq_f64(acc0, acc2) : vminq_f64(acc0, acc2);
double lanes[2];
vst1q_f64(lanes, acc0);
for (int i = 0; i < 2; i++)
result = isMax ? std::max(result, lanes[i]) : std::min(result, lanes[i]);
for (; x < cols; x++)
result = isMax ? std::max(result, src[x]) : std::min(result, src[x]);
dstmat.ptr<double>(y)[0] = result;
}
});
v_cleanup();
}
#endif
#if defined(__aarch64__) || defined(_M_ARM64)
static inline uint32_t reduceSum2_8u_NEON(uint8x16_t v)
{
uint16x8_t lo = vmull_u8(vget_low_u8(v), vget_low_u8(v));
uint16x8_t hi = vmull_u8(vget_high_u8(v), vget_high_u8(v));
return vaddvq_u32(vpaddlq_u16(lo)) + vaddvq_u32(vpaddlq_u16(hi));
}
template<typename DT>
static void sum2_8uC1(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const uchar* src = srcmat.ptr<uchar>(y);
DT* dst = dstmat.ptr<DT>(y);
uint32_t result = 0;
int x = 0;
for (; x <= cols - 16; x += 16)
result += reduceSum2_8u_NEON(vld1q_u8(src + x));
for (; x < cols; x++)
result += (uint32_t)src[x] * src[x];
dst[0] = (DT)(int32_t)result;
}
});
v_cleanup();
}
static void sum2_16u32fC1(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const ushort* src = srcmat.ptr<ushort>(y);
float result = 0;
float32x4_t acc0 = vdupq_n_f32(0), acc1 = acc0, acc2 = acc0, acc3 = acc0;
int x = 0;
for (; x <= cols - 16; x += 16)
{
uint16x8_t v01 = vld1q_u16(src + x);
uint16x8_t v23 = vld1q_u16(src + x + 8);
float32x4_t v0 = vcvtq_f32_u32(vmovl_u16(vget_low_u16(v01)));
float32x4_t v1 = vcvtq_f32_u32(vmovl_u16(vget_high_u16(v01)));
float32x4_t v2 = vcvtq_f32_u32(vmovl_u16(vget_low_u16(v23)));
float32x4_t v3 = vcvtq_f32_u32(vmovl_u16(vget_high_u16(v23)));
acc0 = vmlaq_f32(acc0, v0, v0);
acc1 = vmlaq_f32(acc1, v1, v1);
acc2 = vmlaq_f32(acc2, v2, v2);
acc3 = vmlaq_f32(acc3, v3, v3);
}
acc0 = vaddq_f32(vaddq_f32(acc0, acc1), vaddq_f32(acc2, acc3));
float lanes[4];
vst1q_f32(lanes, acc0);
for (int i = 0; i < 4; i++)
result += lanes[i];
for (; x < cols; x++)
{
float value = (float)src[x];
result += value * value;
}
dstmat.ptr<float>(y)[0] = result;
}
});
v_cleanup();
}
static void sum2_16s32fC1(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const short* src = srcmat.ptr<short>(y);
float result = 0;
float32x4_t acc0 = vdupq_n_f32(0), acc1 = acc0, acc2 = acc0, acc3 = acc0;
int x = 0;
for (; x <= cols - 16; x += 16)
{
int16x8_t v01 = vld1q_s16(src + x);
int16x8_t v23 = vld1q_s16(src + x + 8);
float32x4_t v0 = vcvtq_f32_s32(vmovl_s16(vget_low_s16(v01)));
float32x4_t v1 = vcvtq_f32_s32(vmovl_s16(vget_high_s16(v01)));
float32x4_t v2 = vcvtq_f32_s32(vmovl_s16(vget_low_s16(v23)));
float32x4_t v3 = vcvtq_f32_s32(vmovl_s16(vget_high_s16(v23)));
acc0 = vmlaq_f32(acc0, v0, v0);
acc1 = vmlaq_f32(acc1, v1, v1);
acc2 = vmlaq_f32(acc2, v2, v2);
acc3 = vmlaq_f32(acc3, v3, v3);
}
acc0 = vaddq_f32(vaddq_f32(acc0, acc1), vaddq_f32(acc2, acc3));
float lanes[4];
vst1q_f32(lanes, acc0);
for (int i = 0; i < 4; i++)
result += lanes[i];
for (; x < cols; x++)
{
float value = (float)src[x];
result += value * value;
}
dstmat.ptr<float>(y)[0] = result;
}
});
v_cleanup();
}
static void sum2_16u64fC1(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const ushort* src = srcmat.ptr<ushort>(y);
uint64x2_t acc0 = vdupq_n_u64(0), acc1 = acc0, acc2 = acc0, acc3 = acc0;
int x = 0;
for (; x <= cols - 8; x += 8)
{
uint16x8_t v = vld1q_u16(src + x);
uint32x4_t sq0 = vmull_u16(vget_low_u16(v), vget_low_u16(v));
uint32x4_t sq1 = vmull_u16(vget_high_u16(v), vget_high_u16(v));
acc0 = vaddq_u64(acc0, vmovl_u32(vget_low_u32(sq0)));
acc1 = vaddq_u64(acc1, vmovl_high_u32(sq0));
acc2 = vaddq_u64(acc2, vmovl_u32(vget_low_u32(sq1)));
acc3 = vaddq_u64(acc3, vmovl_high_u32(sq1));
}
acc0 = vaddq_u64(vaddq_u64(acc0, acc1), vaddq_u64(acc2, acc3));
uint64_t lanes[2];
vst1q_u64(lanes, acc0);
uint64_t result = lanes[0] + lanes[1];
for (; x < cols; x++)
result += (uint64_t)src[x] * src[x];
dstmat.ptr<double>(y)[0] = (double)result;
}
});
v_cleanup();
}
static void sum2_16s64fC1(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const short* src = srcmat.ptr<short>(y);
uint64x2_t acc0 = vdupq_n_u64(0), acc1 = acc0, acc2 = acc0, acc3 = acc0;
int x = 0;
for (; x <= cols - 8; x += 8)
{
int16x8_t v = vld1q_s16(src + x);
int32x4_t sq0 = vmull_s16(vget_low_s16(v), vget_low_s16(v));
int32x4_t sq1 = vmull_s16(vget_high_s16(v), vget_high_s16(v));
uint32x4_t usq0 = vreinterpretq_u32_s32(sq0);
uint32x4_t usq1 = vreinterpretq_u32_s32(sq1);
acc0 = vaddq_u64(acc0, vmovl_u32(vget_low_u32(usq0)));
acc1 = vaddq_u64(acc1, vmovl_high_u32(usq0));
acc2 = vaddq_u64(acc2, vmovl_u32(vget_low_u32(usq1)));
acc3 = vaddq_u64(acc3, vmovl_high_u32(usq1));
}
acc0 = vaddq_u64(vaddq_u64(acc0, acc1), vaddq_u64(acc2, acc3));
uint64_t lanes[2];
vst1q_u64(lanes, acc0);
uint64_t result = lanes[0] + lanes[1];
for (; x < cols; x++)
{
int64_t value = src[x];
result += (uint64_t)(value * value);
}
dstmat.ptr<double>(y)[0] = (double)result;
}
});
v_cleanup();
}
static void sum2_32f64fC1(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const float* src = srcmat.ptr<float>(y);
float64x2_t acc0 = vdupq_n_f64(0), acc1 = acc0, acc2 = acc0, acc3 = acc0;
int x = 0;
for (; x <= cols - 8; x += 8)
{
float32x4_t v01 = vld1q_f32(src + x);
float32x4_t v23 = vld1q_f32(src + x + 4);
float64x2_t v0 = vcvt_f64_f32(vget_low_f32(v01));
float64x2_t v1 = vcvt_high_f64_f32(v01);
float64x2_t v2 = vcvt_f64_f32(vget_low_f32(v23));
float64x2_t v3 = vcvt_high_f64_f32(v23);
acc0 = vmlaq_f64(acc0, v0, v0);
acc1 = vmlaq_f64(acc1, v1, v1);
acc2 = vmlaq_f64(acc2, v2, v2);
acc3 = vmlaq_f64(acc3, v3, v3);
}
acc0 = vaddq_f64(vaddq_f64(acc0, acc1), vaddq_f64(acc2, acc3));
double lanes[2];
vst1q_f64(lanes, acc0);
double result = lanes[0] + lanes[1];
for (; x < cols; x++)
{
double value = (double)src[x];
result += value * value;
}
dstmat.ptr<double>(y)[0] = result;
}
});
v_cleanup();
}
static void sum2_64f64fC1(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const double* src = srcmat.ptr<double>(y);
float64x2_t acc0 = vdupq_n_f64(0), acc1 = acc0, acc2 = acc0, acc3 = acc0;
int x = 0;
for (; x <= cols - 8; x += 8)
{
float64x2_t v0 = vld1q_f64(src + x);
float64x2_t v1 = vld1q_f64(src + x + 2);
float64x2_t v2 = vld1q_f64(src + x + 4);
float64x2_t v3 = vld1q_f64(src + x + 6);
acc0 = vmlaq_f64(acc0, v0, v0);
acc1 = vmlaq_f64(acc1, v1, v1);
acc2 = vmlaq_f64(acc2, v2, v2);
acc3 = vmlaq_f64(acc3, v3, v3);
}
acc0 = vaddq_f64(vaddq_f64(acc0, acc1), vaddq_f64(acc2, acc3));
double lanes[2];
vst1q_f64(lanes, acc0);
double result = lanes[0] + lanes[1];
for (; x < cols; x++)
result += src[x] * src[x];
dstmat.ptr<double>(y)[0] = result;
}
});
v_cleanup();
}
template<typename DT>
static void sum2_8uC3(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const uchar* src = srcmat.ptr<uchar>(y);
DT* dst = dstmat.ptr<DT>(y);
uint32_t results[3] = {0, 0, 0};
int x = 0;
for (; x <= cols - 16; x += 16)
{
uint8x16x3_t v = vld3q_u8(src + x * 3);
for (int c = 0; c < 3; c++)
results[c] += reduceSum2_8u_NEON(v.val[c]);
}
for (int c = 0; c < 3; c++)
{
for (int i = x; i < cols; i++)
{
uint32_t value = src[i * 3 + c];
results[c] += value * value;
}
dst[c] = (DT)(int32_t)results[c];
}
}
});
v_cleanup();
}
template<typename DT>
static void sum2_8uC4(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const uchar* src = srcmat.ptr<uchar>(y);
DT* dst = dstmat.ptr<DT>(y);
uint32_t results[4] = {0, 0, 0, 0};
int x = 0;
for (; x <= cols - 16; x += 16)
{
uint8x16x4_t v = vld4q_u8(src + x * 4);
for (int c = 0; c < 4; c++)
results[c] += reduceSum2_8u_NEON(v.val[c]);
}
for (int c = 0; c < 4; c++)
{
for (int i = x; i < cols; i++)
{
uint32_t value = src[i * 4 + c];
results[c] += value * value;
}
dst[c] = (DT)(int32_t)results[c];
}
}
});
v_cleanup();
}
#endif
static void sum2_32fC1(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const float* src = srcmat.ptr<float>(y);
float result = 0;
int x = 0;
float32x4_t acc = vdupq_n_f32(0);
for (; x <= cols - 4; x += 4)
{
float32x4_t v = vld1q_f32(src + x);
acc = vaddq_f32(acc, vmulq_f32(v, v));
}
float lanes[4];
vst1q_f32(lanes, acc);
for (int i = 0; i < 4; i++)
result += lanes[i];
for (; x < cols; x++)
result += src[x] * src[x];
dstmat.ptr<float>(y)[0] = result;
}
});
v_cleanup();
}
static void sum2_32fC4(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const float* src = srcmat.ptr<float>(y);
float* dst = dstmat.ptr<float>(y);
float32x4x4_t acc = {{
vdupq_n_f32(0), vdupq_n_f32(0),
vdupq_n_f32(0), vdupq_n_f32(0)
}};
int x = 0;
for (; x <= cols - 4; x += 4)
{
float32x4x4_t v = vld4q_f32(src + x * 4);
for (int c = 0; c < 4; c++)
acc.val[c] = vmlaq_f32(acc.val[c], v.val[c], v.val[c]);
}
for (int c = 0; c < 4; c++)
{
float lanes[4];
float result = 0;
vst1q_f32(lanes, acc.val[c]);
for (int i = 0; i < 4; i++)
result += lanes[i];
for (int i = x; i < cols; i++)
result += src[i * 4 + c] * src[i * 4 + c];
dst[c] = result;
}
}
});
v_cleanup();
}
} // namespace reduce_c_neon
+866
View File
@@ -0,0 +1,866 @@
// This file is part of OpenCV project.
// It is subject to the license terms in the LICENSE file found in the top-level directory
// of this distribution and at http://opencv.org/license.html
namespace reduce_c_rvv
{
// Optimized ReduceC support in this backend:
//
// | Input -> output type/channel | SUM | AVG | MIN | MAX | SUM2 |
// |------------------------------------|:---:|:---:|:---:|:---:|:----:|
// | 8UC1/C3/C4 -> 8UC1/C3/C4 | - | - | x | x | - |
// | 8UC1/C3/C4 -> 32SC1/C3/C4 | x | x | - | - | x |
// | 8UC1/C3/C4 -> 32FC1/C3/C4 | x | x | - | - | x |
// | 8UC1/C3/C4 -> 64FC1/C3/C4 | - | - | - | - | x |
// | 16UC1/C3/C4 -> 16UC1/C3/C4 | - | - | x | x | - |
// | 16UC1 -> 32FC1 | x | x | - | - | x |
// | 16UC3/C4 -> 32FC3/C4 | x | x | - | - | - |
// | 16UC1 -> 64FC1 | - | - | - | - | x* |
// | 16UC3/C4 -> 64FC3/C4 | - | - | - | - | - |
// | 16SC1/C3/C4 -> 16SC1/C3/C4 | - | - | x | x | - |
// | 16SC1 -> 32FC1 | x | x | - | - | x |
// | 16SC3/C4 -> 32FC3/C4 | x | x | - | - | - |
// | 16SC1 -> 64FC1 | - | - | - | - | x* |
// | 16SC3/C4 -> 64FC3/C4 | - | - | - | - | - |
// | 32FC1/C3/C4 -> 32FC1/C3/C4 | x | x | x | x | x |
// | 32FC1 -> 64FC1 | x* | x* | - | - | x* |
// | 32FC3/C4 -> 64FC3/C4 | x* | x* | - | - | - |
// | 64FC1 -> 64FC1 | x* | x* | x* | x* | x* |
// | 64FC3/C4 -> 64FC3/C4 | x* | x* | - | - | - |
//
// 'x' in SUM/AVG denotes the existing shared universal-intrinsics kernel; 'x'
// in MIN/MAX/SUM2 denotes a native RVV kernel. '*' requires
// CV_SIMD_SCALABLE_64F. For legal MIN/MAX/SUM2 combinations marked '-', and for
// other channel counts, dispatch uses the shared generic fallback.
template<bool isMax>
static void minMax8uC1(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const uchar* src = srcmat.ptr<uchar>(y);
uchar* dst = dstmat.ptr<uchar>(y);
uchar result = isMax ? 0 : UCHAR_MAX;
int x = 0;
const int vlmax = __riscv_vsetvlmax_e8m8();
vuint8m8_t acc = __riscv_vmv_v_x_u8m8(result, vlmax);
for (; x < cols; )
{
const int vl = __riscv_vsetvl_e8m8(cols - x);
vuint8m8_t v = __riscv_vle8_v_u8m8(src + x, vl);
acc = isMax ? __riscv_vmaxu_tu(acc, acc, v, vl)
: __riscv_vminu_tu(acc, acc, v, vl);
x += vl;
}
vuint8m1_t seed = __riscv_vmv_s_x_u8m1(result, __riscv_vsetvlmax_e8m1());
vuint8m1_t reduced = isMax ? __riscv_vredmaxu(acc, seed, vlmax)
: __riscv_vredminu(acc, seed, vlmax);
result = (uchar)__riscv_vmv_x(reduced);
for (; x < cols; x++)
result = reduceScalarMinMax<isMax>(result, src[x]);
dst[0] = result;
}
});
v_cleanup();
}
template<bool isMax>
static void minMax16uC1(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const ushort* src = srcmat.ptr<ushort>(y);
const ushort initial = isMax ? 0 : USHRT_MAX;
const int vlmax = __riscv_vsetvlmax_e16m8();
vuint16m8_t acc = __riscv_vmv_v_x_u16m8(initial, vlmax);
int x = 0;
for (; x < cols; )
{
const int vl = __riscv_vsetvl_e16m8(cols - x);
vuint16m8_t v = __riscv_vle16_v_u16m8(src + x, vl);
acc = isMax ? __riscv_vmaxu_tu(acc, acc, v, vl)
: __riscv_vminu_tu(acc, acc, v, vl);
x += vl;
}
vuint16m1_t seed = __riscv_vmv_s_x_u16m1(initial, __riscv_vsetvlmax_e16m1());
vuint16m1_t reduced = isMax ? __riscv_vredmaxu(acc, seed, vlmax)
: __riscv_vredminu(acc, seed, vlmax);
dstmat.ptr<ushort>(y)[0] = (ushort)__riscv_vmv_x(reduced);
}
});
v_cleanup();
}
template<bool isMax>
static void minMax16sC1(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const short* src = srcmat.ptr<short>(y);
const short initial = isMax ? SHRT_MIN : SHRT_MAX;
const int vlmax = __riscv_vsetvlmax_e16m8();
vint16m8_t acc = __riscv_vmv_v_x_i16m8(initial, vlmax);
int x = 0;
for (; x < cols; )
{
const int vl = __riscv_vsetvl_e16m8(cols - x);
vint16m8_t v = __riscv_vle16_v_i16m8(src + x, vl);
acc = isMax ? __riscv_vmax_tu(acc, acc, v, vl)
: __riscv_vmin_tu(acc, acc, v, vl);
x += vl;
}
vint16m1_t seed = __riscv_vmv_s_x_i16m1(initial, __riscv_vsetvlmax_e16m1());
vint16m1_t reduced = isMax ? __riscv_vredmax(acc, seed, vlmax)
: __riscv_vredmin(acc, seed, vlmax);
dstmat.ptr<short>(y)[0] = (short)__riscv_vmv_x(reduced);
}
});
v_cleanup();
}
template<bool isMax>
static void minMax16uC4(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
const ushort initial = isMax ? 0 : USHRT_MAX;
const int vlmax = __riscv_vsetvlmax_e16m2();
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const ushort* src = srcmat.ptr<ushort>(y);
ushort* dst = dstmat.ptr<ushort>(y);
vuint16m2_t acc0 = __riscv_vmv_v_x_u16m2(initial, vlmax);
vuint16m2_t acc1 = acc0, acc2 = acc0, acc3 = acc0;
int x = 0;
for (; x < cols; )
{
const int vl = __riscv_vsetvl_e16m2(cols - x);
vuint16m2x4_t v = __riscv_vlseg4e16_v_u16m2x4(src + x * 4, vl);
vuint16m2_t v0 = __riscv_vget_v_u16m2x4_u16m2(v, 0);
vuint16m2_t v1 = __riscv_vget_v_u16m2x4_u16m2(v, 1);
vuint16m2_t v2 = __riscv_vget_v_u16m2x4_u16m2(v, 2);
vuint16m2_t v3 = __riscv_vget_v_u16m2x4_u16m2(v, 3);
acc0 = isMax ? __riscv_vmaxu_tu(acc0, acc0, v0, vl)
: __riscv_vminu_tu(acc0, acc0, v0, vl);
acc1 = isMax ? __riscv_vmaxu_tu(acc1, acc1, v1, vl)
: __riscv_vminu_tu(acc1, acc1, v1, vl);
acc2 = isMax ? __riscv_vmaxu_tu(acc2, acc2, v2, vl)
: __riscv_vminu_tu(acc2, acc2, v2, vl);
acc3 = isMax ? __riscv_vmaxu_tu(acc3, acc3, v3, vl)
: __riscv_vminu_tu(acc3, acc3, v3, vl);
x += vl;
}
vuint16m1_t seed = __riscv_vmv_s_x_u16m1(initial, __riscv_vsetvlmax_e16m1());
dst[0] = (ushort)__riscv_vmv_x(isMax ? __riscv_vredmaxu(acc0, seed, vlmax) : __riscv_vredminu(acc0, seed, vlmax));
dst[1] = (ushort)__riscv_vmv_x(isMax ? __riscv_vredmaxu(acc1, seed, vlmax) : __riscv_vredminu(acc1, seed, vlmax));
dst[2] = (ushort)__riscv_vmv_x(isMax ? __riscv_vredmaxu(acc2, seed, vlmax) : __riscv_vredminu(acc2, seed, vlmax));
dst[3] = (ushort)__riscv_vmv_x(isMax ? __riscv_vredmaxu(acc3, seed, vlmax) : __riscv_vredminu(acc3, seed, vlmax));
}
});
v_cleanup();
}
template<bool isMax>
static void minMax16sC4(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
const short initial = isMax ? SHRT_MIN : SHRT_MAX;
const int vlmax = __riscv_vsetvlmax_e16m2();
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const short* src = srcmat.ptr<short>(y);
short* dst = dstmat.ptr<short>(y);
vint16m2_t acc0 = __riscv_vmv_v_x_i16m2(initial, vlmax);
vint16m2_t acc1 = acc0, acc2 = acc0, acc3 = acc0;
int x = 0;
for (; x < cols; )
{
const int vl = __riscv_vsetvl_e16m2(cols - x);
vint16m2x4_t v = __riscv_vlseg4e16_v_i16m2x4(src + x * 4, vl);
vint16m2_t v0 = __riscv_vget_v_i16m2x4_i16m2(v, 0);
vint16m2_t v1 = __riscv_vget_v_i16m2x4_i16m2(v, 1);
vint16m2_t v2 = __riscv_vget_v_i16m2x4_i16m2(v, 2);
vint16m2_t v3 = __riscv_vget_v_i16m2x4_i16m2(v, 3);
acc0 = isMax ? __riscv_vmax_tu(acc0, acc0, v0, vl)
: __riscv_vmin_tu(acc0, acc0, v0, vl);
acc1 = isMax ? __riscv_vmax_tu(acc1, acc1, v1, vl)
: __riscv_vmin_tu(acc1, acc1, v1, vl);
acc2 = isMax ? __riscv_vmax_tu(acc2, acc2, v2, vl)
: __riscv_vmin_tu(acc2, acc2, v2, vl);
acc3 = isMax ? __riscv_vmax_tu(acc3, acc3, v3, vl)
: __riscv_vmin_tu(acc3, acc3, v3, vl);
x += vl;
}
vint16m1_t seed = __riscv_vmv_s_x_i16m1(initial, __riscv_vsetvlmax_e16m1());
dst[0] = (short)__riscv_vmv_x(isMax ? __riscv_vredmax(acc0, seed, vlmax) : __riscv_vredmin(acc0, seed, vlmax));
dst[1] = (short)__riscv_vmv_x(isMax ? __riscv_vredmax(acc1, seed, vlmax) : __riscv_vredmin(acc1, seed, vlmax));
dst[2] = (short)__riscv_vmv_x(isMax ? __riscv_vredmax(acc2, seed, vlmax) : __riscv_vredmin(acc2, seed, vlmax));
dst[3] = (short)__riscv_vmv_x(isMax ? __riscv_vredmax(acc3, seed, vlmax) : __riscv_vredmin(acc3, seed, vlmax));
}
});
v_cleanup();
}
template<bool isMax>
static void minMax16uC3(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
const ushort initial = isMax ? 0 : USHRT_MAX;
const int vlmax = __riscv_vsetvlmax_e16m2();
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const ushort* src = srcmat.ptr<ushort>(y);
ushort* dst = dstmat.ptr<ushort>(y);
vuint16m2_t acc0 = __riscv_vmv_v_x_u16m2(initial, vlmax);
vuint16m2_t acc1 = acc0, acc2 = acc0;
int x = 0;
for (; x < cols; )
{
const int vl = __riscv_vsetvl_e16m2(cols - x);
vuint16m2x3_t v = __riscv_vlseg3e16_v_u16m2x3(src + x * 3, vl);
vuint16m2_t v0 = __riscv_vget_v_u16m2x3_u16m2(v, 0);
vuint16m2_t v1 = __riscv_vget_v_u16m2x3_u16m2(v, 1);
vuint16m2_t v2 = __riscv_vget_v_u16m2x3_u16m2(v, 2);
acc0 = isMax ? __riscv_vmaxu_tu(acc0, acc0, v0, vl) : __riscv_vminu_tu(acc0, acc0, v0, vl);
acc1 = isMax ? __riscv_vmaxu_tu(acc1, acc1, v1, vl) : __riscv_vminu_tu(acc1, acc1, v1, vl);
acc2 = isMax ? __riscv_vmaxu_tu(acc2, acc2, v2, vl) : __riscv_vminu_tu(acc2, acc2, v2, vl);
x += vl;
}
vuint16m1_t seed = __riscv_vmv_s_x_u16m1(initial, __riscv_vsetvlmax_e16m1());
dst[0] = (ushort)__riscv_vmv_x(isMax ? __riscv_vredmaxu(acc0, seed, vlmax) : __riscv_vredminu(acc0, seed, vlmax));
dst[1] = (ushort)__riscv_vmv_x(isMax ? __riscv_vredmaxu(acc1, seed, vlmax) : __riscv_vredminu(acc1, seed, vlmax));
dst[2] = (ushort)__riscv_vmv_x(isMax ? __riscv_vredmaxu(acc2, seed, vlmax) : __riscv_vredminu(acc2, seed, vlmax));
}
});
v_cleanup();
}
template<bool isMax>
static void minMax16sC3(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
const short initial = isMax ? SHRT_MIN : SHRT_MAX;
const int vlmax = __riscv_vsetvlmax_e16m2();
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const short* src = srcmat.ptr<short>(y);
short* dst = dstmat.ptr<short>(y);
vint16m2_t acc0 = __riscv_vmv_v_x_i16m2(initial, vlmax);
vint16m2_t acc1 = acc0, acc2 = acc0;
int x = 0;
for (; x < cols; )
{
const int vl = __riscv_vsetvl_e16m2(cols - x);
vint16m2x3_t v = __riscv_vlseg3e16_v_i16m2x3(src + x * 3, vl);
vint16m2_t v0 = __riscv_vget_v_i16m2x3_i16m2(v, 0);
vint16m2_t v1 = __riscv_vget_v_i16m2x3_i16m2(v, 1);
vint16m2_t v2 = __riscv_vget_v_i16m2x3_i16m2(v, 2);
acc0 = isMax ? __riscv_vmax_tu(acc0, acc0, v0, vl) : __riscv_vmin_tu(acc0, acc0, v0, vl);
acc1 = isMax ? __riscv_vmax_tu(acc1, acc1, v1, vl) : __riscv_vmin_tu(acc1, acc1, v1, vl);
acc2 = isMax ? __riscv_vmax_tu(acc2, acc2, v2, vl) : __riscv_vmin_tu(acc2, acc2, v2, vl);
x += vl;
}
vint16m1_t seed = __riscv_vmv_s_x_i16m1(initial, __riscv_vsetvlmax_e16m1());
dst[0] = (short)__riscv_vmv_x(isMax ? __riscv_vredmax(acc0, seed, vlmax) : __riscv_vredmin(acc0, seed, vlmax));
dst[1] = (short)__riscv_vmv_x(isMax ? __riscv_vredmax(acc1, seed, vlmax) : __riscv_vredmin(acc1, seed, vlmax));
dst[2] = (short)__riscv_vmv_x(isMax ? __riscv_vredmax(acc2, seed, vlmax) : __riscv_vredmin(acc2, seed, vlmax));
}
});
v_cleanup();
}
template<bool isMax>
static void minMax8uC3(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
const uchar initial = isMax ? 0 : UCHAR_MAX;
const int vlmax = __riscv_vsetvlmax_e8m1();
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const uchar* src = srcmat.ptr<uchar>(y);
uchar* dst = dstmat.ptr<uchar>(y);
vuint8m1_t acc0 = __riscv_vmv_v_x_u8m1(initial, vlmax);
vuint8m1_t acc1 = acc0, acc2 = acc0;
int x = 0;
for (; x < cols; )
{
const int vl = __riscv_vsetvl_e8m1(cols - x);
vuint8m1x3_t v = __riscv_vlseg3e8_v_u8m1x3(src + x * 3, vl);
vuint8m1_t v0 = __riscv_vget_v_u8m1x3_u8m1(v, 0);
vuint8m1_t v1 = __riscv_vget_v_u8m1x3_u8m1(v, 1);
vuint8m1_t v2 = __riscv_vget_v_u8m1x3_u8m1(v, 2);
acc0 = isMax ? __riscv_vmaxu_tu(acc0, acc0, v0, vl)
: __riscv_vminu_tu(acc0, acc0, v0, vl);
acc1 = isMax ? __riscv_vmaxu_tu(acc1, acc1, v1, vl)
: __riscv_vminu_tu(acc1, acc1, v1, vl);
acc2 = isMax ? __riscv_vmaxu_tu(acc2, acc2, v2, vl)
: __riscv_vminu_tu(acc2, acc2, v2, vl);
x += vl;
}
vuint8m1_t seed = __riscv_vmv_s_x_u8m1(initial, vlmax);
dst[0] = (uchar)__riscv_vmv_x(isMax ? __riscv_vredmaxu(acc0, seed, vlmax)
: __riscv_vredminu(acc0, seed, vlmax));
dst[1] = (uchar)__riscv_vmv_x(isMax ? __riscv_vredmaxu(acc1, seed, vlmax)
: __riscv_vredminu(acc1, seed, vlmax));
dst[2] = (uchar)__riscv_vmv_x(isMax ? __riscv_vredmaxu(acc2, seed, vlmax)
: __riscv_vredminu(acc2, seed, vlmax));
}
});
v_cleanup();
}
template<bool isMax>
static void minMax8uC4(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
const uchar initial = isMax ? 0 : UCHAR_MAX;
const int vlmax = __riscv_vsetvlmax_e8m1();
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const uchar* src = srcmat.ptr<uchar>(y);
uchar* dst = dstmat.ptr<uchar>(y);
vuint8m1_t acc0 = __riscv_vmv_v_x_u8m1(initial, vlmax);
vuint8m1_t acc1 = acc0, acc2 = acc0, acc3 = acc0;
int x = 0;
for (; x < cols; )
{
const int vl = __riscv_vsetvl_e8m1(cols - x);
vuint8m1x4_t v = __riscv_vlseg4e8_v_u8m1x4(src + x * 4, vl);
vuint8m1_t v0 = __riscv_vget_v_u8m1x4_u8m1(v, 0);
vuint8m1_t v1 = __riscv_vget_v_u8m1x4_u8m1(v, 1);
vuint8m1_t v2 = __riscv_vget_v_u8m1x4_u8m1(v, 2);
vuint8m1_t v3 = __riscv_vget_v_u8m1x4_u8m1(v, 3);
acc0 = isMax ? __riscv_vmaxu_tu(acc0, acc0, v0, vl)
: __riscv_vminu_tu(acc0, acc0, v0, vl);
acc1 = isMax ? __riscv_vmaxu_tu(acc1, acc1, v1, vl)
: __riscv_vminu_tu(acc1, acc1, v1, vl);
acc2 = isMax ? __riscv_vmaxu_tu(acc2, acc2, v2, vl)
: __riscv_vminu_tu(acc2, acc2, v2, vl);
acc3 = isMax ? __riscv_vmaxu_tu(acc3, acc3, v3, vl)
: __riscv_vminu_tu(acc3, acc3, v3, vl);
x += vl;
}
vuint8m1_t seed = __riscv_vmv_s_x_u8m1(initial, vlmax);
dst[0] = (uchar)__riscv_vmv_x(isMax ? __riscv_vredmaxu(acc0, seed, vlmax)
: __riscv_vredminu(acc0, seed, vlmax));
dst[1] = (uchar)__riscv_vmv_x(isMax ? __riscv_vredmaxu(acc1, seed, vlmax)
: __riscv_vredminu(acc1, seed, vlmax));
dst[2] = (uchar)__riscv_vmv_x(isMax ? __riscv_vredmaxu(acc2, seed, vlmax)
: __riscv_vredminu(acc2, seed, vlmax));
dst[3] = (uchar)__riscv_vmv_x(isMax ? __riscv_vredmaxu(acc3, seed, vlmax)
: __riscv_vredminu(acc3, seed, vlmax));
}
});
v_cleanup();
}
template<bool isMax>
static void minMax32fC1(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const float* src = srcmat.ptr<float>(y);
float result = src[0];
int x = 0;
const int vlmax = __riscv_vsetvlmax_e32m8();
vfloat32m8_t acc = __riscv_vfmv_v_f_f32m8(result, vlmax);
for (; x < cols; )
{
const int vl = __riscv_vsetvl_e32m8(cols - x);
vfloat32m8_t v = __riscv_vle32_v_f32m8(src + x, vl);
acc = isMax ? __riscv_vfmax_tu(acc, acc, v, vl)
: __riscv_vfmin_tu(acc, acc, v, vl);
x += vl;
}
vfloat32m1_t seed = __riscv_vfmv_s_f_f32m1(result, __riscv_vsetvlmax_e32m1());
vfloat32m1_t reduced = isMax ? __riscv_vfredmax(acc, seed, vlmax)
: __riscv_vfredmin(acc, seed, vlmax);
result = __riscv_vfmv_f(reduced);
for (; x < cols; x++)
result = reduceScalarMinMax<isMax>(result, src[x]);
dstmat.ptr<float>(y)[0] = result;
}
});
v_cleanup();
}
template<bool isMax>
static void minMax32fC3(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
const float initial = isMax ? std::numeric_limits<float>::lowest() : std::numeric_limits<float>::max();
const int vlmax = __riscv_vsetvlmax_e32m2();
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const float* src = srcmat.ptr<float>(y);
float* dst = dstmat.ptr<float>(y);
vfloat32m2_t acc0 = __riscv_vfmv_v_f_f32m2(initial, vlmax);
vfloat32m2_t acc1 = acc0, acc2 = acc0;
int x = 0;
for (; x < cols; )
{
const int vl = __riscv_vsetvl_e32m2(cols - x);
vfloat32m2x3_t v = __riscv_vlseg3e32_v_f32m2x3(src + x * 3, vl);
vfloat32m2_t v0 = __riscv_vget_v_f32m2x3_f32m2(v, 0);
vfloat32m2_t v1 = __riscv_vget_v_f32m2x3_f32m2(v, 1);
vfloat32m2_t v2 = __riscv_vget_v_f32m2x3_f32m2(v, 2);
acc0 = isMax ? __riscv_vfmax_tu(acc0, acc0, v0, vl)
: __riscv_vfmin_tu(acc0, acc0, v0, vl);
acc1 = isMax ? __riscv_vfmax_tu(acc1, acc1, v1, vl)
: __riscv_vfmin_tu(acc1, acc1, v1, vl);
acc2 = isMax ? __riscv_vfmax_tu(acc2, acc2, v2, vl)
: __riscv_vfmin_tu(acc2, acc2, v2, vl);
x += vl;
}
vfloat32m1_t seed = __riscv_vfmv_s_f_f32m1(initial, __riscv_vsetvlmax_e32m1());
dst[0] = __riscv_vfmv_f(isMax ? __riscv_vfredmax(acc0, seed, vlmax)
: __riscv_vfredmin(acc0, seed, vlmax));
dst[1] = __riscv_vfmv_f(isMax ? __riscv_vfredmax(acc1, seed, vlmax)
: __riscv_vfredmin(acc1, seed, vlmax));
dst[2] = __riscv_vfmv_f(isMax ? __riscv_vfredmax(acc2, seed, vlmax)
: __riscv_vfredmin(acc2, seed, vlmax));
}
});
v_cleanup();
}
template<bool isMax>
static void minMax32fC4(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
const float initial = isMax ? std::numeric_limits<float>::lowest() : std::numeric_limits<float>::max();
const int vlmax = __riscv_vsetvlmax_e32m2();
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const float* src = srcmat.ptr<float>(y);
float* dst = dstmat.ptr<float>(y);
vfloat32m2_t acc0 = __riscv_vfmv_v_f_f32m2(initial, vlmax);
vfloat32m2_t acc1 = acc0, acc2 = acc0, acc3 = acc0;
int x = 0;
for (; x < cols; )
{
const int vl = __riscv_vsetvl_e32m2(cols - x);
vfloat32m2x4_t v = __riscv_vlseg4e32_v_f32m2x4(src + x * 4, vl);
vfloat32m2_t v0 = __riscv_vget_v_f32m2x4_f32m2(v, 0);
vfloat32m2_t v1 = __riscv_vget_v_f32m2x4_f32m2(v, 1);
vfloat32m2_t v2 = __riscv_vget_v_f32m2x4_f32m2(v, 2);
vfloat32m2_t v3 = __riscv_vget_v_f32m2x4_f32m2(v, 3);
acc0 = isMax ? __riscv_vfmax_tu(acc0, acc0, v0, vl)
: __riscv_vfmin_tu(acc0, acc0, v0, vl);
acc1 = isMax ? __riscv_vfmax_tu(acc1, acc1, v1, vl)
: __riscv_vfmin_tu(acc1, acc1, v1, vl);
acc2 = isMax ? __riscv_vfmax_tu(acc2, acc2, v2, vl)
: __riscv_vfmin_tu(acc2, acc2, v2, vl);
acc3 = isMax ? __riscv_vfmax_tu(acc3, acc3, v3, vl)
: __riscv_vfmin_tu(acc3, acc3, v3, vl);
x += vl;
}
vfloat32m1_t seed = __riscv_vfmv_s_f_f32m1(initial, __riscv_vsetvlmax_e32m1());
dst[0] = __riscv_vfmv_f(isMax ? __riscv_vfredmax(acc0, seed, vlmax)
: __riscv_vfredmin(acc0, seed, vlmax));
dst[1] = __riscv_vfmv_f(isMax ? __riscv_vfredmax(acc1, seed, vlmax)
: __riscv_vfredmin(acc1, seed, vlmax));
dst[2] = __riscv_vfmv_f(isMax ? __riscv_vfredmax(acc2, seed, vlmax)
: __riscv_vfredmin(acc2, seed, vlmax));
dst[3] = __riscv_vfmv_f(isMax ? __riscv_vfredmax(acc3, seed, vlmax)
: __riscv_vfredmin(acc3, seed, vlmax));
}
});
v_cleanup();
}
#if CV_SIMD_SCALABLE_64F
template<bool isMax>
static void minMax64fC1(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
const double initial = isMax ? std::numeric_limits<double>::lowest()
: std::numeric_limits<double>::max();
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const double* src = srcmat.ptr<double>(y);
const int vlmax = __riscv_vsetvlmax_e64m8();
vfloat64m8_t acc = __riscv_vfmv_v_f_f64m8(initial, vlmax);
int x = 0;
for (; x < cols; )
{
const int vl = __riscv_vsetvl_e64m8(cols - x);
vfloat64m8_t v = __riscv_vle64_v_f64m8(src + x, vl);
acc = isMax ? __riscv_vfmax_tu(acc, acc, v, vl)
: __riscv_vfmin_tu(acc, acc, v, vl);
x += vl;
}
vfloat64m1_t seed = __riscv_vfmv_s_f_f64m1(initial, __riscv_vsetvlmax_e64m1());
vfloat64m1_t reduced = isMax ? __riscv_vfredmax(acc, seed, vlmax)
: __riscv_vfredmin(acc, seed, vlmax);
dstmat.ptr<double>(y)[0] = __riscv_vfmv_f(reduced);
}
});
v_cleanup();
}
#endif
template<typename DT>
static void sum2_8uC1(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const uchar* src = srcmat.ptr<uchar>(y);
DT* dst = dstmat.ptr<DT>(y);
uint32_t result = 0;
int x = 0;
vuint32m1_t acc = __riscv_vmv_v_x_u32m1(0, __riscv_vsetvlmax_e32m1());
for (; x < cols; )
{
const int vl = __riscv_vsetvl_e8m4(cols - x);
vuint8m4_t v = __riscv_vle8_v_u8m4(src + x, vl);
acc = __riscv_vwredsumu(__riscv_vwmulu(v, v, vl), acc, vl);
x += vl;
}
result = (uint32_t)__riscv_vmv_x(acc);
for (; x < cols; x++)
result += (uint32_t)src[x] * src[x];
dst[0] = (DT)(int32_t)result;
}
});
v_cleanup();
}
static void sum2_16u32fC1(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const ushort* src = srcmat.ptr<ushort>(y);
const int vlmax = __riscv_vsetvlmax_e32m8();
vfloat32m8_t acc = __riscv_vfmv_v_f_f32m8(0, vlmax);
int x = 0;
for (; x < cols; )
{
const int vl = __riscv_vsetvl_e16m4(cols - x);
vuint16m4_t v = __riscv_vle16_v_u16m4(src + x, vl);
vfloat32m8_t vf = __riscv_vfwcvt_f_xu_v_f32m8(v, vl);
acc = __riscv_vfmacc_tu(acc, vf, vf, vl);
x += vl;
}
vfloat32m1_t zero = __riscv_vfmv_s_f_f32m1(0, __riscv_vsetvlmax_e32m1());
dstmat.ptr<float>(y)[0] = __riscv_vfmv_f(__riscv_vfredusum(acc, zero, vlmax));
}
});
v_cleanup();
}
static void sum2_16s32fC1(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const short* src = srcmat.ptr<short>(y);
const int vlmax = __riscv_vsetvlmax_e32m8();
vfloat32m8_t acc = __riscv_vfmv_v_f_f32m8(0, vlmax);
int x = 0;
for (; x < cols; )
{
const int vl = __riscv_vsetvl_e16m4(cols - x);
vint16m4_t v = __riscv_vle16_v_i16m4(src + x, vl);
vfloat32m8_t vf = __riscv_vfwcvt_f_x_v_f32m8(v, vl);
acc = __riscv_vfmacc_tu(acc, vf, vf, vl);
x += vl;
}
vfloat32m1_t zero = __riscv_vfmv_s_f_f32m1(0, __riscv_vsetvlmax_e32m1());
dstmat.ptr<float>(y)[0] = __riscv_vfmv_f(__riscv_vfredusum(acc, zero, vlmax));
}
});
v_cleanup();
}
#if CV_SIMD_SCALABLE_64F
static void sum2_16u64fC1(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const ushort* src = srcmat.ptr<ushort>(y);
const int vlmax = __riscv_vsetvlmax_e64m8();
vfloat64m8_t acc = __riscv_vfmv_v_f_f64m8(0, vlmax);
int x = 0;
for (; x < cols; )
{
const int vl = __riscv_vsetvl_e16m2(cols - x);
vuint16m2_t v = __riscv_vle16_v_u16m2(src + x, vl);
vfloat32m4_t vf = __riscv_vfwcvt_f_xu_v_f32m4(v, vl);
vfloat64m8_t vd = __riscv_vfwcvt_f_f_v_f64m8(vf, vl);
acc = __riscv_vfmacc_tu(acc, vd, vd, vl);
x += vl;
}
vfloat64m1_t zero = __riscv_vfmv_s_f_f64m1(0, __riscv_vsetvlmax_e64m1());
dstmat.ptr<double>(y)[0] = __riscv_vfmv_f(__riscv_vfredusum(acc, zero, vlmax));
}
});
v_cleanup();
}
static void sum2_16s64fC1(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const short* src = srcmat.ptr<short>(y);
const int vlmax = __riscv_vsetvlmax_e64m8();
vfloat64m8_t acc = __riscv_vfmv_v_f_f64m8(0, vlmax);
int x = 0;
for (; x < cols; )
{
const int vl = __riscv_vsetvl_e16m2(cols - x);
vint16m2_t v = __riscv_vle16_v_i16m2(src + x, vl);
vfloat32m4_t vf = __riscv_vfwcvt_f_x_v_f32m4(v, vl);
vfloat64m8_t vd = __riscv_vfwcvt_f_f_v_f64m8(vf, vl);
acc = __riscv_vfmacc_tu(acc, vd, vd, vl);
x += vl;
}
vfloat64m1_t zero = __riscv_vfmv_s_f_f64m1(0, __riscv_vsetvlmax_e64m1());
dstmat.ptr<double>(y)[0] = __riscv_vfmv_f(__riscv_vfredusum(acc, zero, vlmax));
}
});
v_cleanup();
}
static void sum2_32f64fC1(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const float* src = srcmat.ptr<float>(y);
const int vlmax = __riscv_vsetvlmax_e64m8();
vfloat64m8_t acc = __riscv_vfmv_v_f_f64m8(0, vlmax);
int x = 0;
for (; x < cols; )
{
const int vl = __riscv_vsetvl_e32m4(cols - x);
vfloat32m4_t v = __riscv_vle32_v_f32m4(src + x, vl);
vfloat64m8_t vd = __riscv_vfwcvt_f_f_v_f64m8(v, vl);
acc = __riscv_vfmacc_tu(acc, vd, vd, vl);
x += vl;
}
vfloat64m1_t zero = __riscv_vfmv_s_f_f64m1(0, __riscv_vsetvlmax_e64m1());
dstmat.ptr<double>(y)[0] = __riscv_vfmv_f(__riscv_vfredusum(acc, zero, vlmax));
}
});
v_cleanup();
}
static void sum2_64f64fC1(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const double* src = srcmat.ptr<double>(y);
const int vlmax = __riscv_vsetvlmax_e64m8();
vfloat64m8_t acc = __riscv_vfmv_v_f_f64m8(0, vlmax);
int x = 0;
for (; x < cols; )
{
const int vl = __riscv_vsetvl_e64m8(cols - x);
vfloat64m8_t v = __riscv_vle64_v_f64m8(src + x, vl);
acc = __riscv_vfmacc_tu(acc, v, v, vl);
x += vl;
}
vfloat64m1_t zero = __riscv_vfmv_s_f_f64m1(0, __riscv_vsetvlmax_e64m1());
dstmat.ptr<double>(y)[0] = __riscv_vfmv_f(__riscv_vfredusum(acc, zero, vlmax));
}
});
v_cleanup();
}
#endif
template<typename DT>
static void sum2_8uC3(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const uchar* src = srcmat.ptr<uchar>(y);
DT* dst = dstmat.ptr<DT>(y);
vuint32m1_t acc0 = __riscv_vmv_v_x_u32m1(0, __riscv_vsetvlmax_e32m1());
vuint32m1_t acc1 = acc0, acc2 = acc0;
int x = 0;
for (; x < cols; )
{
const int vl = __riscv_vsetvl_e8m1(cols - x);
vuint8m1x3_t v = __riscv_vlseg3e8_v_u8m1x3(src + x * 3, vl);
vuint8m1_t v0 = __riscv_vget_v_u8m1x3_u8m1(v, 0);
vuint8m1_t v1 = __riscv_vget_v_u8m1x3_u8m1(v, 1);
vuint8m1_t v2 = __riscv_vget_v_u8m1x3_u8m1(v, 2);
acc0 = __riscv_vwredsumu(__riscv_vwmulu(v0, v0, vl), acc0, vl);
acc1 = __riscv_vwredsumu(__riscv_vwmulu(v1, v1, vl), acc1, vl);
acc2 = __riscv_vwredsumu(__riscv_vwmulu(v2, v2, vl), acc2, vl);
x += vl;
}
dst[0] = (DT)(int32_t)(uint32_t)__riscv_vmv_x(acc0);
dst[1] = (DT)(int32_t)(uint32_t)__riscv_vmv_x(acc1);
dst[2] = (DT)(int32_t)(uint32_t)__riscv_vmv_x(acc2);
}
});
v_cleanup();
}
template<typename DT>
static void sum2_8uC4(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const uchar* src = srcmat.ptr<uchar>(y);
DT* dst = dstmat.ptr<DT>(y);
vuint32m1_t acc0 = __riscv_vmv_v_x_u32m1(0, __riscv_vsetvlmax_e32m1());
vuint32m1_t acc1 = acc0, acc2 = acc0, acc3 = acc0;
int x = 0;
for (; x < cols; )
{
const int vl = __riscv_vsetvl_e8m1(cols - x);
vuint8m1x4_t v = __riscv_vlseg4e8_v_u8m1x4(src + x * 4, vl);
vuint8m1_t v0 = __riscv_vget_v_u8m1x4_u8m1(v, 0);
vuint8m1_t v1 = __riscv_vget_v_u8m1x4_u8m1(v, 1);
vuint8m1_t v2 = __riscv_vget_v_u8m1x4_u8m1(v, 2);
vuint8m1_t v3 = __riscv_vget_v_u8m1x4_u8m1(v, 3);
acc0 = __riscv_vwredsumu(__riscv_vwmulu(v0, v0, vl), acc0, vl);
acc1 = __riscv_vwredsumu(__riscv_vwmulu(v1, v1, vl), acc1, vl);
acc2 = __riscv_vwredsumu(__riscv_vwmulu(v2, v2, vl), acc2, vl);
acc3 = __riscv_vwredsumu(__riscv_vwmulu(v3, v3, vl), acc3, vl);
x += vl;
}
dst[0] = (DT)(int32_t)(uint32_t)__riscv_vmv_x(acc0);
dst[1] = (DT)(int32_t)(uint32_t)__riscv_vmv_x(acc1);
dst[2] = (DT)(int32_t)(uint32_t)__riscv_vmv_x(acc2);
dst[3] = (DT)(int32_t)(uint32_t)__riscv_vmv_x(acc3);
}
});
v_cleanup();
}
static void sum2_32fC1(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const float* src = srcmat.ptr<float>(y);
float result = 0;
int x = 0;
const int vlmax = __riscv_vsetvlmax_e32m8();
vfloat32m8_t acc = __riscv_vfmv_v_f_f32m8(0, vlmax);
for (; x < cols; )
{
const int vl = __riscv_vsetvl_e32m8(cols - x);
vfloat32m8_t v = __riscv_vle32_v_f32m8(src + x, vl);
acc = __riscv_vfmacc_tu(acc, v, v, vl);
x += vl;
}
vfloat32m1_t zero = __riscv_vfmv_s_f_f32m1(0, __riscv_vsetvlmax_e32m1());
result = __riscv_vfmv_f(__riscv_vfredusum(acc, zero, vlmax));
for (; x < cols; x++)
result += src[x] * src[x];
dstmat.ptr<float>(y)[0] = result;
}
});
v_cleanup();
}
static void sum2_32fC3(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
const int vlmax = __riscv_vsetvlmax_e32m2();
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const float* src = srcmat.ptr<float>(y);
float* dst = dstmat.ptr<float>(y);
vfloat32m2_t acc0 = __riscv_vfmv_v_f_f32m2(0, vlmax);
vfloat32m2_t acc1 = acc0, acc2 = acc0;
int x = 0;
for (; x < cols; )
{
const int vl = __riscv_vsetvl_e32m2(cols - x);
vfloat32m2x3_t v = __riscv_vlseg3e32_v_f32m2x3(src + x * 3, vl);
vfloat32m2_t v0 = __riscv_vget_v_f32m2x3_f32m2(v, 0);
vfloat32m2_t v1 = __riscv_vget_v_f32m2x3_f32m2(v, 1);
vfloat32m2_t v2 = __riscv_vget_v_f32m2x3_f32m2(v, 2);
acc0 = __riscv_vfmacc_tu(acc0, v0, v0, vl);
acc1 = __riscv_vfmacc_tu(acc1, v1, v1, vl);
acc2 = __riscv_vfmacc_tu(acc2, v2, v2, vl);
x += vl;
}
vfloat32m1_t zero = __riscv_vfmv_s_f_f32m1(0, __riscv_vsetvlmax_e32m1());
dst[0] = __riscv_vfmv_f(__riscv_vfredusum(acc0, zero, vlmax));
dst[1] = __riscv_vfmv_f(__riscv_vfredusum(acc1, zero, vlmax));
dst[2] = __riscv_vfmv_f(__riscv_vfredusum(acc2, zero, vlmax));
}
});
v_cleanup();
}
static void sum2_32fC4(const Mat& srcmat, Mat& dstmat)
{
const int cols = srcmat.cols;
const int vlmax = __riscv_vsetvlmax_e32m2();
parallel_for_(Range(0, srcmat.rows), [&](const Range& range) {
for (int y = range.start; y < range.end; y++)
{
const float* src = srcmat.ptr<float>(y);
float* dst = dstmat.ptr<float>(y);
vfloat32m2_t acc0 = __riscv_vfmv_v_f_f32m2(0, vlmax);
vfloat32m2_t acc1 = acc0, acc2 = acc0, acc3 = acc0;
int x = 0;
for (; x < cols; )
{
const int vl = __riscv_vsetvl_e32m2(cols - x);
vfloat32m2x4_t v = __riscv_vlseg4e32_v_f32m2x4(src + x * 4, vl);
vfloat32m2_t v0 = __riscv_vget_v_f32m2x4_f32m2(v, 0);
vfloat32m2_t v1 = __riscv_vget_v_f32m2x4_f32m2(v, 1);
vfloat32m2_t v2 = __riscv_vget_v_f32m2x4_f32m2(v, 2);
vfloat32m2_t v3 = __riscv_vget_v_f32m2x4_f32m2(v, 3);
acc0 = __riscv_vfmacc_tu(acc0, v0, v0, vl);
acc1 = __riscv_vfmacc_tu(acc1, v1, v1, vl);
acc2 = __riscv_vfmacc_tu(acc2, v2, v2, vl);
acc3 = __riscv_vfmacc_tu(acc3, v3, v3, vl);
x += vl;
}
vfloat32m1_t zero = __riscv_vfmv_s_f_f32m1(0, __riscv_vsetvlmax_e32m1());
dst[0] = __riscv_vfmv_f(__riscv_vfredusum(acc0, zero, vlmax));
dst[1] = __riscv_vfmv_f(__riscv_vfredusum(acc1, zero, vlmax));
dst[2] = __riscv_vfmv_f(__riscv_vfredusum(acc2, zero, vlmax));
dst[3] = __riscv_vfmv_f(__riscv_vfredusum(acc3, zero, vlmax));
}
});
v_cleanup();
}
} // namespace reduce_c_rvv
+1 -1
View File
@@ -170,7 +170,7 @@ int Core_ReduceTest::checkOp( const Mat& src, int dstType, int opType, const Mat
getMatTypeStr( dstType, dstTypeStr ); getMatTypeStr( dstType, dstTypeStr );
const char* dimStr = dim == 0 ? "ROWS" : "COLS"; const char* dimStr = dim == 0 ? "ROWS" : "COLS";
snprintf( msg, sizeof(msg), "bad accuracy with srcType = %s, dstType = %s, opType = %s, dim = %s", snprintf( msg, sizeof(msg), "bad accuracy with srcType = %s, dstType = %s, opType = %s, dim = %s\n",
srcTypeStr.c_str(), dstTypeStr.c_str(), opTypeStr, dimStr ); srcTypeStr.c_str(), dstTypeStr.c_str(), opTypeStr, dimStr );
ts->printf( cvtest::TS::LOG, msg ); ts->printf( cvtest::TS::LOG, msg );
return cvtest::TS::FAIL_BAD_ACCURACY; return cvtest::TS::FAIL_BAD_ACCURACY;
+27
View File
@@ -1602,6 +1602,33 @@ TEST(Core_Arithm, scalar_handling_19599) // https://github.com/opencv/opencv/is
EXPECT_EQ(1, c.rows); EXPECT_EQ(1, c.rows);
} }
TEST(Core_ExtractChannel, twoChannel8uSubmatrix)
{
Mat sourceStorage(5, 37, CV_8UC2);
randu(sourceStorage, 0, 256);
Mat source = sourceStorage(Rect(3, 1, 31, 3));
ASSERT_FALSE(source.isContinuous());
for (int channel = 0; channel < 2; ++channel)
{
Mat destinationStorage(5, 37, CV_8UC1, Scalar::all(0));
Mat destination = destinationStorage(Rect(3, 1, source.cols, source.rows));
Mat expected(source.size(), CV_8UC1);
ASSERT_FALSE(destination.isContinuous());
for (int row = 0; row < source.rows; ++row)
{
const uchar* sourceRow = source.ptr<uchar>(row);
uchar* expectedRow = expected.ptr<uchar>(row);
for (int column = 0; column < source.cols; ++column)
expectedRow[column] = sourceRow[column * 2 + channel];
}
extractChannel(source, destination, channel);
EXPECT_EQ(0, cvtest::norm(expected, destination, NORM_INF));
}
}
// https://github.com/opencv/opencv/issues/24163 // https://github.com/opencv/opencv/issues/24163
typedef tuple<perf::MatDepth,int,int,int> Arith_Regression24163Param; typedef tuple<perf::MatDepth,int,int,int> Arith_Regression24163Param;
typedef testing::TestWithParam<Arith_Regression24163Param> Core_Arith_Regression24163; typedef testing::TestWithParam<Arith_Regression24163Param> Core_Arith_Regression24163;
+7 -1
View File
@@ -6,6 +6,8 @@
#include "npy_blob.hpp" #include "npy_blob.hpp"
#include <opencv2/dnn/shape_utils.hpp> #include <opencv2/dnn/shape_utils.hpp>
#include <opencv2/dnn/all_layers.hpp> #include <opencv2/dnn/all_layers.hpp>
#include <iostream>
namespace opencv_test { namespace { namespace opencv_test { namespace {
testing::internal::ParamGenerator< tuple<Backend, Target> > dnnBackendsAndTargetsInt8() testing::internal::ParamGenerator< tuple<Backend, Target> > dnnBackendsAndTargetsInt8()
@@ -34,6 +36,7 @@ public:
int numInps = 1, int numOuts = 1, bool useCaffeModel = false, int numInps = 1, int numOuts = 1, bool useCaffeModel = false,
bool useCommonInputBlob = true, bool hasText = false, bool perChannel = true) bool useCommonInputBlob = true, bool hasText = false, bool perChannel = true)
{ {
std::cout << "Testning layer " << basename << std::endl;
CV_Assert_N(numInps >= 1, numInps <= 10, numOuts >= 1, numOuts <= 10); CV_Assert_N(numInps >= 1, numInps <= 10, numOuts >= 1, numOuts <= 10);
std::vector<Mat> inps(numInps), inps_int8(numInps); std::vector<Mat> inps(numInps), inps_int8(numInps);
std::vector<Mat> refs(numOuts), outs_int8(numOuts), outs_dequantized(numOuts); std::vector<Mat> refs(numOuts), outs_int8(numOuts), outs_dequantized(numOuts);
@@ -239,7 +242,10 @@ TEST_P(Test_Int8_layers, MaxPooling)
TEST_P(Test_Int8_layers, Reduce) TEST_P(Test_Int8_layers, Reduce)
{ {
testLayer("reduce_mean", "TensorFlow", 0.0005, 0.0014); // Test fails on some CI hosts
if (backend != DNN_BACKEND_INFERENCE_ENGINE_NGRAPH)
testLayer("reduce_mean", "TensorFlow", 0.0005, 0.0014);
testLayer("reduce_mean", "ONNX", 0.00062, 0.0014); testLayer("reduce_mean", "ONNX", 0.00062, 0.0014);
testLayer("reduce_mean_axis1", "ONNX", 0.00032, 0.0007); testLayer("reduce_mean_axis1", "ONNX", 0.00032, 0.0007);
testLayer("reduce_mean_axis2", "ONNX", 0.00033, 0.001); testLayer("reduce_mean_axis2", "ONNX", 0.00033, 0.001);
@@ -328,6 +328,42 @@ TEST(Features2d_FLANN_Composite, regression) { CV_FlannCompositeIndexTest test;
TEST(Features2d_FLANN_Auto, regression) { CV_FlannAutotunedIndexTest test; test.safe_run(); } TEST(Features2d_FLANN_Auto, regression) { CV_FlannAutotunedIndexTest test; test.safe_run(); }
TEST(Features2d_FLANN_Saved, regression) { CV_FlannSavedIndexTest test; test.safe_run(); } TEST(Features2d_FLANN_Saved, regression) { CV_FlannSavedIndexTest test; test.safe_run(); }
// A saved KD-tree index whose serialized leaf node carries a point index
// outside the dataset must be rejected on load. Before the added validation
// the malformed index loaded silently and the out-of-range index was
// dereferenced during search (heap out-of-bounds access).
TEST(Features2d_FLANN_KDTree, load_rejects_out_of_range_leaf_index)
{
Mat features(1, 4, CV_32F);
features.at<float>(0, 0) = 1.f; features.at<float>(0, 1) = 2.f;
features.at<float>(0, 2) = 3.f; features.at<float>(0, 3) = 4.f;
const String filename = tempfile();
{
Index index(features, KDTreeIndexParams(1));
index.save(filename);
}
// Overwrite the single leaf node's divfeat (the first int of the last
// serialized Node record) with an index far outside the 1-row dataset.
{
FILE* f = fopen(filename.c_str(), "r+b");
ASSERT_TRUE(f != NULL);
ASSERT_EQ(0, fseek(f, 0, SEEK_END));
const long node_size = (long)(sizeof(int) + sizeof(float) + 2 * sizeof(void*));
const long size = ftell(f);
ASSERT_GT(size, node_size);
ASSERT_EQ(0, fseek(f, size - node_size, SEEK_SET));
const int out_of_range = 1 << 28;
ASSERT_EQ((size_t)1, fwrite(&out_of_range, sizeof(int), 1, f));
fclose(f);
}
Index loaded;
EXPECT_THROW(loaded.load(features, filename), cv::Exception);
remove(filename.c_str());
}
#endif #endif
}} // namespace }} // namespace
@@ -266,11 +266,25 @@ private:
{ {
tree = pool_.allocate<Node>(); tree = pool_.allocate<Node>();
load_value(stream, *tree); load_value(stream, *tree);
if (tree->child1!=NULL) { if (tree->child1!=NULL || tree->child2!=NULL) {
load_tree(stream, tree->child1); // Internal node: divfeat is the split dimension and is used to index
// the query vector during search, so it must be a valid dimension.
if (tree->divfeat < 0 || (size_t)tree->divfeat >= veclen_) {
FLANN_THROW(cv::Error::StsParseError, "FLANN kd-tree index: split dimension is out of range");
}
if (tree->child1!=NULL) {
load_tree(stream, tree->child1);
}
if (tree->child2!=NULL) {
load_tree(stream, tree->child2);
}
} }
if (tree->child2!=NULL) { else {
load_tree(stream, tree->child2); // Leaf node: divfeat is a dataset point index dereferenced during
// search, so it must fall inside the dataset.
if (tree->divfeat < 0 || (size_t)tree->divfeat >= size_) {
FLANN_THROW(cv::Error::StsParseError, "FLANN kd-tree index: leaf feature index is out of range");
}
} }
} }
@@ -158,12 +158,33 @@ public:
{ {
load_value(stream, size_); load_value(stream, size_);
load_value(stream, dim_); load_value(stream, dim_);
// The dataset the index is attached to is fixed by the caller, so a
// saved index whose stored size/dim disagree with it is malformed.
if (size_ != dataset_.rows || dim_ != dataset_.cols) {
FLANN_THROW(cv::Error::StsParseError, "FLANN kd-tree(single) index: saved dataset dimensions do not match");
}
load_value(stream, root_bbox_); load_value(stream, root_bbox_);
if (root_bbox_.size() != dim_) {
FLANN_THROW(cv::Error::StsParseError, "FLANN kd-tree(single) index: bounding box has wrong length");
}
load_value(stream, reorder_); load_value(stream, reorder_);
load_value(stream, leaf_max_size_); load_value(stream, leaf_max_size_);
load_value(stream, vind_); load_value(stream, vind_);
// vind_ holds one dataset point index per row and every entry is
// dereferenced during search.
if (vind_.size() != size_) {
FLANN_THROW(cv::Error::StsParseError, "FLANN kd-tree(single) index: index permutation has wrong length");
}
for (size_t i = 0; i < vind_.size(); ++i) {
if (vind_[i] < 0 || (size_t)vind_[i] >= size_) {
FLANN_THROW(cv::Error::StsParseError, "FLANN kd-tree(single) index: point index is out of range");
}
}
if (reorder_) { if (reorder_) {
load_value(stream, data_); load_value(stream, data_);
if (data_.rows != size_ || data_.cols != dim_) {
FLANN_THROW(cv::Error::StsParseError, "FLANN kd-tree(single) index: reordered data has wrong shape");
}
} }
else { else {
data_ = dataset_; data_ = dataset_;
@@ -303,11 +324,25 @@ private:
{ {
tree = pool_.allocate<Node>(); tree = pool_.allocate<Node>();
load_value(stream, *tree); load_value(stream, *tree);
if (tree->child1!=NULL) { if (tree->child1!=NULL || tree->child2!=NULL) {
load_tree(stream, tree->child1); // Internal node: divfeat is the split dimension, used to index the
// query vector and the per-dimension distance array during search.
if (tree->divfeat < 0 || (size_t)tree->divfeat >= dim_) {
FLANN_THROW(cv::Error::StsParseError, "FLANN kd-tree(single) index: split dimension is out of range");
}
if (tree->child1!=NULL) {
load_tree(stream, tree->child1);
}
if (tree->child2!=NULL) {
load_tree(stream, tree->child2);
}
} }
if (tree->child2!=NULL) { else {
load_tree(stream, tree->child2); // Leaf node: [left, right) is a range of point slots dereferenced
// during search, so it must stay inside the dataset.
if (tree->left < 0 || tree->right < tree->left || (size_t)tree->right > size_) {
FLANN_THROW(cv::Error::StsParseError, "FLANN kd-tree(single) index: leaf point range is out of range");
}
} }
} }
@@ -77,7 +77,7 @@ TEST_P(GStreamerSourceTest, AccuracyTest)
EXPECT_FALSE(ccomp.running()); EXPECT_FALSE(ccomp.running());
EXPECT_EQ(streamLength, framesCount); EXPECT_NEAR(streamLength, framesCount, 1);
} }
TEST_P(GStreamerSourceTest, TimestampsTest) TEST_P(GStreamerSourceTest, TimestampsTest)
@@ -124,12 +124,12 @@ TEST_P(GStreamerSourceTest, TimestampsTest)
EXPECT_FALSE(ccomp.running()); EXPECT_FALSE(ccomp.running());
EXPECT_EQ(0L, allSeqIds.front()); EXPECT_EQ(0L, allSeqIds.front());
EXPECT_EQ(int64_t(streamLength) - 1, allSeqIds.back()); EXPECT_NEAR(int64_t(streamLength) - 1, allSeqIds.back(), 1);
EXPECT_EQ(streamLength, allSeqIds.size()); EXPECT_NEAR(streamLength, allSeqIds.size(), 1);
EXPECT_TRUE(std::is_sorted(allSeqIds.begin(), allSeqIds.end())); EXPECT_TRUE(std::is_sorted(allSeqIds.begin(), allSeqIds.end()));
EXPECT_EQ(allSeqIds.size(), std::set<int64_t>(allSeqIds.begin(), allSeqIds.end()).size()); EXPECT_EQ(allSeqIds.size(), std::set<int64_t>(allSeqIds.begin(), allSeqIds.end()).size());
EXPECT_EQ(streamLength, allTimestamps.size()); EXPECT_NEAR(streamLength, allTimestamps.size(), 1);
EXPECT_TRUE(std::is_sorted(allTimestamps.begin(), allTimestamps.end())); EXPECT_TRUE(std::is_sorted(allTimestamps.begin(), allTimestamps.end()));
} }
@@ -270,7 +270,7 @@ TEST_P(GStreamerSourceTest, GFrameTest)
EXPECT_FALSE(ccomp.running()); EXPECT_FALSE(ccomp.running());
EXPECT_EQ(streamLength, framesCount); EXPECT_NEAR(streamLength, framesCount, 1);
} }
+6 -6
View File
@@ -169,13 +169,13 @@ void icvCvt_Gray2BGR_16u_C1C3R( const ushort* gray, int gray_step,
ushort* bgr, int bgr_step, Size size ) ushort* bgr, int bgr_step, Size size )
{ {
int i; int i;
for( ; size.height--; gray += gray_step/sizeof(gray[0]) ) for( ; size.height--; gray += gray_step/(int)sizeof(gray[0]) )
{ {
for( i = 0; i < size.width; i++, bgr += 3 ) for( i = 0; i < size.width; i++, bgr += 3 )
{ {
bgr[0] = bgr[1] = bgr[2] = gray[i]; bgr[0] = bgr[1] = bgr[2] = gray[i];
} }
bgr += bgr_step/sizeof(bgr[0]) - size.width*3; bgr += bgr_step/(int)sizeof(bgr[0]) - size.width*3;
} }
} }
@@ -214,8 +214,8 @@ void icvCvt_BGRA2BGR_16u_C4C3R( const ushort* bgra, int bgra_step,
bgr[0] = t0; bgr[1] = t1; bgr[0] = t0; bgr[1] = t1;
t0 = bgra[swap_rb^2]; bgr[2] = t0; t0 = bgra[swap_rb^2]; bgr[2] = t0;
} }
bgr += bgr_step/sizeof(bgr[0]) - size.width*3; bgr += bgr_step/(int)sizeof(bgr[0]) - size.width*3;
bgra += bgra_step/sizeof(bgra[0]) - size.width*4; bgra += bgra_step/(int)sizeof(bgra[0]) - size.width*4;
} }
} }
@@ -252,8 +252,8 @@ void icvCvt_BGRA2RGBA_16u_C4R( const ushort* bgra, int bgra_step,
rgba[0] = t2; rgba[1] = t1; rgba[0] = t2; rgba[1] = t1;
rgba[2] = t0; rgba[3] = t3; rgba[2] = t0; rgba[3] = t3;
} }
bgra += bgra_step/sizeof(bgra[0]) - size.width*4; bgra += bgra_step/(int)sizeof(bgra[0]) - size.width*4;
rgba += rgba_step/sizeof(rgba[0]) - size.width*4; rgba += rgba_step/(int)sizeof(rgba[0]) - size.width*4;
} }
} }
+22
View File
@@ -689,6 +689,28 @@ TEST(Imgcodecs_Tiff, readWrite_unsigned)
EXPECT_EQ(0, remove(filenameOutput.c_str())); EXPECT_EQ(0, remove(filenameOutput.c_str()));
} }
// See https://github.com/opencv/opencv/issues/29615
// Decoding a 16-bit 4-channel TIFF used to form an out of range pointer in
// icvCvt_BGRA2RGBA_16u_C4R because the byte step was divided by an unsigned
// sizeof and then had size.width*4 subtracted, which wraps around for a zero
// step. This just checks that a 16UC4 TIFF round-trips correctly; the value is
// mainly that the sanitizer builds no longer report the pointer overflow.
TEST(Imgcodecs_Tiff, regression_29615_16UC4)
{
Mat img(4, 3, CV_16UC4);
randu(img, Scalar::all(0), Scalar::all(65535));
vector<uchar> buf;
ASSERT_NO_THROW(ASSERT_TRUE(imencode(".tiff", img, buf)));
Mat decoded;
ASSERT_NO_THROW(decoded = imdecode(buf, IMREAD_UNCHANGED));
ASSERT_FALSE(decoded.empty());
ASSERT_EQ(CV_16UC4, decoded.type());
ASSERT_EQ(img.size(), decoded.size());
EXPECT_EQ(0, cvtest::norm(img, decoded, NORM_INF));
}
TEST(Imgcodecs_Tiff, readWrite_32FC1) TEST(Imgcodecs_Tiff, readWrite_32FC1)
{ {
const string root = cvtest::TS::ptr()->get_data_path(); const string root = cvtest::TS::ptr()->get_data_path();
+2
View File
@@ -11,6 +11,8 @@ ocv_add_dispatched_file(morph SSE2 SSE4_1 AVX2)
ocv_add_dispatched_file(smooth SSE2 SSE4_1 AVX2 AVX512_ICL) ocv_add_dispatched_file(smooth SSE2 SSE4_1 AVX2 AVX512_ICL)
ocv_add_dispatched_file(sumpixels SSE2 AVX2 AVX512_SKX) ocv_add_dispatched_file(sumpixels SSE2 AVX2 AVX512_SKX)
ocv_add_dispatched_file(equalize_hist AVX512_ICL) ocv_add_dispatched_file(equalize_hist AVX512_ICL)
ocv_add_dispatched_file(imgwarp SSE4_1 AVX2 AVX512_SKX AVX512_ICL)
ocv_add_dispatched_file(pyramids_avx512_vbmi AVX512_ICL)
ocv_define_module(imgproc opencv_core WRAP java objc python js) ocv_define_module(imgproc opencv_core WRAP java objc python js)
if(OPENCV_CORE_EXCLUDE_C_API) if(OPENCV_CORE_EXCLUDE_C_API)
-203
View File
@@ -688,152 +688,6 @@ calcHist_8u( std::vector<uchar*>& _ptrs, const std::vector<int>& _deltas,
} }
} }
#ifdef HAVE_IPP
typedef IppStatus(CV_STDCALL * IppiHistogram_C1)(const void* pSrc, int srcStep,
IppiSize roiSize, Ipp32u* pHist, const IppiHistogramSpec* pSpec, Ipp8u* pBuffer);
static IppiHistogram_C1 getIppiHistogramFunction_C1(int type)
{
IppiHistogram_C1 ippFunction =
(type == CV_8UC1) ? (IppiHistogram_C1)ippiHistogram_8u_C1R :
(type == CV_16UC1) ? (IppiHistogram_C1)ippiHistogram_16u_C1R :
(type == CV_32FC1) ? (IppiHistogram_C1)ippiHistogram_32f_C1R :
NULL;
return ippFunction;
}
class ipp_calcHistParallelTLS
{
public:
ipp_calcHistParallelTLS() {}
IppAutoBuffer<IppiHistogramSpec> spec;
IppAutoBuffer<Ipp8u> buffer;
IppAutoBuffer<Ipp32u> thist;
};
class ipp_calcHistParallel: public ParallelLoopBody
{
public:
ipp_calcHistParallel(const Mat &src, Mat &hist, Ipp32s histSize, const float *ranges, bool uniform, bool &ok):
ParallelLoopBody(), m_src(src), m_hist(hist), m_ok(ok)
{
ok = true;
m_uniform = uniform;
m_ranges = ranges;
m_histSize = histSize;
m_type = ippiGetDataType(src.type());
m_levelsNum = histSize+1;
ippiHistogram_C1 = getIppiHistogramFunction_C1(src.type());
m_fullRoi = ippiSize(src.size());
m_bufferSize = 0;
m_specSize = 0;
if(!ippiHistogram_C1)
{
ok = false;
return;
}
if(ippiHistogramGetBufferSize(m_type, m_fullRoi, &m_levelsNum, 1, 1, &m_specSize, &m_bufferSize) < 0)
{
ok = false;
return;
}
hist.setTo(0);
}
virtual void operator() (const Range & range) const CV_OVERRIDE
{
CV_INSTRUMENT_REGION_IPP();
if(!m_ok)
return;
ipp_calcHistParallelTLS *pTls = m_tls.get();
IppiSize roi = {m_src.cols, range.end - range.start };
bool mtLoop = false;
if(m_fullRoi.height != roi.height)
mtLoop = true;
if(!pTls->spec)
{
pTls->spec.allocate(m_specSize);
if(!pTls->spec.get())
{
m_ok = false;
return;
}
pTls->buffer.allocate(m_bufferSize);
if(!pTls->buffer.get() && m_bufferSize)
{
m_ok = false;
return;
}
if(m_uniform)
{
if(ippiHistogramUniformInit(m_type, (Ipp32f*)&m_ranges[0], (Ipp32f*)&m_ranges[1], (Ipp32s*)&m_levelsNum, 1, pTls->spec) < 0)
{
m_ok = false;
return;
}
}
else
{
if(ippiHistogramInit(m_type, (const Ipp32f**)&m_ranges, (Ipp32s*)&m_levelsNum, 1, pTls->spec) < 0)
{
m_ok = false;
return;
}
}
pTls->thist.allocate(m_histSize*sizeof(Ipp32u));
}
if(CV_INSTRUMENT_FUN_IPP(ippiHistogram_C1, m_src.ptr(range.start), (int)m_src.step, roi, pTls->thist, pTls->spec, pTls->buffer) < 0)
{
m_ok = false;
return;
}
if(mtLoop)
{
for(int i = 0; i < m_histSize; i++)
CV_XADD((int*)(m_hist.ptr(i)), *(int*)((Ipp32u*)pTls->thist + i));
}
else
ippiCopy_32s_C1R((Ipp32s*)pTls->thist.get(), sizeof(Ipp32u), (Ipp32s*)m_hist.ptr(), (int)m_hist.step, ippiSize(1, m_histSize));
}
private:
const Mat &m_src;
Mat &m_hist;
Ipp32s m_histSize;
const float *m_ranges;
bool m_uniform;
IppiHistogram_C1 ippiHistogram_C1;
IppiSize m_fullRoi;
IppDataType m_type;
Ipp32s m_levelsNum;
int m_bufferSize;
int m_specSize;
mutable Mutex m_syncMutex;
TLSData<ipp_calcHistParallelTLS> m_tls;
volatile bool &m_ok;
const ipp_calcHistParallel & operator = (const ipp_calcHistParallel & );
};
#endif
} }
#ifdef HAVE_OPENVX #ifdef HAVE_OPENVX
@@ -894,58 +748,6 @@ namespace cv
} }
#endif #endif
#ifdef HAVE_IPP
#define IPP_HISTOGRAM_PARALLEL 1
namespace cv
{
static bool ipp_calchist(const Mat &image, Mat &hist, int histSize, const float** ranges, bool uniform, bool accumulate)
{
CV_INSTRUMENT_REGION_IPP();
#if IPP_VERSION_X100 < 201801
// No SSE42 optimization for uniform 32f
if(uniform && image.depth() == CV_32F && cv::ipp::getIppTopFeatures() == ippCPUID_SSE42)
return false;
#endif
// IPP_DISABLE_HISTOGRAM - https://github.com/opencv/opencv/issues/11544
// and https://github.com/opencv/opencv/issues/21595
if ((uniform && (ranges[0][1] - ranges[0][0]) != histSize) || abs(ranges[0][0]) != cvFloor(ranges[0][0]))
return false;
Mat ihist = hist;
if(accumulate)
ihist.create(1, &histSize, CV_32S);
bool ok = true;
int threads = ippiSuggestThreadsNum(image, (1+((double)ihist.total()/image.total()))*2);
Range range(0, image.rows);
ipp_calcHistParallel invoker(image, ihist, histSize, ranges[0], uniform, ok);
if(!ok)
return false;
if(IPP_HISTOGRAM_PARALLEL && threads > 1)
parallel_for_(range, invoker, threads*2);
else
invoker(range);
if(ok)
{
if(accumulate)
{
IppiSize histRoi = ippiSize(1, histSize);
IppAutoBuffer<Ipp32f> fhist(histSize*sizeof(Ipp32f));
CV_INSTRUMENT_FUN_IPP(ippiConvert_32s32f_C1R, (Ipp32s*)ihist.ptr(), (int)ihist.step, (Ipp32f*)fhist, sizeof(Ipp32f), histRoi);
CV_INSTRUMENT_FUN_IPP(ippiAdd_32f_C1IR, (Ipp32f*)fhist, sizeof(Ipp32f), (Ipp32f*)hist.ptr(), (int)hist.step, histRoi);
}
else
CV_INSTRUMENT_FUN_IPP(ippiConvert_32s32f_C1R, (Ipp32s*)ihist.ptr(), (int)ihist.step, (Ipp32f*)hist.ptr(), (int)hist.step, ippiSize(1, histSize));
}
return ok;
}
}
#endif
void cv::calcHist( const Mat* images, int nimages, const int* channels, void cv::calcHist( const Mat* images, int nimages, const int* channels,
InputArray _mask, OutputArray _hist, int dims, const int* histSize, InputArray _mask, OutputArray _hist, int dims, const int* histSize,
const float** ranges, bool uniform, bool accumulate ) const float** ranges, bool uniform, bool accumulate )
@@ -973,11 +775,6 @@ void cv::calcHist( const Mat* images, int nimages, const int* channels,
if(histdata != hist.data) if(histdata != hist.data)
accumulate = false; accumulate = false;
CV_IPP_RUN(
nimages == 1 && dims == 1 && channels && channels[0] == 0
&& _mask.empty() && images[0].dims <= 2 && ranges && ranges[0],
ipp_calchist(images[0], hist, histSize[0], ranges, uniform, accumulate));
if (nimages == 1 && dims == 1 && channels && channels[0] == 0 && _mask.empty() && images[0].dims <= 2 && ranges && ranges[0]) { if (nimages == 1 && dims == 1 && channels && channels[0] == 0 && _mask.empty() && images[0].dims <= 2 && ranges && ranges[0]) {
CALL_HAL(calcHist, cv_hal_calcHist, images[0].data, images[0].step, images[0].type(), images[0].cols, images[0].rows, CALL_HAL(calcHist, cv_hal_calcHist, images[0].data, images[0].step, images[0].type(), images[0].cols, images[0].rows,
hist.ptr<float>(), histSize[0], ranges, uniform, accumulate); hist.ptr<float>(), histSize[0], ranges, uniform, accumulate);
+4
View File
@@ -13,6 +13,7 @@
// Copyright (C) 2000-2008, Intel Corporation, all rights reserved. // Copyright (C) 2000-2008, Intel Corporation, all rights reserved.
// Copyright (C) 2009, Willow Garage Inc., all rights reserved. // Copyright (C) 2009, Willow Garage Inc., all rights reserved.
// Copyright (C) 2014-2015, Itseez Inc., all rights reserved. // Copyright (C) 2014-2015, Itseez Inc., all rights reserved.
// Copyright (C) 2026, Advanced Micro Devices, all rights reserved.
// Third party copyrights are property of their respective owners. // Third party copyrights are property of their respective owners.
// //
// Redistribution and use in source and binary forms, with or without modification, // Redistribution and use in source and binary forms, with or without modification,
@@ -95,4 +96,7 @@ int warpAffineBlockline(int *adelta, int *bdelta, short* xy, short* alpha, int X
} }
} }
#include "imgwarp.simd.hpp"
/* End of file. */ /* End of file. */
+219 -40
View File
@@ -13,6 +13,7 @@
// Copyright (C) 2000-2008, Intel Corporation, all rights reserved. // Copyright (C) 2000-2008, Intel Corporation, all rights reserved.
// Copyright (C) 2009, Willow Garage Inc., all rights reserved. // Copyright (C) 2009, Willow Garage Inc., all rights reserved.
// Copyright (C) 2014-2015, Itseez Inc., all rights reserved. // Copyright (C) 2014-2015, Itseez Inc., all rights reserved.
// Copyright (C) 2026, Advanced Micro Devices, all rights reserved.
// Third party copyrights are property of their respective owners. // Third party copyrights are property of their respective owners.
// //
// Redistribution and use in source and binary forms, with or without modification, // Redistribution and use in source and binary forms, with or without modification,
@@ -55,6 +56,9 @@
#include "opencv2/core/softfloat.hpp" #include "opencv2/core/softfloat.hpp"
#include "imgwarp.hpp" #include "imgwarp.hpp"
#include "imgwarp.simd.hpp"
#include "imgwarp.simd_declarations.hpp" // defines CV_CPU_DISPATCH_MODES_ALL=AVX512_ICL,...,BASELINE based on CMakeLists.txt content
using namespace cv; using namespace cv;
namespace cv namespace cv
@@ -611,6 +615,81 @@ template<bool isRelative> using RemapVec_8u = RemapNoVec<isRelative>;
#endif #endif
template<typename T, typename AT>
struct RemapBilinearVecC1
{
int operator()(const T*, size_t, T*, const short*, const ushort*,
const AT*, int, int, int) const { return 0; }
};
template<>
struct RemapBilinearVecC1<float, float>
{
int operator()(const float* S0, size_t sstep, float* D, const short* XY,
const ushort* FXY, const float* wtab, int dx, int X1, int off_y) const
{
CV_CPU_DISPATCH(remapBilinearC1_simd,
(CV_32F, (const uchar*)S0, sstep, (uchar*)D, XY, FXY, wtab, dx, X1, off_y),
CV_CPU_DISPATCH_MODES_ALL);
}
};
template<>
struct RemapBilinearVecC1<ushort, float>
{
int operator()(const ushort* S0, size_t sstep, ushort* D, const short* XY,
const ushort* FXY, const float* wtab, int dx, int X1, int off_y) const
{
CV_CPU_DISPATCH(remapBilinearC1_simd,
(CV_16U, (const uchar*)S0, sstep, (uchar*)D, XY, FXY, wtab, dx, X1, off_y),
CV_CPU_DISPATCH_MODES_ALL);
}
};
template<>
struct RemapBilinearVecC1<short, float>
{
int operator()(const short* S0, size_t sstep, short* D, const short* XY,
const ushort* FXY, const float* wtab, int dx, int X1, int off_y) const
{
CV_CPU_DISPATCH(remapBilinearC1_simd,
(CV_16S, (const uchar*)S0, sstep, (uchar*)D, XY, FXY, wtab, dx, X1, off_y),
CV_CPU_DISPATCH_MODES_ALL);
}
};
static inline int remapBilinearSameRun( const short* XY, int dx, int end,
unsigned width1, unsigned height1, bool inl )
{
int n = 0;
#if (CV_SIMD || CV_SIMD_SCALABLE)
const int span = VTraits<v_int16>::vlanes();
const v_int16 vw = vx_setall_s16((short)std::min<unsigned>(width1, 0x7fff));
const v_int16 vh = vx_setall_s16((short)std::min<unsigned>(height1, 0x7fff));
const v_int16 vm1 = vx_setall_s16(-1);
for( ; dx + n + span <= end; n += span )
{
v_int16 sx, sy;
v_load_deinterleave(XY + (dx + n) * 2, sx, sy);
// in-bounds: 0 <= sx < width1 && 0 <= sy < height1
v_int16 inb = v_and(v_and(v_gt(sx, vm1), v_lt(sx, vw)),
v_and(v_gt(sy, vm1), v_lt(sy, vh)));
const bool allSame = inl ? v_check_all(inb) : !v_check_any(inb);
if( !allSame )
break;
}
vx_cleanup();
#endif
for( ; dx + n < end; n++ )
{
const int sx = XY[(dx + n) * 2], sy = XY[(dx + n) * 2 + 1];
const bool ib = (unsigned)sx < width1 && (unsigned)sy < height1;
if( ib != inl )
break;
}
return n;
}
template<class CastOp, class VecOp, typename AT, bool isRelative> template<class CastOp, class VecOp, typename AT, bool isRelative>
static void remapBilinear( const Mat& _src, Mat& _dst, const Mat& _xy, static void remapBilinear( const Mat& _src, Mat& _dst, const Mat& _xy,
const Mat& _fxy, const void* _wtab, const Mat& _fxy, const void* _wtab,
@@ -647,6 +726,12 @@ static void remapBilinear( const Mat& _src, Mat& _dst, const Mat& _xy,
const int off_y = (isRelative ? (_offset.y+dy) : 0); const int off_y = (isRelative ? (_offset.y+dy) : 0);
for(int dx = 0; dx <= dsize.width; dx++ ) for(int dx = 0; dx <= dsize.width; dx++ )
{ {
if( !isRelative && dx < dsize.width )
{
int n = remapBilinearSameRun(XY, dx, dsize.width, width1, height1, prevInlier);
if( n > 0 )
dx += n - 1;
}
bool curInlier = dx < dsize.width ? bool curInlier = dx < dsize.width ?
(unsigned)XY[dx*2]+(isRelative ? (_offset.x+dx) : 0) < width1 && (unsigned)XY[dx*2]+(isRelative ? (_offset.x+dx) : 0) < width1 &&
(unsigned)XY[dx*2+1]+off_y < height1 : !prevInlier; (unsigned)XY[dx*2+1]+off_y < height1 : !prevInlier;
@@ -667,6 +752,11 @@ static void remapBilinear( const Mat& _src, Mat& _dst, const Mat& _xy,
if( cn == 1 ) if( cn == 1 )
{ {
if( !isRelative )
{
int n = RemapBilinearVecC1<T, AT>()(S0, sstep, D, XY, FXY, wtab, dx, X1, off_y);
D += n; dx += n;
}
for( ; dx < X1; dx++, D++ ) for( ; dx < X1; dx++, D++ )
{ {
int sx = XY[dx*2]+(isRelative ? (_offset.x+dx) : 0), sy = XY[dx*2+1]+off_y; int sx = XY[dx*2]+(isRelative ? (_offset.x+dx) : 0), sy = XY[dx*2+1]+off_y;
@@ -843,6 +933,53 @@ static void remapBilinear( const Mat& _src, Mat& _dst, const Mat& _xy,
} }
// Dispatch shim for the single-channel non-relative bicubic in-bounds fast path (32F only).
template<typename T, typename AT>
struct RemapBicubicVecC1
{
int operator()(const T*, size_t, T*, const short*, const ushort*, const AT*,
int, int, unsigned, unsigned, int) const { return 0; }
};
template<>
struct RemapBicubicVecC1<float, float>
{
int operator()(const float* S0, size_t sstep, float* D, const short* XY,
const ushort* FXY, const float* wtab, int dx, int dwidth, unsigned width1,
unsigned height1, int off_y) const
{
CV_CPU_DISPATCH(remapBicubicC1wp_simd,
(CV_32F, (const uchar*)S0, sstep, (uchar*)D, XY, FXY, wtab, dx, dwidth, width1, height1, off_y),
CV_CPU_DISPATCH_MODES_ALL);
}
};
template<>
struct RemapBicubicVecC1<ushort, float>
{
int operator()(const ushort* S0, size_t sstep, ushort* D, const short* XY,
const ushort* FXY, const float* wtab, int dx, int dwidth, unsigned width1,
unsigned height1, int off_y) const
{
CV_CPU_DISPATCH(remapBicubicC1wp_simd,
(CV_16U, (const uchar*)S0, sstep, (uchar*)D, XY, FXY, wtab, dx, dwidth, width1, height1, off_y),
CV_CPU_DISPATCH_MODES_ALL);
}
};
template<>
struct RemapBicubicVecC1<short, float>
{
int operator()(const short* S0, size_t sstep, short* D, const short* XY,
const ushort* FXY, const float* wtab, int dx, int dwidth, unsigned width1,
unsigned height1, int off_y) const
{
CV_CPU_DISPATCH(remapBicubicC1wp_simd,
(CV_16S, (const uchar*)S0, sstep, (uchar*)D, XY, FXY, wtab, dx, dwidth, width1, height1, off_y),
CV_CPU_DISPATCH_MODES_ALL);
}
};
template<class CastOp, typename AT, int ONE, bool isRelative> template<class CastOp, typename AT, int ONE, bool isRelative>
static void remapBicubic( const Mat& _src, Mat& _dst, const Mat& _xy, static void remapBicubic( const Mat& _src, Mat& _dst, const Mat& _xy,
const Mat& _fxy, const void* _wtab, const Mat& _fxy, const void* _wtab,
@@ -879,6 +1016,12 @@ static void remapBicubic( const Mat& _src, Mat& _dst, const Mat& _xy,
const int off_y = isRelative ? (_offset.y+dy) : 0; const int off_y = isRelative ? (_offset.y+dy) : 0;
for(int dx = 0; dx < dsize.width; dx++, D += cn ) for(int dx = 0; dx < dsize.width; dx++, D += cn )
{ {
if( cn == 1 && !isRelative )
{
int n = RemapBicubicVecC1<T, AT>()(S0, sstep, D, XY, FXY, wtab, dx,
dsize.width, width1, height1, off_y);
if( n > 0 ) { D += (n - 1)*cn; dx += n - 1; continue; }
}
const int off_x = isRelative ? (_offset.x+dx) : 0; const int off_x = isRelative ? (_offset.x+dx) : 0;
int sx = XY[dx*2]-1+off_x, sy = XY[dx*2+1]-1+off_y; int sx = XY[dx*2]-1+off_x, sy = XY[dx*2+1]-1+off_y;
const AT* w = wtab + FXY[dx]*16; const AT* w = wtab + FXY[dx]*16;
@@ -948,6 +1091,33 @@ static void remapBicubic( const Mat& _src, Mat& _dst, const Mat& _xy,
} }
template<typename T, typename AT>
struct RemapLanczos4VecC1
{
int operator()(const T*, size_t, T*, const short*, const ushort*, const AT*,
int, int, unsigned, unsigned, int) const { return 0; }
};
#define CV_REMAP_LANCZOS4_SHIM(T, DEPTH) \
template<> struct RemapLanczos4VecC1<T, float> \
{ \
int operator()(const T* S0, size_t sstep, T* D, const short* XY, \
const ushort* FXY, const float* wtab, int dx, int dwidth, \
unsigned width1, unsigned height1, int off_y) const \
{ \
CV_CPU_DISPATCH(remapLanczos4C1_simd, \
(DEPTH, (const uchar*)S0, sstep, (uchar*)D, XY, FXY, wtab, \
dx, dwidth, width1, height1, off_y), \
CV_CPU_DISPATCH_MODES_ALL); \
} \
};
// 32F is intentionally not shimmed: its vectorized accumulation deviates beyond
// the float accuracy tolerance, so it stays on the scalar loop. Emitting a shim
// would add a per-pixel dispatch call that returns 0 and slows the scalar path.
CV_REMAP_LANCZOS4_SHIM(ushort, CV_16U)
CV_REMAP_LANCZOS4_SHIM(short, CV_16S)
#undef CV_REMAP_LANCZOS4_SHIM
template<class CastOp, typename AT, int ONE, bool isRelative> template<class CastOp, typename AT, int ONE, bool isRelative>
static void remapLanczos4( const Mat& _src, Mat& _dst, const Mat& _xy, static void remapLanczos4( const Mat& _src, Mat& _dst, const Mat& _xy,
const Mat& _fxy, const void* _wtab, const Mat& _fxy, const void* _wtab,
@@ -984,6 +1154,12 @@ static void remapLanczos4( const Mat& _src, Mat& _dst, const Mat& _xy,
const int off_y = isRelative ? (_offset.y+dy) : 0; const int off_y = isRelative ? (_offset.y+dy) : 0;
for(int dx = 0; dx < dsize.width; dx++, D += cn ) for(int dx = 0; dx < dsize.width; dx++, D += cn )
{ {
if( cn == 1 && !isRelative )
{
int n = RemapLanczos4VecC1<T, AT>()(S0, sstep, D, XY, FXY, wtab, dx,
dsize.width, width1, height1, off_y);
if( n > 0 ) { D += (n - 1)*cn; dx += n - 1; continue; }
}
const int off_x = isRelative ? (_offset.x+dx) : 0; const int off_x = isRelative ? (_offset.x+dx) : 0;
int sx = XY[dx*2]-3+off_x, sy = XY[dx*2+1]-3+off_y; int sx = XY[dx*2]-3+off_x, sy = XY[dx*2+1]-3+off_y;
const AT* w = wtab + FXY[dx]*64; const AT* w = wtab + FXY[dx]*64;
@@ -1131,21 +1307,21 @@ public:
const float* sY = m2->ptr<float>(y+y1) + x; const float* sY = m2->ptr<float>(y+y1) + x;
x1 = 0; x1 = 0;
#if CV_SIMD128 #if (CV_SIMD || CV_SIMD_SCALABLE)
{ {
int span = VTraits<v_float32x4>::vlanes(); int span = VTraits<v_float32>::vlanes();
for( ; x1 <= bcols - span * 2; x1 += span * 2 ) for( ; x1 <= bcols - span * 2; x1 += span * 2 )
{ {
v_int32x4 ix0 = v_round(v_load(sX + x1)); v_int32 ix0 = v_round(vx_load(sX + x1));
v_int32x4 iy0 = v_round(v_load(sY + x1)); v_int32 iy0 = v_round(vx_load(sY + x1));
v_int32x4 ix1 = v_round(v_load(sX + x1 + span)); v_int32 ix1 = v_round(vx_load(sX + x1 + span));
v_int32x4 iy1 = v_round(v_load(sY + x1 + span)); v_int32 iy1 = v_round(vx_load(sY + x1 + span));
v_int16x8 dx, dy; v_int16 dx = v_pack(ix0, ix1);
dx = v_pack(ix0, ix1); v_int16 dy = v_pack(iy0, iy1);
dy = v_pack(iy0, iy1);
v_store_interleave(XY + x1 * 2, dx, dy); v_store_interleave(XY + x1 * 2, dx, dy);
} }
vx_cleanup();
} }
#endif #endif
for( ; x1 < bcols; x1++ ) for( ; x1 < bcols; x1++ )
@@ -1172,12 +1348,13 @@ public:
const ushort* sA = m2->ptr<ushort>(y+y1) + x; const ushort* sA = m2->ptr<ushort>(y+y1) + x;
x1 = 0; x1 = 0;
#if CV_SIMD128 #if (CV_SIMD || CV_SIMD_SCALABLE)
{ {
v_uint16x8 v_scale = v_setall_u16(INTER_TAB_SIZE2 - 1); v_uint16 v_scale = vx_setall_u16(INTER_TAB_SIZE2 - 1);
int span = VTraits<v_uint16x8>::vlanes(); int span = VTraits<v_uint16>::vlanes();
for( ; x1 <= bcols - span; x1 += span ) for( ; x1 <= bcols - span; x1 += span )
v_store((unsigned short*)(A + x1), v_and(v_load(sA + x1), v_scale)); v_store((unsigned short*)(A + x1), v_and(vx_load(sA + x1), v_scale));
vx_cleanup();
} }
#endif #endif
for( ; x1 < bcols; x1++ ) for( ; x1 < bcols; x1++ )
@@ -1189,26 +1366,27 @@ public:
const float* sY = m2->ptr<float>(y+y1) + x; const float* sY = m2->ptr<float>(y+y1) + x;
x1 = 0; x1 = 0;
#if CV_SIMD128 #if (CV_SIMD || CV_SIMD_SCALABLE)
{ {
v_float32x4 v_scale = v_setall_f32((float)INTER_TAB_SIZE); v_float32 v_scale = vx_setall_f32((float)INTER_TAB_SIZE);
v_int32x4 v_scale2 = v_setall_s32(INTER_TAB_SIZE - 1); v_int32 v_scale2 = vx_setall_s32(INTER_TAB_SIZE - 1);
int span = VTraits<v_float32x4>::vlanes(); int span = VTraits<v_float32>::vlanes();
for( ; x1 <= bcols - span * 2; x1 += span * 2 ) for( ; x1 <= bcols - span * 2; x1 += span * 2 )
{ {
v_int32x4 v_sx0 = v_round(v_mul(v_scale, v_load(sX + x1))); v_int32 v_sx0 = v_round(v_mul(v_scale, vx_load(sX + x1)));
v_int32x4 v_sy0 = v_round(v_mul(v_scale, v_load(sY + x1))); v_int32 v_sy0 = v_round(v_mul(v_scale, vx_load(sY + x1)));
v_int32x4 v_sx1 = v_round(v_mul(v_scale, v_load(sX + x1 + span))); v_int32 v_sx1 = v_round(v_mul(v_scale, vx_load(sX + x1 + span)));
v_int32x4 v_sy1 = v_round(v_mul(v_scale, v_load(sY + x1 + span))); v_int32 v_sy1 = v_round(v_mul(v_scale, vx_load(sY + x1 + span)));
v_uint16x8 v_sx8 = v_reinterpret_as_u16(v_pack(v_and(v_sx0, v_scale2), v_and(v_sx1, v_scale2))); v_uint16 v_sx8 = v_reinterpret_as_u16(v_pack(v_and(v_sx0, v_scale2), v_and(v_sx1, v_scale2)));
v_uint16x8 v_sy8 = v_reinterpret_as_u16(v_pack(v_and(v_sy0, v_scale2), v_and(v_sy1, v_scale2))); v_uint16 v_sy8 = v_reinterpret_as_u16(v_pack(v_and(v_sy0, v_scale2), v_and(v_sy1, v_scale2)));
v_uint16x8 v_v = v_or(v_shl<INTER_BITS>(v_sy8), v_sx8); v_uint16 v_v = v_or(v_shl<INTER_BITS>(v_sy8), v_sx8);
v_store(A + x1, v_v); v_store(A + x1, v_v);
v_int16x8 v_d0 = v_pack(v_shr<INTER_BITS>(v_sx0), v_shr<INTER_BITS>(v_sx1)); v_int16 v_d0 = v_pack(v_shr<INTER_BITS>(v_sx0), v_shr<INTER_BITS>(v_sx1));
v_int16x8 v_d1 = v_pack(v_shr<INTER_BITS>(v_sy0), v_shr<INTER_BITS>(v_sy1)); v_int16 v_d1 = v_pack(v_shr<INTER_BITS>(v_sy0), v_shr<INTER_BITS>(v_sy1));
v_store_interleave(XY + (x1 << 1), v_d0, v_d1); v_store_interleave(XY + (x1 << 1), v_d0, v_d1);
} }
vx_cleanup();
} }
#endif #endif
for( ; x1 < bcols; x1++ ) for( ; x1 < bcols; x1++ )
@@ -1226,28 +1404,29 @@ public:
const float* sXY = m1->ptr<float>(y+y1) + x*2; const float* sXY = m1->ptr<float>(y+y1) + x*2;
x1 = 0; x1 = 0;
#if CV_SIMD128 #if (CV_SIMD || CV_SIMD_SCALABLE)
{ {
v_float32x4 v_scale = v_setall_f32((float)INTER_TAB_SIZE); v_float32 v_scale = vx_setall_f32((float)INTER_TAB_SIZE);
v_int32x4 v_scale2 = v_setall_s32(INTER_TAB_SIZE - 1), v_scale3 = v_setall_s32(INTER_TAB_SIZE); v_int32 v_scale2 = vx_setall_s32(INTER_TAB_SIZE - 1), v_scale3 = vx_setall_s32(INTER_TAB_SIZE);
int span = VTraits<v_float32x4>::vlanes(); int span = VTraits<v_float32>::vlanes();
for( ; x1 <= bcols - span * 2; x1 += span * 2 ) for( ; x1 <= bcols - span * 2; x1 += span * 2 )
{ {
v_float32x4 v_fx, v_fy; v_float32 v_fx, v_fy;
v_load_deinterleave(sXY + (x1 << 1), v_fx, v_fy); v_load_deinterleave(sXY + (x1 << 1), v_fx, v_fy);
v_int32x4 v_sx0 = v_round(v_mul(v_fx, v_scale)); v_int32 v_sx0 = v_round(v_mul(v_fx, v_scale));
v_int32x4 v_sy0 = v_round(v_mul(v_fy, v_scale)); v_int32 v_sy0 = v_round(v_mul(v_fy, v_scale));
v_load_deinterleave(sXY + ((x1 + span) << 1), v_fx, v_fy); v_load_deinterleave(sXY + ((x1 + span) << 1), v_fx, v_fy);
v_int32x4 v_sx1 = v_round(v_mul(v_fx, v_scale)); v_int32 v_sx1 = v_round(v_mul(v_fx, v_scale));
v_int32x4 v_sy1 = v_round(v_mul(v_fy, v_scale)); v_int32 v_sy1 = v_round(v_mul(v_fy, v_scale));
v_int32x4 v_v0 = v_muladd(v_scale3, (v_and(v_sy0, v_scale2)), (v_and(v_sx0, v_scale2))); v_int32 v_v0 = v_muladd(v_scale3, (v_and(v_sy0, v_scale2)), (v_and(v_sx0, v_scale2)));
v_int32x4 v_v1 = v_muladd(v_scale3, (v_and(v_sy1, v_scale2)), (v_and(v_sx1, v_scale2))); v_int32 v_v1 = v_muladd(v_scale3, (v_and(v_sy1, v_scale2)), (v_and(v_sx1, v_scale2)));
v_uint16x8 v_v8 = v_reinterpret_as_u16(v_pack(v_v0, v_v1)); v_uint16 v_v8 = v_reinterpret_as_u16(v_pack(v_v0, v_v1));
v_store(A + x1, v_v8); v_store(A + x1, v_v8);
v_int16x8 v_dx = v_pack(v_shr<INTER_BITS>(v_sx0), v_shr<INTER_BITS>(v_sx1)); v_int16 v_dx = v_pack(v_shr<INTER_BITS>(v_sx0), v_shr<INTER_BITS>(v_sx1));
v_int16x8 v_dy = v_pack(v_shr<INTER_BITS>(v_sy0), v_shr<INTER_BITS>(v_sy1)); v_int16 v_dy = v_pack(v_shr<INTER_BITS>(v_sy0), v_shr<INTER_BITS>(v_sy1));
v_store_interleave(XY + (x1 << 1), v_dx, v_dy); v_store_interleave(XY + (x1 << 1), v_dx, v_dy);
} }
vx_cleanup();
} }
#endif #endif
+478
View File
@@ -0,0 +1,478 @@
// This file is part of OpenCV project.
// It is subject to the license terms in the LICENSE file found in the top-level directory
// of this distribution and at http://opencv.org/license.html.
//
// Copyright (C) 2026, Advanced Micro Devices, all rights reserved.
#include "opencv2/core/hal/intrin.hpp"
namespace cv {
CV_CPU_OPTIMIZATION_NAMESPACE_BEGIN
int remapBilinearC1_simd(int depth, const uchar* S0, size_t sstep, uchar* D,
const short* XY, const ushort* FXY, const float* wtab,
int dx, int X1, int off_y);
int remapBicubicC1_simd(int depth, const uchar* S0, size_t sstep, uchar* D,
const short* XY, const ushort* FXY,
int dx, int dwidth, unsigned width1, unsigned height1, int off_y);
int remapLanczos4C1_simd(int depth, const uchar* S0, size_t sstep, uchar* D,
const short* XY, const ushort* FXY, const float* wtab,
int dx, int dwidth, unsigned width1, unsigned height1, int off_y);
int remapBicubicC1wp_simd(int depth, const uchar* S0, size_t sstep, uchar* D,
const short* XY, const ushort* FXY, const float* wtab,
int dx, int dwidth, unsigned width1, unsigned height1, int off_y);
#ifndef CV_CPU_OPTIMIZATION_DECLARATIONS_ONLY
#if (CV_SIMD || CV_SIMD_SCALABLE)
static inline v_float32 remapGatherF32(const float* base, const int* ofs)
{
float CV_DECL_ALIGNED(CV_SIMD_WIDTH) buf[VTraits<v_float32>::max_nlanes];
const int n = VTraits<v_float32>::vlanes();
for (int k = 0; k < n; k++)
buf[k] = base[ofs[k]];
return vx_load(buf);
}
static inline v_float32 remapGatherF32(const ushort* base, const int* ofs)
{
float CV_DECL_ALIGNED(CV_SIMD_WIDTH) buf[VTraits<v_float32>::max_nlanes];
const int n = VTraits<v_float32>::vlanes();
for (int k = 0; k < n; k++)
buf[k] = (float)base[ofs[k]];
return vx_load(buf);
}
static inline v_float32 remapGatherF32(const short* base, const int* ofs)
{
float CV_DECL_ALIGNED(CV_SIMD_WIDTH) buf[VTraits<v_float32>::max_nlanes];
const int n = VTraits<v_float32>::vlanes();
for (int k = 0; k < n; k++)
buf[k] = (float)base[ofs[k]];
return vx_load(buf);
}
static CV_ALWAYS_INLINE void remapCorners(const short* S0, size_t sstep, const int* ofs,
v_float32& s0, v_float32& s1,
v_float32& s2, v_float32& s3)
{
int CV_DECL_ALIGNED(CV_SIMD_WIDTH) topbuf[VTraits<v_float32>::max_nlanes];
int CV_DECL_ALIGNED(CV_SIMD_WIDTH) botbuf[VTraits<v_float32>::max_nlanes];
const int n = VTraits<v_float32>::vlanes();
for (int k = 0; k < n; k++)
{
const short* p = S0 + ofs[k];
int t, b;
memcpy(&t, p, sizeof(t));
memcpy(&b, p + sstep, sizeof(b));
topbuf[k] = t; botbuf[k] = b;
}
v_int32 top = vx_load(topbuf), bot = vx_load(botbuf);
s0 = v_cvt_f32(v_shr<16>(v_shl<16>(top))); // low 16 bits (sign-extended)
s1 = v_cvt_f32(v_shr<16>(top)); // high 16 bits (sign-extended)
s2 = v_cvt_f32(v_shr<16>(v_shl<16>(bot)));
s3 = v_cvt_f32(v_shr<16>(bot));
}
static CV_ALWAYS_INLINE void remapCorners(const ushort* S0, size_t sstep, const int* ofs,
v_float32& s0, v_float32& s1,
v_float32& s2, v_float32& s3)
{
int CV_DECL_ALIGNED(CV_SIMD_WIDTH) topbuf[VTraits<v_float32>::max_nlanes];
int CV_DECL_ALIGNED(CV_SIMD_WIDTH) botbuf[VTraits<v_float32>::max_nlanes];
const int n = VTraits<v_float32>::vlanes();
for (int k = 0; k < n; k++)
{
const ushort* p = S0 + ofs[k];
int t, b;
memcpy(&t, p, sizeof(t));
memcpy(&b, p + sstep, sizeof(b));
topbuf[k] = t; botbuf[k] = b;
}
const v_uint32 lo16 = vx_setall_u32(0xffff);
v_uint32 top = v_reinterpret_as_u32(vx_load(topbuf));
v_uint32 bot = v_reinterpret_as_u32(vx_load(botbuf));
s0 = v_cvt_f32(v_reinterpret_as_s32(v_and(top, lo16))); // low 16 (zero-ext)
s1 = v_cvt_f32(v_reinterpret_as_s32(v_shr<16>(top))); // high 16 (zero-ext)
s2 = v_cvt_f32(v_reinterpret_as_s32(v_and(bot, lo16)));
s3 = v_cvt_f32(v_reinterpret_as_s32(v_shr<16>(bot)));
}
static inline void remapStoreC1(float* D, const v_float32& res)
{
v_store(D, res);
}
static inline void remapStoreC1(ushort* D, const v_float32& res)
{
v_pack_u_store(D, v_round(res));
}
static inline void remapStoreC1(short* D, const v_float32& res)
{
v_pack_store(D, v_round(res));
}
template<typename T>
static int remapBilinearC1_run(const T* S0, size_t sstep, T* D,
const short* XY, const ushort* FXY,
const float* wtab, int dx, int X1, int off_y)
{
CV_UNUSED(wtab);
const int vlanes = VTraits<v_float32>::vlanes();
const int dx0 = dx;
const v_float32 vone = vx_setall_f32(1.f);
const v_float32 vscale = vx_setall_f32(1.f / INTER_TAB_SIZE);
const v_int32 vmask = vx_setall_s32(INTER_TAB_SIZE - 1);
int CV_DECL_ALIGNED(CV_SIMD_WIDTH) ofs[VTraits<v_float32>::max_nlanes];
for( ; dx <= X1 - vlanes; dx += vlanes )
{
for( int k = 0; k < vlanes; k++ )
{
const int sx = XY[(dx + k) * 2];
const int sy = XY[(dx + k) * 2 + 1] + off_y;
ofs[k] = sy * (int)sstep + sx;
}
v_float32 s0, s1, s2, s3;
remapCorners(S0, sstep, ofs, s0, s1, s2, s3);
v_int32 fxy = v_reinterpret_as_s32(vx_load_expand(FXY + dx));
v_float32 fx = v_mul(v_cvt_f32(v_and(fxy, vmask)), vscale);
v_float32 fy = v_mul(v_cvt_f32(v_shr<INTER_BITS>(fxy)), vscale);
v_float32 cx0 = v_sub(vone, fx);
v_float32 cy0 = v_sub(vone, fy);
v_float32 w0 = v_mul(cx0, cy0);
v_float32 w1 = v_mul(fx, cy0);
v_float32 w2 = v_mul(cx0, fy);
v_float32 w3 = v_mul(fx, fy);
v_float32 res = v_fma(s0, w0, v_fma(s1, w1, v_fma(s2, w2, v_mul(s3, w3))));
remapStoreC1(D + (dx - dx0), res);
}
vx_cleanup();
return dx - dx0;
}
static int remapBilinearF32_run(const float* S0, size_t sstep, float* D,
const short* XY, const ushort* FXY,
const float* wtab, int dx, int X1, int off_y)
{
CV_UNUSED(wtab);
const int vlanes = VTraits<v_float32>::vlanes();
const int dx0 = dx;
const float* S1 = S0 + sstep;
const v_float32 vone = vx_setall_f32(1.f);
const v_float32 vscale = vx_setall_f32(1.f / INTER_TAB_SIZE);
const v_int32 vmask = vx_setall_s32(INTER_TAB_SIZE - 1);
int CV_DECL_ALIGNED(CV_SIMD_WIDTH) ofs[VTraits<v_float32>::max_nlanes];
for( ; dx <= X1 - vlanes; dx += vlanes )
{
for( int k = 0; k < vlanes; k++ )
{
const int sx = XY[(dx + k) * 2];
const int sy = XY[(dx + k) * 2 + 1] + off_y;
ofs[k] = sy * (int)sstep + sx;
}
v_float32 s0 = remapGatherF32(S0, ofs);
v_float32 s1 = remapGatherF32(S0 + 1, ofs);
v_float32 s2 = remapGatherF32(S1, ofs);
v_float32 s3 = remapGatherF32(S1 + 1, ofs);
v_int32 fxy = v_reinterpret_as_s32(vx_load_expand(FXY + dx));
v_float32 fx = v_mul(v_cvt_f32(v_and(fxy, vmask)), vscale);
v_float32 fy = v_mul(v_cvt_f32(v_shr<INTER_BITS>(fxy)), vscale);
v_float32 cx0 = v_sub(vone, fx);
v_float32 cy0 = v_sub(vone, fy);
v_float32 w0 = v_mul(cx0, cy0);
v_float32 w1 = v_mul(fx, cy0);
v_float32 w2 = v_mul(cx0, fy);
v_float32 w3 = v_mul(fx, fy);
v_float32 res = v_fma(s0, w0, v_fma(s1, w1, v_fma(s2, w2, v_mul(s3, w3))));
v_store(D + (dx - dx0), res);
}
vx_cleanup();
return dx - dx0;
}
// Evaluate the four cubic interpolation coefficients for a vector of fractional
// positions, matching interpolateCubic() (A = -0.75). The strict remap test
// tolerates an absolute error of 1.0 for bicubic, so FMA contraction is fine.
static inline void interpolateCubicV(const v_float32& x,
v_float32& c0, v_float32& c1,
v_float32& c2, v_float32& c3)
{
const v_float32 A = vx_setall_f32(-0.75f);
const v_float32 A5 = vx_setall_f32(-3.75f); // 5*A
const v_float32 A8 = vx_setall_f32(-6.0f); // 8*A
const v_float32 A4 = vx_setall_f32(-3.0f); // 4*A
const v_float32 Ap2 = vx_setall_f32(1.25f); // A+2
const v_float32 Ap3 = vx_setall_f32(2.25f); // A+3
const v_float32 one = vx_setall_f32(1.f);
v_float32 xp1 = v_add(x, one);
c0 = v_sub(v_mul(v_add(v_mul(v_sub(v_mul(A, xp1), A5), xp1), A8), xp1), A4);
c1 = v_add(v_mul(v_mul(v_sub(v_mul(Ap2, x), Ap3), x), x), one);
v_float32 u = v_sub(one, x);
c2 = v_add(v_mul(v_mul(v_sub(v_mul(Ap2, u), Ap3), u), u), one);
c3 = v_sub(v_sub(v_sub(one, c0), c1), c2);
}
// Select one of four vectors by runtime index. Used instead of an array of
// vector types, which is not valid for sizeless RVV vector types.
static inline v_float32 selectV4(int i, const v_float32& a, const v_float32& b,
const v_float32& c, const v_float32& d)
{
return i == 0 ? a : (i == 1 ? b : (i == 2 ? c : d));
}
template<typename T>
static int remapBicubicC1_run(const T* S0, size_t sstep, T* D, const short* XY,
const ushort* FXY, int dx, int dwidth,
unsigned width1, unsigned height1, int off_y)
{
const int vlanes = VTraits<v_float32>::vlanes();
const int dx0 = dx;
const v_float32 vscale = vx_setall_f32(1.f / INTER_TAB_SIZE);
const v_int32 vmask = vx_setall_s32(INTER_TAB_SIZE - 1);
int CV_DECL_ALIGNED(CV_SIMD_WIDTH) ofs[VTraits<v_float32>::max_nlanes];
float CV_DECL_ALIGNED(CV_SIMD_WIDTH) buf[VTraits<v_float32>::max_nlanes];
for( ; dx <= dwidth - vlanes; dx += vlanes )
{
bool allIn = true;
for( int k = 0; k < vlanes; k++ )
{
const unsigned sx = (unsigned)(XY[(dx + k) * 2] - 1);
const unsigned sy = (unsigned)(XY[(dx + k) * 2 + 1] - 1 + off_y);
if( sx >= width1 || sy >= height1 ) { allIn = false; break; }
ofs[k] = (int)((XY[(dx + k) * 2 + 1] - 1 + off_y) * (int)sstep
+ (XY[(dx + k) * 2] - 1));
}
if( !allIn )
break;
v_int32 fxy = v_reinterpret_as_s32(vx_load_expand(FXY + dx));
v_float32 fx = v_mul(v_cvt_f32(v_and(fxy, vmask)), vscale);
v_float32 fy = v_mul(v_cvt_f32(v_shr<INTER_BITS>(fxy)), vscale);
v_float32 vx0, vx1, vx2, vx3, vy0, vy1, vy2, vy3;
interpolateCubicV(fx, vx0, vx1, vx2, vx3);
interpolateCubicV(fy, vy0, vy1, vy2, vy3);
v_float32 acc = vx_setzero_f32();
for( int r = 0; r < 4; r++ )
{
const int roff = r * (int)sstep;
const v_float32 vyr = selectV4(r, vy0, vy1, vy2, vy3);
for( int c = 0; c < 4; c++ )
{
for( int k = 0; k < vlanes; k++ )
buf[k] = (float)S0[ofs[k] + roff + c];
acc = v_fma(vx_load(buf), v_mul(vyr, selectV4(c, vx0, vx1, vx2, vx3)), acc);
}
}
remapStoreC1(D + (dx - dx0), acc);
}
vx_cleanup();
return dx - dx0;
}
#if CV_SIMD128
static inline void remapLoad8(const float* S, v_float32x4& lo, v_float32x4& hi)
{
lo = v_load(S); hi = v_load(S + 4);
}
static inline void remapLoad8(const ushort* S, v_float32x4& lo, v_float32x4& hi)
{
v_uint16x8 v = v_load(S);
v_uint32x4 a, b; v_expand(v, a, b);
lo = v_cvt_f32(v_reinterpret_as_s32(a));
hi = v_cvt_f32(v_reinterpret_as_s32(b));
}
static inline void remapLoad8(const short* S, v_float32x4& lo, v_float32x4& hi)
{
v_int16x8 v = v_load(S);
v_int32x4 a, b; v_expand(v, a, b);
lo = v_cvt_f32(a); hi = v_cvt_f32(b);
}
static inline void remapStoreScalar(float* D, float v) { *D = v; }
static inline void remapStoreScalar(ushort* D, float v) { *D = saturate_cast<ushort>(v); }
static inline void remapStoreScalar(short* D, float v) { *D = saturate_cast<short>(v); }
static inline v_float32x4 remapLoad4(const float* S) { return v_load(S); }
static inline v_float32x4 remapLoad4(const ushort* S) { return v_cvt_f32(v_reinterpret_as_s32(v_load_expand(S))); }
static inline v_float32x4 remapLoad4(const short* S) { return v_cvt_f32(v_load_expand(S)); }
template<typename T>
static int remapBicubicC1wp_run(const T* S0, size_t sstep, T* D, const short* XY,
const ushort* FXY, const float* wtab, int dx,
int dwidth, unsigned width1, unsigned height1, int off_y)
{
const int dx0 = dx;
for( ; dx < dwidth; dx++ )
{
const unsigned sx = (unsigned)(XY[dx * 2] - 1);
const unsigned sy = (unsigned)(XY[dx * 2 + 1] - 1 + off_y);
if( sx >= width1 || sy >= height1 )
break;
const float* w = wtab + FXY[dx] * 16;
const T* S = S0 + (size_t)sy * sstep + sx;
v_float32x4 acc = v_setzero_f32();
for( int r = 0; r < 4; r++, S += sstep, w += 4 )
acc = v_fma(remapLoad4(S), v_load(w), acc);
remapStoreScalar(D + (dx - dx0), v_reduce_sum(acc));
}
vx_cleanup();
return dx - dx0;
}
template<typename T>
static int remapLanczos4C1_run(const T* S0, size_t sstep, T* D, const short* XY,
const ushort* FXY, const float* wtab, int dx,
int dwidth, unsigned width1, unsigned height1, int off_y)
{
const int dx0 = dx;
for( ; dx < dwidth; dx++ )
{
const unsigned sx = (unsigned)(XY[dx * 2] - 3);
const unsigned sy = (unsigned)(XY[dx * 2 + 1] - 3 + off_y);
if( sx >= width1 || sy >= height1 )
break;
const float* w = wtab + FXY[dx] * 64;
const T* S = S0 + (size_t)sy * sstep + sx;
v_float32x4 acc = v_setzero_f32();
for( int r = 0; r < 8; r++, S += sstep, w += 8 )
{
v_float32x4 s_lo, s_hi;
remapLoad8(S, s_lo, s_hi);
acc = v_fma(s_lo, v_load(w), acc);
acc = v_fma(s_hi, v_load(w + 4), acc);
}
remapStoreScalar(D + (dx - dx0), v_reduce_sum(acc));
}
vx_cleanup();
return dx - dx0;
}
#endif // CV_SIMD128
#endif // CV_SIMD
int remapBilinearC1_simd(int depth, const uchar* S0, size_t sstep, uchar* D,
const short* XY, const ushort* FXY, const float* wtab,
int dx, int X1, int off_y)
{
#if (CV_SIMD || CV_SIMD_SCALABLE)
switch (depth)
{
case CV_32F:
return remapBilinearF32_run((const float*)S0, sstep, (float*)D,
XY, FXY, wtab, dx, X1, off_y);
case CV_16U:
return remapBilinearC1_run<ushort>((const ushort*)S0, sstep, (ushort*)D,
XY, FXY, wtab, dx, X1, off_y);
case CV_16S:
return remapBilinearC1_run<short>((const short*)S0, sstep, (short*)D,
XY, FXY, wtab, dx, X1, off_y);
default:
return 0;
}
#else
CV_UNUSED(depth); CV_UNUSED(S0); CV_UNUSED(sstep); CV_UNUSED(D);
CV_UNUSED(XY); CV_UNUSED(FXY); CV_UNUSED(wtab);
CV_UNUSED(dx); CV_UNUSED(X1); CV_UNUSED(off_y);
return 0;
#endif
}
int remapBicubicC1_simd(int depth, const uchar* S0, size_t sstep, uchar* D,
const short* XY, const ushort* FXY,
int dx, int dwidth, unsigned width1, unsigned height1, int off_y)
{
#if (CV_SIMD || CV_SIMD_SCALABLE)
switch (depth)
{
case CV_32F:
return remapBicubicC1_run<float>((const float*)S0, sstep, (float*)D,
XY, FXY, dx, dwidth, width1, height1, off_y);
case CV_16U:
return remapBicubicC1_run<ushort>((const ushort*)S0, sstep, (ushort*)D,
XY, FXY, dx, dwidth, width1, height1, off_y);
case CV_16S:
return remapBicubicC1_run<short>((const short*)S0, sstep, (short*)D,
XY, FXY, dx, dwidth, width1, height1, off_y);
default:
return 0;
}
#else
CV_UNUSED(depth); CV_UNUSED(S0); CV_UNUSED(sstep); CV_UNUSED(D);
CV_UNUSED(XY); CV_UNUSED(FXY); CV_UNUSED(dx); CV_UNUSED(dwidth);
CV_UNUSED(width1); CV_UNUSED(height1); CV_UNUSED(off_y);
return 0;
#endif
}
int remapLanczos4C1_simd(int depth, const uchar* S0, size_t sstep, uchar* D,
const short* XY, const ushort* FXY, const float* wtab,
int dx, int dwidth, unsigned width1, unsigned height1, int off_y)
{
#if CV_SIMD128
switch (depth)
{
// CV_32F is intentionally omitted: the vectorized 8x8 accumulation reorders
// the 64-tap sum, so on the tight 1e-3 float tolerance it diverges from the
// scalar path (used for relative maps). 32F lanczos4 stays on the scalar loop.
case CV_16U:
return remapLanczos4C1_run<ushort>((const ushort*)S0, sstep, (ushort*)D,
XY, FXY, wtab, dx, dwidth, width1, height1, off_y);
case CV_16S:
return remapLanczos4C1_run<short>((const short*)S0, sstep, (short*)D,
XY, FXY, wtab, dx, dwidth, width1, height1, off_y);
default:
return 0;
}
#else
CV_UNUSED(depth); CV_UNUSED(S0); CV_UNUSED(sstep); CV_UNUSED(D);
CV_UNUSED(XY); CV_UNUSED(FXY); CV_UNUSED(wtab); CV_UNUSED(dx); CV_UNUSED(dwidth);
CV_UNUSED(width1); CV_UNUSED(height1); CV_UNUSED(off_y);
return 0;
#endif
}
int remapBicubicC1wp_simd(int depth, const uchar* S0, size_t sstep, uchar* D,
const short* XY, const ushort* FXY, const float* wtab,
int dx, int dwidth, unsigned width1, unsigned height1, int off_y)
{
#if CV_SIMD128
switch (depth)
{
case CV_32F:
return remapBicubicC1wp_run<float>((const float*)S0, sstep, (float*)D,
XY, FXY, wtab, dx, dwidth, width1, height1, off_y);
case CV_16U:
return remapBicubicC1wp_run<ushort>((const ushort*)S0, sstep, (ushort*)D,
XY, FXY, wtab, dx, dwidth, width1, height1, off_y);
case CV_16S:
return remapBicubicC1wp_run<short>((const short*)S0, sstep, (short*)D,
XY, FXY, wtab, dx, dwidth, width1, height1, off_y);
default:
return 0;
}
#else
CV_UNUSED(depth); CV_UNUSED(S0); CV_UNUSED(sstep); CV_UNUSED(D);
CV_UNUSED(XY); CV_UNUSED(FXY); CV_UNUSED(wtab); CV_UNUSED(dx); CV_UNUSED(dwidth);
CV_UNUSED(width1); CV_UNUSED(height1); CV_UNUSED(off_y);
return 0;
#endif
}
#endif // CV_CPU_OPTIMIZATION_DECLARATIONS_ONLY
CV_CPU_OPTIMIZATION_NAMESPACE_END
} // namespace cv
+4
View File
@@ -13,6 +13,7 @@
// Copyright (C) 2000-2008, Intel Corporation, all rights reserved. // Copyright (C) 2000-2008, Intel Corporation, all rights reserved.
// Copyright (C) 2009, Willow Garage Inc., all rights reserved. // Copyright (C) 2009, Willow Garage Inc., all rights reserved.
// Copyright (C) 2014-2015, Itseez Inc., all rights reserved. // Copyright (C) 2014-2015, Itseez Inc., all rights reserved.
// Copyright (C) 2026, Advanced Micro Devices, all rights reserved.
// Third party copyrights are property of their respective owners. // Third party copyrights are property of their respective owners.
// //
// Redistribution and use in source and binary forms, with or without modification, // Redistribution and use in source and binary forms, with or without modification,
@@ -466,4 +467,7 @@ Ptr<WarpPerspectiveLine_SSE4> WarpPerspectiveLine_SSE4::getImpl(const double *M)
} }
} }
#include "imgwarp.simd.hpp"
/* End of file. */ /* End of file. */
+7 -6
View File
@@ -437,8 +437,9 @@ __kernel void remap_2_32FC1(__global const uchar * srcptr, int src_step, int src
#if defined BORDER_CONSTANT #if defined BORDER_CONSTANT
float xf = map1[0], yf = map2[0]; float xf = map1[0], yf = map2[0];
int sx = (convert_int_sat_rtz(mad(xf, (float)INTER_TAB_SIZE, 0.5f)) >> INTER_BITS); // Coordinates can be negative, so truncation toward zero gives incorrect indices.
int sy = (convert_int_sat_rtz(mad(yf, (float)INTER_TAB_SIZE, 0.5f)) >> INTER_BITS); int sx = (convert_int_sat_rte(xf * (float)INTER_TAB_SIZE) >> INTER_BITS);
int sy = (convert_int_sat_rte(yf * (float)INTER_TAB_SIZE) >> INTER_BITS);
#if WARP_RELATIVE #if WARP_RELATIVE
sx += x; sx += x;
sy += y; sy += y;
@@ -464,7 +465,7 @@ __kernel void remap_2_32FC1(__global const uchar * srcptr, int src_step, int src
#else #else
#pragma unroll #pragma unroll
for (int xp = 0; xp < 2; ++xp) for (int xp = 0; xp < 2; ++xp)
xsum = fma(CONVERT_TO_WT(loadpix(srcptr + mad24(xp, TSIZE, src_index))), coeffs_x[xp], xsum); xsum = fma(CONVERT_TO_WT(loadpix(srcptr + mad24(xp, TSIZE, src_index))), (WT)(coeffs_x[xp]), xsum);
#endif #endif
} }
else else
@@ -472,12 +473,12 @@ __kernel void remap_2_32FC1(__global const uchar * srcptr, int src_step, int src
#pragma unroll #pragma unroll
for (int xp = 0; xp < 2; ++xp) for (int xp = 0; xp < 2; ++xp)
xsum = fma(sx + xp >= 0 && sx + xp < src_cols ? xsum = fma(sx + xp >= 0 && sx + xp < src_cols ?
CONVERT_TO_WT(loadpix(srcptr + mad24(xp, TSIZE, src_index))) : scalar, coeffs_x[xp], xsum); CONVERT_TO_WT(loadpix(srcptr + mad24(xp, TSIZE, src_index))) : scalar, (WT)(coeffs_x[xp]), xsum);
} }
sum = fma(xsum, coeffs_y[yp], sum); sum = fma(xsum, (WT)(coeffs_y[yp]), sum);
} }
else else
sum = fma(scalar, coeffs_y[yp], sum); sum = fma(scalar, (WT)(coeffs_y[yp]), sum);
} }
storepix(CONVERT_TO_T(sum), dst); storepix(CONVERT_TO_T(sum), dst);
+91 -1
View File
@@ -13,6 +13,7 @@
// Copyright (C) 2000-2008, Intel Corporation, all rights reserved. // Copyright (C) 2000-2008, Intel Corporation, all rights reserved.
// Copyright (C) 2009, Willow Garage Inc., all rights reserved. // Copyright (C) 2009, Willow Garage Inc., all rights reserved.
// Copyright (C) 2014-2015, Itseez Inc., all rights reserved. // Copyright (C) 2014-2015, Itseez Inc., all rights reserved.
// Copyright (C) 2026, Advanced Micro Devices, Inc., all rights reserved.
// Third party copyrights are property of their respective owners. // Third party copyrights are property of their respective owners.
// //
// Redistribution and use in source and binary forms, with or without modification, // Redistribution and use in source and binary forms, with or without modification,
@@ -54,6 +55,12 @@ template<typename T, int shift> struct FixPtCast
typedef T rtype; typedef T rtype;
rtype operator ()(type1 arg) const { return (T)((arg + (1 << (shift-1))) >> shift); } rtype operator ()(type1 arg) const { return (T)((arg + (1 << (shift-1))) >> shift); }
}; };
template<typename T, int shift> struct FixPtUcharCast
{
typedef ushort type1;
typedef T rtype;
rtype operator ()(type1 arg) const { return (T)((arg + (1 << (shift-1))) >> shift); }
};
template<typename T, int shift> struct FltCast template<typename T, int shift> struct FltCast
{ {
@@ -83,6 +90,76 @@ template<typename T1, typename T2> int PyrUpVecV(T1**, T2**, int) { return 0; }
template<typename T1, typename T2> int PyrUpVecVOneRow(T1**, T2*, int) { return 0; } template<typename T1, typename T2> int PyrUpVecVOneRow(T1**, T2*, int) { return 0; }
#if (CV_SIMD || CV_SIMD_SCALABLE) #if (CV_SIMD || CV_SIMD_SCALABLE)
// Dispatched AVX-512 VBMI implementations
int PyrDownVecH_uchar_ushort_1_dispatch(const uchar* src, ushort* row, int width);
int PyrDownVecH_uchar_ushort_2_dispatch(const uchar* src, ushort* row, int width);
int PyrDownVecH_uchar_ushort_3_dispatch(const uchar* src, ushort* row, int width);
int PyrDownVecH_uchar_ushort_4_dispatch(const uchar* src, ushort* row, int width);
// uchar -> ushort intermediate storage implementations
template<> int PyrDownVecH<uchar, ushort, 1>(const uchar* src, ushort* row, int width)
{
int x = PyrDownVecH_uchar_ushort_1_dispatch(src, row, width);
return x;
}
template<> int PyrDownVecH<uchar, ushort, 2>(const uchar* src, ushort* row, int width)
{
int x = PyrDownVecH_uchar_ushort_2_dispatch(src, row, width);
return x;
}
template<> int PyrDownVecH<uchar, ushort, 3>(const uchar* src, ushort* row, int width)
{
int x = PyrDownVecH_uchar_ushort_3_dispatch(src, row, width);
return x;
}
template<> int PyrDownVecH<uchar, ushort, 4>(const uchar* src, ushort* row, int width)
{
int x = PyrDownVecH_uchar_ushort_4_dispatch(src, row, width);
return x;
}
template<> int PyrDownVecV<ushort, uchar>(ushort** src, uchar* dst, int width)
{
int x = 0;
const ushort *row0 = src[0], *row1 = src[1], *row2 = src[2], *row3 = src[3], *row4 = src[4];
for( ; x <= width - VTraits<v_uint8>::vlanes(); x += VTraits<v_uint8>::vlanes() )
{
v_uint16 r0, r1, r2, r3, r4, t0, t1;
r0 = vx_load(row0 + x);
r1 = vx_load(row1 + x);
r2 = vx_load(row2 + x);
r3 = vx_load(row3 + x);
r4 = vx_load(row4 + x);
t0 = v_add(v_add(v_add(r0, r4), v_add(r2, r2)), v_shl<2>(v_add(v_add(r1, r3), r2)));
r0 = vx_load(row0 + x + VTraits<v_uint16>::vlanes());
r1 = vx_load(row1 + x + VTraits<v_uint16>::vlanes());
r2 = vx_load(row2 + x + VTraits<v_uint16>::vlanes());
r3 = vx_load(row3 + x + VTraits<v_uint16>::vlanes());
r4 = vx_load(row4 + x + VTraits<v_uint16>::vlanes());
t1 = v_add(v_add(v_add(r0, r4), v_add(r2, r2)), v_shl<2>(v_add(v_add(r1, r3), r2)));
v_store(dst + x, v_rshr_pack<8>(t0, t1));
}
if (x <= width - VTraits<v_uint16>::vlanes())
{
v_uint16 r0, r1, r2, r3, r4, t0;
r0 = vx_load(row0 + x);
r1 = vx_load(row1 + x);
r2 = vx_load(row2 + x);
r3 = vx_load(row3 + x);
r4 = vx_load(row4 + x);
t0 = v_add(v_add(v_add(r0, r4), v_add(r2, r2)), v_shl<2>(v_add(v_add(r1, r3), r2)));
v_rshr_pack_store<8>(dst + x, t0);
x += VTraits<v_uint16>::vlanes();
}
vx_cleanup();
return x;
}
template<> int PyrDownVecH<uchar, int, 1>(const uchar* src, int* row, int width) template<> int PyrDownVecH<uchar, int, 1>(const uchar* src, int* row, int width)
{ {
@@ -1291,10 +1368,23 @@ void cv::pyrDown( InputArray _src, OutputArray _dst, const Size& _dsz, int borde
{ {
CALL_HAL(pyrDown, cv_hal_pyrdown, src.data, src.step, src.cols, src.rows, dst.data, dst.step, dst.cols, dst.rows, depth, src.channels(), borderType); CALL_HAL(pyrDown, cv_hal_pyrdown, src.data, src.step, src.cols, src.rows, dst.data, dst.step, dst.cols, dst.rows, depth, src.channels(), borderType);
} }
bool use_avx512vbmi = checkHardwareSupport(CPU_AVX_512VBMI);
PyrFunc func = 0; PyrFunc func = 0;
if( depth == CV_8U ) if( depth == CV_8U )
func = pyrDown_< FixPtCast<uchar, 8> >; {
if(use_avx512vbmi)
{
// intermediate storage in 16bit only improves when used along with AVX512_VBMI ISA.
// AVX2: Usage with AVX2 has negligible measurable uplift in performance.
// ARM: Using 16bit intermediate storage has shown 5 to 10% degradation on ARM platform.
func = pyrDown_< FixPtUcharCast<uchar, 8> >;
}
else
{
func = pyrDown_< FixPtCast<uchar, 8> >;
}
}
else if( depth == CV_16S ) else if( depth == CV_16S )
func = pyrDown_< FixPtCast<short, 8> >; func = pyrDown_< FixPtCast<short, 8> >;
else if( depth == CV_16U ) else if( depth == CV_16U )
@@ -0,0 +1,44 @@
// This file is part of OpenCV project.
// It is subject to the license terms in the LICENSE file found in the top-level directory
// of this distribution and at http://opencv.org/license.html.
// Copyright (C) 2026, Advanced Micro Devices, Inc., all rights reserved.
#include "precomp.hpp"
#include "opencv2/core/hal/intrin.hpp"
#include "pyramids_avx512_vbmi.simd.hpp"
#include "pyramids_avx512_vbmi.simd_declarations.hpp" // defines CV_CPU_DISPATCH_MODES_ALL=AVX512_ICL,...,BASELINE
namespace cv {
// declared in pyramids.cpp
int PyrDownVecH_uchar_ushort_1_dispatch(const uchar* src, ushort* row, int width);
int PyrDownVecH_uchar_ushort_2_dispatch(const uchar* src, ushort* row, int width);
int PyrDownVecH_uchar_ushort_3_dispatch(const uchar* src, ushort* row, int width);
int PyrDownVecH_uchar_ushort_4_dispatch(const uchar* src, ushort* row, int width);
int PyrDownVecH_uchar_ushort_1_dispatch(const uchar* src, ushort* row, int width)
{
CV_CPU_DISPATCH(PyrDownVecH_uchar_ushort_1_vbmi, (src, row, width),
CV_CPU_DISPATCH_MODES_ALL);
}
int PyrDownVecH_uchar_ushort_2_dispatch(const uchar* src, ushort* row, int width)
{
CV_CPU_DISPATCH(PyrDownVecH_uchar_ushort_2_vbmi, (src, row, width),
CV_CPU_DISPATCH_MODES_ALL);
}
int PyrDownVecH_uchar_ushort_3_dispatch(const uchar* src, ushort* row, int width)
{
CV_CPU_DISPATCH(PyrDownVecH_uchar_ushort_3_vbmi, (src, row, width),
CV_CPU_DISPATCH_MODES_ALL);
}
int PyrDownVecH_uchar_ushort_4_dispatch(const uchar* src, ushort* row, int width)
{
CV_CPU_DISPATCH(PyrDownVecH_uchar_ushort_4_vbmi, (src, row, width),
CV_CPU_DISPATCH_MODES_ALL);
}
} // namespace cv
@@ -0,0 +1,259 @@
// This file is part of OpenCV project.
// It is subject to the license terms in the LICENSE file found in the top-level directory
// of this distribution and at http://opencv.org/license.html.
// Copyright (C) 2026, Advanced Micro Devices, Inc., all rights reserved.
#include "opencv2/core/hal/intrin.hpp"
namespace cv {
CV_CPU_OPTIMIZATION_NAMESPACE_BEGIN
// AVX-512 VBMI vpermb-based PyrDownVecH for uchar->ushort.
int PyrDownVecH_uchar_ushort_1_vbmi(const uchar* src, ushort* row, int width);
int PyrDownVecH_uchar_ushort_2_vbmi(const uchar* src, ushort* row, int width);
int PyrDownVecH_uchar_ushort_3_vbmi(const uchar* src, ushort* row, int width);
int PyrDownVecH_uchar_ushort_4_vbmi(const uchar* src, ushort* row, int width);
#ifndef CV_CPU_OPTIMIZATION_DECLARATIONS_ONLY
int PyrDownVecH_uchar_ushort_1_vbmi(const uchar* src, ushort* row, int width)
{
int x = 0;
#if !CV_AVX_512VBMI
CV_UNUSED(src); CV_UNUSED(row); CV_UNUSED(width);
#else
// cn=1: 32 output pixels per iteration.
// Source stride = 2 bytes/pixel. Tap k for pixel i: src[2*i + k], k=0..4.
// Max source byte: 2*31 + 4 = 66. Fits in 2 zmm (128 bytes).
__m512i vidx0 = _v512_set_epu8(
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
62,60,58,56,54,52,50,48,46,44,42,40,38,36,34,32,
30,28,26,24,22,20,18,16,14,12,10, 8, 6, 4, 2, 0);
__m512i vidx1 = _mm512_add_epi8(vidx0, _mm512_set1_epi8(1));
__m512i vidx2 = _mm512_add_epi8(vidx0, _mm512_set1_epi8(2));
__m512i vidx3 = _mm512_add_epi8(vidx0, _mm512_set1_epi8(3));
__m512i vidx4 = _mm512_add_epi8(vidx0, _mm512_set1_epi8(4));
__m512i v6 = _mm512_set1_epi16(6);
for (; x <= width - 32; x += 32, src += 64, row += 32)
{
__m512i sA = _mm512_loadu_si512(src);
__m512i sB = _mm512_loadu_si512(src + 64);
__m512i t0 = _mm512_permutex2var_epi8(sA, vidx0, sB);
__m512i t1 = _mm512_permutex2var_epi8(sA, vidx1, sB);
__m512i t2 = _mm512_permutex2var_epi8(sA, vidx2, sB);
__m512i t3 = _mm512_permutex2var_epi8(sA, vidx3, sB);
__m512i t4 = _mm512_permutex2var_epi8(sA, vidx4, sB);
__m512i w0 = _mm512_cvtepu8_epi16(_mm512_castsi512_si256(t0));
__m512i w1 = _mm512_cvtepu8_epi16(_mm512_castsi512_si256(t1));
__m512i w2 = _mm512_cvtepu8_epi16(_mm512_castsi512_si256(t2));
__m512i w3 = _mm512_cvtepu8_epi16(_mm512_castsi512_si256(t3));
__m512i w4 = _mm512_cvtepu8_epi16(_mm512_castsi512_si256(t4));
__m512i res = _mm512_add_epi16(
_mm512_add_epi16(_mm512_mullo_epi16(w2, v6),
_mm512_slli_epi16(_mm512_add_epi16(w1, w3), 2)),
_mm512_add_epi16(w0, w4));
_mm512_storeu_si512(row, res);
}
_mm256_zeroupper();
#endif
return x;
}
int PyrDownVecH_uchar_ushort_2_vbmi(const uchar* src, ushort* row, int width)
{
int x = 0;
#if !CV_AVX_512VBMI
CV_UNUSED(src); CV_UNUSED(row); CV_UNUSED(width);
#else
// cn=2: 16 output pixels x 2 channels = 32 output ushorts per iteration.
// Source stride = 4 bytes/pixel. Tap k for pixel i, ch j: src[4*i + 2*k + j].
// Max source byte: 4*15 + 8 + 1 = 69. Fits in 2 zmm.
__m512i vidx0 = _v512_set_epu8(
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
61,60,57,56,53,52,49,48,45,44,41,40,37,36,33,32,
29,28,25,24,21,20,17,16,13,12, 9, 8, 5, 4, 1, 0);
__m512i vidx1 = _mm512_add_epi8(vidx0, _mm512_set1_epi8(2));
__m512i vidx2 = _mm512_add_epi8(vidx0, _mm512_set1_epi8(4));
__m512i vidx3 = _mm512_add_epi8(vidx0, _mm512_set1_epi8(6));
__m512i vidx4 = _mm512_add_epi8(vidx0, _mm512_set1_epi8(8));
__m512i v6 = _mm512_set1_epi16(6);
for (; x <= width - 32; x += 32, src += 64, row += 32)
{
__m512i sA = _mm512_loadu_si512(src);
__m512i sB = _mm512_loadu_si512(src + 64);
__m512i t0 = _mm512_permutex2var_epi8(sA, vidx0, sB);
__m512i t1 = _mm512_permutex2var_epi8(sA, vidx1, sB);
__m512i t2 = _mm512_permutex2var_epi8(sA, vidx2, sB);
__m512i t3 = _mm512_permutex2var_epi8(sA, vidx3, sB);
__m512i t4 = _mm512_permutex2var_epi8(sA, vidx4, sB);
__m512i w0 = _mm512_cvtepu8_epi16(_mm512_castsi512_si256(t0));
__m512i w1 = _mm512_cvtepu8_epi16(_mm512_castsi512_si256(t1));
__m512i w2 = _mm512_cvtepu8_epi16(_mm512_castsi512_si256(t2));
__m512i w3 = _mm512_cvtepu8_epi16(_mm512_castsi512_si256(t3));
__m512i w4 = _mm512_cvtepu8_epi16(_mm512_castsi512_si256(t4));
__m512i res = _mm512_add_epi16(
_mm512_add_epi16(_mm512_mullo_epi16(w2, v6),
_mm512_slli_epi16(_mm512_add_epi16(w1, w3), 2)),
_mm512_add_epi16(w0, w4));
_mm512_storeu_si512(row, res);
}
_mm256_zeroupper();
#endif
return x;
}
int PyrDownVecH_uchar_ushort_3_vbmi(const uchar* src, ushort* row, int width)
{
int x = 0;
#if !CV_AVX_512VBMI
CV_UNUSED(src); CV_UNUSED(row); CV_UNUSED(width);
#else
// Each iteration: 16 output pixels × 3 channels = 48 output ushorts.
// Source span: 16×6 + 4*3 = 108 bytes. Load 128 contiguous bytes in 2 zmm.
__m512i vidx0 = _v512_set_epu8(
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
92,91,90, 86,85,84, 80,79,78, 74,73,72, 68,67,66, 62,
61,60, 56,55,54, 50,49,48, 44,43,42, 38,37,36, 32,31,
30, 26,25,24, 20,19,18, 14,13,12, 8, 7, 6, 2, 1, 0);
__m512i vidx1 = _v512_set_epu8(
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
95,94,93, 89,88,87, 83,82,81, 77,76,75, 71,70,69, 65,
64,63, 59,58,57, 53,52,51, 47,46,45, 41,40,39, 35,34,
33, 29,28,27, 23,22,21, 17,16,15, 11,10, 9, 5, 4, 3);
__m512i vidx2 = _v512_set_epu8(
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
98,97,96, 92,91,90, 86,85,84, 80,79,78, 74,73,72, 68,
67,66, 62,61,60, 56,55,54, 50,49,48, 44,43,42, 38,37,
36, 32,31,30, 26,25,24, 20,19,18, 14,13,12, 8, 7, 6);
__m512i vidx3 = _v512_set_epu8(
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
101,100,99, 95,94,93, 89,88,87, 83,82,81, 77,76,75, 71,
70,69, 65,64,63, 59,58,57, 53,52,51, 47,46,45, 41,40,
39, 35,34,33, 29,28,27, 23,22,21, 17,16,15, 11,10, 9);
__m512i vidx4 = _v512_set_epu8(
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
104,103,102, 98,97,96, 92,91,90, 86,85,84, 80,79,78, 74,
73,72, 68,67,66, 62,61,60, 56,55,54, 50,49,48, 44,43,
42, 38,37,36, 32,31,30, 26,25,24, 20,19,18, 14,13,12);
__m512i v6 = _mm512_set1_epi16(6);
//16 output pixels × 3 channels
for (; x <= width - 48; x += 48, src += 96, row += 48)
{
__m512i sA = _mm512_loadu_si512(src);
__m512i sB = _mm512_loadu_si512(src + 64);
__m512i t0 = _mm512_permutex2var_epi8(sA, vidx0, sB);
__m512i t1 = _mm512_permutex2var_epi8(sA, vidx1, sB);
__m512i t2 = _mm512_permutex2var_epi8(sA, vidx2, sB);
__m512i t3 = _mm512_permutex2var_epi8(sA, vidx3, sB);
__m512i t4 = _mm512_permutex2var_epi8(sA, vidx4, sB);
// Expand lower 32 bytes of each tap to u16 (first 32 of 48 bytes)
__m512i t0_lo = _mm512_cvtepu8_epi16(_mm512_castsi512_si256(t0));
__m512i t1_lo = _mm512_cvtepu8_epi16(_mm512_castsi512_si256(t1));
__m512i t2_lo = _mm512_cvtepu8_epi16(_mm512_castsi512_si256(t2));
__m512i t3_lo = _mm512_cvtepu8_epi16(_mm512_castsi512_si256(t3));
__m512i t4_lo = _mm512_cvtepu8_epi16(_mm512_castsi512_si256(t4));
// Expand upper 16 bytes (bytes 32..47) to u16
__m256i t0_hi = _mm256_cvtepu8_epi16(_mm512_extracti32x4_epi32(t0, 2));
__m256i t1_hi = _mm256_cvtepu8_epi16(_mm512_extracti32x4_epi32(t1, 2));
__m256i t2_hi = _mm256_cvtepu8_epi16(_mm512_extracti32x4_epi32(t2, 2));
__m256i t3_hi = _mm256_cvtepu8_epi16(_mm512_extracti32x4_epi32(t3, 2));
__m256i t4_hi = _mm256_cvtepu8_epi16(_mm512_extracti32x4_epi32(t4, 2));
// Compute low half (32xu16 values):
__m512i sum13_lo = _mm512_add_epi16(t1_lo, t3_lo);
__m512i res_lo = _mm512_add_epi16(
_mm512_add_epi16(_mm512_mullo_epi16(t2_lo, v6),
_mm512_slli_epi16(sum13_lo, 2)),
_mm512_add_epi16(t0_lo, t4_lo));
// Compute high half (16xu16 values):
__m256i v6_256 = _mm256_set1_epi16(6);
__m256i sum13_hi = _mm256_add_epi16(t1_hi, t3_hi);
__m256i res_hi = _mm256_add_epi16(
_mm256_add_epi16(_mm256_mullo_epi16(t2_hi, v6_256),
_mm256_slli_epi16(sum13_hi, 2)),
_mm256_add_epi16(t0_hi, t4_hi));
_mm512_storeu_si512(row, res_lo); //32 ushort
_mm256_storeu_si256((__m256i*)(row + 32), res_hi); //16 ushort
}
_mm256_zeroupper();
#endif // CV_AVX_512VBMI
return x;
}
int PyrDownVecH_uchar_ushort_4_vbmi(const uchar* src, ushort* row, int width)
{
int x = 0;
#if !CV_AVX_512VBMI
CV_UNUSED(src); CV_UNUSED(row); CV_UNUSED(width);
#else
// cn=4: 8 output pixels x 4 channels = 32 output ushorts per iteration.
// Source stride = 8 bytes/pixel. Tap k for pixel i, ch j: src[8*i + 4*k + j].
// Max source byte: 8*7 + 4*4 + 3 = 75. Fits in 2 zmm.
__m512i vidx0 = _v512_set_epu8(
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
59,58,57,56,51,50,49,48,43,42,41,40,35,34,33,32,
27,26,25,24,19,18,17,16,11,10, 9, 8, 3, 2, 1, 0);
__m512i vidx1 = _mm512_add_epi8(vidx0, _mm512_set1_epi8(4));
__m512i vidx2 = _mm512_add_epi8(vidx0, _mm512_set1_epi8(8));
__m512i vidx3 = _mm512_add_epi8(vidx0, _mm512_set1_epi8(12));
__m512i vidx4 = _mm512_add_epi8(vidx0, _mm512_set1_epi8(16));
__m512i v6 = _mm512_set1_epi16(6);
for (; x <= width - 32; x += 32, src += 64, row += 32)
{
__m512i sA = _mm512_loadu_si512(src);
__m512i sB = _mm512_loadu_si512(src + 64);
__m512i t0 = _mm512_permutex2var_epi8(sA, vidx0, sB);
__m512i t1 = _mm512_permutex2var_epi8(sA, vidx1, sB);
__m512i t2 = _mm512_permutex2var_epi8(sA, vidx2, sB);
__m512i t3 = _mm512_permutex2var_epi8(sA, vidx3, sB);
__m512i t4 = _mm512_permutex2var_epi8(sA, vidx4, sB);
__m512i w0 = _mm512_cvtepu8_epi16(_mm512_castsi512_si256(t0));
__m512i w1 = _mm512_cvtepu8_epi16(_mm512_castsi512_si256(t1));
__m512i w2 = _mm512_cvtepu8_epi16(_mm512_castsi512_si256(t2));
__m512i w3 = _mm512_cvtepu8_epi16(_mm512_castsi512_si256(t3));
__m512i w4 = _mm512_cvtepu8_epi16(_mm512_castsi512_si256(t4));
__m512i res = _mm512_add_epi16(
_mm512_add_epi16(_mm512_mullo_epi16(w2, v6),
_mm512_slli_epi16(_mm512_add_epi16(w1, w3), 2)),
_mm512_add_epi16(w0, w4));
_mm512_storeu_si512(row, res);
}
_mm256_zeroupper();
#endif
return x;
}
#endif // CV_CPU_OPTIMIZATION_DECLARATIONS_ONLY
CV_CPU_OPTIMIZATION_NAMESPACE_END
} // namespace cv
+277
View File
@@ -52,8 +52,12 @@
//M*/ //M*/
#include "../test_precomp.hpp" #include "../test_precomp.hpp"
#include "opencv2/core/types.hpp"
#include "opencv2/ts/ocl_test.hpp" #include "opencv2/ts/ocl_test.hpp"
#include <cmath>
#include <limits>
#ifdef HAVE_OPENCL #ifdef HAVE_OPENCL
namespace opencv_test { namespace opencv_test {
@@ -458,6 +462,279 @@ OCL_TEST_P(Remap_INTER_LINEAR, Mat)
} }
} }
template <typename T>
class RemapBorderConstant : public ::testing::Test
{
public:
RemapBorderConstant()
{
constexpr float k1 = -0.08f;
constexpr float coordinate_offset = -0.03f;
undistortedCanvas = calculateUndistortedCanvas(source_size, k1);
mapx.create(undistortedCanvas.size());
mapy.create(undistortedCanvas.size());
for (int y = 0; y < undistortedCanvas.height; ++y)
{
for (int x = 0; x < undistortedCanvas.width; ++x)
{
const Point2f undistorted(undistortedCanvas.tl() + cv::Point2i(x, y));
const Point2f distorted = barrelDistort(undistorted, source_size,
k1);
mapx.at<float>(y, x) = x == 0 ? coordinate_offset : distorted.x;
mapy.at<float>(y, x) = y == 0 ? coordinate_offset : distorted.y;
}
}
}
protected:
void runTensorProductTest()
{
const Mat_<T> source = createTensorProductImage(source_size);
runTest(createBarrelDistortedImage(source, barrel_coefficient));
}
void runCheckerboardTest()
{
const Mat_<T> source = createCheckerboardImage(source_size);
runTest(createBarrelDistortedImage(source, barrel_coefficient));
}
private:
static const Size source_size;
static constexpr float barrel_coefficient = -0.08f;
void runTest(const Mat_<T>& src)
{
constexpr float border_value = 2.0f;
constexpr float epsilon = 0.02f;
const Mat_<T> expected = remapReference(src, mapx, mapy,
border_value);
Mat_<T> cpuResult;
OCL_OFF(cv::remap(src, cpuResult, mapx, mapy, INTER_LINEAR,
BORDER_CONSTANT, Scalar::all(border_value)));
UMat usrc, umapx, umapy, uResult;
OCL_ON(src.copyTo(usrc));
OCL_ON(mapx.copyTo(umapx));
OCL_ON(mapy.copyTo(umapy));
OCL_ON(cv::remap(usrc, uResult, umapx, umapy, INTER_LINEAR,
BORDER_CONSTANT, Scalar::all(border_value)));
Mat_<T> oclResult;
uResult.copyTo(oclResult);
EXPECT_MAT_NEAR(expected, cpuResult, epsilon);
EXPECT_MAT_NEAR(expected, oclResult, epsilon);
EXPECT_MAT_NEAR(cpuResult, oclResult, epsilon);
}
static Mat_<T> createTensorProductImage(const Size& image_size)
{
constexpr float first_coordinate = 0.0f;
constexpr float last_coordinate = 1.0f;
Mat_<T> image(image_size);
for (int y = 0; y < image.rows; ++y)
{
const T yValue = first_coordinate +
static_cast<T>(y) * (last_coordinate - first_coordinate) /
static_cast<T>(image.rows - 1);
for (int x = 0; x < image.cols; ++x)
{
const T xValue = first_coordinate +
static_cast<T>(x) * (last_coordinate - first_coordinate) /
static_cast<T>(image.cols - 1);
image.template at<T>(y, x) = xValue * yValue;
}
}
return image;
}
static Mat_<T> createCheckerboardImage(const Size& image_size)
{
constexpr int checkerboard_period = 2;
constexpr float first_coordinate = 0.0f;
constexpr float last_coordinate = 1.0f;
Mat_<T> image(image_size);
for (int y = 0; y < image.rows; ++y)
{
for (int x = 0; x < image.cols; ++x)
{
image.template at<T>(y, x) = (x + y) % checkerboard_period == 0 ?
first_coordinate : last_coordinate;
}
}
return image;
}
static Mat_<T> remapReference(const Mat_<T>& src, const Mat1f& mapx,
const Mat1f& mapy, T borderValue)
{
CV_Assert(src.type() == DataType<T>::type);
CV_Assert(mapx.size() == mapy.size());
Mat_<T> dst(mapx.size());
for (int y = 0; y < dst.rows; ++y)
{
for (int x = 0; x < dst.cols; ++x)
{
const float fx = mapx.at<float>(y, x);
const float fy = mapy.at<float>(y, x);
const int scaledX = static_cast<int>(std::floor(fx * INTER_TAB_SIZE + 0.5f));
const int scaledY = static_cast<int>(std::floor(fy * INTER_TAB_SIZE + 0.5f));
const int sx = scaledX >> INTER_BITS;
const int sy = scaledY >> INTER_BITS;
const T wx = static_cast<T>(scaledX & (INTER_TAB_SIZE - 1)) /
static_cast<T>(INTER_TAB_SIZE);
const T wy = static_cast<T>(scaledY & (INTER_TAB_SIZE - 1)) /
static_cast<T>(INTER_TAB_SIZE);
T value = 0;
for (int yp = 0; yp < 2; ++yp)
{
for (int xp = 0; xp < 2; ++xp)
{
const int sourceX = sx + xp;
const int sourceY = sy + yp;
const bool inBounds = sourceX >= 0 && sourceX < src.cols &&
sourceY >= 0 && sourceY < src.rows;
const T sourceValue = inBounds ? src.template at<T>(sourceY, sourceX) : borderValue;
const T xWeight = xp == 0 ? 1 - wx : wx;
const T yWeight = yp == 0 ? 1 - wy : wy;
value += sourceValue * xWeight * yWeight;
}
}
dst.template at<T>(y, x) = value;
}
}
return dst;
}
static Point2f barrelDistort(const Point2f& point, const Size& imageSize, float k1)
{
const Point2f center = cv::Point2i(imageSize - cv::Size(1, 1)) / 2.0f;
const float scale = std::max(center.x, center.y);
const Point2f normalized = (point - center) / scale;
const float radiusSquared = normalized.dot(normalized);
const float factor = 1.0f + k1 * radiusSquared;
return center + normalized * scale * factor;
}
static Point2f barrelUndistort(const Point2f& point, const Size& imageSize, float k1)
{
const Point2f center = cv::Point2i(imageSize - cv::Size(1, 1)) / 2.0f;
const float scale = std::max(center.x, center.y);
const Point2f normalized = (point - center) / scale;
const float distortedRadius = std::hypot(normalized.x, normalized.y);
if (std::fpclassify(distortedRadius) == FP_ZERO)
return point;
float undistortedRadius = distortedRadius;
constexpr int inverse_iterations = 8;
for (int i = 0; i < inverse_iterations; ++i)
{
const float radiusSquared = undistortedRadius * undistortedRadius;
const float function = undistortedRadius * (1.0f + k1 * radiusSquared) -
distortedRadius;
const float derivative = 1.0f + 3.0f * k1 * radiusSquared;
undistortedRadius -= function / derivative;
}
const float radialScale = undistortedRadius / distortedRadius;
return center + normalized * radialScale * scale;
}
static Rect calculateUndistortedCanvas(const Size& sourceSize, float k1)
{
const Point2f corners[] = {
Point2f(0.0f, 0.0f),
Point2f(static_cast<float>(sourceSize.width - 1), 0.0f),
Point2f(0.0f, static_cast<float>(sourceSize.height - 1)),
Point2f(static_cast<float>(sourceSize.width - 1),
static_cast<float>(sourceSize.height - 1))
};
float minX = std::numeric_limits<float>::max();
float minY = std::numeric_limits<float>::max();
float maxX = std::numeric_limits<float>::lowest();
float maxY = std::numeric_limits<float>::lowest();
for (const Point2f& corner : corners)
{
const Point2f undistortedCorner = barrelUndistort(corner, sourceSize, k1);
minX = std::min(minX, undistortedCorner.x);
minY = std::min(minY, undistortedCorner.y);
maxX = std::max(maxX, undistortedCorner.x);
maxY = std::max(maxY, undistortedCorner.y);
}
const int left = static_cast<int>(std::floor(minX));
const int top = static_cast<int>(std::floor(minY));
const int right = static_cast<int>(std::ceil(maxX));
const int bottom = static_cast<int>(std::ceil(maxY));
return Rect(left, top, right - left + 1, bottom - top + 1);
}
static Mat_<T> createBarrelDistortedImage(const Mat_<T>& image, float k1)
{
Mat1f mapx(image.size());
Mat1f mapy(image.size());
for (int y = 0; y < image.rows; ++y)
{
for (int x = 0; x < image.cols; ++x)
{
const Point2f undistorted = barrelUndistort(Point2i(x, y),
image.size(), k1);
mapx.at<float>(y, x) = undistorted.x;
mapy.at<float>(y, x) = undistorted.y;
}
}
constexpr float first_coordinate = 0.0f;
return remapReference(image, mapx, mapy, first_coordinate);
}
Mat1f mapx;
Mat1f mapy;
Rect undistortedCanvas;
};
template<class T>
const Size RemapBorderConstant<T>::source_size{30, 20};
template<class T>
constexpr float RemapBorderConstant<T>::barrel_coefficient;
using RemapBorderConstantTypes = ::testing::Types<float, double>;
TYPED_TEST_CASE(RemapBorderConstant, RemapBorderConstantTypes);
// The smooth tensor-product image exposes artifacts at the image edges.
TYPED_TEST(RemapBorderConstant, tensor_product)
{
this->runTensorProductTest();
}
// The checkerboard image exposes interpolation artifacts within the image.
TYPED_TEST(RemapBorderConstant, checkerboard)
{
this->runCheckerboardTest();
}
///////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////////////////////////////////////////////////////////
// remap relative // remap relative
+5 -1
View File
@@ -1839,12 +1839,14 @@ int DTreesImpl::readSplit( const FileNode& fn )
Split split; Split split;
int vi = (int)fn["var"]; int vi = (int)fn["var"];
CV_Assert( 0 <= vi && vi <= (int)varType.size() ); CV_Assert( 0 <= vi && vi < (int)varMapping.size() );
vi = varMapping[vi]; // convert to varIdx if needed vi = varMapping[vi]; // convert to varIdx if needed
CV_Assert( 0 <= vi && vi < (int)varType.size() );
split.varIdx = vi; split.varIdx = vi;
if( varType[vi] == VAR_CATEGORICAL ) // split on categorical var if( varType[vi] == VAR_CATEGORICAL ) // split on categorical var
{ {
CV_Assert( vi < (int)catOfs.size() );
int i, val, ssize = getSubsetSize(vi); int i, val, ssize = getSubsetSize(vi);
split.subsetOfs = (int)subsets.size(); split.subsetOfs = (int)subsets.size();
for( i = 0; i < ssize; i++ ) for( i = 0; i < ssize; i++ )
@@ -1860,6 +1862,7 @@ int DTreesImpl::readSplit( const FileNode& fn )
if( fns.isInt() ) if( fns.isInt() )
{ {
val = (int)fns; val = (int)fns;
CV_Assert( 0 <= val && (val >> 5) < ssize );
subset[val >> 5] |= 1 << (val & 31); subset[val >> 5] |= 1 << (val & 31);
} }
else else
@@ -1869,6 +1872,7 @@ int DTreesImpl::readSplit( const FileNode& fn )
for( i = 0; i < n; i++, ++it ) for( i = 0; i < n; i++, ++it )
{ {
val = (int)*it; val = (int)*it;
CV_Assert( 0 <= val && (val >> 5) < ssize );
subset[val >> 5] |= 1 << (val & 31); subset[val >> 5] |= 1 << (val & 31);
} }
} }
+55
View File
@@ -96,6 +96,61 @@ ML_Legacy_Param param_list[] = {
INSTANTIATE_TEST_CASE_P(/**/, ML_Legacy_Params, testing::ValuesIn(param_list)); INSTANTIATE_TEST_CASE_P(/**/, ML_Legacy_Params, testing::ValuesIn(param_list));
TEST(ML_DTrees, load_bad_categorical_split)
{
// Train a tree with a categorical input so the model carries a
// categorical split serialized as an "in"/"not_in" value list.
const int n = 40;
Mat samples(n, 1, CV_32F);
Mat responses(n, 1, CV_32S);
for (int i = 0; i < n; i++)
{
int cat = i % 4;
samples.at<float>(i, 0) = (float)cat;
responses.at<int>(i, 0) = (cat == 1 || cat == 2) ? 1 : 0;
}
Mat varType(2, 1, CV_8U);
varType.at<uchar>(0) = ml::VAR_CATEGORICAL;
varType.at<uchar>(1) = ml::VAR_CATEGORICAL;
Ptr<ml::TrainData> td = ml::TrainData::create(samples, ml::ROW_SAMPLE, responses,
noArray(), noArray(), noArray(), varType);
Ptr<ml::DTrees> dt = ml::DTrees::create();
dt->setMaxDepth(4);
dt->setCVFolds(0);
dt->setMaxCategories(4);
dt->setMinSampleCount(1);
dt->train(td);
const string filename = cv::tempfile(".yml");
dt->save(filename);
string model;
{
std::ifstream in(filename.c_str());
std::stringstream ss;
ss << in.rdbuf();
model = ss.str();
}
// Category values in the split list index a per-split subset bitmask.
// Injecting a value larger than the number of categories used to write
// past that bitmask; loading such a model must be rejected, not crash.
size_t pos = model.find("in:");
ASSERT_NE(pos, string::npos);
size_t br = model.find('[', pos);
ASSERT_NE(br, string::npos);
model.insert(br + 1, "1000000,");
{
std::ofstream out(filename.c_str());
out << model;
}
Ptr<ml::DTrees> bad;
EXPECT_THROW(bad = ml::DTrees::load(filename), Exception);
remove(filename.c_str());
}
/*TEST(ML_SVM, throw_exception_when_save_untrained_model) /*TEST(ML_SVM, throw_exception_when_save_untrained_model)
{ {
Ptr<cv::ml::SVM> svm; Ptr<cv::ml::SVM> svm;
@@ -109,6 +109,8 @@ bool Dictionary::readDictionary(const cv::FileNode& fn) {
int nMarkers = 0, _markerSize = 0; int nMarkers = 0, _markerSize = 0;
if (fn.empty() || !readParameter("nmarkers", nMarkers, fn) || !readParameter("markersize", _markerSize, fn)) if (fn.empty() || !readParameter("nmarkers", nMarkers, fn) || !readParameter("markersize", _markerSize, fn))
return false; return false;
if (_markerSize <= 0)
return false;
Mat bytes(0, 0, CV_8UC1), marker(_markerSize, _markerSize, CV_8UC1); Mat bytes(0, 0, CV_8UC1), marker(_markerSize, _markerSize, CV_8UC1);
std::string markerString; std::string markerString;
for (int i = 0; i < nMarkers; i++) { for (int i = 0; i < nMarkers; i++) {
@@ -116,6 +118,8 @@ bool Dictionary::readDictionary(const cv::FileNode& fn) {
ostr << i; ostr << i;
if (!readParameter("marker_" + ostr.str(), markerString, fn)) if (!readParameter("marker_" + ostr.str(), markerString, fn))
return false; return false;
if (markerString.size() != (size_t)_markerSize * _markerSize)
return false;
for (int j = 0; j < (int) markerString.size(); j++) for (int j = 0; j < (int) markerString.size(); j++)
marker.at<unsigned char>(j) = (markerString[j] == '0') ? 0 : 1; marker.at<unsigned char>(j) = (markerString[j] == '0') ? 0 : 1;
bytes.push_back(Dictionary::getByteListFromBits(marker)); bytes.push_back(Dictionary::getByteListFromBits(marker));
+1 -1
View File
@@ -723,7 +723,7 @@ void HOGCache::init(const HOGDescriptor* _descriptor,
#if CV_SIMD128 #if CV_SIMD128
idx = v_float32x4(0.0f, 1.0f, 2.0f, 3.0f); idx = v_float32x4(0.0f, 1.0f, 2.0f, 3.0f);
for (; j <= blockSize.height - 4; j += 4) for (; j <= blockSize.width - 4; j += 4)
{ {
v_float32x4 t = v_sub(idx, _bw); v_float32x4 t = v_sub(idx, _bw);
t = v_mul(t, t); t = v_mul(t, t);
@@ -1409,6 +1409,46 @@ TEST(CV_ArucoMultiDict, serialization)
} }
TEST(CV_ArucoDictionary, readDictionary_oversized_marker)
{
// A marker string longer than markersize*markersize must be rejected instead of
// being written past the markersize x markersize buffer in readDictionary.
std::string serialized;
{
FileStorage fs_out(".json", FileStorage::WRITE + FileStorage::MEMORY);
ASSERT_TRUE(fs_out.isOpened());
fs_out << "nmarkers" << 1;
fs_out << "markersize" << 5;
fs_out << "marker_0" << std::string(2000, '1');
serialized = fs_out.releaseAndGetString();
}
FileStorage fs_in(serialized, FileStorage::READ + FileStorage::MEMORY);
ASSERT_TRUE(fs_in.isOpened());
aruco::Dictionary dict;
bool ok = true;
ASSERT_NO_THROW(ok = dict.readDictionary(fs_in.root()));
EXPECT_FALSE(ok);
}
TEST(CV_ArucoDictionary, readDictionary_roundtrip)
{
aruco::Dictionary dict = aruco::getPredefinedDictionary(aruco::DICT_5X5_50);
std::string serialized;
{
FileStorage fs_out(".json", FileStorage::WRITE + FileStorage::MEMORY);
ASSERT_TRUE(fs_out.isOpened());
dict.writeDictionary(fs_out);
serialized = fs_out.releaseAndGetString();
}
FileStorage fs_in(serialized, FileStorage::READ + FileStorage::MEMORY);
ASSERT_TRUE(fs_in.isOpened());
aruco::Dictionary loaded;
ASSERT_TRUE(loaded.readDictionary(fs_in.root()));
EXPECT_EQ(dict, loaded);
}
struct ArucoThreading: public testing::TestWithParam<aruco::CornerRefineMethod> struct ArucoThreading: public testing::TestWithParam<aruco::CornerRefineMethod>
{ {
struct NumThreadsSetter { struct NumThreadsSetter {
@@ -1364,4 +1364,22 @@ TEST(Objdetect_CascadeDetector, small_img)
} }
} }
// See https://github.com/opencv/opencv/issues/23580
// HOGDescriptor::compute() overflowed the gaussian weights buffer when the block
// was taller than it was wide: the SIMD store loop for the width buffer (_dj) was
// bounded by blockSize.height instead of blockSize.width. The block is chosen wide
// enough that the buffer is heap allocated, so the overflow is a real out-of-bounds
// write. "Done" is simply that compute() runs without crashing.
TEST(Objdetect_HOGDescriptor, issue_23580_tall_block_no_overflow)
{
Size winSize(272, 2048); // height > width, width > AutoBuffer stack size
HOGDescriptor hog(winSize, /*blockSize*/ winSize, /*blockStride*/ Size(8, 8),
/*cellSize*/ Size(8, 8), /*nbins*/ 9);
Mat src(winSize, CV_8UC1, Scalar::all(0));
std::vector<float> descriptors;
ASSERT_NO_THROW(hog.compute(src, descriptors));
EXPECT_FALSE(descriptors.empty());
}
}} // namespace }} // namespace
-7
View File
@@ -845,8 +845,6 @@ TEST(videoio_ffmpeg, create_with_property_badarg)
EXPECT_FALSE(cap.isOpened()); EXPECT_FALSE(cap.isOpened());
} }
// requires FFmpeg wrapper rebuild on Windows
#ifndef _WIN32
TEST(videoio_ffmpeg, open_with_format_cv8uc3) TEST(videoio_ffmpeg, open_with_format_cv8uc3)
{ {
if (!videoio_registry::hasBackend(CAP_FFMPEG)) if (!videoio_registry::hasBackend(CAP_FFMPEG))
@@ -862,7 +860,6 @@ TEST(videoio_ffmpeg, open_with_format_cv8uc3)
ASSERT_TRUE(cap.read(frame)); ASSERT_TRUE(cap.read(frame));
EXPECT_EQ(frame.channels(), 3); EXPECT_EQ(frame.channels(), 3);
} }
#endif
// related issue: https://github.com/opencv/opencv/issues/16821 // related issue: https://github.com/opencv/opencv/issues/16821
TEST(videoio_ffmpeg, DISABLED_open_from_web) TEST(videoio_ffmpeg, DISABLED_open_from_web)
@@ -1049,12 +1046,9 @@ inline static std::string videoio_ffmpeg_mismatch_name_printer(const testing::Te
INSTANTIATE_TEST_CASE_P(/**/, videoio_ffmpeg_channel_mismatch, testing::ValuesIn(mismatch_cases), videoio_ffmpeg_mismatch_name_printer); INSTANTIATE_TEST_CASE_P(/**/, videoio_ffmpeg_channel_mismatch, testing::ValuesIn(mismatch_cases), videoio_ffmpeg_mismatch_name_printer);
#ifndef _WIN32
typedef tuple<string, string> AlphaChannelParams; typedef tuple<string, string> AlphaChannelParams;
typedef testing::TestWithParam< AlphaChannelParams > videoio_ffmpeg_alpha_channel; typedef testing::TestWithParam< AlphaChannelParams > videoio_ffmpeg_alpha_channel;
// New feature in https://github.com/opencv/opencv/pull/28751 requires FFmpeg wrapper rebuild on Windows
TEST_P(videoio_ffmpeg_alpha_channel, write_read) TEST_P(videoio_ffmpeg_alpha_channel, write_read)
{ {
if (!videoio_registry::hasBackend(CAP_FFMPEG)) if (!videoio_registry::hasBackend(CAP_FFMPEG))
@@ -1116,7 +1110,6 @@ AlphaChannelParams alpha_params[] =
}; };
INSTANTIATE_TEST_CASE_P(/**/, videoio_ffmpeg_alpha_channel, testing::ValuesIn(alpha_params)); INSTANTIATE_TEST_CASE_P(/**/, videoio_ffmpeg_alpha_channel, testing::ValuesIn(alpha_params));
#endif
// related issue: https://github.com/opencv/opencv/issues/23088 // related issue: https://github.com/opencv/opencv/issues/23088
TEST(ffmpeg_cap_properties, set_pos_get_msec) TEST(ffmpeg_cap_properties, set_pos_get_msec)
@@ -30,6 +30,12 @@ int main( int argc, char** argv )
cout << "Usage: " << argv[0] << " <Input image>" << endl; cout << "Usage: " << argv[0] << " <Input image>" << endl;
return -1; return -1;
} }
if (image.channels() != 3)
{
cout << "The tutorial expects 3 channel image as input!\n" << endl;
cout << "Usage: " << argv[0] << " <Input image>" << endl;
return -1;
}
//! [basic-linear-transform-load] //! [basic-linear-transform-load]
//! [basic-linear-transform-output] //! [basic-linear-transform-output]
@@ -54,7 +60,7 @@ int main( int argc, char** argv )
//! [basic-linear-transform-operation] //! [basic-linear-transform-operation]
for( int y = 0; y < image.rows; y++ ) { for( int y = 0; y < image.rows; y++ ) {
for( int x = 0; x < image.cols; x++ ) { for( int x = 0; x < image.cols; x++ ) {
for( int c = 0; c < image.channels(); c++ ) { for( int c = 0; c < 3; c++ ) {
new_image.at<Vec3b>(y,x)[c] = new_image.at<Vec3b>(y,x)[c] =
saturate_cast<uchar>( alpha*image.at<Vec3b>(y,x)[c] + beta ); saturate_cast<uchar>( alpha*image.at<Vec3b>(y,x)[c] + beta );
} }