mirror of
https://github.com/opencv/opencv.git
synced 2026-07-29 07:13:02 +04:00
Merge pull request #24231 from fengyuentau:halide_cleanup_5.x
dnn: cleanup of halide backend for 5.x #24231 Merge with https://github.com/opencv/opencv_extra/pull/1092. ### Pull Request Readiness Checklist See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request - [x] I agree to contribute to the project under Apache 2 License. - [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV - [x] The PR is proposed to the proper branch - [x] There is a reference to the original bug report and related work - [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable Patch to opencv_extra has the same branch name. - [x] The feature is well documented and sample code can be built with the project CMake
This commit is contained in:
Yuantao Feng
GitHub
@@ -3,7 +3,7 @@ Load Caffe framework models {#tutorial_dnn_googlenet}
|
||||
|
||||
@tableofcontents
|
||||
|
||||
@next_tutorial{tutorial_dnn_halide}
|
||||
@next_tutorial{tutorial_dnn_openvino}
|
||||
|
||||
| | |
|
||||
| -: | :- |
|
||||
|
||||
@@ -1,88 +0,0 @@
|
||||
# How to enable Halide backend for improve efficiency {#tutorial_dnn_halide}
|
||||
|
||||
@tableofcontents
|
||||
|
||||
@prev_tutorial{tutorial_dnn_googlenet}
|
||||
@next_tutorial{tutorial_dnn_halide_scheduling}
|
||||
|
||||
| | |
|
||||
| -: | :- |
|
||||
| Original author | Dmitry Kurtaev |
|
||||
| Compatibility | OpenCV >= 3.3 |
|
||||
|
||||
## Introduction
|
||||
This tutorial guidelines how to run your models in OpenCV deep learning module
|
||||
using Halide language backend. Halide is an open-source project that let us
|
||||
write image processing algorithms in well-readable format, schedule computations
|
||||
according to specific device and evaluate it with a quite good efficiency.
|
||||
|
||||
An official website of the Halide project: http://halide-lang.org/.
|
||||
|
||||
An up to date efficiency comparison: https://github.com/opencv/opencv/wiki/DNN-Efficiency
|
||||
|
||||
## Requirements
|
||||
### LLVM compiler
|
||||
|
||||
@note LLVM compilation might take a long time.
|
||||
|
||||
- Download LLVM source code from http://releases.llvm.org/4.0.0/llvm-4.0.0.src.tar.xz.
|
||||
Unpack it. Let **llvm_root** is a root directory of source code.
|
||||
|
||||
- Create directory **llvm_root**/tools/clang
|
||||
|
||||
- Download Clang with the same version as LLVM. In our case it will be from
|
||||
http://releases.llvm.org/4.0.0/cfe-4.0.0.src.tar.xz. Unpack it into
|
||||
**llvm_root**/tools/clang. Note that it should be a root for Clang source code.
|
||||
|
||||
- Build LLVM on Linux
|
||||
@code
|
||||
cd llvm_root
|
||||
mkdir build && cd build
|
||||
cmake -DLLVM_ENABLE_TERMINFO=OFF -DLLVM_TARGETS_TO_BUILD="X86" -DLLVM_ENABLE_ASSERTIONS=ON -DCMAKE_BUILD_TYPE=Release ..
|
||||
make -j4
|
||||
@endcode
|
||||
|
||||
- Build LLVM on Windows (Developer Command Prompt)
|
||||
@code
|
||||
mkdir \\path-to-llvm-build\\ && cd \\path-to-llvm-build\\
|
||||
cmake.exe -DLLVM_ENABLE_TERMINFO=OFF -DLLVM_TARGETS_TO_BUILD=X86 -DLLVM_ENABLE_ASSERTIONS=ON -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=\\path-to-llvm-install\\ -G "Visual Studio 14 Win64" \\path-to-llvm-src\\
|
||||
MSBuild.exe /m:4 /t:Build /p:Configuration=Release .\\INSTALL.vcxproj
|
||||
@endcode
|
||||
|
||||
@note `\\path-to-llvm-build\\` and `\\path-to-llvm-install\\` are different directories.
|
||||
|
||||
### Halide language.
|
||||
|
||||
- Download source code from GitHub repository, https://github.com/halide/Halide
|
||||
or using git. The root directory will be a **halide_root**.
|
||||
@code
|
||||
git clone https://github.com/halide/Halide.git
|
||||
@endcode
|
||||
|
||||
- Build Halide on Linux
|
||||
@code
|
||||
cd halide_root
|
||||
mkdir build && cd build
|
||||
cmake -DLLVM_DIR=llvm_root/build/lib/cmake/llvm -DCMAKE_BUILD_TYPE=Release -DLLVM_VERSION=40 -DWITH_TESTS=OFF -DWITH_APPS=OFF -DWITH_TUTORIALS=OFF ..
|
||||
make -j4
|
||||
@endcode
|
||||
|
||||
- Build Halide on Windows (Developer Command Prompt)
|
||||
@code
|
||||
cd halide_root
|
||||
mkdir build && cd build
|
||||
cmake.exe -DLLVM_DIR=\\path-to-llvm-install\\lib\\cmake\\llvm -DLLVM_VERSION=40 -DWITH_TESTS=OFF -DWITH_APPS=OFF -DWITH_TUTORIALS=OFF -DCMAKE_BUILD_TYPE=Release -G "Visual Studio 14 Win64" ..
|
||||
MSBuild.exe /m:4 /t:Build /p:Configuration=Release .\\ALL_BUILD.vcxproj
|
||||
@endcode
|
||||
|
||||
## Build OpenCV with Halide backend
|
||||
When you build OpenCV add the following configuration flags:
|
||||
|
||||
- `WITH_HALIDE` - enable Halide linkage
|
||||
|
||||
- `HALIDE_ROOT_DIR` - path to Halide build directory
|
||||
|
||||
## Set Halide as a preferable backend
|
||||
@code
|
||||
net.setPreferableBackend(DNN_BACKEND_HALIDE);
|
||||
@endcode
|
||||
@@ -1,92 +0,0 @@
|
||||
# How to schedule your network for Halide backend {#tutorial_dnn_halide_scheduling}
|
||||
|
||||
@tableofcontents
|
||||
|
||||
@prev_tutorial{tutorial_dnn_halide}
|
||||
@next_tutorial{tutorial_dnn_openvino}
|
||||
|
||||
| | |
|
||||
| -: | :- |
|
||||
| Original author | Dmitry Kurtaev |
|
||||
| Compatibility | OpenCV >= 3.3 |
|
||||
|
||||
## Introduction
|
||||
Halide code is the same for every device we use. But for achieving the satisfied
|
||||
efficiency we should schedule computations properly. In this tutorial we describe
|
||||
the ways to schedule your networks using Halide backend in OpenCV deep learning module.
|
||||
|
||||
For better understanding of Halide scheduling you might want to read tutorials @ http://halide-lang.org/tutorials.
|
||||
|
||||
If it's your first meeting with Halide in OpenCV, we recommend to start from @ref tutorial_dnn_halide.
|
||||
|
||||
## Configuration files
|
||||
You can schedule computations of Halide pipeline by writing textual configuration files.
|
||||
It means that you can easily vectorize, parallelize and manage loops order of
|
||||
layers computation. Pass path to file with scheduling directives for specific
|
||||
device into ```cv::dnn::Net::setHalideScheduler``` before the first ```cv::dnn::Net::forward``` call.
|
||||
|
||||
Scheduling configuration files represented as YAML files where each node is a
|
||||
scheduled function or a scheduling directive.
|
||||
@code
|
||||
relu1:
|
||||
reorder: [x, c, y]
|
||||
split: { y: 2, c: 8 }
|
||||
parallel: [yo, co]
|
||||
unroll: yi
|
||||
vectorize: { x: 4 }
|
||||
conv1_constant_exterior:
|
||||
compute_at: { relu1: yi }
|
||||
@endcode
|
||||
|
||||
Considered use variables `n` for batch dimension, `c` for channels,
|
||||
`y` for rows and `x` for columns. For variables after split are used names
|
||||
with the same prefix but `o` and `i` suffixes for outer and inner variables
|
||||
correspondingly. In example, for variable `x` in range `[0, 10)` directive
|
||||
`split: { x: 2 }` gives new ones `xo` in range `[0, 5)` and `xi` in range `[0, 2)`.
|
||||
Variable name `x` is no longer available in the same scheduling node.
|
||||
|
||||
You can find scheduling examples at [opencv_extra/testdata/dnn](https://github.com/opencv/opencv_extra/tree/5.x/testdata/dnn)
|
||||
and use it for schedule your networks.
|
||||
|
||||
## Layers fusing
|
||||
Thanks to layers fusing we can schedule only the top layers of fused sets.
|
||||
Because for every output value we use the fused formula.
|
||||
In example, if you have three layers Convolution + Scale + ReLU one by one,
|
||||
@code
|
||||
conv(x, y, c, n) = sum(...) + bias(c);
|
||||
scale(x, y, c, n) = conv(x, y, c, n) * weights(c);
|
||||
relu(x, y, c, n) = max(scale(x, y, c, n), 0);
|
||||
@endcode
|
||||
|
||||
fused function is something like
|
||||
@code
|
||||
relu(x, y, c, n) = max((sum(...) + bias(c)) * weights(c), 0);
|
||||
@endcode
|
||||
|
||||
So only function called `relu` require scheduling.
|
||||
|
||||
## Scheduling patterns
|
||||
Sometimes networks built using blocked structure that means some layer are
|
||||
identical or quite similar. If you want to apply the same scheduling for
|
||||
different layers accurate to tiling or vectorization factors, define scheduling
|
||||
patterns in section `patterns` at the beginning of scheduling file.
|
||||
Also, your patterns may use some parametric variables.
|
||||
@code
|
||||
# At the beginning of the file
|
||||
patterns:
|
||||
fully_connected:
|
||||
split: { c: c_split }
|
||||
fuse: { src: [x, y, co], dst: block }
|
||||
parallel: block
|
||||
vectorize: { ci: c_split }
|
||||
# Somewhere below
|
||||
fc8:
|
||||
pattern: fully_connected
|
||||
params: { c_split: 8 }
|
||||
@endcode
|
||||
|
||||
## Automatic scheduling
|
||||
You can let DNN to schedule layers automatically. Just skip call of ```cv::dnn::Net::setHalideScheduler```. Sometimes it might be even more efficient than manual scheduling.
|
||||
But if specific layers require be scheduled manually, you would be able to
|
||||
mix both manual and automatic scheduling ways. Write scheduling file
|
||||
and skip layers that you want to be scheduled automatically.
|
||||
@@ -1,7 +1,7 @@
|
||||
OpenCV usage with OpenVINO {#tutorial_dnn_openvino}
|
||||
=====================
|
||||
|
||||
@prev_tutorial{tutorial_dnn_halide_scheduling}
|
||||
@prev_tutorial{tutorial_dnn_googlenet}
|
||||
@next_tutorial{tutorial_dnn_android}
|
||||
|
||||
| | |
|
||||
|
||||
@@ -2,8 +2,6 @@ Deep Neural Networks (dnn module) {#tutorial_table_of_content_dnn}
|
||||
=====================================
|
||||
|
||||
- @subpage tutorial_dnn_googlenet
|
||||
- @subpage tutorial_dnn_halide
|
||||
- @subpage tutorial_dnn_halide_scheduling
|
||||
- @subpage tutorial_dnn_openvino
|
||||
- @subpage tutorial_dnn_android
|
||||
- @subpage tutorial_dnn_yolo
|
||||
|
||||
@@ -492,7 +492,6 @@ OpenCV have own DNN inference module which have own build-in engine, but can als
|
||||
| `WITH_NGRAPH` | _OFF_ | **Deprecated since OpenVINO 2022.1** Enables Intel NGraph library support. This library is part of Inference Engine backend which allows executing arbitrary networks read from files in multiple formats supported by OpenCV: Caffe, TensorFlow, PyTorch, Darknet, etc.. NGraph library must be installed, it is included into Inference Engine. |
|
||||
| `WITH_OPENVINO` | _OFF_ | Enable Intel OpenVINO Toolkit support. Should be used for OpenVINO>=2022.1 instead of `WITH_INF_ENGINE` and `WITH_NGRAPH`. |
|
||||
| `OPENCV_DNN_CUDA` | _OFF_ | Enable CUDA backend. [CUDA](https://en.wikipedia.org/wiki/CUDA), CUBLAS and [CUDNN](https://developer.nvidia.com/cudnn) must be installed. |
|
||||
| `WITH_HALIDE` | _OFF_ | Use experimental [Halide](https://en.wikipedia.org/wiki/Halide_(programming_language)) backend which can generate optimized code for dnn-layers at runtime. Halide must be installed. |
|
||||
| `WITH_VULKAN` | _OFF_ | Enable experimental [Vulkan](https://en.wikipedia.org/wiki/Vulkan_(API)) backend. Does not require additional dependencies, but can use external Vulkan headers (`VULKAN_INCLUDE_DIRS`). |
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user