mirror of
https://github.com/opencv/opencv.git
synced 2026-07-29 15:23:05 +04:00
Merge pull request #9233 from LaurentBerger:dnndoc
This commit is contained in:
Alexander Alekhin
Binary file not shown.
|
Before Width: | Height: | Size: 27 KiB |
@@ -1,66 +0,0 @@
|
||||
Load Caffe framework models {#tutorial_dnn_googlenet}
|
||||
===========================
|
||||
|
||||
Introduction
|
||||
------------
|
||||
|
||||
In this tutorial you will learn how to use opencv_dnn module for image classification by using
|
||||
GoogLeNet trained network from [Caffe model zoo](http://caffe.berkeleyvision.org/model_zoo.html).
|
||||
|
||||
We will demonstrate results of this example on the following picture.
|
||||
|
||||
|
||||
Source Code
|
||||
-----------
|
||||
|
||||
We will be using snippets from the example application, that can be downloaded [here](https://github.com/opencv/opencv/blob/master/samples/dnn/caffe_googlenet.cpp).
|
||||
|
||||
@include dnn/caffe_googlenet.cpp
|
||||
|
||||
Explanation
|
||||
-----------
|
||||
|
||||
-# Firstly, download GoogLeNet model files:
|
||||
[bvlc_googlenet.prototxt ](https://raw.githubusercontent.com/opencv/opencv/master/samples/data/dnn/bvlc_googlenet.prototxt) and
|
||||
[bvlc_googlenet.caffemodel](http://dl.caffe.berkeleyvision.org/bvlc_googlenet.caffemodel)
|
||||
|
||||
Also you need file with names of [ILSVRC2012](http://image-net.org/challenges/LSVRC/2012/browse-synsets) classes:
|
||||
[synset_words.txt](https://raw.githubusercontent.com/opencv/opencv/master/samples/data/dnn/synset_words.txt).
|
||||
|
||||
Put these files into working dir of this program example.
|
||||
|
||||
-# Read and initialize network using path to .prototxt and .caffemodel files
|
||||
@snippet dnn/caffe_googlenet.cpp Read and initialize network
|
||||
|
||||
-# Check that network was read successfully
|
||||
@snippet dnn/caffe_googlenet.cpp Check that network was read successfully
|
||||
|
||||
-# Read input image and convert to the blob, acceptable by GoogleNet
|
||||
@snippet dnn/caffe_googlenet.cpp Prepare blob
|
||||
Firstly, we resize the image and change its channel sequence order.
|
||||
|
||||
Now image is actually a 3-dimensional array with 224x224x3 shape.
|
||||
|
||||
Next, we convert the image to 4-dimensional blob (so-called batch) with 1x3x224x224 shape by using special cv::dnn::blobFromImages constructor.
|
||||
|
||||
-# Pass the blob to the network
|
||||
@snippet dnn/caffe_googlenet.cpp Set input blob
|
||||
In bvlc_googlenet.prototxt the network input blob named as "data", therefore this blob labeled as ".data" in opencv_dnn API.
|
||||
|
||||
Other blobs labeled as "name_of_layer.name_of_layer_output".
|
||||
|
||||
-# Make forward pass
|
||||
@snippet dnn/caffe_googlenet.cpp Make forward pass
|
||||
During the forward pass output of each network layer is computed, but in this example we need output from "prob" layer only.
|
||||
|
||||
-# Determine the best class
|
||||
@snippet dnn/caffe_googlenet.cpp Gather output
|
||||
We put the output of "prob" layer, which contain probabilities for each of 1000 ILSVRC2012 image classes, to the `prob` blob.
|
||||
And find the index of element with maximal value in this one. This index correspond to the class of the image.
|
||||
|
||||
-# Print results
|
||||
@snippet dnn/caffe_googlenet.cpp Print results
|
||||
For our image we get:
|
||||
> Best class: #812 'space shuttle'
|
||||
>
|
||||
> Probability: 99.6378%
|
||||
@@ -1,130 +0,0 @@
|
||||
# How to enable Halide backend for improve efficiency {#tutorial_dnn_halide}
|
||||
|
||||
## Introduction
|
||||
This tutorial guidelines how to run your models in OpenCV deep learning module
|
||||
using Halide language backend. Halide is an open-source project that let us
|
||||
write image processing algorithms in well-readable format, schedule computations
|
||||
according to specific device and evaluate it with a quite good efficiency.
|
||||
|
||||
An official website of the Halide project: http://halide-lang.org/.
|
||||
|
||||
## Efficiency comparison
|
||||
Measured on Intel® Core™ i7-6700K CPU @ 4.00GHz x 8.
|
||||
|
||||
Single image forward pass (in milliseconds):
|
||||
|
||||
| Architecture | MKL backend | Halide backend | Speed Up ratio |
|
||||
|-----------------:|------------:|---------------:|---------------:|
|
||||
| AlexNet | 16.55 | 22.38 | x0.73 |
|
||||
| ResNet-50 | 63.69 | 73.91 | x0.86 |
|
||||
| SqueezeNet v1.1 | 10.11 | 8.21 | x1.23 |
|
||||
| Inception-5h | 35.38 | 37.06 | x0.95 |
|
||||
| ENet @ 3x512x256 | 82.26 | 41.21 | x1.99 |
|
||||
|
||||
Scheduling directives might be found @ [opencv_extra/testdata/dnn](https://github.com/opencv/opencv_extra/tree/master/testdata/dnn).
|
||||
|
||||
## Requirements
|
||||
### LLVM compiler
|
||||
|
||||
@note LLVM compilation might take a long time.
|
||||
|
||||
- Download LLVM source code from http://releases.llvm.org/4.0.0/llvm-4.0.0.src.tar.xz.
|
||||
Unpack it. Let **llvm_root** is a root directory of source code.
|
||||
|
||||
- Create directory **llvm_root**/tools/clang
|
||||
|
||||
- Download Clang with the same version as LLVM. In our case it will be from
|
||||
http://releases.llvm.org/4.0.0/cfe-4.0.0.src.tar.xz. Unpack it into
|
||||
**llvm_root**/tools/clang. Note that it should be a root for Clang source code.
|
||||
|
||||
- Build LLVM on Linux
|
||||
@code
|
||||
cd llvm_root
|
||||
mkdir build && cd build
|
||||
cmake -DLLVM_ENABLE_TERMINFO=OFF -DLLVM_TARGETS_TO_BUILD="X86" -DLLVM_ENABLE_ASSERTIONS=ON -DCMAKE_BUILD_TYPE=Release ..
|
||||
make -j4
|
||||
@endcode
|
||||
|
||||
- Build LLVM on Windows (Developer Command Prompt)
|
||||
@code
|
||||
mkdir \\path-to-llvm-build\\ && cd \\path-to-llvm-build\\
|
||||
cmake.exe -DLLVM_ENABLE_TERMINFO=OFF -DLLVM_TARGETS_TO_BUILD=X86 -DLLVM_ENABLE_ASSERTIONS=ON -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=\\path-to-llvm-install\\ -G "Visual Studio 14 Win64" \\path-to-llvm-src\\
|
||||
MSBuild.exe /m:4 /t:Build /p:Configuration=Release .\\INSTALL.vcxproj
|
||||
@endcode
|
||||
|
||||
@note `\\path-to-llvm-build\\` and `\\path-to-llvm-install\\` are different directories.
|
||||
|
||||
### Halide language.
|
||||
|
||||
- Download source code from GitHub repository, https://github.com/halide/Halide
|
||||
or using git. The root directory will be a **halide_root**.
|
||||
@code
|
||||
git clone https://github.com/halide/Halide.git
|
||||
@endcode
|
||||
|
||||
- Build Halide on Linux
|
||||
@code
|
||||
cd halide_root
|
||||
mkdir build && cd build
|
||||
cmake -DLLVM_DIR=llvm_root/build/lib/cmake/llvm -DCMAKE_BUILD_TYPE=Release -DLLVM_VERSION=40 -DWITH_TESTS=OFF -DWITH_APPS=OFF -DWITH_TUTORIALS=OFF ..
|
||||
make -j4
|
||||
@endcode
|
||||
|
||||
- Build Halide on Windows (Developer Command Prompt)
|
||||
@code
|
||||
cd halide_root
|
||||
mkdir build && cd build
|
||||
cmake.exe -DLLVM_DIR=\\path-to-llvm-install\\lib\\cmake\\llvm -DLLVM_VERSION=40 -DWITH_TESTS=OFF -DWITH_APPS=OFF -DWITH_TUTORIALS=OFF -DCMAKE_BUILD_TYPE=Release -G "Visual Studio 14 Win64" ..
|
||||
MSBuild.exe /m:4 /t:Build /p:Configuration=Release .\\ALL_BUILD.vcxproj
|
||||
@endcode
|
||||
|
||||
## Build OpenCV with Halide backend
|
||||
When you build OpenCV add the following configuration flags:
|
||||
|
||||
- `WITH_HALIDE` - enable Halide linkage
|
||||
|
||||
- `HALIDE_ROOT_DIR` - path to Halide build directory
|
||||
|
||||
## Sample
|
||||
|
||||
@include dnn/squeezenet_halide.cpp
|
||||
|
||||
## Explanation
|
||||
Download Caffe model from SqueezeNet repository: [train_val.prototxt](https://github.com/DeepScale/SqueezeNet/blob/master/SqueezeNet_v1.1/train_val.prototxt) and [squeezenet_v1.1.caffemodel](https://github.com/DeepScale/SqueezeNet/blob/master/SqueezeNet_v1.1/squeezenet_v1.1.caffemodel).
|
||||
|
||||
Also you need file with names of [ILSVRC2012](http://image-net.org/challenges/LSVRC/2012/browse-synsets) classes:
|
||||
[synset_words.txt](https://raw.githubusercontent.com/opencv/opencv/master/samples/data/dnn/synset_words.txt).
|
||||
|
||||
Put these files into working dir of this program example.
|
||||
|
||||
-# Read and initialize network using path to .prototxt and .caffemodel files
|
||||
@snippet dnn/squeezenet_halide.cpp Read and initialize network
|
||||
|
||||
-# Check that network was read successfully
|
||||
@snippet dnn/squeezenet_halide.cpp Check that network was read successfully
|
||||
|
||||
-# Read input image and convert to the 4-dimensional blob, acceptable by SqueezeNet v1.1
|
||||
@snippet dnn/squeezenet_halide.cpp Prepare blob
|
||||
|
||||
-# Pass the blob to the network
|
||||
@snippet dnn/squeezenet_halide.cpp Set input blob
|
||||
|
||||
-# Enable Halide backend for layers where it is implemented
|
||||
@snippet dnn/squeezenet_halide.cpp Enable Halide backend
|
||||
|
||||
-# Make forward pass
|
||||
@snippet dnn/squeezenet_halide.cpp Make forward pass
|
||||
Remember that the first forward pass after initialization require quite more
|
||||
time that the next ones. It's because of runtime compilation of Halide pipelines
|
||||
at the first invocation.
|
||||
|
||||
-# Determine the best class
|
||||
@snippet dnn/squeezenet_halide.cpp Determine the best class
|
||||
|
||||
-# Print results
|
||||
@snippet dnn/squeezenet_halide.cpp Print results
|
||||
For our image we get:
|
||||
|
||||
> Best class: #812 'space shuttle'
|
||||
>
|
||||
> Probability: 97.9812%
|
||||
@@ -1,82 +0,0 @@
|
||||
# How to schedule your network for Halide backend {#tutorial_dnn_halide_scheduling}
|
||||
|
||||
## Introduction
|
||||
Halide code is the same for every device we use. But for achieving the satisfied
|
||||
efficiency we should schedule computations properly. In this tutorial we describe
|
||||
the ways to schedule your networks using Halide backend in OpenCV deep learning module.
|
||||
|
||||
For better understanding of Halide scheduling you might want to read tutorials @ http://halide-lang.org/tutorials.
|
||||
|
||||
If it's your first meeting with Halide in OpenCV, we recommend to start from @ref tutorial_dnn_halide.
|
||||
|
||||
## Configuration files
|
||||
You can schedule computations of Halide pipeline by writing textual configuration files.
|
||||
It means that you can easily vectorize, parallelize and manage loops order of
|
||||
layers computation. Pass path to file with scheduling directives for specific
|
||||
device into ```cv::dnn::Net::setHalideScheduler``` before the first ```cv::dnn::Net::forward``` call.
|
||||
|
||||
Scheduling configuration files represented as YAML files where each node is a
|
||||
scheduled function or a scheduling directive.
|
||||
@code
|
||||
relu1:
|
||||
reorder: [x, c, y]
|
||||
split: { y: 2, c: 8 }
|
||||
parallel: [yo, co]
|
||||
unroll: yi
|
||||
vectorize: { x: 4 }
|
||||
conv1_constant_exterior:
|
||||
compute_at: { relu1: yi }
|
||||
@endcode
|
||||
|
||||
Considered use variables `n` for batch dimension, `c` for channels,
|
||||
`y` for rows and `x` for columns. For variables after split are used names
|
||||
with the same prefix but `o` and `i` suffixes for outer and inner variables
|
||||
correspondingly. In example, for variable `x` in range `[0, 10)` directive
|
||||
`split: { x: 2 }` gives new ones `xo` in range `[0, 5)` and `xi` in range `[0, 2)`.
|
||||
Variable name `x` is no longer available in the same scheduling node.
|
||||
|
||||
You can find scheduling examples at [opencv_extra/testdata/dnn](https://github.com/opencv/opencv_extra/tree/master/testdata/dnn)
|
||||
and use it for schedule your networks.
|
||||
|
||||
## Layers fusing
|
||||
Thanks to layers fusing we can schedule only the top layers of fused sets.
|
||||
Because for every output value we use the fused formula.
|
||||
In example, if you have three layers Convolution + Scale + ReLU one by one,
|
||||
@code
|
||||
conv(x, y, c, n) = sum(...) + bias(c);
|
||||
scale(x, y, c, n) = conv(x, y, c, n) * weights(c);
|
||||
relu(x, y, c, n) = max(scale(x, y, c, n), 0);
|
||||
@endcode
|
||||
|
||||
fused function is something like
|
||||
@code
|
||||
relu(x, y, c, n) = max((sum(...) + bias(c)) * weights(c), 0);
|
||||
@endcode
|
||||
|
||||
So only function called `relu` require scheduling.
|
||||
|
||||
## Scheduling patterns
|
||||
Sometimes networks built using blocked structure that means some layer are
|
||||
identical or quite similar. If you want to apply the same scheduling for
|
||||
different layers accurate to tiling or vectorization factors, define scheduling
|
||||
patterns in section `patterns` at the beginning of scheduling file.
|
||||
Also, your patters may use some parametric variables.
|
||||
@code
|
||||
# At the beginning of the file
|
||||
patterns:
|
||||
fully_connected:
|
||||
split: { c: c_split }
|
||||
fuse: { src: [x, y, co], dst: block }
|
||||
parallel: block
|
||||
vectorize: { ci: c_split }
|
||||
# Somewhere below
|
||||
fc8:
|
||||
pattern: fully_connected
|
||||
params: { c_split: 8 }
|
||||
@endcode
|
||||
|
||||
## Automatic scheduling
|
||||
You can let DNN to schedule layers automatically. Just skip call of ```cv::dnn::Net::setHalideScheduler```. Sometimes it might be even more efficient than manual scheduling.
|
||||
But if specific layers require be scheduled manually, you would be able to
|
||||
mix both manual and automatic scheduling ways. Write scheduling file
|
||||
and skip layers that you want to be scheduled automatically.
|
||||
Reference in New Issue
Block a user