mirror of
https://github.com/opencv/opencv.git
synced 2026-07-30 07:43:03 +04:00
Merge branch 4.x
This commit is contained in:
Alexander Alekhin
@@ -188,7 +188,7 @@ implementation below.
|
||||
|
||||
This will return a similarity index for each channel of the image. This value is between zero and
|
||||
one, where one corresponds to perfect fit. Unfortunately, the many Gaussian blurring is quite
|
||||
costly, so while the PSNR may work in a real time like environment (24 frame per second) this will
|
||||
costly, so while the PSNR may work in a real time like environment (24 frames per second) this will
|
||||
take significantly more than to accomplish similar performance results.
|
||||
|
||||
Therefore, the source code presented at the start of the tutorial will perform the PSNR measurement
|
||||
|
||||
@@ -36,6 +36,10 @@ create a circle board pattern in file acircleboard.svg with 7 rows, 5 columns an
|
||||
|
||||
python gen_pattern.py -o acircleboard.svg --rows 7 --columns 5 --type acircles --square_size 10 --radius_rate 2
|
||||
|
||||
create a radon checkerboard for findChessboardCornersSB() with markers in (7 4), (7 5), (8 5) cells:
|
||||
|
||||
python gen_pattern.py -o radon_checkerboard.svg --rows 10 --columns 15 --type radon_checkerboard -s 12.1 -m 7 4 7 5 8 5
|
||||
|
||||
If you want to change unit use -u option (mm inches, px, m)
|
||||
|
||||
If you want to change page size use -w and -h options
|
||||
|
||||
+1
-1
@@ -4,7 +4,7 @@ File Input and Output using XML and YAML files {#tutorial_file_input_output_with
|
||||
@tableofcontents
|
||||
|
||||
@prev_tutorial{tutorial_discrete_fourier_transform}
|
||||
@next_tutorial{tutorial_how_to_use_OpenCV_parallel_for_}
|
||||
@next_tutorial{tutorial_how_to_use_OpenCV_parallel_for_new}
|
||||
|
||||
| | |
|
||||
| -: | :- |
|
||||
|
||||
+166
@@ -0,0 +1,166 @@
|
||||
How to use the OpenCV parallel_for_ to parallelize your code {#tutorial_how_to_use_OpenCV_parallel_for_new}
|
||||
==================================================================
|
||||
|
||||
@tableofcontents
|
||||
|
||||
@prev_tutorial{tutorial_file_input_output_with_xml_yml}
|
||||
@next_tutorial{tutorial_univ_intrin}
|
||||
|
||||
| | |
|
||||
| -: | :- |
|
||||
| Compatibility | OpenCV >= 3.0 |
|
||||
|
||||
Goal
|
||||
----
|
||||
|
||||
The goal of this tutorial is to demonstrate the use of the OpenCV `parallel_for_` framework to easily parallelize your code. To illustrate the concept, we will write a program to perform convolution operation over an image.
|
||||
The full tutorial code is [here](https://github.com/opencv/opencv/tree/master/samples/cpp/tutorial_code/how_to_use_OpenCV_parallel_for_/how_to_use_OpenCV_parallel_for_new.cpp).
|
||||
|
||||
Precondition
|
||||
----
|
||||
|
||||
### Parallel Frameworks
|
||||
The first precondition is to have OpenCV built with a parallel framework.
|
||||
In OpenCV 4.5, the following parallel frameworks are available in that order:
|
||||
|
||||
* Intel Threading Building Blocks (3rdparty library, should be explicitly enabled)
|
||||
* OpenMP (integrated to compiler, should be explicitly enabled)
|
||||
* APPLE GCD (system wide, used automatically (APPLE only))
|
||||
* Windows RT concurrency (system wide, used automatically (Windows RT only))
|
||||
* Windows concurrency (part of runtime, used automatically (Windows only - MSVC++ >= 10))
|
||||
* Pthreads
|
||||
|
||||
As you can see, several parallel frameworks can be used in the OpenCV library. Some parallel libraries are third party libraries and have to be explicitly enabled in CMake before building, while others are automatically available with the platform (e.g. APPLE GCD).
|
||||
|
||||
### Race Conditions
|
||||
Race conditions occur when more than one thread try to write *or* read and write to a particular memory location simultaneously.
|
||||
Based on that, we can broadly classify algorithms into two categories:-
|
||||
1. Algorithms in which only a single thread writes data to a particular memory location.
|
||||
* In *convolution*, for example, even though multiple threads may read from a pixel at a particular time, only a single thread *writes* to a particular pixel.
|
||||
|
||||
2. Algorithms in which multiple threads may write to a single memory location.
|
||||
* Finding contours, features, etc. Such algorithms may require each thread to add data to a global variable simultaneously. For example, when detecting features, each thread will add features of their respective parts of the image to a common vector, thus creating a race condition.
|
||||
|
||||
Convolution
|
||||
-----------
|
||||
|
||||
We will use the example of performing a convolution to demonstrate the use of `parallel_for_` to parallelize the computation. This is an example of an algorithm which does not lead to a race condition.
|
||||
|
||||
Theory
|
||||
------
|
||||
Convolution is a simple mathematical operation widely used in image processing. Here, we slide a smaller matrix, called the *kernel*, over an image and a sum of the product of pixel values and corresponding values in the kernel gives us the value of the particular pixel in the output (called the anchor point of the kernel). Based on the values in the kernel, we get different results.
|
||||
In the example below, we use a 3x3 kernel (anchored at its center) and convolve over a 5x5 matrix to produce a 3x3 matrix. The size of the output can be altered by padding the input with suitable values.
|
||||
|
||||
|
||||
For more information about different kernels and what they do, look [here](https://en.wikipedia.org/wiki/Kernel_(image_processing))
|
||||
|
||||
For the purpose of this tutorial, we will implement the simplest form of the function which takes a grayscale image (1 channel) and an odd length square kernel and produces an output image.
|
||||
The operation will not be performed in-place.
|
||||
@note We can store a few of the relevant pixels temporarily to make sure we use the original values during the convolution and then do it in-place. However, the purpose of this tutorial is to introduce parallel_for_ function and an inplace implementation may be too complicated.
|
||||
|
||||
Pseudocode
|
||||
-----------
|
||||
|
||||
InputImage src, OutputImage dst, kernel(size n)
|
||||
makeborder(src, n/2)
|
||||
for each pixel (i, j) strictly inside borders, do:
|
||||
{
|
||||
value := 0
|
||||
for k := -n/2 to n/2, do:
|
||||
for l := -n/2 to n/2, do:
|
||||
value += kernel[n/2 + k][n/2 + l]*src[i + k][j + l]
|
||||
|
||||
dst[i][j] := value
|
||||
}
|
||||
|
||||
For an *n-sized kernel*, we will add a border of size *n/2* to handle edge cases.
|
||||
We then run two loops to move along the kernel and add the products to sum
|
||||
|
||||
Implementation
|
||||
--------------
|
||||
|
||||
### Sequential implementation
|
||||
|
||||
@snippet how_to_use_OpenCV_parallel_for_new.cpp convolution-sequential
|
||||
|
||||
We first make an output matrix(dst) with the same size as src and add borders to the src image(to handle edge cases).
|
||||
@snippet how_to_use_OpenCV_parallel_for_new.cpp convolution-make-borders
|
||||
|
||||
We then sequentially iterate over the pixels in the src image and compute the value over the kernel and the neighbouring pixel values.
|
||||
We then fill value to the corresponding pixel in the dst image.
|
||||
@snippet how_to_use_OpenCV_parallel_for_new.cpp convolution-kernel-loop
|
||||
|
||||
### Parallel implementation
|
||||
|
||||
When looking at the sequential implementation, we can notice that each pixel depends on multiple neighbouring pixels but only one pixel is edited at a time. Thus, to optimize the computation, we can split the image into stripes and parallely perform convolution on each, by exploiting the multi-core architecture of modern processor. The OpenCV @ref cv::parallel_for_ framework automatically decides how to split the computation efficiently and does most of the work for us.
|
||||
|
||||
@note Although values of a pixel in a particular stripe may depend on pixel values outside the stripe, these are only read only operations and hence will not cause undefined behaviour.
|
||||
|
||||
|
||||
We first declare a custom class that inherits from @ref cv::ParallelLoopBody and override the `virtual void operator ()(const cv::Range& range) const`.
|
||||
@snippet how_to_use_OpenCV_parallel_for_new.cpp convolution-parallel
|
||||
|
||||
The range in the `operator ()` represents the subset of values that will be treated by an individual thread. Based on the requirement, there may be different ways of splitting the range which in turn changes the computation.
|
||||
|
||||
For example, we can either
|
||||
1. Split the entire traversal of the image and obtain the [row, col] coordinate in the following way (as shown in the above code):
|
||||
|
||||
@snippet how_to_use_OpenCV_parallel_for_new.cpp overload-full
|
||||
|
||||
We would then call the parallel_for_ function in the following way:
|
||||
@snippet how_to_use_OpenCV_parallel_for_new.cpp convolution-parallel-function
|
||||
<br>
|
||||
|
||||
2. Split the rows and compute for each row:
|
||||
|
||||
@snippet how_to_use_OpenCV_parallel_for_new.cpp overload-row-split
|
||||
|
||||
In this case, we call the parallel_for_ function with a different range:
|
||||
@snippet how_to_use_OpenCV_parallel_for_new.cpp convolution-parallel-function-row
|
||||
|
||||
@note In our case, both implementations perform similarly. Some cases may allow better memory access patterns or other performance benefits.
|
||||
|
||||
To set the number of threads, you can use: @ref cv::setNumThreads. You can also specify the number of splitting using the nstripes parameter in @ref cv::parallel_for_. For instance, if your processor has 4 threads, setting `cv::setNumThreads(2)` or setting `nstripes=2` should be the same as by default it will use all the processor threads available but will split the workload only on two threads.
|
||||
|
||||
@note C++ 11 standard allows to simplify the parallel implementation by get rid of the `parallelConvolution` class and replacing it with lambda expression:
|
||||
|
||||
@snippet how_to_use_OpenCV_parallel_for_new.cpp convolution-parallel-cxx11
|
||||
|
||||
Results
|
||||
-----------
|
||||
|
||||
The resulting time taken for execution of the two implementations on a
|
||||
* *512x512 input* with a *5x5 kernel*:
|
||||
|
||||
This program shows how to use the OpenCV parallel_for_ function and
|
||||
compares the performance of the sequential and parallel implementations for a
|
||||
convolution operation
|
||||
Usage:
|
||||
./a.out [image_path -- default lena.jpg]
|
||||
|
||||
Sequential Implementation: 0.0953564s
|
||||
Parallel Implementation: 0.0246762s
|
||||
Parallel Implementation(Row Split): 0.0248722s
|
||||
|
||||
<br>
|
||||
|
||||
* *512x512 input with a 3x3 kernel*
|
||||
|
||||
This program shows how to use the OpenCV parallel_for_ function and
|
||||
compares the performance of the sequential and parallel implementations for a
|
||||
convolution operation
|
||||
Usage:
|
||||
./a.out [image_path -- default lena.jpg]
|
||||
|
||||
Sequential Implementation: 0.0301325s
|
||||
Parallel Implementation: 0.0117053s
|
||||
Parallel Implementation(Row Split): 0.0117894s
|
||||
|
||||
The performance of the parallel implementation depends on the type of CPU you have. For instance, on 4 cores - 8 threads CPU, runtime may be 6x to 7x faster than a sequential implementation. There are many factors to explain why we do not achieve a speed-up of 8x:
|
||||
* the overhead to create and manage the threads,
|
||||
* background processes running in parallel,
|
||||
* the difference between 4 hardware cores with 2 logical threads for each core and 8 hardware cores.
|
||||
|
||||
In the tutorial, we used a horizontal gradient filter(as shown in the animation above), which produces an image highlighting the vertical edges.
|
||||
|
||||
|
||||
BIN
Binary file not shown.
|
After Width: | Height: | Size: 147 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 64 KiB |
+9
-9
@@ -91,8 +91,8 @@ a new header with the new boundaries:
|
||||
Mat D (A, Rect(10, 10, 100, 100) ); // using a rectangle
|
||||
Mat E = A(Range::all(), Range(1,3)); // using row and column boundaries
|
||||
@endcode
|
||||
Now you may ask -- if the matrix itself may belong to multiple *Mat* objects who takes responsibility
|
||||
for cleaning it up when it's no longer needed. The short answer is: the last object that used it.
|
||||
Now you may ask -- if the matrix itself may belong to multiple *Mat* objects, who takes responsibility
|
||||
for cleaning it up when it's no longer needed? The short answer is: the last object that used it.
|
||||
This is handled by using a reference counting mechanism. Whenever somebody copies a header of a
|
||||
*Mat* object, a counter is increased for the matrix. Whenever a header is cleaned, this counter
|
||||
is decreased. When the counter reaches zero the matrix is freed. Sometimes you will want to copy
|
||||
@@ -102,12 +102,12 @@ Mat F = A.clone();
|
||||
Mat G;
|
||||
A.copyTo(G);
|
||||
@endcode
|
||||
Now modifying *F* or *G* will not affect the matrix pointed by the *A*'s header. What you need to
|
||||
Now modifying *F* or *G* will not affect the matrix pointed to by the *A*'s header. What you need to
|
||||
remember from all this is that:
|
||||
|
||||
- Output image allocation for OpenCV functions is automatic (unless specified otherwise).
|
||||
- You do not need to think about memory management with OpenCV's C++ interface.
|
||||
- The assignment operator and the copy constructor only copies the header.
|
||||
- The assignment operator and the copy constructor only copy the header.
|
||||
- The underlying matrix of an image may be copied using the @ref cv::Mat::clone() and @ref cv::Mat::copyTo()
|
||||
functions.
|
||||
|
||||
@@ -122,10 +122,10 @@ of these allows us to create many shades of gray.
|
||||
For *colorful* ways we have a lot more methods to choose from. Each of them breaks it down to three
|
||||
or four basic components and we can use the combination of these to create the others. The most
|
||||
popular one is RGB, mainly because this is also how our eye builds up colors. Its base colors are
|
||||
red, green and blue. To code the transparency of a color sometimes a fourth element: alpha (A) is
|
||||
red, green and blue. To code the transparency of a color sometimes a fourth element, alpha (A), is
|
||||
added.
|
||||
|
||||
There are, however, many other color systems each with their own advantages:
|
||||
There are, however, many other color systems, each with their own advantages:
|
||||
|
||||
- RGB is the most common as our eyes use something similar, however keep in mind that OpenCV standard display
|
||||
system composes colors using the BGR color space (red and blue channels are swapped places).
|
||||
@@ -139,11 +139,11 @@ There are, however, many other color systems each with their own advantages:
|
||||
Each of the building components has its own valid domains. This leads to the data type used. How
|
||||
we store a component defines the control we have over its domain. The smallest data type possible is
|
||||
*char*, which means one byte or 8 bits. This may be unsigned (so can store values from 0 to 255) or
|
||||
signed (values from -127 to +127). Although in case of three components this already gives 16
|
||||
million possible colors to represent (like in case of RGB) we may acquire an even finer control by
|
||||
signed (values from -127 to +127). Although this width, in the case of three components (like RGB), already gives 16
|
||||
million possible colors to represent, we may acquire an even finer control by
|
||||
using the float (4 byte = 32 bit) or double (8 byte = 64 bit) data types for each component.
|
||||
Nevertheless, remember that increasing the size of a component also increases the size of the whole
|
||||
picture in the memory.
|
||||
picture in memory.
|
||||
|
||||
Creating a Mat object explicitly
|
||||
----------------------------------
|
||||
|
||||
@@ -9,4 +9,5 @@ The Core Functionality (core module) {#tutorial_table_of_content_core}
|
||||
- @subpage tutorial_basic_linear_transform
|
||||
- @subpage tutorial_discrete_fourier_transform
|
||||
- @subpage tutorial_file_input_output_with_xml_yml
|
||||
- @subpage tutorial_how_to_use_OpenCV_parallel_for_
|
||||
- @subpage tutorial_how_to_use_OpenCV_parallel_for_new
|
||||
- @subpage tutorial_univ_intrin
|
||||
|
||||
@@ -0,0 +1,334 @@
|
||||
Vectorizing your code using Universal Intrinsics {#tutorial_univ_intrin}
|
||||
==================================================================
|
||||
|
||||
@tableofcontents
|
||||
|
||||
@prev_tutorial{tutorial_how_to_use_OpenCV_parallel_for_new}
|
||||
|
||||
| | |
|
||||
| -: | :- |
|
||||
| Compatibility | OpenCV >= 3.0 |
|
||||
|
||||
Goal
|
||||
----
|
||||
|
||||
The goal of this tutorial is to provide a guide to using the @ref core_hal_intrin feature to vectorize your C++ code for a faster runtime.
|
||||
We'll briefly look into _SIMD intrinsics_ and how to work with wide _registers_, followed by a tutorial on the basic operations using wide registers.
|
||||
|
||||
Theory
|
||||
------
|
||||
|
||||
In this section, we will briefly look into a few concepts to better help understand the functionality.
|
||||
|
||||
### Intrinsics
|
||||
Intrinsics are functions which are separately handled by the compiler. These functions are often optimized to perform in the most efficient ways possible and hence run faster than normal implementations. However, since these functions depend on the compiler, it makes it difficult to write portable applications.
|
||||
|
||||
### SIMD
|
||||
SIMD stands for **Single Instruction, Multiple Data**. SIMD Intrinsics allow the processor to vectorize calculations. The data is stored in what are known as *registers*. A *register* may be *128-bits*, *256-bits* or *512-bits* wide. Each *register* stores **multiple values** of the **same data type**. The size of the register and the size of each value determines the number of values stored in total.
|
||||
|
||||
Depending on what *Instruction Sets* your CPU supports, you may be able to use the different registers. To learn more, look [here](https://en.wikipedia.org/wiki/Instruction_set_architecture)
|
||||
|
||||
Universal Intrinsics
|
||||
--------------------
|
||||
|
||||
OpenCVs universal intrinsics provides an abstraction to SIMD vectorization methods and allows the user to use intrinsics without the need to write system specific code.
|
||||
|
||||
OpenCV Universal Intrinsics support the following instruction sets:
|
||||
* *128 bit* registers of various types support is implemented for a wide range of architectures including
|
||||
* x86(SSE/SSE2/SSE4.2),
|
||||
* ARM(NEON),
|
||||
* PowerPC(VSX),
|
||||
* MIPS(MSA).
|
||||
* *256 bit* registers are supported on x86(AVX2) and
|
||||
* *512 bit* registers are supported on x86(AVX512)
|
||||
|
||||
**We will now introduce the available structures and functions:**
|
||||
* Register structures
|
||||
* Load and store
|
||||
* Mathematical Operations
|
||||
* Reduce and Mask
|
||||
|
||||
### Register Structures
|
||||
|
||||
The Universal Intrinsics set implements every register as a structure based on the particular SIMD register.
|
||||
All types contain the `nlanes` enumeration which gives the exact number of values that the type can hold. This eliminates the need to hardcode the number of values during implementations.
|
||||
|
||||
@note Each register structure is under the `cv` namespace.
|
||||
|
||||
There are **two types** of registers:
|
||||
|
||||
* **Variable sized registers**: These structures do not have a fixed size and their exact bit length is deduced during compilation, based on the available SIMD capabilities. Consequently, the value of the `nlanes` enum is determined in compile time.
|
||||
<br>
|
||||
|
||||
Each structure follows the following convention:
|
||||
|
||||
v_[type of value][size of each value in bits]
|
||||
|
||||
For instance, **v_uint8 holds 8-bit unsigned integers** and **v_float32 holds 32-bit floating point values**. We then declare a register like we would declare any object in C++
|
||||
|
||||
Based on the available SIMD instruction set, a particular register will hold different number of values.
|
||||
For example: If your computer supports a maximum of 256bit registers,
|
||||
* *v_uint8* will hold 32 8-bit unsigned integers
|
||||
* *v_float64* will hold 4 64-bit floats (doubles)
|
||||
|
||||
v_uint8 a; // a is a register supporting uint8(char) data
|
||||
int n = a.nlanes; // n holds 32
|
||||
|
||||
Available data type and sizes:
|
||||
|Type|Size in bits|
|
||||
|-:|:-|
|
||||
|uint| 8, 16, 32, 64|
|
||||
|int | 8, 16, 32, 64|
|
||||
|float | 32, 64|
|
||||
|
||||
* **Constant sized registers**: These structures have a fixed bit size and hold a constant number of values. We need to know what SIMD instruction set is supported by the system and select compatible registers. Use these only if exact bit length is necessary.
|
||||
<br>
|
||||
|
||||
Each structure follows the convention:
|
||||
|
||||
v_[type of value][size of each value in bits]x[number of values]
|
||||
|
||||
Suppose we want to store
|
||||
* 32-bit(*size in bits*) signed integers in a **128 bit register**. Since the register size is already known, we can find out the *number of data points in register* (*128/32 = 4*):
|
||||
|
||||
v_int32x8 reg1 // holds 8 32-bit signed integers.
|
||||
|
||||
* 64-bit floats in 512 bit register:
|
||||
|
||||
v_float64x8 reg2 // reg2.nlanes = 8
|
||||
|
||||
### Load and Store operations
|
||||
|
||||
Now that we know how registers work, let us look at the functions used for filling these registers with values.
|
||||
|
||||
* **Load**: Load functions allow you to *load* values into a register.
|
||||
* *Constructors* - When declaring a register structure, we can either provide a memory address from where the register will pick up contiguous values, or provide the values explicitly as multiple arguments (Explicit multiple arguments is available only for Constant Sized Registers):
|
||||
|
||||
float ptr[32] = {1, 2, 3 ..., 32}; // ptr is a pointer to a contiguous memory block of 32 floats
|
||||
|
||||
// Variable Sized Registers //
|
||||
int x = v_float32().nlanes; // set x as the number of values the register can hold
|
||||
|
||||
v_float32 reg1(ptr); // reg1 stores first x values according to the maximum register size available.
|
||||
v_float32 reg2(ptr + x); // reg stores the next x values
|
||||
|
||||
// Constant Sized Registers //
|
||||
v_float32x4 reg1(ptr); // reg1 stores the first 4 floats (1, 2, 3, 4)
|
||||
v_float32x4 reg2(ptr + 4); // reg2 stores the next 4 floats (5, 6, 7, 8)
|
||||
|
||||
// Or we can explicitly write down the values.
|
||||
v_float32x4(1, 2, 3, 4);
|
||||
|
||||
<br>
|
||||
* *Load Function* - We can use the load method and provide the memory address of the data:
|
||||
|
||||
float ptr[32] = {1, 2, 3, ..., 32};
|
||||
v_float32 reg_var;
|
||||
reg_var = vx_load(ptr); // loads values from ptr[0] upto ptr[reg_var.nlanes - 1]
|
||||
|
||||
v_float32x4 reg_128;
|
||||
reg_128 = v_load(ptr); // loads values from ptr[0] upto ptr[3]
|
||||
|
||||
v_float32x8 reg_256;
|
||||
reg_256 = v256_load(ptr); // loads values from ptr[0] upto ptr[7]
|
||||
|
||||
v_float32x16 reg_512;
|
||||
reg_512 = v512_load(ptr); // loads values from ptr[0] upto ptr[15]
|
||||
|
||||
@note The load function assumes data is unaligned. If your data is aligned, you may use the `vx_load_aligned()` function.
|
||||
<br>
|
||||
|
||||
* **Store**: Store functions allow you to *store* the values from a register into a particular memory location.
|
||||
* To store values from a register into a memory location, you may use the *v_store()* function:
|
||||
|
||||
float ptr[4];
|
||||
v_store(ptr, reg); // store the first 128 bits(interpreted as 4x32-bit floats) of reg into ptr.
|
||||
<br>
|
||||
@note Ensure **ptr** has the same type as register. You can also cast the register into the proper type before carrying out operations. Simply typecasting the pointer to a particular type will lead wrong interpretation of data.
|
||||
|
||||
### Binary and Unary Operators
|
||||
|
||||
The universal intrinsics set provides element wise binary and unary operations.
|
||||
|
||||
* **Arithmetics**: We can add, subtract, multiply and divide two registers element-wise. The registers must be of the same width and hold the same type. To multiply two registers, for example:
|
||||
|
||||
v_float32 a, b; // {a1, ..., an}, {b1, ..., bn}
|
||||
v_float32 c;
|
||||
c = a + b // {a1 + b1, ..., an + bn}
|
||||
c = a * b; // {a1 * b1, ..., an * bn}
|
||||
|
||||
<br>
|
||||
|
||||
* **Bitwise Logic and Shifts**: We can left shift or right shift the bits of each element of the register. We can also apply bitwise &, |, ^ and ~ operators between two registers element-wise:
|
||||
|
||||
v_int32 as; // {a1, ..., an}
|
||||
v_int32 al = as << 2; // {a1 << 2, ..., an << 2}
|
||||
v_int32 bl = as >> 2; // {a1 >> 2, ..., an >> 2}
|
||||
|
||||
v_int32 a, b;
|
||||
v_int32 a_and_b = a & b; // {a1 & b1, ..., an & bn}
|
||||
|
||||
<br>
|
||||
|
||||
* **Comparison Operators**: We can compare values between two registers using the <, >, <= , >=, == and != operators. Since each register contains multiple values, we don't get a single bool for these operations. Instead, for true values, all bits are converted to one (0xff for 8 bits, 0xffff for 16 bits, etc), while false values return bits converted to zero.
|
||||
|
||||
// let us consider the following code is run in a 128-bit register
|
||||
v_uint8 a; // a = {0, 1, 2, ..., 15}
|
||||
v_uint8 b; // b = {15, 14, 13, ..., 0}
|
||||
|
||||
v_uint8 c = a < b;
|
||||
|
||||
/*
|
||||
let us look at the first 4 values in binary
|
||||
|
||||
a = |00000000|00000001|00000010|00000011|
|
||||
b = |00001111|00001110|00001101|00001100|
|
||||
c = |11111111|11111111|11111111|11111111|
|
||||
|
||||
If we store the values of c and print them as integers, we will get 255 for true values and 0 for false values.
|
||||
*/
|
||||
---
|
||||
// In a computer supporting 256-bit registers
|
||||
v_int32 a; // a = {1, 2, 3, 4, 5, 6, 7, 8}
|
||||
v_int32 b; // b = {8, 7, 6, 5, 4, 3, 2, 1}
|
||||
|
||||
v_int32 c = (a < b); // c = {-1, -1, -1, -1, 0, 0, 0, 0}
|
||||
|
||||
/*
|
||||
The true values are 0xffffffff, which in signed 32-bit integer representation is equal to -1.
|
||||
*/
|
||||
<br>
|
||||
|
||||
* **Min/Max operations**: We can use the *v_min()* and *v_max()* functions to return registers containing element-wise min, or max, of the two registers:
|
||||
|
||||
v_int32 a; // {a1, ..., an}
|
||||
v_int32 b; // {b1, ..., bn}
|
||||
|
||||
v_int32 mn = v_min(a, b); // {min(a1, b1), ..., min(an, bn)}
|
||||
v_int32 mx = v_max(a, b); // {max(a1, b1), ..., max(an, bn)}
|
||||
<br>
|
||||
|
||||
@note Comparison and Min/Max operators are not available for 64 bit integers. Bitwise shift and logic operators are available only for integer values. Bitwise shift is available only for 16, 32 and 64 bit registers.
|
||||
|
||||
### Reduce and Mask
|
||||
|
||||
* **Reduce Operations**: The *v_reduce_min()*, *v_reduce_max()* and *v_reduce_sum()* return a single value denoting the min, max or sum of the entire register:
|
||||
|
||||
v_int32 a; // a = {a1, ..., a4}
|
||||
int mn = v_reduce_min(a); // mn = min(a1, ..., an)
|
||||
int sum = v_reduce_sum(a); // sum = a1 + ... + an
|
||||
<br>
|
||||
|
||||
* **Mask Operations**: Mask operations allow us to replicate conditionals in wide registers. These include:
|
||||
* *v_check_all()* - Returns a bool, which is true if all the values in the register are less than zero.
|
||||
* *v_check_any()* - Returns a bool, which is true if any value in the register is less than zero.
|
||||
* *v_select()* - Returns a register, which blends two registers, based on a mask.
|
||||
|
||||
v_uint8 a; // {a1, .., an}
|
||||
v_uint8 b; // {b1, ..., bn}
|
||||
|
||||
v_int32x4 mask: // {0xff, 0, 0, 0xff, ..., 0xff, 0}
|
||||
|
||||
v_uint8 Res = v_select(mask, a, b) // {a1, b2, b3, a4, ..., an-1, bn}
|
||||
|
||||
/*
|
||||
"Res" will contain the value from "a" if mask is true (all bits set to 1),
|
||||
and value from "b" if mask is false (all bits set to 0)
|
||||
|
||||
We can use comparison operators to generate mask and v_select to obtain results based on conditionals.
|
||||
It is common to set all values of b to 0. Thus, v_select will give values of "a" or 0 based on the mask.
|
||||
*/
|
||||
|
||||
## Demonstration
|
||||
In the following section, we will vectorize a simple convolution function for single channel and compare the results to a scalar implementation.
|
||||
@note Not all algorithms are improved by manual vectorization. In fact, in certain cases, the compiler may *autovectorize* the code, thus producing faster results for scalar implementations.
|
||||
|
||||
You may learn more about convolution from the previous tutorial. We use the same naive implementation from the previous tutorial and compare it to the vectorized version.
|
||||
|
||||
The full tutorial code is [here](https://github.com/opencv/opencv/tree/master/samples/cpp/tutorial_code/univ_intrin/univ_intrin.cpp).
|
||||
|
||||
### Vectorizing Convolution
|
||||
|
||||
We will first implement a 1-D convolution and then vectorize it. The 2-D vectorized convolution will perform 1-D convolution across the rows to produce the correct results.
|
||||
|
||||
#### 1-D Convolution: Scalar
|
||||
@snippet univ_intrin.cpp convolution-1D-scalar
|
||||
|
||||
1. We first set up variables and make a border on both sides of the src matrix, to take care of edge cases.
|
||||
@snippet univ_intrin.cpp convolution-1D-border
|
||||
|
||||
2. For the main loop, we select an index *i* and offset it on both sides along with the kernel, using the k variable. We store the value in *value* and add it to the *dst* matrix.
|
||||
@snippet univ_intrin.cpp convolution-1D-scalar-main
|
||||
|
||||
#### 1-D Convolution: Vector
|
||||
|
||||
We will now look at the vectorized version of 1-D convolution.
|
||||
@snippet univ_intrin.cpp convolution-1D-vector
|
||||
|
||||
1. In our case, the kernel is a float. Since the kernel's datatype is the largest, we convert src to float32, forming *src_32*. We also make a border like we did for the naive case.
|
||||
@snippet univ_intrin.cpp convolution-1D-convert
|
||||
|
||||
2. Now, for each column in the *kernel*, we calculate the scalar product of the value with all *window* vectors of length `step`. We add these values to the already stored values in ans
|
||||
@snippet univ_intrin.cpp convolution-1D-main
|
||||
|
||||
* We declare a pointer to the src_32 and kernel and run a loop for each kernel element
|
||||
@snippet univ_intrin.cpp convolution-1D-main-h1
|
||||
|
||||
* We load a register with the current kernel element. A window is shifted from *0* to *len - step* and its product with the kernel_wide array is added to the values stored in *ans*. We store the values back into *ans*
|
||||
@snippet univ_intrin.cpp convolution-1D-main-h2
|
||||
|
||||
* Since the length might not be divisible by steps, we take care of the remaining values directly. The number of *tail* values will always be less than *step* and will not affect the performance significantly. We store all the values to *ans* which is a float pointer. We can also directly store them in a `Mat` object
|
||||
@snippet univ_intrin.cpp convolution-1D-main-h3
|
||||
|
||||
* Here is an iterative example:
|
||||
|
||||
For example:
|
||||
kernel: {k1, k2, k3}
|
||||
src: ...|a1|a2|a3|a4|...
|
||||
|
||||
|
||||
iter1:
|
||||
for each idx i in (0, len), 'step' idx at a time
|
||||
kernel_wide: |k1|k1|k1|k1|
|
||||
window: |a0|a1|a2|a3|
|
||||
ans: ...| 0| 0| 0| 0|...
|
||||
sum = ans + window * kernel_wide
|
||||
= |a0 * k1|a1 * k1|a2 * k1|a3 * k1|
|
||||
|
||||
iter2:
|
||||
kernel_wide: |k2|k2|k2|k2|
|
||||
window: |a1|a2|a3|a4|
|
||||
ans: ...|a0 * k1|a1 * k1|a2 * k1|a3 * k1|...
|
||||
sum = ans + window * kernel_wide
|
||||
= |a0 * k1 + a1 * k2|a1 * k1 + a2 * k2|a2 * k1 + a3 * k2|a3 * k1 + a4 * k2|
|
||||
|
||||
iter3:
|
||||
kernel_wide: |k3|k3|k3|k3|
|
||||
window: |a2|a3|a4|a5|
|
||||
ans: ...|a0 * k1 + a1 * k2|a1 * k1 + a2 * k2|a2 * k1 + a3 * k2|a3 * k1 + a4 * k2|...
|
||||
sum = sum + window * kernel_wide
|
||||
= |a0*k1 + a1*k2 + a2*k3|a1*k1 + a2*k2 + a3*k3|a2*k1 + a3*k2 + a4*k3|a3*k1 + a4*k2 + a5*k3|
|
||||
|
||||
|
||||
@note The function parameters also include *row*, *rowk* and *len*. These values are used when using the function as an intermediate step of 2-D convolution
|
||||
|
||||
#### 2-D Convolution
|
||||
|
||||
Suppose our kernel has *ksize* rows. To compute the values for a particular row, we compute the 1-D convolution of the previous *ksize/2* and the next *ksize/2* rows, with the corresponding kernel row. The final values is simply the sum of the individual 1-D convolutions
|
||||
@snippet univ_intrin.cpp convolution-2D
|
||||
|
||||
1. We first initialize variables and make a border above and below the *src* matrix. The left and right sides are handled by the 1-D convolution function.
|
||||
@snippet univ_intrin.cpp convolution-2D-init
|
||||
|
||||
2. For each row, we calculate the 1-D convolution of the rows above and below it. we then add the values to the *dst* matrix.
|
||||
@snippet univ_intrin.cpp convolution-2D-main
|
||||
|
||||
3. We finally convert the *dst* matrix to a *8-bit* `unsigned char` matrix
|
||||
@snippet univ_intrin.cpp convolution-2D-conv
|
||||
|
||||
Results
|
||||
-------
|
||||
|
||||
In the tutorial, we used a horizontal gradient kernel. We obtain the same output image for both methods.
|
||||
|
||||
Improvement in runtime varies and will depend on the SIMD capabilities available in your CPU.
|
||||
@@ -0,0 +1,95 @@
|
||||
# DNN-based Face Detection And Recognition {#tutorial_dnn_face}
|
||||
|
||||
@tableofcontents
|
||||
|
||||
@prev_tutorial{tutorial_dnn_text_spotting}
|
||||
@next_tutorial{pytorch_cls_tutorial_dnn_conversion}
|
||||
|
||||
| | |
|
||||
| -: | :- |
|
||||
| Original Author | Chengrui Wang, Yuantao Feng |
|
||||
| Compatibility | OpenCV >= 4.5.1 |
|
||||
|
||||
## Introduction
|
||||
|
||||
In this section, we introduce the DNN-based module for face detection and face recognition. Models can be obtained in [Models](#Models). The usage of `FaceDetectorYN` and `FaceRecognizer` are presented in [Usage](#Usage).
|
||||
|
||||
## Models
|
||||
|
||||
There are two models (ONNX format) pre-trained and required for this module:
|
||||
- [Face Detection](https://github.com/ShiqiYu/libfacedetection.train/tree/master/tasks/task1/onnx):
|
||||
- Size: 337KB
|
||||
- Results on WIDER Face Val set: 0.830(easy), 0.824(medium), 0.708(hard)
|
||||
- [Face Recognition](https://drive.google.com/file/d/1ClK9WiB492c5OZFKveF3XiHCejoOxINW/view?usp=sharing)
|
||||
- Size: 36.9MB
|
||||
- Results:
|
||||
|
||||
| Database | Accuracy | Threshold (normL2) | Threshold (cosine) |
|
||||
| -------- | -------- | ------------------ | ------------------ |
|
||||
| LFW | 99.60% | 1.128 | 0.363 |
|
||||
| CALFW | 93.95% | 1.149 | 0.340 |
|
||||
| CPLFW | 91.05% | 1.204 | 0.275 |
|
||||
| AgeDB-30 | 94.90% | 1.202 | 0.277 |
|
||||
| CFP-FP | 94.80% | 1.253 | 0.212 |
|
||||
|
||||
## Usage
|
||||
|
||||
### DNNFaceDetector
|
||||
|
||||
```cpp
|
||||
// Initialize FaceDetectorYN
|
||||
Ptr<FaceDetectorYN> faceDetector = FaceDetectorYN::create(onnx_path, "", image.size(), score_thresh, nms_thresh, top_k);
|
||||
|
||||
// Forward
|
||||
Mat faces;
|
||||
faceDetector->detect(image, faces);
|
||||
```
|
||||
|
||||
The detection output `faces` is a two-dimension array of type CV_32F, whose rows are the detected face instances, columns are the location of a face and 5 facial landmarks. The format of each row is as follows:
|
||||
|
||||
```
|
||||
x1, y1, w, h, x_re, y_re, x_le, y_le, x_nt, y_nt, x_rcm, y_rcm, x_lcm, y_lcm
|
||||
```
|
||||
, where `x1, y1, w, h` are the top-left coordinates, width and height of the face bounding box, `{x, y}_{re, le, nt, rcm, lcm}` stands for the coordinates of right eye, left eye, nose tip, the right corner and left corner of the mouth respectively.
|
||||
|
||||
|
||||
### Face Recognition
|
||||
|
||||
Following Face Detection, run codes below to extract face feature from facial image.
|
||||
|
||||
```cpp
|
||||
// Initialize FaceRecognizer with model path (cv::String)
|
||||
Ptr<FaceRecognizer> faceRecognizer = FaceRecognizer::create(model_path, "");
|
||||
|
||||
// Aligning and cropping facial image through the first face of faces detected by dnn_face::DNNFaceDetector
|
||||
Mat aligned_face;
|
||||
faceRecognizer->alignCrop(image, faces.row(0), aligned_face);
|
||||
|
||||
// Run feature extraction with given aligned_face (cv::Mat)
|
||||
Mat feature;
|
||||
faceRecognizer->feature(aligned_face, feature);
|
||||
feature = feature.clone();
|
||||
```
|
||||
|
||||
After obtaining face features *feature1* and *feature2* of two facial images, run codes below to calculate the identity discrepancy between the two faces.
|
||||
|
||||
```cpp
|
||||
// Calculating the discrepancy between two face features by using cosine distance.
|
||||
double cos_score = faceRecognizer->match(feature1, feature2, FaceRecognizer::DisType::COSINE);
|
||||
// Calculating the discrepancy between two face features by using normL2 distance.
|
||||
double L2_score = faceRecognizer->match(feature1, feature2, FaceRecognizer::DisType::NORM_L2);
|
||||
```
|
||||
|
||||
For example, two faces have same identity if the cosine distance is greater than or equal to 0.363, or the normL2 distance is less than or equal to 1.128.
|
||||
|
||||
## Reference:
|
||||
|
||||
- https://github.com/ShiqiYu/libfacedetection
|
||||
- https://github.com/ShiqiYu/libfacedetection.train
|
||||
- https://github.com/zhongyy/SFace
|
||||
|
||||
## Acknowledgement
|
||||
|
||||
Thanks [Professor Shiqi Yu](https://github.com/ShiqiYu/) and [Yuantao Feng](https://github.com/fengyuentau) for training and providing the face detection model.
|
||||
|
||||
Thanks [Professor Deng](http://www.whdeng.cn/), [PhD Candidate Zhong](https://github.com/zhongyy/) and [Master Candidate Wang](https://github.com/crywang/) for training and providing the face recognition model.
|
||||
@@ -3,7 +3,7 @@
|
||||
@tableofcontents
|
||||
|
||||
@prev_tutorial{tutorial_dnn_OCR}
|
||||
@next_tutorial{pytorch_cls_tutorial_dnn_conversion}
|
||||
@next_tutorial{tutorial_dnn_face}
|
||||
|
||||
| | |
|
||||
| -: | :- |
|
||||
@@ -26,6 +26,11 @@ Before recognition, you should `setVocabulary` and `setDecodeType`.
|
||||
- `T` is the sequence length
|
||||
- `B` is the batch size (only support `B=1` in inference)
|
||||
- and `Dim` is the length of vocabulary +1('Blank' of CTC is at the index=0 of Dim).
|
||||
- "CTC-prefix-beam-search", the output of the text recognition model should be a probability matrix same with "CTC-greedy".
|
||||
- The algorithm is proposed at Hannun's [paper](https://arxiv.org/abs/1408.2873).
|
||||
- `setDecodeOptsCTCPrefixBeamSearch` could be used to control the beam size in search step.
|
||||
- To futher optimize for big vocabulary, a new option `vocPruneSize` is introduced to avoid iterate the whole vocbulary
|
||||
but only the number of `vocPruneSize` tokens with top probabilty.
|
||||
|
||||
@ref cv::dnn::TextRecognitionModel::recognize() is the main function for text recognition.
|
||||
- The input image should be a cropped text image or an image with `roiRects`
|
||||
|
||||
@@ -10,6 +10,7 @@ Deep Neural Networks (dnn module) {#tutorial_table_of_content_dnn}
|
||||
- @subpage tutorial_dnn_custom_layers
|
||||
- @subpage tutorial_dnn_OCR
|
||||
- @subpage tutorial_dnn_text_spotting
|
||||
- @subpage tutorial_dnn_face
|
||||
|
||||
#### PyTorch models with OpenCV
|
||||
In this section you will find the guides, which describe how to run classification, segmentation and detection PyTorch DNN models with OpenCV.
|
||||
|
||||
@@ -589,6 +589,14 @@ Some features have been added specifically for automated build environments, lik
|
||||
| `OPENCV_CMAKE_HOOKS_DIR` | _empty_ | OpenCV allows to customize configuration process by adding custom hook scripts at each stage and substage. cmake scripts with predefined names located in the directory set by this variable will be included before and after various configuration stages. Examples of file names: _CMAKE_INIT.cmake_, _PRE_CMAKE_BOOTSTRAP.cmake_, _POST_CMAKE_BOOTSTRAP.cmake_, etc.. Other names are not documented and can be found in the project cmake files by searching for the _ocv_cmake_hook_ macro calls. |
|
||||
| `OPENCV_DUMP_HOOKS_FLOW` | _OFF_ | Enables a debug message print on each cmake hook script call. |
|
||||
|
||||
## Contrib Modules
|
||||
|
||||
Following build options are utilized in `opencv_contrib` modules, as stated [previously](#tutorial_config_reference_general_contrib), these extra modules can be added to your final build by setting `DOPENCV_EXTRA_MODULES_PATH` option.
|
||||
|
||||
| Option | Default | Description |
|
||||
| ------ | ------- | ----------- |
|
||||
| `WITH_CLP` | _OFF_ | Will add [coinor](https://projects.coin-or.org/Clp) linear programming library build support which is required in `videostab` module. Make sure to install the development libraries of coinor-clp. |
|
||||
|
||||
|
||||
# Other non-documented options
|
||||
|
||||
@@ -605,7 +613,6 @@ Some features have been added specifically for automated build environments, lik
|
||||
`WITH_CPUFEATURES`
|
||||
`WITH_EIGEN`
|
||||
`WITH_OPENVX`
|
||||
`WITH_CLP`
|
||||
`WITH_DIRECTX`
|
||||
`WITH_VA`
|
||||
`WITH_LAPACK`
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
Using OpenCV with gcc and CMake {#tutorial_linux_gcc_cmake}
|
||||
===============================
|
||||
|
||||
@prev_tutorial{tutorial_linux_install}
|
||||
@prev_tutorial{tutorial_linux_gdb_pretty_printer}
|
||||
@next_tutorial{tutorial_linux_eclipse}
|
||||
|
||||
| | |
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 35 KiB |
@@ -0,0 +1,38 @@
|
||||
Using OpenCV with gdb-powered IDEs {#tutorial_linux_gdb_pretty_printer}
|
||||
=====================
|
||||
|
||||
@prev_tutorial{tutorial_linux_install}
|
||||
@next_tutorial{tutorial_linux_gcc_cmake}
|
||||
|
||||
| | |
|
||||
| -: | :- |
|
||||
| Original author | Egor Smirnov |
|
||||
| Compatibility | OpenCV >= 4.0 |
|
||||
|
||||
@tableofcontents
|
||||
|
||||
# Capabilities {#tutorial_linux_gdb_pretty_printer_capabilities}
|
||||
|
||||
This pretty-printer can show element type, `is_continuous`, `is_submatrix` flags and (possibly truncated) matrix. It is known to work in Clion, VS Code and gdb.
|
||||
|
||||
|
||||
|
||||
|
||||
# Installation {#tutorial_linux_gdb_pretty_printer_installation}
|
||||
|
||||
Move into `opencv/samples/gdb/`. Place `mat_pretty_printer.py` in a convinient place, rename `gdbinit` to `.gdbinit` and move it into your home folder. Change 'source' line of `.gdbinit` to point to your `mat_pretty_printer.py` path.
|
||||
|
||||
In order to check version of python bundled with your gdb, use the following commands from the gdb shell:
|
||||
|
||||
python
|
||||
import sys
|
||||
print(sys.version_info)
|
||||
end
|
||||
|
||||
If the version of python 3 installed in your system doesn't match the version in gdb, create a new virtual environment with the exact same version, install `numpy` and change the path to python3 in `.gdbinit` accordingly.
|
||||
|
||||
|
||||
# Usage {#tutorial_linux_gdb_pretty_printer_usage}
|
||||
|
||||
The fields in a debugger prefixed with `view_` are pseudo-fields added for convinience, the rest are left as is.
|
||||
If you feel that the number of elements in truncated view is too low, you can edit `mat_pretty_printer.py` - `np.set_printoptions` controlls everything matrix display-related.
|
||||
@@ -1,7 +1,7 @@
|
||||
Installation in Linux {#tutorial_linux_install}
|
||||
=====================
|
||||
|
||||
@next_tutorial{tutorial_linux_gcc_cmake}
|
||||
@next_tutorial{tutorial_linux_gdb_pretty_printer}
|
||||
|
||||
| | |
|
||||
| -: | :- |
|
||||
|
||||
@@ -6,6 +6,7 @@ Introduction to OpenCV {#tutorial_table_of_content_introduction}
|
||||
|
||||
##### Linux
|
||||
- @subpage tutorial_linux_install
|
||||
- @subpage tutorial_linux_gdb_pretty_printer
|
||||
- @subpage tutorial_linux_gcc_cmake
|
||||
- @subpage tutorial_linux_eclipse
|
||||
|
||||
|
||||
@@ -13,6 +13,8 @@ Working with a boosted cascade of weak classifiers includes two major stages: th
|
||||
|
||||
To support this tutorial, several official OpenCV applications will be used: [opencv_createsamples](https://github.com/opencv/opencv/tree/master/apps/createsamples), [opencv_annotation](https://github.com/opencv/opencv/tree/master/apps/annotation), [opencv_traincascade](https://github.com/opencv/opencv/tree/master/apps/traincascade) and [opencv_visualisation](https://github.com/opencv/opencv/tree/master/apps/visualisation).
|
||||
|
||||
@note Createsamples and traincascade are disabled since OpenCV 4.0. Consider using these apps for training from 3.4 branch for Cascade Classifier. Model format is the same between 3.4 and 4.x.
|
||||
|
||||
### Important notes
|
||||
|
||||
- If you come across any tutorial mentioning the old opencv_haartraining tool <i>(which is deprecated and still using the OpenCV1.x interface)</i>, then please ignore that tutorial and stick to the opencv_traincascade tool. This tool is a newer version, written in C++ in accordance to the OpenCV 2.x and OpenCV 3.x API. The opencv_traincascade supports both HAAR like wavelet features @cite Viola01 and LBP (Local Binary Patterns) @cite Liao2007 features. LBP features yield integer precision in contrast to HAAR features, yielding floating point precision, so both training and detection with LBP are several times faster then with HAAR features. Regarding the LBP and HAAR detection quality, it mainly depends on the training data used and the training parameters selected. It's possible to train a LBP-based classifier that will provide almost the same quality as HAAR-based one, within a percentage of the training time.
|
||||
|
||||
Reference in New Issue
Block a user