Merge branch 4.x

2026-07-31 08:13:04 +04:00 · 2021-04-09 10:30:38 +00:00
parent 263de1f86e bfb10d74eb
commit fc628014bb
1114 changed files with 64039 additions and 14611 deletions
@@ -1,6 +1,14 @@
 # How to run custom OCR model {#tutorial_dnn_OCR}

+@tableofcontents
+
@prev_tutorial{tutorial_dnn_custom_layers}
+@next_tutorial{tutorial_dnn_text_spotting}
+
+|    |    |
+| -: | :- |
+| Original author | Zihao Mu |
+| Compatibility | OpenCV >= 4.3 |

 ## Introduction

@@ -43,4 +51,4 @@ The input of text recognition model is the output of the text detection model, w

 DenseNet_CTC has the smallest parameters and best FPS, and it is suitable for edge devices, which are very sensitive to the cost of calculation. If you have limited computing resources and want to achieve better accuracy, VGG_CTC is a good choice.

-CRNN_VGG_BiLSTM_CTC is suitable for scenarios that require high recognition accuracy.
+CRNN_VGG_BiLSTM_CTC is suitable for scenarios that require high recognition accuracy.
@@ -1,8 +1,15 @@
 # How to run deep networks on Android device {#tutorial_dnn_android}

+@tableofcontents
+
@prev_tutorial{tutorial_dnn_halide_scheduling}
@next_tutorial{tutorial_dnn_yolo}

+|    |    |
+| -: | :- |
+| Original author | Dmitry Kurtaev |
+| Compatibility | OpenCV >= 3.3 |
+
 ## Introduction
 In this tutorial you'll know how to run deep learning networks on Android device
 using OpenCV deep learning module.
@@ -1,8 +1,15 @@
 # Custom deep learning layers support {#tutorial_dnn_custom_layers}

+@tableofcontents
+
@prev_tutorial{tutorial_dnn_javascript}
@next_tutorial{tutorial_dnn_OCR}

+|    |    |
+| -: | :- |
+| Original author | Dmitry Kurtaev |
+| Compatibility | OpenCV >= 3.4.1 |
+
 ## Introduction
 Deep learning is a fast growing area. The new approaches to build neural networks
 usually introduce new types of layers. They could be modifications of existing
@@ -1,8 +1,15 @@
 Load Caffe framework models  {#tutorial_dnn_googlenet}
 ===========================

+@tableofcontents
+
@next_tutorial{tutorial_dnn_halide}

+|    |    |
+| -: | :- |
+| Original author | Vitaliy Lyudvichenko |
+| Compatibility | OpenCV >= 3.3 |
+
 Introduction
 ------------

@@ -1,8 +1,15 @@
 # How to enable Halide backend for improve efficiency  {#tutorial_dnn_halide}

+@tableofcontents
+
@prev_tutorial{tutorial_dnn_googlenet}
@next_tutorial{tutorial_dnn_halide_scheduling}

+|    |    |
+| -: | :- |
+| Original author | Dmitry Kurtaev |
+| Compatibility | OpenCV >= 3.3 |
+
 ## Introduction
 This tutorial guidelines how to run your models in OpenCV deep learning module
 using Halide language backend. Halide is an open-source project that let us
@@ -1,8 +1,15 @@
 # How to schedule your network for Halide backend {#tutorial_dnn_halide_scheduling}

+@tableofcontents
+
@prev_tutorial{tutorial_dnn_halide}
@next_tutorial{tutorial_dnn_android}

+|    |    |
+| -: | :- |
+| Original author | Dmitry Kurtaev |
+| Compatibility | OpenCV >= 3.3 |
+
 ## Introduction
 Halide code is the same for every device we use. But for achieving the satisfied
 efficiency we should schedule computations properly. In this tutorial we describe
@@ -1,8 +1,15 @@
 # How to run deep networks in browser {#tutorial_dnn_javascript}

+@tableofcontents
+
@prev_tutorial{tutorial_dnn_yolo}
@next_tutorial{tutorial_dnn_custom_layers}

+|    |    |
+| -: | :- |
+| Original author | Dmitry Kurtaev |
+| Compatibility | OpenCV >= 3.3.1 |
+
 ## Introduction
 This tutorial will show us how to run deep learning models using OpenCV.js right
 in a browser. Tutorial refers a sample of face detection and face recognition
@@ -0,0 +1,220 @@
+# Conversion of PyTorch Classification Models and Launch with OpenCV C++ {#pytorch_cls_c_tutorial_dnn_conversion}
+
+@prev_tutorial{pytorch_cls_tutorial_dnn_conversion}
+
+|    |    |
+| -: | :- |
+| Original author | Anastasia Murzova |
+| Compatibility | OpenCV >= 4.5 |
+
+## Goals
+In this tutorial you will learn how to:
+* convert PyTorch classification models into ONNX format
+* run converted PyTorch model with OpenCV C/C++ API
+* provide model inference
+
+We will explore the above-listed points by the example of ResNet-50 architecture.
+
+## Introduction
+Let's briefly view the key concepts involved in the pipeline of PyTorch models transition with OpenCV API. The initial step in conversion of PyTorch models into cv::dnn::Net
+is model transferring into [ONNX](https://onnx.ai/about.html) format. ONNX aims at the interchangeability of the neural networks between various frameworks. There is a built-in function in PyTorch for ONNX conversion: [``torch.onnx.export``](https://pytorch.org/docs/stable/onnx.html#torch.onnx.export).
+Further the obtained ``.onnx`` model is passed into cv::dnn::readNetFromONNX or cv::dnn::readNet.
+
+## Requirements
+To be able to experiment with the below code you will need to install a set of libraries. We will use a virtual environment with python3.7+ for this:
+
+```console
+virtualenv -p /usr/bin/python3.7 <env_dir_path>
+source <env_dir_path>/bin/activate
+```
+
+For OpenCV-Python building from source, follow the corresponding instructions from the @ref tutorial_py_table_of_contents_setup.
+
+Before you start the installation of the libraries, you can customize the [requirements.txt](https://github.com/opencv/opencv/tree/master/samples/dnn/dnn_model_runner/dnn_conversion/requirements.txt), excluding or including (for example, ``opencv-python``) some dependencies.
+The below line initiates requirements installation into the previously activated virtual environment:
+
+```console
+pip install -r requirements.txt
+```
+
+## Practice
+In this part we are going to cover the following points:
+1. create a classification model conversion pipeline
+2. provide the inference, process prediction results
+
+### Model Conversion Pipeline
+The code in this subchapter is located in the ``samples/dnn/dnn_model_runner`` module and can be executed with the line:
+
+```console
+python -m dnn_model_runner.dnn_conversion.pytorch.classification.py_to_py_resnet50_onnx
+```
+
+The following code contains the description of the below-listed steps:
+1. instantiate PyTorch model
+2. convert PyTorch model into ``.onnx``
+
+```python
+# initialize PyTorch ResNet-50 model
+original_model = models.resnet50(pretrained=True)
+
+# get the path to the converted into ONNX PyTorch model
+full_model_path = get_pytorch_onnx_model(original_model)
+print("PyTorch ResNet-50 model was successfully converted: ", full_model_path)
+```
+
+``get_pytorch_onnx_model(original_model)`` function is based on ``torch.onnx.export(...)`` call:
+
+```python
+# define the directory for further converted model save
+onnx_model_path = "models"
+# define the name of further converted model
+onnx_model_name = "resnet50.onnx"
+
+# create directory for further converted model
+os.makedirs(onnx_model_path, exist_ok=True)
+
+# get full path to the converted model
+full_model_path = os.path.join(onnx_model_path, onnx_model_name)
+
+# generate model input
+generated_input = Variable(
+    torch.randn(1, 3, 224, 224)
+)
+
+# model export into ONNX format
+torch.onnx.export(
+    original_model,
+    generated_input,
+    full_model_path,
+    verbose=True,
+    input_names=["input"],
+    output_names=["output"],
+    opset_version=11
+)
+```
+
+After the successful execution of the above code we will get the following output:
+
+```console
+PyTorch ResNet-50 model was successfully converted: models/resnet50.onnx
+```
+
+The proposed in ``dnn/samples`` module ``dnn_model_runner`` allows us to reproduce the above conversion steps for the following PyTorch classification models:
+* alexnet
+* vgg11
+* vgg13
+* vgg16
+* vgg19
+* resnet18
+* resnet34
+* resnet50
+* resnet101
+* resnet152
+* squeezenet1_0
+* squeezenet1_1
+* resnext50_32x4d
+* resnext101_32x8d
+* wide_resnet50_2
+* wide_resnet101_2
+
+To obtain the converted model, the following line should be executed:
+
+```
+python -m dnn_model_runner.dnn_conversion.pytorch.classification.py_to_py_cls --model_name <pytorch_cls_model_name> --evaluate False
+```
+
+For the ResNet-50 case the below line should be run:
+
+```
+python -m dnn_model_runner.dnn_conversion.pytorch.classification.py_to_py_cls --model_name resnet50 --evaluate False
+```
+
+The default root directory for the converted model storage is defined in module ``CommonConfig``:
+
+```python
+@dataclass
+class CommonConfig:
+    output_data_root_dir: str = "dnn_model_runner/dnn_conversion"
+```
+
+Thus, the converted ResNet-50 will be saved in ``dnn_model_runner/dnn_conversion/models``.
+
+### Inference Pipeline
+Now we can use ```models/resnet50.onnx``` for the inference pipeline using OpenCV C/C++ API. The implemented pipeline can be found in [samples/dnn/classification.cpp](https://github.com/opencv/opencv/blob/master/samples/dnn/classification.cpp).
+After the build of samples (``BUILD_EXAMPLES`` flag value should be ``ON``), the appropriate ``example_dnn_classification`` executable file will be provided.
+
+To provide model inference we will use the below [squirrel photo](https://www.pexels.com/photo/brown-squirrel-eating-1564292) (under [CC0](https://www.pexels.com/terms-of-service/) license) corresponding to ImageNet class ID 335:
+```console
+fox squirrel, eastern fox squirrel, Sciurus niger
+```
+
+![Classification model input image](images/squirrel_cls.jpg)
+
+For the label decoding of the obtained prediction, we also need ``imagenet_classes.txt`` file, which contains the full list of the ImageNet classes.
+
+In this tutorial we will run the inference process for the converted PyTorch ResNet-50 model from the build (``samples/build``) directory:
+
+```
+./dnn/example_dnn_classification --model=../dnn/models/resnet50.onnx --input=../data/squirrel_cls.jpg --width=224 --height=224 --rgb=true --scale="0.003921569" --mean="123.675 116.28 103.53" --std="0.229 0.224 0.225" --crop=true --initial_width=256 --initial_height=256 --classes=../data/dnn/classification_classes_ILSVRC2012.txt
+```
+
+Let's explore ``classification.cpp`` key points step by step:
+
+1. read the model with cv::dnn::readNet, initialize the network:
+
+```cpp
+Net net = readNet(model, config, framework);
+```
+
+The ``model`` parameter value is taken from ``--model`` key. In our case, it is ``resnet50.onnx``.
+
+* preprocess input image:
+
+```cpp
+if (rszWidth != 0 && rszHeight != 0)
+{
+    resize(frame, frame, Size(rszWidth, rszHeight));
+}
+
+// Create a 4D blob from a frame
+blobFromImage(frame, blob, scale, Size(inpWidth, inpHeight), mean, swapRB, crop);
+
+// Check std values.
+if (std.val[0] != 0.0 && std.val[1] != 0.0 && std.val[2] != 0.0)
+{
+    // Divide blob by std.
+    divide(blob, std, blob);
+}
+```
+
+In this step we use cv::dnn::blobFromImage function to prepare model input.
+We set ``Size(rszWidth, rszHeight)`` with  ``--initial_width=256 --initial_height=256`` for the initial image resize as it's described in [PyTorch ResNet inference pipeline](https://pytorch.org/hub/pytorch_vision_resnet/).
+
+It should be noted that firstly in cv::dnn::blobFromImage mean value is subtracted and only then pixel values are multiplied by scale.
+Thus, we use ``--mean="123.675 116.28 103.53"``, which is equivalent to ``[0.485, 0.456, 0.406]`` multiplied by ``255.0`` to reproduce the original image preprocessing order for PyTorch classification models:
+
+```python
+img /= 255.0
+img -= [0.485, 0.456, 0.406]
+img /= [0.229, 0.224, 0.225]
+```
+
+* make forward pass:
+
+```cpp
+net.setInput(blob);
+Mat prob = net.forward();
+```
+
+* process the prediction:
+
+```cpp
+Point classIdPoint;
+double confidence;
+minMaxLoc(prob.reshape(1, 1), 0, &confidence, 0, &classIdPoint);
+int classId = classIdPoint.x;
+```
+
+Here we choose the most likely object class. The ``classId`` result for our case is 335 - fox squirrel, eastern fox squirrel, Sciurus niger:
+
+![ResNet50 OpenCV C++ inference output](images/opencv_resnet50_test_res_c.jpg)
@@ -0,0 +1,362 @@
+# Conversion of PyTorch Classification Models and Launch with OpenCV Python {#pytorch_cls_tutorial_dnn_conversion}
+
+@prev_tutorial{tutorial_dnn_OCR}
+@next_tutorial{pytorch_cls_c_tutorial_dnn_conversion}
+
+|    |    |
+| -: | :- |
+| Original author | Anastasia Murzova |
+| Compatibility | OpenCV >= 4.5 |
+
+## Goals
+In this tutorial you will learn how to:
+* convert PyTorch classification models into ONNX format
+* run converted PyTorch model with OpenCV Python API
+* obtain an evaluation of the PyTorch and OpenCV DNN models.
+
+We will explore the above-listed points by the example of the ResNet-50 architecture.
+
+## Introduction
+Let's briefly view the key concepts involved in the pipeline of PyTorch models transition with OpenCV API. The initial step in conversion of PyTorch models into cv.dnn.Net
+is model transferring into [ONNX](https://onnx.ai/about.html) format. ONNX aims at the interchangeability of the neural networks between various frameworks. There is a built-in function in PyTorch for ONNX conversion: [``torch.onnx.export``](https://pytorch.org/docs/stable/onnx.html#torch.onnx.export).
+Further the obtained ``.onnx`` model is passed into cv.dnn.readNetFromONNX.
+
+## Requirements
+To be able to experiment with the below code you will need to install a set of libraries. We will use a virtual environment with python3.7+ for this:
+
+```console
+virtualenv -p /usr/bin/python3.7 <env_dir_path>
+source <env_dir_path>/bin/activate
+```
+
+For OpenCV-Python building from source, follow the corresponding instructions from the @ref tutorial_py_table_of_contents_setup.
+
+Before you start the installation of the libraries, you can customize the [requirements.txt](https://github.com/opencv/opencv/tree/master/samples/dnn/dnn_model_runner/dnn_conversion/requirements.txt), excluding or including (for example, ``opencv-python``) some dependencies.
+The below line initiates requirements installation into the previously activated virtual environment:
+
+```console
+pip install -r requirements.txt
+```
+
+## Practice
+In this part we are going to cover the following points:
+1. create a classification model conversion pipeline and provide the inference
+2. evaluate and test classification models
+
+If you'd like merely to run evaluation or test model pipelines, the "Model Conversion Pipeline" part can be skipped.
+
+### Model Conversion Pipeline
+The code in this subchapter is located in the ``dnn_model_runner`` module and can be executed with the line:
+
+```console
+python -m dnn_model_runner.dnn_conversion.pytorch.classification.py_to_py_resnet50
+```
+
+The following code contains the description of the below-listed steps:
+1. instantiate PyTorch model
+2. convert PyTorch model into ``.onnx``
+3. read the transferred network with OpenCV API
+4. prepare input data
+5. provide inference
+
+```python
+# initialize PyTorch ResNet-50 model
+original_model = models.resnet50(pretrained=True)
+
+# get the path to the converted into ONNX PyTorch model
+full_model_path = get_pytorch_onnx_model(original_model)
+
+# read converted .onnx model with OpenCV API
+opencv_net = cv2.dnn.readNetFromONNX(full_model_path)
+print("OpenCV model was successfully read. Layer IDs: \n", opencv_net.getLayerNames())
+
+# get preprocessed image
+input_img = get_preprocessed_img("../data/squirrel_cls.jpg")
+
+# get ImageNet labels
+imagenet_labels = get_imagenet_labels("../data/dnn/classification_classes_ILSVRC2012.txt")
+
+# obtain OpenCV DNN predictions
+get_opencv_dnn_prediction(opencv_net, input_img, imagenet_labels)
+
+# obtain original PyTorch ResNet50 predictions
+get_pytorch_dnn_prediction(original_model, input_img, imagenet_labels)
+```
+
+To provide model inference we will use the below [squirrel photo](https://www.pexels.com/photo/brown-squirrel-eating-1564292) (under [CC0](https://www.pexels.com/terms-of-service/) license) corresponding to ImageNet class ID 335:
+```console
+fox squirrel, eastern fox squirrel, Sciurus niger
+```
+
+![Classification model input image](images/squirrel_cls.jpg)
+
+For the label decoding of the obtained prediction, we also need ``imagenet_classes.txt`` file, which contains the full list of the ImageNet classes.
+
+Let's go deeper into each step by the example of pretrained PyTorch ResNet-50:
+*  instantiate PyTorch ResNet-50 model:
+
+```python
+# initialize PyTorch ResNet-50 model
+original_model = models.resnet50(pretrained=True)
+```
+
+*  convert PyTorch model into ONNX:
+
+```python
+# define the directory for further converted model save
+onnx_model_path = "models"
+# define the name of further converted model
+onnx_model_name = "resnet50.onnx"
+
+# create directory for further converted model
+os.makedirs(onnx_model_path, exist_ok=True)
+
+# get full path to the converted model
+full_model_path = os.path.join(onnx_model_path, onnx_model_name)
+
+# generate model input
+generated_input = Variable(
+    torch.randn(1, 3, 224, 224)
+)
+
+# model export into ONNX format
+torch.onnx.export(
+    original_model,
+    generated_input,
+    full_model_path,
+    verbose=True,
+    input_names=["input"],
+    output_names=["output"],
+    opset_version=11
+)
+```
+
+After the successful execution of the above code, we will get ``models/resnet50.onnx``.
+
+* read the transferred network with cv.dnn.readNetFromONNX passing the obtained in the previous step ONNX model into it:
+
+```python
+# read converted .onnx model with OpenCV API
+opencv_net = cv2.dnn.readNetFromONNX(full_model_path)
+```
+
+* prepare input data:
+
+```python
+# read the image
+input_img = cv2.imread(img_path, cv2.IMREAD_COLOR)
+input_img = input_img.astype(np.float32)
+
+input_img = cv2.resize(input_img, (256, 256))
+
+# define preprocess parameters
+mean = np.array([0.485, 0.456, 0.406]) * 255.0
+scale = 1 / 255.0
+std = [0.229, 0.224, 0.225]
+
+# prepare input blob to fit the model input:
+# 1. subtract mean
+# 2. scale to set pixel values from 0 to 1
+input_blob = cv2.dnn.blobFromImage(
+    image=input_img,
+    scalefactor=scale,
+    size=(224, 224),  # img target size
+    mean=mean,
+    swapRB=True,  # BGR -> RGB
+    crop=True  # center crop
+)
+# 3. divide by std
+input_blob[0] /= np.asarray(std, dtype=np.float32).reshape(3, 1, 1)
+```
+
+In this step we read the image and prepare model input with cv.dnn.blobFromImage function, which returns 4-dimensional blob.
+It should be noted that firstly in cv.dnn.blobFromImage mean value is subtracted and only then pixel values are multiplied by scale. Thus, ``mean`` is multiplied by ``255.0`` to reproduce the original image preprocessing order:
+
+```python
+img /= 255.0
+img -= [0.485, 0.456, 0.406]
+img /= [0.229, 0.224, 0.225]
+```
+
+* OpenCV cv.dnn.Net inference:
+
+```python
+# set OpenCV DNN input
+opencv_net.setInput(preproc_img)
+
+# OpenCV DNN inference
+out = opencv_net.forward()
+print("OpenCV DNN prediction: \n")
+print("* shape: ", out.shape)
+
+# get the predicted class ID
+imagenet_class_id = np.argmax(out)
+
+# get confidence
+confidence = out[0][imagenet_class_id]
+print("* class ID: {}, label: {}".format(imagenet_class_id, imagenet_labels[imagenet_class_id]))
+print("* confidence: {:.4f}".format(confidence))
+```
+
+After the above code execution we will get the following output:
+
+```console
+OpenCV DNN prediction:
+* shape:  (1, 1000)
+* class ID: 335, label: fox squirrel, eastern fox squirrel, Sciurus niger
+* confidence: 14.8308
+```
+
+* PyTorch ResNet-50 model inference:
+
+```python
+original_net.eval()
+preproc_img = torch.FloatTensor(preproc_img)
+
+# inference
+out = original_net(preproc_img)
+print("\nPyTorch model prediction: \n")
+print("* shape: ", out.shape)
+
+# get the predicted class ID
+imagenet_class_id = torch.argmax(out, axis=1).item()
+print("* class ID: {}, label: {}".format(imagenet_class_id, imagenet_labels[imagenet_class_id]))
+
+# get confidence
+confidence = out[0][imagenet_class_id]
+print("* confidence: {:.4f}".format(confidence.item()))
+```
+
+After the above code launching we will get the following output:
+
+```console
+PyTorch model prediction:
+* shape:  torch.Size([1, 1000])
+* class ID: 335, label: fox squirrel, eastern fox squirrel, Sciurus niger
+* confidence: 14.8308
+```
+
+The inference results of the original ResNet-50 model and cv.dnn.Net are equal. For the extended evaluation of the models we can use ``py_to_py_cls`` of the ``dnn_model_runner`` module. This module part will be described in the next subchapter.
+
+### Evaluation of the Models
+
+The proposed in ``samples/dnn`` ``dnn_model_runner`` module allows to run the full evaluation pipeline on the ImageNet dataset and test execution for the following PyTorch classification models:
+* alexnet
+* vgg11
+* vgg13
+* vgg16
+* vgg19
+* resnet18
+* resnet34
+* resnet50
+* resnet101
+* resnet152
+* squeezenet1_0
+* squeezenet1_1
+* resnext50_32x4d
+* resnext101_32x8d
+* wide_resnet50_2
+* wide_resnet101_2
+
+This list can be also extended with further appropriate evaluation pipeline configuration.
+
+#### Evaluation Mode
+
+The below line represents running of the module in the evaluation mode:
+
+```console
+python -m dnn_model_runner.dnn_conversion.pytorch.classification.py_to_py_cls --model_name <pytorch_cls_model_name>
+```
+
+Chosen from the list classification model will be read into OpenCV cv.dnn.Net object. Evaluation results of PyTorch and OpenCV models (accuracy, inference time, L1) will be written into the log file. Inference time values will be also depicted in a chart to generalize the obtained model information.
+
+Necessary evaluation configurations are defined in the [test_config.py](https://github.com/opencv/opencv/tree/master/samples/dnn/dnn_model_runner/dnn_conversion/common/test/configs/test_config.py) and can be modified in accordance with actual paths of data location:
+
+```python
+@dataclass
+class TestClsConfig:
+    batch_size: int = 50
+    frame_size: int = 224
+    img_root_dir: str = "./ILSVRC2012_img_val"
+    # location of image-class matching
+    img_cls_file: str = "./val.txt"
+    bgr_to_rgb: bool = True
+```
+
+To initiate the evaluation of the PyTorch ResNet-50, run the following line:
+
+```console
+python -m dnn_model_runner.dnn_conversion.pytorch.classification.py_to_py_cls --model_name resnet50
+```
+
+After script launch, the log file with evaluation data will be generated in ``dnn_model_runner/dnn_conversion/logs``:
+
+```console
+The model PyTorch resnet50 was successfully obtained and converted to OpenCV DNN resnet50
+===== Running evaluation of the model with the following params:
+    * val data location: ./ILSVRC2012_img_val
+    * log file location: dnn_model_runner/dnn_conversion/logs/PyTorch_resnet50_log.txt
+```
+
+#### Test Mode
+
+The below line represents running of the module in the test mode, namely it provides the steps for the model inference:
+
+```console
+python -m dnn_model_runner.dnn_conversion.pytorch.classification.py_to_py_cls --model_name <pytorch_cls_model_name> --test True --default_img_preprocess <True/False> --evaluate False
+```
+
+Here ``default_img_preprocess`` key defines whether you'd like to parametrize the model test process with some particular values or use the default values, for example, ``scale``, ``mean`` or ``std``.
+
+Test configuration is represented in [test_config.py](https://github.com/opencv/opencv/tree/master/samples/dnn/dnn_model_runner/dnn_conversion/common/test/configs/test_config.py) ``TestClsModuleConfig`` class:
+
+```python
+@dataclass
+class TestClsModuleConfig:
+    cls_test_data_dir: str = "../data"
+    test_module_name: str = "classification"
+    test_module_path: str = "classification.py"
+    input_img: str = os.path.join(cls_test_data_dir, "squirrel_cls.jpg")
+    model: str = ""
+
+    frame_height: str = str(TestClsConfig.frame_size)
+    frame_width: str = str(TestClsConfig.frame_size)
+    scale: str = "1.0"
+    mean: List[str] = field(default_factory=lambda: ["0.0", "0.0", "0.0"])
+    std: List[str] = field(default_factory=list)
+    crop: str = "False"
+    rgb: str = "True"
+    rsz_height: str = ""
+    rsz_width: str = ""
+    classes: str = os.path.join(cls_test_data_dir, "dnn", "classification_classes_ILSVRC2012.txt")
+```
+
+The default image preprocessing options are defined in [default_preprocess_config.py](https://github.com/opencv/opencv/tree/master/samples/dnn/dnn_model_runner/dnn_conversion/common/test/configs/default_preprocess_config.py). For instance:
+
+```python
+BASE_IMG_SCALE_FACTOR = 1 / 255.0
+PYTORCH_RSZ_HEIGHT = 256
+PYTORCH_RSZ_WIDTH = 256
+
+pytorch_resize_input_blob = {
+    "mean": ["123.675", "116.28", "103.53"],
+    "scale": str(BASE_IMG_SCALE_FACTOR),
+    "std": ["0.229", "0.224", "0.225"],
+    "crop": "True",
+    "rgb": "True",
+    "rsz_height": str(PYTORCH_RSZ_HEIGHT),
+    "rsz_width": str(PYTORCH_RSZ_WIDTH)
+}
+```
+
+The basis of the model testing is represented in [samples/dnn/classification.py](https://github.com/opencv/opencv/blob/master/samples/dnn/classification.py).  ``classification.py`` can be executed autonomously with provided converted model in ``--input`` and populated parameters for cv.dnn.blobFromImage.
+
+To reproduce from scratch the described in "Model Conversion Pipeline" OpenCV steps with ``dnn_model_runner`` execute the below line:
+
+```console
+python -m dnn_model_runner.dnn_conversion.pytorch.classification.py_to_py_cls --model_name resnet50 --test True --default_img_preprocess True --evaluate False
+```
+
+The network prediction is depicted in the top left corner of the output window:
+
+![ResNet50 OpenCV inference output](images/pytorch_resnet50_opencv_test_res.jpg)
@@ -0,0 +1,360 @@
+# Conversion of TensorFlow Classification Models and Launch with OpenCV Python {#tf_cls_tutorial_dnn_conversion}
+
+|    |    |
+| -: | :- |
+| Original author | Anastasia Murzova |
+| Compatibility | OpenCV >= 4.5 |
+
+## Goals
+In this tutorial you will learn how to:
+* obtain frozen graphs of TensorFlow (TF) classification models
+* run converted TensorFlow model with OpenCV Python API
+* obtain an evaluation of the TensorFlow and OpenCV DNN models
+
+We will explore the above-listed points by the example of MobileNet architecture.
+
+## Introduction
+Let's briefly view the key concepts involved in the pipeline of TensorFlow models transition with OpenCV API. The initial step in conversion of TensorFlow models into cv.dnn.Net
+is obtaining the frozen TF model graph. Frozen graph defines the combination of the model graph structure with kept values of the required variables, for example, weights. Usually the frozen graph is saved in [protobuf](https://en.wikipedia.org/wiki/Protocol_Buffers) (```.pb```) files.
+After the model ``.pb`` file was generated it can be read with cv.dnn.readNetFromTensorflow function.
+
+## Requirements
+To be able to experiment with the below code you will need to install a set of libraries. We will use a virtual environment with python3.7+ for this:
+
+```console
+virtualenv -p /usr/bin/python3.7 <env_dir_path>
+source <env_dir_path>/bin/activate
+```
+
+For OpenCV-Python building from source, follow the corresponding instructions from the @ref tutorial_py_table_of_contents_setup.
+
+Before you start the installation of the libraries, you can customize the [requirements.txt](https://github.com/opencv/opencv/tree/master/samples/dnn/dnn_model_runner/dnn_conversion/requirements.txt), excluding or including (for example, ``opencv-python``) some dependencies.
+The below line initiates requirements installation into the previously activated virtual environment:
+
+```console
+pip install -r requirements.txt
+```
+
+## Practice
+In this part we are going to cover the following points:
+1. create a TF classification model conversion pipeline and provide the inference
+2. evaluate and test TF classification models
+
+If you'd like merely to run evaluation or test model pipelines, the "Model Conversion Pipeline" tutorial part can be skipped.
+
+### Model Conversion Pipeline
+The code in this subchapter is located in the ``dnn_model_runner`` module and can be executed with the line:
+
+```console
+python -m dnn_model_runner.dnn_conversion.tf.classification.py_to_py_mobilenet
+```
+
+The following code contains the description of the below-listed steps:
+1. instantiate TF model
+2. create TF frozen graph
+3. read TF frozen graph with OpenCV API
+4. prepare input data
+5. provide inference
+
+```python
+# initialize TF MobileNet model
+original_tf_model = MobileNet(
+    include_top=True,
+    weights="imagenet"
+)
+
+# get TF frozen graph path
+full_pb_path = get_tf_model_proto(original_tf_model)
+
+# read frozen graph with OpenCV API
+opencv_net = cv2.dnn.readNetFromTensorflow(full_pb_path)
+print("OpenCV model was successfully read. Model layers: \n", opencv_net.getLayerNames())
+
+# get preprocessed image
+input_img = get_preprocessed_img("../data/squirrel_cls.jpg")
+
+# get ImageNet labels
+imagenet_labels = get_imagenet_labels("../data/dnn/classification_classes_ILSVRC2012.txt")
+
+# obtain OpenCV DNN predictions
+get_opencv_dnn_prediction(opencv_net, input_img, imagenet_labels)
+
+# obtain TF model predictions
+get_tf_dnn_prediction(original_tf_model, input_img, imagenet_labels)
+```
+
+To provide model inference we will use the below [squirrel photo](https://www.pexels.com/photo/brown-squirrel-eating-1564292) (under [CC0](https://www.pexels.com/terms-of-service/) license) corresponding to ImageNet class ID 335:
+```console
+fox squirrel, eastern fox squirrel, Sciurus niger
+```
+
+![Classification model input image](images/squirrel_cls.jpg)
+
+For the label decoding of the obtained prediction, we also need ``imagenet_classes.txt`` file, which contains the full list of the ImageNet classes.
+
+Let's go deeper into each step by the example of pretrained TF MobileNet:
+* instantiate TF model:
+
+```python
+# initialize TF MobileNet model
+original_tf_model = MobileNet(
+    include_top=True,
+    weights="imagenet"
+)
+```
+
+* create TF frozen graph
+
+```python
+# define the directory for .pb model
+pb_model_path = "models"
+
+# define the name of .pb model
+pb_model_name = "mobilenet.pb"
+
+# create directory for further converted model
+os.makedirs(pb_model_path, exist_ok=True)
+
+# get model TF graph
+tf_model_graph = tf.function(lambda x: tf_model(x))
+
+# get concrete function
+tf_model_graph = tf_model_graph.get_concrete_function(
+    tf.TensorSpec(tf_model.inputs[0].shape, tf_model.inputs[0].dtype))
+
+# obtain frozen concrete function
+frozen_tf_func = convert_variables_to_constants_v2(tf_model_graph)
+# get frozen graph
+frozen_tf_func.graph.as_graph_def()
+
+# save full tf model
+tf.io.write_graph(graph_or_graph_def=frozen_tf_func.graph,
+                  logdir=pb_model_path,
+                  name=pb_model_name,
+                  as_text=False)
+```
+
+After the successful execution of the above code, we will get a frozen graph in ``models/mobilenet.pb``.
+
+* read TF frozen graph with with cv.dnn.readNetFromTensorflow passing the obtained in the previous step ``mobilenet.pb`` into it:
+
+```python
+# get TF frozen graph path
+full_pb_path = get_tf_model_proto(original_tf_model)
+```
+
+* prepare input data with cv2.dnn.blobFromImage function:
+
+```python
+# read the image
+input_img = cv2.imread(img_path, cv2.IMREAD_COLOR)
+input_img = input_img.astype(np.float32)
+
+# define preprocess parameters
+mean = np.array([1.0, 1.0, 1.0]) * 127.5
+scale = 1 / 127.5
+
+# prepare input blob to fit the model input:
+# 1. subtract mean
+# 2. scale to set pixel values from 0 to 1
+input_blob = cv2.dnn.blobFromImage(
+    image=input_img,
+    scalefactor=scale,
+    size=(224, 224),  # img target size
+    mean=mean,
+    swapRB=True,  # BGR -> RGB
+    crop=True  # center crop
+)
+print("Input blob shape: {}\n".format(input_blob.shape))
+```
+
+Please, pay attention at the preprocessing order in the cv2.dnn.blobFromImage function. Firstly, the mean value is subtracted and only then pixel values are multiplied by the defined scale.
+Therefore, to reproduce the image preprocessing pipeline from the TF [``mobilenet.preprocess_input``](https://github.com/tensorflow/tensorflow/blob/02032fb477e9417197132648ec81e75beee9063a/tensorflow/python/keras/applications/mobilenet.py#L443-L445) function, we multiply ``mean`` by ``127.5``.
+
+As a result, 4-dimensional ``input_blob`` was obtained:
+
+ ``Input blob shape: (1, 3, 224, 224)``
+
+* provide OpenCV cv.dnn.Net inference:
+
+```python
+# set OpenCV DNN input
+opencv_net.setInput(preproc_img)
+
+# OpenCV DNN inference
+out = opencv_net.forward()
+print("OpenCV DNN prediction: \n")
+print("* shape: ", out.shape)
+
+# get the predicted class ID
+imagenet_class_id = np.argmax(out)
+
+# get confidence
+confidence = out[0][imagenet_class_id]
+print("* class ID: {}, label: {}".format(imagenet_class_id, imagenet_labels[imagenet_class_id]))
+print("* confidence: {:.4f}\n".format(confidence))
+```
+
+After the above code execution we will get the following output:
+
+```console
+OpenCV DNN prediction:
+* shape:  (1, 1000)
+* class ID: 335, label: fox squirrel, eastern fox squirrel, Sciurus niger
+* confidence: 0.9525
+```
+
+* provide TF MobileNet inference:
+
+```python
+# inference
+preproc_img = preproc_img.transpose(0, 2, 3, 1)
+print("TF input blob shape: {}\n".format(preproc_img.shape))
+
+out = original_net(preproc_img)
+
+print("\nTensorFlow model prediction: \n")
+print("* shape: ", out.shape)
+
+# get the predicted class ID
+imagenet_class_id = np.argmax(out)
+print("* class ID: {}, label: {}".format(imagenet_class_id, imagenet_labels[imagenet_class_id]))
+
+# get confidence
+confidence = out[0][imagenet_class_id]
+print("* confidence: {:.4f}".format(confidence))
+```
+
+To fit TF model input, ``input_blob`` was transposed:
+
+```console
+TF input blob shape: (1, 224, 224, 3)
+```
+
+TF inference results are the following:
+
+```console
+TensorFlow model prediction:
+* shape:  (1, 1000)
+* class ID: 335, label: fox squirrel, eastern fox squirrel, Sciurus niger
+* confidence: 0.9525
+```
+
+As it can be seen from the experiments OpenCV and TF inference results are equal.
+
+### Evaluation of the Models
+
+The proposed in ``dnn/samples`` ``dnn_model_runner`` module allows to run the full evaluation pipeline on the ImageNet dataset and test execution for the following TensorFlow classification models:
+* vgg16
+* vgg19
+* resnet50
+* resnet101
+* resnet152
+* densenet121
+* densenet169
+* densenet201
+* inceptionresnetv2
+* inceptionv3
+* mobilenet
+* mobilenetv2
+* nasnetlarge
+* nasnetmobile
+* xception
+
+This list can be also extended with further appropriate evaluation pipeline configuration.
+
+#### Evaluation Mode
+
+To below line represents running of the module in the evaluation mode:
+
+```console
+python -m dnn_model_runner.dnn_conversion.tf.classification.py_to_py_cls --model_name <tf_cls_model_name>
+```
+
+Chosen from the list classification model will be read into OpenCV ``cv.dnn_Net`` object. Evaluation results of TF and OpenCV models (accuracy, inference time, L1) will be written into the log file. Inference time values will be also depicted in a chart to generalize the obtained model information.
+
+Necessary evaluation configurations are defined in the [test_config.py](https://github.com/opencv/opencv/tree/master/samples/dnn/dnn_model_runner/dnn_conversion/common/test/configs/test_config.py) and can be modified in accordance with actual paths of data location::
+
+```python
+@dataclass
+class TestClsConfig:
+    batch_size: int = 50
+    frame_size: int = 224
+    img_root_dir: str = "./ILSVRC2012_img_val"
+    # location of image-class matching
+    img_cls_file: str = "./val.txt"
+    bgr_to_rgb: bool = True
+```
+
+The values from ``TestClsConfig`` can be customized in accordance with chosen model.
+
+To initiate the evaluation of the TensorFlow MobileNet, run the following line:
+
+```console
+python -m dnn_model_runner.dnn_conversion.tf.classification.py_to_py_cls --model_name mobilenet
+```
+
+After script launch, the log file with evaluation data will be generated in ``dnn_model_runner/dnn_conversion/logs``:
+
+```console
+===== Running evaluation of the model with the following params:
+    * val data location: ./ILSVRC2012_img_val
+    * log file location: dnn_model_runner/dnn_conversion/logs/TF_mobilenet_log.txt
+```
+
+#### Test Mode
+
+The below line represents running of the module in the test mode, namely it provides the steps for the model inference:
+
+```console
+python -m dnn_model_runner.dnn_conversion.tf.classification.py_to_py_cls --model_name <tf_cls_model_name> --test True --default_img_preprocess <True/False> --evaluate False
+```
+
+Here ``default_img_preprocess`` key defines whether you'd like to parametrize the model test process with some particular values or use the default values, for example, ``scale``, ``mean`` or ``std``.
+
+Test configuration is represented in [test_config.py](https://github.com/opencv/opencv/tree/master/samples/dnn/dnn_model_runner/dnn_conversion/common/test/configs/test_config.py) ``TestClsModuleConfig`` class:
+
+```python
+@dataclass
+class TestClsModuleConfig:
+    cls_test_data_dir: str = "../data"
+    test_module_name: str = "classification"
+    test_module_path: str = "classification.py"
+    input_img: str = os.path.join(cls_test_data_dir, "squirrel_cls.jpg")
+    model: str = ""
+
+    frame_height: str = str(TestClsConfig.frame_size)
+    frame_width: str = str(TestClsConfig.frame_size)
+    scale: str = "1.0"
+    mean: List[str] = field(default_factory=lambda: ["0.0", "0.0", "0.0"])
+    std: List[str] = field(default_factory=list)
+    crop: str = "False"
+    rgb: str = "True"
+    rsz_height: str = ""
+    rsz_width: str = ""
+    classes: str = os.path.join(cls_test_data_dir, "dnn", "classification_classes_ILSVRC2012.txt")
+```
+
+The default image preprocessing options are defined in ``default_preprocess_config.py``. For instance, for MobileNet:
+
+```python
+tf_input_blob = {
+    "mean": ["127.5", "127.5", "127.5"],
+    "scale": str(1 / 127.5),
+    "std": [],
+    "crop": "True",
+    "rgb": "True"
+}
+```
+
+The basis of the model testing is represented in [samples/dnn/classification.py](https://github.com/opencv/opencv/blob/master/samples/dnn/classification.py). ``classification.py`` can be executed autonomously with provided converted model in ``--input`` and populated parameters for cv.dnn.blobFromImage.
+
+To reproduce from scratch the described in "Model Conversion Pipeline" OpenCV steps with ``dnn_model_runner`` execute the below line:
+
+```console
+python -m dnn_model_runner.dnn_conversion.tf.classification.py_to_py_cls --model_name mobilenet --test True --default_img_preprocess True --evaluate False
+```
+
+The network prediction is depicted in the top left corner of the output window:
+
+![TF MobileNet OpenCV inference output](images/tf_mobilenet_opencv_test_res.jpg)
@@ -0,0 +1,140 @@
+# Conversion of TensorFlow Detection Models and Launch with OpenCV Python {#tf_det_tutorial_dnn_conversion}
+
+|    |    |
+| -: | :- |
+| Original author | Anastasia Murzova |
+| Compatibility | OpenCV >= 4.5 |
+
+## Goals
+In this tutorial you will learn how to:
+* obtain frozen graphs of TensorFlow (TF) detection models
+* run converted TensorFlow model with OpenCV Python API
+
+We will explore the above-listed points by the example of SSD MobileNetV1.
+
+## Introduction
+Let's briefly view the key concepts involved in the pipeline of TensorFlow models transition with OpenCV API. The initial step in the conversion of TensorFlow models into cv.dnn.Net
+is obtaining the frozen TF model graph. A frozen graph defines the combination of the model graph structure with kept values of the required variables, for example, weights. The frozen graph is saved in [protobuf](https://en.wikipedia.org/wiki/Protocol_Buffers) (```.pb```) files.
+There are special functions for reading ``.pb`` graphs in OpenCV: cv.dnn.readNetFromTensorflow and cv.dnn.readNet.
+
+## Requirements
+To be able to experiment with the below code you will need to install a set of libraries. We will use a virtual environment with python3.7+ for this:
+
+```console
+virtualenv -p /usr/bin/python3.7 <env_dir_path>
+source <env_dir_path>/bin/activate
+```
+
+For OpenCV-Python building from source, follow the corresponding instructions from the @ref tutorial_py_table_of_contents_setup.
+
+Before you start the installation of the libraries, you can customize the [requirements.txt](https://github.com/opencv/opencv/tree/master/samples/dnn/dnn_model_runner/dnn_conversion/requirements.txt), excluding or including (for example, ``opencv-python``) some dependencies.
+The below line initiates requirements installation into the previously activated virtual environment:
+
+```console
+pip install -r requirements.txt
+```
+
+## Practice
+In this part we are going to cover the following points:
+1. create a TF classification model conversion pipeline and provide the inference
+2. provide the inference, process prediction results
+
+### Model Preparation
+The code in this subchapter is located in the ``samples/dnn/dnn_model_runner`` module and can be executed with the below line:
+
+```console
+python -m dnn_model_runner.dnn_conversion.tf.detection.py_to_py_ssd_mobilenet
+```
+
+The following code contains the steps of the TF SSD MobileNetV1 model retrieval:
+
+```python
+    tf_model_name = 'ssd_mobilenet_v1_coco_2017_11_17'
+    graph_extraction_dir = "./"
+    frozen_graph_path = extract_tf_frozen_graph(tf_model_name, graph_extraction_dir)
+    print("Frozen graph path for {}: {}".format(tf_model_name, frozen_graph_path))
+```
+
+In ``extract_tf_frozen_graph`` function we extract the provided in model archive ``frozen_inference_graph.pb`` for its further processing:
+
+```python
+# define model archive name
+tf_model_tar = model_name + '.tar.gz'
+# define link to retrieve model archive
+model_link = DETECTION_MODELS_URL + tf_model_tar
+
+tf_frozen_graph_name = 'frozen_inference_graph'
+
+try:
+    urllib.request.urlretrieve(model_link, tf_model_tar)
+except Exception:
+    print("TF {} was not retrieved: {}".format(model_name, model_link))
+    return
+
+print("TF {} was retrieved.".format(model_name))
+
+tf_model_tar = tarfile.open(tf_model_tar)
+frozen_graph_path = ""
+
+for model_tar_elem in tf_model_tar.getmembers():
+    if tf_frozen_graph_name in os.path.basename(model_tar_elem.name):
+        tf_model_tar.extract(model_tar_elem, extracted_model_path)
+        frozen_graph_path = os.path.join(extracted_model_path, model_tar_elem.name)
+        break
+tf_model_tar.close()
+```
+
+After the successful execution of the above code we will get the following output:
+
+```console
+TF ssd_mobilenet_v1_coco_2017_11_17 was retrieved.
+Frozen graph path for ssd_mobilenet_v1_coco_2017_11_17: ./ssd_mobilenet_v1_coco_2017_11_17/frozen_inference_graph.pb
+```
+
+To provide model inference we will use the below [double-decker bus photo](https://www.pexels.com/photo/bus-and-car-on-one-way-street-3626589/) (under [Pexels](https://www.pexels.com/license/) license):
+
+![Double-decker bus](images/pexels_double_decker_bus.jpg)
+
+To initiate the test process we need to provide an appropriate model configuration. We will use [``ssd_mobilenet_v1_coco.config``](https://github.com/tensorflow/models/blob/master/research/object_detection/samples/configs/ssd_mobilenet_v1_coco.config) from [TensorFlow Object Detection API](https://github.com/tensorflow/models/tree/master/research/object_detection#tensorflow-object-detection-api).
+TensorFlow Object Detection API framework contains helpful mechanisms for object detection model manipulations.
+
+We will use this configuration to provide a text graph representation. To generate ``.pbtxt`` we will use the corresponding [``samples/dnn/tf_text_graph_ssd.py``](https://github.com/opencv/opencv/blob/master/samples/dnn/tf_text_graph_ssd.py) script:
+
+```console
+python tf_text_graph_ssd.py --input ssd_mobilenet_v1_coco_2017_11_17/frozen_inference_graph.pb --config ssd_mobilenet_v1_coco_2017_11_17/ssd_mobilenet_v1_coco.config --output ssd_mobilenet_v1_coco_2017_11_17.pbtxt
+```
+
+After successful execution ``ssd_mobilenet_v1_coco_2017_11_17.pbtxt`` will be created.
+
+Before we run ``object_detection.py``, let's have a look at the default values for the SSD MobileNetV1 test process configuration. They are located in [``models.yml``](https://github.com/opencv/opencv/blob/master/samples/dnn/models.yml):
+
+```yml
+ssd_tf:
+  model: "ssd_mobilenet_v1_coco_2017_11_17.pb"
+  config: "ssd_mobilenet_v1_coco_2017_11_17.pbtxt"
+  mean: [0, 0, 0]
+  scale: 1.0
+  width: 300
+  height: 300
+  rgb: true
+  classes: "object_detection_classes_coco.txt"
+  sample: "object_detection"
+```
+
+To fetch these values we need to provide frozen graph ``ssd_mobilenet_v1_coco_2017_11_17.pb`` model and text graph ``ssd_mobilenet_v1_coco_2017_11_17.pbtxt``:
+
+```console
+python object_detection.py ssd_tf --input ../data/pexels_double_decker_bus.jpg
+```
+
+This line is equivalent to:
+
+```console
+python object_detection.py --model ssd_mobilenet_v1_coco_2017_11_17.pb --config  ssd_mobilenet_v1_coco_2017_11_17.pbtxt  --input ../data/pexels_double_decker_bus.jpg --width 300 --height 300 --classes ../data/dnn/object_detection_classes_coco.txt
+```
+
+The result is:
+
+![OpenCV SSD bus result](images/opencv_bus_res.jpg)
+
+There are several helpful parameters, which can be also customized for result corrections: threshold (``--thr``) and non-maximum suppression (``--nms``) values.
@@ -0,0 +1,332 @@
+# Conversion of PyTorch Segmentation Models and Launch with OpenCV {#pytorch_segm_tutorial_dnn_conversion}
+
+## Goals
+In this tutorial you will learn how to:
+* convert PyTorch segmentation models
+* run converted PyTorch model with OpenCV
+* obtain an evaluation of the PyTorch and OpenCV DNN models
+
+We will explore the above-listed points by the example of the FCN ResNet-50 architecture.
+
+## Introduction
+The key points involved in the transition pipeline of the [PyTorch classification](https://link_to_cls_tutorial) and segmentation models with OpenCV API are equal. The first step is model transferring into [ONNX](https://onnx.ai/about.html) format with PyTorch [``torch.onnx.export``](https://pytorch.org/docs/stable/onnx.html#torch.onnx.export) built-in function.
+Further the obtained ``.onnx`` model is passed into cv.dnn.readNetFromONNX, which returns cv.dnn.Net object ready for DNN manipulations.
+
+## Practice
+In this part we are going to cover the following points:
+1. create a segmentation model conversion pipeline and provide the inference
+2. evaluate and test segmentation models
+
+If you'd like merely to run evaluation or test model pipelines, the "Model Conversion Pipeline" part can be skipped.
+
+### Model Conversion Pipeline
+The code in this subchapter is located in the ``dnn_model_runner`` module and can be executed with the line:
+
+``
+python -m dnn_model_runner.dnn_conversion.pytorch.segmentation.py_to_py_fcnresnet50
+``
+
+The following code contains the description of the below-listed steps:
+1. instantiate PyTorch model
+2. convert PyTorch model into ``.onnx``
+3. read the transferred network with OpenCV API
+4. prepare input data
+5. provide inference
+6. get colored masks from predictions
+7. visualize results
+
+```python
+# initialize PyTorch FCN ResNet-50 model
+original_model = models.segmentation.fcn_resnet50(pretrained=True)
+
+# get the path to the converted into ONNX PyTorch model
+full_model_path = get_pytorch_onnx_model(original_model)
+
+# read converted .onnx model with OpenCV API
+opencv_net = cv2.dnn.readNetFromONNX(full_model_path)
+print("OpenCV model was successfully read. Layer IDs: \n", opencv_net.getLayerNames())
+
+# get preprocessed image
+img, input_img = get_processed_imgs("test_data/sem_segm/2007_000033.jpg")
+
+# obtain OpenCV DNN predictions
+opencv_prediction = get_opencv_dnn_prediction(opencv_net, input_img)
+
+# obtain original PyTorch ResNet50 predictions
+pytorch_prediction = get_pytorch_dnn_prediction(original_model, input_img)
+
+pascal_voc_classes, pascal_voc_colors = read_colors_info("test_data/sem_segm/pascal-classes.txt")
+
+# obtain colored segmentation masks
+opencv_colored_mask = get_colored_mask(img.shape, opencv_prediction, pascal_voc_colors)
+pytorch_colored_mask = get_colored_mask(img.shape, pytorch_prediction, pascal_voc_colors)
+
+# obtain palette of PASCAL VOC colors
+color_legend = get_legend(pascal_voc_classes, pascal_voc_colors)
+
+cv2.imshow('PyTorch Colored Mask', pytorch_colored_mask)
+cv2.imshow('OpenCV DNN Colored Mask', opencv_colored_mask)
+cv2.imshow('Color Legend', color_legend)
+
+cv2.waitKey(0)
+```
+
+To provide the model inference we will use the below picture from the [PASCAL VOC](http://host.robots.ox.ac.uk/pascal/VOC/) validation dataset:
+
+![PASCAL VOC img](images/2007_000033.jpg)
+
+The target segmented result is:
+
+![PASCAL VOC ground truth](images/2007_000033.png)
+
+For the PASCAL VOC colors decoding and its mapping with the predicted masks, we also need ``pascal-classes.txt`` file, which contains the full list of the PASCAL VOC classes and corresponding colors.
+
+Let's go deeper into each code step by the example of pretrained PyTorch FCN ResNet-50:
+*  instantiate PyTorch FCN ResNet-50 model:
+
+```python
+# initialize PyTorch FCN ResNet-50 model
+original_model = models.segmentation.fcn_resnet50(pretrained=True)
+```
+
+*  convert PyTorch model into ONNX format:
+
+```python
+# define the directory for further converted model save
+onnx_model_path = "models"
+# define the name of further converted model
+onnx_model_name = "fcnresnet50.onnx"
+
+# create directory for further converted model
+os.makedirs(onnx_model_path, exist_ok=True)
+
+# get full path to the converted model
+full_model_path = os.path.join(onnx_model_path, onnx_model_name)
+
+# generate model input to build the graph
+generated_input = Variable(
+    torch.randn(1, 3, 500, 500)
+)
+
+# model export into ONNX format
+torch.onnx.export(
+    original_model,
+    generated_input,
+    full_model_path,
+    verbose=True,
+    input_names=["input"],
+    output_names=["output"],
+    opset_version=11
+)
+```
+
+The code from this step does not differ from the classification conversion case. Thus, after the successful execution of the above code, we will get ``models/fcnresnet50.onnx``.
+
+* read the transferred network with cv.dnn.readNetFromONNX passing the obtained in the previous step ONNX model into it:
+
+```python
+# read converted .onnx model with OpenCV API
+opencv_net = cv2.dnn.readNetFromONNX(full_model_path)
+```
+
+* prepare input data:
+
+```python
+# read the image
+input_img = cv2.imread(img_path, cv2.IMREAD_COLOR)
+input_img = input_img.astype(np.float32)
+
+# target image sizes
+img_height = input_img.shape[0]
+img_width = input_img.shape[1]
+
+# define preprocess parameters
+mean = np.array([0.485, 0.456, 0.406]) * 255.0
+scale = 1 / 255.0
+std = [0.229, 0.224, 0.225]
+
+# prepare input blob to fit the model input:
+# 1. subtract mean
+# 2. scale to set pixel values from 0 to 1
+input_blob = cv2.dnn.blobFromImage(
+    image=input_img,
+    scalefactor=scale,
+    size=(img_width, img_height),  # img target size
+    mean=mean,
+    swapRB=True,  # BGR -> RGB
+    crop=False  # center crop
+)
+# 3. divide by std
+input_blob[0] /= np.asarray(std, dtype=np.float32).reshape(3, 1, 1)
+```
+
+In this step we read the image and prepare model input with cv2.dnn.blobFromImage function, which returns 4-dimensional blob.
+It should be noted that firstly in ``cv2.dnn.blobFromImage`` mean value is subtracted and only then pixel values are scaled. Thus, ``mean`` is multiplied by ``255.0`` to reproduce the original image preprocessing order:
+
+```python
+img /= 255.0
+img -= [0.485, 0.456, 0.406]
+img /= [0.229, 0.224, 0.225]
+```
+
+* OpenCV ``cv.dnn_Net`` inference:
+
+```python
+# set OpenCV DNN input
+opencv_net.setInput(preproc_img)
+
+# OpenCV DNN inference
+out = opencv_net.forward()
+print("OpenCV DNN segmentation prediction: \n")
+print("* shape: ", out.shape)
+
+# get IDs of predicted classes
+out_predictions = np.argmax(out[0], axis=0)
+```
+
+After the above code execution we will get the following output:
+
+```
+OpenCV DNN segmentation prediction:
+* shape:  (1, 21, 500, 500)
+```
+
+Each prediction channel out of 21, where 21 represents the number of PASCAL VOC classes, contains probabilities, which indicate how likely the pixel corresponds to the PASCAL VOC class.
+
+* PyTorch FCN ResNet-50 model inference:
+
+```python
+original_net.eval()
+preproc_img = torch.FloatTensor(preproc_img)
+
+with torch.no_grad():
+    # obtaining unnormalized probabilities for each class
+    out = original_net(preproc_img)['out']
+
+print("\nPyTorch segmentation model prediction: \n")
+print("* shape: ", out.shape)
+
+# get IDs of predicted classes
+out_predictions = out[0].argmax(dim=0)
+```
+
+After the above code launching we will get the following output:
+
+```
+PyTorch segmentation model prediction:
+* shape:  torch.Size([1, 21, 366, 500])
+```
+
+PyTorch prediction also contains probabilities corresponding to each class prediction.
+
+* get colored masks from predictions:
+
+```python
+# convert mask values into PASCAL VOC colors
+processed_mask = np.stack([colors[color_id] for color_id in segm_mask.flatten()])
+
+# reshape mask into 3-channel image
+processed_mask = processed_mask.reshape(mask_height, mask_width, 3)
+processed_mask = cv2.resize(processed_mask, (img_width, img_height), interpolation=cv2.INTER_NEAREST).astype(
+    np.uint8)
+
+# convert colored mask from BGR to RGB for compatibility with PASCAL VOC colors
+processed_mask = cv2.cvtColor(processed_mask, cv2.COLOR_BGR2RGB)
+```
+
+In this step we map the probabilities from segmentation masks with appropriate colors of the predicted classes. Let's have a look at the results:
+
+![OpenCV Colored Mask](images/legend_opencv_color_mask.png)
+
+For the extended evaluation of the models, we can use ``py_to_py_segm`` script of the ``dnn_model_runner`` module. This module part will be described in the next subchapter.
+
+### Evaluation of the Models
+
+The proposed in ``dnn/samples`` ``dnn_model_runner`` module allows to run the full evaluation pipeline on the PASCAL VOC dataset and test execution for the following PyTorch segmentation models:
+* FCN ResNet-50
+* FCN ResNet-101
+
+This list can be also extended with further appropriate evaluation pipeline configuration.
+
+#### Evaluation Mode
+
+The below line represents running of the module in the evaluation mode:
+
+```
+python -m dnn_model_runner.dnn_conversion.pytorch.segmentation.py_to_py_segm --model_name <pytorch_segm_model_name>
+```
+
+Chosen from the list segmentation model will be read into OpenCV ``cv.dnn_Net`` object. Evaluation results of PyTorch and OpenCV models (pixel accuracy, mean IoU, inference time) will be written into the log file. Inference time values will be also depicted in a chart to generalize the obtained model information.
+
+Necessary evaluation configurations are defined in the [``test_config.py``](https://github.com/opencv/opencv/tree/master/samples/dnn/dnn_model_runner/dnn_conversion/common/test/configs/test_config.py):
+
+```python
+@dataclass
+class TestSegmConfig:
+    frame_size: int = 500
+    img_root_dir: str = "./VOC2012"
+    img_dir: str = os.path.join(img_root_dir, "JPEGImages/")
+    img_segm_gt_dir: str = os.path.join(img_root_dir, "SegmentationClass/")
+    # reduced val: https://github.com/shelhamer/fcn.berkeleyvision.org/blob/master/data/pascal/seg11valid.txt
+    segm_val_file: str = os.path.join(img_root_dir, "ImageSets/Segmentation/seg11valid.txt")
+    colour_file_cls: str = os.path.join(img_root_dir, "ImageSets/Segmentation/pascal-classes.txt")
+```
+
+These values can be modified in accordance with chosen model pipeline.
+
+To initiate the evaluation of the PyTorch FCN ResNet-50, run the following line:
+
+```
+python -m dnn_model_runner.dnn_conversion.pytorch.segmentation.py_to_py_segm --model_name fcnresnet50
+```
+
+#### Test Mode
+
+The below line represents running of the module in the test mode, which provides the steps for the model inference:
+
+```
+python -m dnn_model_runner.dnn_conversion.pytorch.segmentation.py_to_py_segm --model_name <pytorch_segm_model_name> --test True --default_img_preprocess <True/False> --evaluate False
+```
+
+Here ``default_img_preprocess`` key defines whether you'd like to parametrize the model test process with some particular values or use the default values, for example, ``scale``, ``mean`` or ``std``.
+
+Test configuration is represented in [``test_config.py``](https://github.com/opencv/opencv/tree/master/samples/dnn/dnn_model_runner/dnn_conversion/common/test/configs/test_config.py) ``TestSegmModuleConfig`` class:
+
+```python
+@dataclass
+class TestSegmModuleConfig:
+    segm_test_data_dir: str = "test_data/sem_segm"
+    test_module_name: str = "segmentation"
+    test_module_path: str = "segmentation.py"
+    input_img: str = os.path.join(segm_test_data_dir, "2007_000033.jpg")
+    model: str = ""
+
+    frame_height: str = str(TestSegmConfig.frame_size)
+    frame_width: str = str(TestSegmConfig.frame_size)
+    scale: float = 1.0
+    mean: List[float] = field(default_factory=lambda: [0.0, 0.0, 0.0])
+    std: List[float] = field(default_factory=list)
+    crop: bool = False
+    rgb: bool = True
+    classes: str = os.path.join(segm_test_data_dir, "pascal-classes.txt")
+```
+
+The default image preprocessing options are defined in ``default_preprocess_config.py``:
+
+```python
+pytorch_segm_input_blob = {
+    "mean": ["123.675", "116.28", "103.53"],
+    "scale": str(1 / 255.0),
+    "std": ["0.229", "0.224", "0.225"],
+    "crop": "False",
+    "rgb": "True"
+}
+```
+
+The basis of the model testing is represented in ``samples/dnn/segmentation.py``.  ``segmentation.py`` can be executed autonomously with provided converted model in ``--input`` and populated parameters for ``cv2.dnn.blobFromImage``.
+
+To reproduce from scratch the described in "Model Conversion Pipeline" OpenCV steps with ``dnn_model_runner`` execute the below line:
+
+```
+python -m dnn_model_runner.dnn_conversion.pytorch.segmentation.py_to_py_segm --model_name fcnresnet50 --test True --default_img_preprocess True --evaluate False
+```
@@ -0,0 +1,406 @@
+# Conversion of TensorFlow Segmentation Models and Launch with OpenCV {#tf_segm_tutorial_dnn_conversion}
+
+## Goals
+In this tutorial you will learn how to:
+* convert TensorFlow (TF) segmentation models
+* run converted TensorFlow model with OpenCV
+* obtain an evaluation of the TensorFlow and OpenCV DNN models
+
+We will explore the above-listed points by the example of the DeepLab architecture.
+
+## Introduction
+The key concepts involved in the transition pipeline of the [TensorFlow classification](https://link_to_cls_tutorial) and segmentation models with OpenCV API are almost equal excepting the phase of graph optimization. The initial step in conversion of TensorFlow models into cv.dnn.Net
+is obtaining the frozen TF model graph. Frozen graph defines the combination of the model graph structure with kept values of the required variables, for example, weights. Usually the frozen graph is saved in [protobuf](https://en.wikipedia.org/wiki/Protocol_Buffers) (```.pb```) files.
+To read the generated segmentation model ``.pb`` file with cv.dnn.readNetFromTensorflow, it is needed to modify the graph with TF [graph transform tool](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/tools/graph_transforms).
+
+## Practice
+In this part we are going to cover the following points:
+1. create a TF classification model conversion pipeline and provide the inference
+2. evaluate and test TF classification models
+
+If you'd like merely to run evaluation or test model pipelines, the "Model Conversion Pipeline" tutorial part can be skipped.
+
+### Model Conversion Pipeline
+The code in this subchapter is located in the ``dnn_model_runner`` module and can be executed with the line:
+
+```
+python -m dnn_model_runner.dnn_conversion.tf.segmentation.py_to_py_deeplab
+```
+
+TensorFlow segmentation models can be found in [TensorFlow Research Models](https://github.com/tensorflow/models/tree/master/research/#tensorflow-research-models) section, which contains the implementations of models on the basis of published research papers.
+We will retrieve the archive with the pre-trained TF DeepLabV3 from the below link:
+
+```
+http://download.tensorflow.org/models/deeplabv3_mnv2_pascal_trainval_2018_01_29.tar.gz
+```
+
+The full frozen graph obtaining pipeline is described in ``deeplab_retrievement.py``:
+
+```python
+def get_deeplab_frozen_graph():
+    # define model path to download
+    models_url = 'http://download.tensorflow.org/models/'
+    mobilenetv2_voctrainval = 'deeplabv3_mnv2_pascal_trainval_2018_01_29.tar.gz'
+
+    # construct model link to download
+    model_link = models_url + mobilenetv2_voctrainval
+
+    try:
+        urllib.request.urlretrieve(model_link, mobilenetv2_voctrainval)
+    except Exception:
+        print("TF DeepLabV3 was not retrieved: {}".format(model_link))
+        return
+
+    tf_model_tar = tarfile.open(mobilenetv2_voctrainval)
+
+    # iterate the obtained model archive
+    for model_tar_elem in tf_model_tar.getmembers():
+        # check whether the model archive contains frozen graph
+        if TF_FROZEN_GRAPH_NAME in os.path.basename(model_tar_elem.name):
+            # extract frozen graph
+            tf_model_tar.extract(model_tar_elem, FROZEN_GRAPH_PATH)
+
+    tf_model_tar.close()
+```
+
+After running this script:
+
+```
+python -m dnn_model_runner.dnn_conversion.tf.segmentation.deeplab_retrievement
+```
+
+we will get ``frozen_inference_graph.pb`` in ``deeplab/deeplabv3_mnv2_pascal_trainval``.
+
+Before going to the network loading with OpenCV it is needed to optimize the extracted ``frozen_inference_graph.pb``.
+To optimize the graph we use TF ``TransformGraph`` with default parameters:
+
+```python
+DEFAULT_OPT_GRAPH_NAME = "optimized_frozen_inference_graph.pb"
+DEFAULT_INPUTS = "sub_7"
+DEFAULT_OUTPUTS = "ResizeBilinear_3"
+DEFAULT_TRANSFORMS = "remove_nodes(op=Identity)" \
+                     " merge_duplicate_nodes" \
+                     " strip_unused_nodes" \
+                     " fold_constants(ignore_errors=true)" \
+                     " fold_batch_norms" \
+                     " fold_old_batch_norms"
+
+
+def optimize_tf_graph(
+        in_graph,
+        out_graph=DEFAULT_OPT_GRAPH_NAME,
+        inputs=DEFAULT_INPUTS,
+        outputs=DEFAULT_OUTPUTS,
+        transforms=DEFAULT_TRANSFORMS,
+        is_manual=True,
+        was_optimized=True
+):
+    # ...
+
+    tf_opt_graph = TransformGraph(
+        tf_graph,
+        inputs,
+        outputs,
+        transforms
+    )
+```
+
+To run graph optimization process, execute the line:
+
+```
+python -m dnn_model_runner.dnn_conversion.tf.segmentation.tf_graph_optimizer --in_graph deeplab/deeplabv3_mnv2_pascal_trainval/frozen_inference_graph.pb
+```
+
+As a result ``deeplab/deeplabv3_mnv2_pascal_trainval`` directory will contain ``optimized_frozen_inference_graph.pb``.
+
+After we have obtained the model graphs, let's examine the below-listed steps:
+1. read TF ``frozen_inference_graph.pb`` graph
+2. read optimized TF frozen graph with OpenCV API
+3. prepare input data
+4. provide inference
+5. get colored masks from predictions
+6. visualize results
+
+```python
+# get TF model graph from the obtained frozen graph
+deeplab_graph = read_deeplab_frozen_graph(deeplab_frozen_graph_path)
+
+# read DeepLab frozen graph with OpenCV API
+opencv_net = cv2.dnn.readNetFromTensorflow(opt_deeplab_frozen_graph_path)
+print("OpenCV model was successfully read. Model layers: \n", opencv_net.getLayerNames())
+
+# get processed image
+original_img_shape, tf_input_blob, opencv_input_img = get_processed_imgs("test_data/sem_segm/2007_000033.jpg")
+
+# obtain OpenCV DNN predictions
+opencv_prediction = get_opencv_dnn_prediction(opencv_net, opencv_input_img)
+
+# obtain TF model predictions
+tf_prediction = get_tf_dnn_prediction(deeplab_graph, tf_input_blob)
+
+# get PASCAL VOC classes and colors
+pascal_voc_classes, pascal_voc_colors = read_colors_info("test_data/sem_segm/pascal-classes.txt")
+
+# obtain colored segmentation masks
+opencv_colored_mask = get_colored_mask(original_img_shape, opencv_prediction, pascal_voc_colors)
+tf_colored_mask = get_tf_colored_mask(original_img_shape, tf_prediction, pascal_voc_colors)
+
+# obtain palette of PASCAL VOC colors
+color_legend = get_legend(pascal_voc_classes, pascal_voc_colors)
+
+cv2.imshow('TensorFlow Colored Mask', tf_colored_mask)
+cv2.imshow('OpenCV DNN Colored Mask', opencv_colored_mask)
+
+cv2.imshow('Color Legend', color_legend)
+```
+
+To provide the model inference we will use the below picture from the [PASCAL VOC](http://host.robots.ox.ac.uk/pascal/VOC/) validation dataset:
+
+![PASCAL VOC img](images/2007_000033.jpg)
+
+The target segmented result is:
+
+![PASCAL VOC ground truth](images/2007_000033.png)
+
+For the PASCAL VOC colors decoding and its mapping with the predicted masks, we also need ``pascal-classes.txt`` file, which contains the full list of the PASCAL VOC classes and corresponding colors.
+
+Let's go deeper into each step by the example of pretrained TF DeepLabV3 MobileNetV2:
+
+* read TF ``frozen_inference_graph.pb`` graph :
+
+```python
+# init deeplab model graph
+model_graph = tf.Graph()
+
+# obtain
+with tf.io.gfile.GFile(frozen_graph_path, 'rb') as graph_file:
+    tf_model_graph = GraphDef()
+tf_model_graph.ParseFromString(graph_file.read())
+
+with model_graph.as_default():
+    tf.import_graph_def(tf_model_graph, name='')
+```
+
+* read optimized TF frozen graph with OpenCV API:
+
+```python
+# read DeepLab frozen graph with OpenCV API
+opencv_net = cv2.dnn.readNetFromTensorflow(opt_deeplab_frozen_graph_path)
+```
+
+* prepare input data with cv2.dnn.blobFromImage function:
+
+```python
+# read the image
+input_img = cv2.imread(img_path, cv2.IMREAD_COLOR)
+input_img = input_img.astype(np.float32)
+
+# preprocess image for TF model input
+tf_preproc_img = cv2.resize(input_img, (513, 513))
+tf_preproc_img = cv2.cvtColor(tf_preproc_img, cv2.COLOR_BGR2RGB)
+
+# define preprocess parameters for OpenCV DNN
+mean = np.array([1.0, 1.0, 1.0]) * 127.5
+scale = 1 / 127.5
+
+# prepare input blob to fit the model input:
+# 1. subtract mean
+# 2. scale to set pixel values from 0 to 1
+input_blob = cv2.dnn.blobFromImage(
+    image=input_img,
+    scalefactor=scale,
+    size=(513, 513),  # img target size
+    mean=mean,
+    swapRB=True,  # BGR -> RGB
+    crop=False  # center crop
+)
+```
+
+Please, pay attention at the preprocessing order in the ``cv2.dnn.blobFromImage`` function. Firstly, the mean value is subtracted and only then pixel values are multiplied by the defined scale.
+Therefore, to reproduce TF image preprocessing pipeline, we multiply ``mean`` by ``127.5``.
+Another important point is image preprocessing for TF DeepLab. To pass the image into TF model we need only to construct an appropriate shape, the rest image preprocessing is described in [feature_extractor.py](https://github.com/tensorflow/models/blob/master/research/deeplab/core/feature_extractor.py) and will be invoked automatically.
+
+* provide OpenCV ``cv.dnn_Net`` inference:
+
+```python
+# set OpenCV DNN input
+opencv_net.setInput(preproc_img)
+
+# OpenCV DNN inference
+out = opencv_net.forward()
+print("OpenCV DNN segmentation prediction: \n")
+print("* shape: ", out.shape)
+
+# get IDs of predicted classes
+out_predictions = np.argmax(out[0], axis=0)
+```
+
+After the above code execution we will get the following output:
+
+```
+OpenCV DNN segmentation prediction:
+* shape:  (1, 21, 513, 513)
+
+```
+
+Each prediction channel out of 21, where 21 represents the number of PASCAL VOC classes, contains probabilities, which indicate how likely the pixel corresponds to the PASCAL VOC class.
+
+* provide TF model inference:
+
+```python
+preproc_img = np.expand_dims(preproc_img, 0)
+
+# init TF session
+tf_session = Session(graph=model_graph)
+
+input_tensor_name = "ImageTensor:0",
+output_tensor_name = "SemanticPredictions:0"
+
+# run inference
+out = tf_session.run(
+    output_tensor_name,
+    feed_dict={input_tensor_name: [preproc_img]}
+)
+
+print("TF segmentation model prediction: \n")
+print("* shape: ", out.shape)
+```
+
+TF inference results are the following:
+
+```
+TF segmentation model prediction:
+* shape:  (1, 513, 513)
+```
+
+TensorFlow prediction contains the indexes of corresponding PASCAL VOC classes.
+
+* transform OpenCV prediction into colored mask:
+
+```python
+mask_height = segm_mask.shape[0]
+mask_width = segm_mask.shape[1]
+
+img_height = original_img_shape[0]
+img_width = original_img_shape[1]
+
+# convert mask values into PASCAL VOC colors
+processed_mask = np.stack([colors[color_id] for color_id in segm_mask.flatten()])
+
+# reshape mask into 3-channel image
+processed_mask = processed_mask.reshape(mask_height, mask_width, 3)
+processed_mask = cv2.resize(processed_mask, (img_width, img_height), interpolation=cv2.INTER_NEAREST).astype(
+    np.uint8)
+
+# convert colored mask from BGR to RGB
+processed_mask = cv2.cvtColor(processed_mask, cv2.COLOR_BGR2RGB)
+```
+
+In this step we map the probabilities from segmentation masks with appropriate colors of the predicted classes. Let's have a look at the results:
+
+![Color Legend](images/colors_legend.png)
+
+![OpenCV Colored Mask](images/deeplab_opencv_colored_mask.png)
+
+* transform TF prediction into colored mask:
+
+```python
+colors = np.array(colors)
+processed_mask = colors[segm_mask[0]]
+
+img_height = original_img_shape[0]
+img_width = original_img_shape[1]
+
+processed_mask = cv2.resize(processed_mask, (img_width, img_height), interpolation=cv2.INTER_NEAREST).astype(
+    np.uint8)
+
+# convert colored mask from BGR to RGB for compatibility with PASCAL VOC colors
+processed_mask = cv2.cvtColor(processed_mask, cv2.COLOR_BGR2RGB)
+```
+
+The result is:
+
+![TF Colored Mask](images/deeplab_tf_colored_mask.png)
+
+As a result, we get two equal segmentation masks.
+
+### Evaluation of the Models
+
+The proposed in ``dnn/samples`` ``dnn_model_runner`` module allows to run the full evaluation pipeline on the PASCAL VOC dataset and test execution for the DeepLab MobileNet model.
+
+#### Evaluation Mode
+
+To below line represents running of the module in the evaluation mode:
+
+```
+python -m dnn_model_runner.dnn_conversion.tf.segmentation.py_to_py_segm
+```
+
+The model will be read into OpenCV ``cv.dnn_Net`` object. Evaluation results of TF and OpenCV models  (pixel accuracy, mean IoU, inference time) will be written into the log file. Inference time values will be also depicted in a chart to generalize the obtained model information.
+
+Necessary evaluation configurations are defined in the [``test_config.py``](https://github.com/opencv/opencv/tree/master/samples/dnn/dnn_model_runner/dnn_conversion/common/test/configs/test_config.py):
+
+```python
+@dataclass
+class TestSegmConfig:
+    frame_size: int = 500
+    img_root_dir: str = "./VOC2012"
+    img_dir: str = os.path.join(img_root_dir, "JPEGImages/")
+    img_segm_gt_dir: str = os.path.join(img_root_dir, "SegmentationClass/")
+    # reduced val: https://github.com/shelhamer/fcn.berkeleyvision.org/blob/master/data/pascal/seg11valid.txt
+    segm_val_file: str = os.path.join(img_root_dir, "ImageSets/Segmentation/seg11valid.txt")
+    colour_file_cls: str = os.path.join(img_root_dir, "ImageSets/Segmentation/pascal-classes.txt")
+```
+
+These values can be modified in accordance with chosen model pipeline.
+
+#### Test Mode
+
+The below line represents running of the module in the test mode, which provides the steps for the model inference:
+
+```
+python -m dnn_model_runner.dnn_conversion.tf.segmentation.py_to_py_segm --test True --default_img_preprocess <True/False> --evaluate False
+```
+
+Here ``default_img_preprocess`` key defines whether you'd like to parametrize the model test process with some particular values or use the default values, for example, ``scale``, ``mean`` or ``std``.
+
+Test configuration is represented in [``test_config.py``](https://github.com/opencv/opencv/tree/master/samples/dnn/dnn_model_runner/dnn_conversion/common/test/configs/test_config.py) ``TestSegmModuleConfig`` class:
+
+```python
+@dataclass
+class TestSegmModuleConfig:
+    segm_test_data_dir: str = "test_data/sem_segm"
+    test_module_name: str = "segmentation"
+    test_module_path: str = "segmentation.py"
+    input_img: str = os.path.join(segm_test_data_dir, "2007_000033.jpg")
+    model: str = ""
+
+    frame_height: str = str(TestSegmConfig.frame_size)
+    frame_width: str = str(TestSegmConfig.frame_size)
+    scale: float = 1.0
+    mean: List[float] = field(default_factory=lambda: [0.0, 0.0, 0.0])
+    std: List[float] = field(default_factory=list)
+    crop: bool = False
+    rgb: bool = True
+    classes: str = os.path.join(segm_test_data_dir, "pascal-classes.txt")
+```
+
+The default image preprocessing options are defined in ``default_preprocess_config.py``:
+
+```python
+tf_segm_input_blob = {
+    "scale": str(1 / 127.5),
+    "mean": ["127.5", "127.5", "127.5"],
+    "std": [],
+    "crop": "False",
+    "rgb": "True"
+}
+```
+
+The basis of the model testing is represented in ``samples/dnn/segmentation.py``.  ``segmentation.py`` can be executed autonomously with provided converted model in ``--input`` and populated parameters for ``cv2.dnn.blobFromImage``.
+
+To reproduce from scratch the described in "Model Conversion Pipeline" OpenCV steps with ``dnn_model_runner`` execute the below line:
+
+```
+python -m dnn_model_runner.dnn_conversion.tf.segmentation.py_to_py_segm --test True --default_img_preprocess True --evaluate False
+```
@@ -0,0 +1,324 @@
+# High Level API: TextDetectionModel and TextRecognitionModel {#tutorial_dnn_text_spotting}
+
+@tableofcontents
+
+@prev_tutorial{tutorial_dnn_OCR}
+@next_tutorial{pytorch_cls_tutorial_dnn_conversion}
+
+|    |    |
+| -: | :- |
+| Original author | Wenqing Zhang |
+| Compatibility | OpenCV >= 4.5 |
+
+## Introduction
+In this tutorial, we will introduce the APIs for TextRecognitionModel and TextDetectionModel in detail.
+
+---
+#### TextRecognitionModel:
+
+In the current version, @ref cv::dnn::TextRecognitionModel only supports CNN+RNN+CTC based algorithms,
+and the greedy decoding method for CTC is provided.
+For more information, please refer to the [original paper](https://arxiv.org/abs/1507.05717)
+
+Before recognition, you should `setVocabulary` and `setDecodeType`.
+- "CTC-greedy", the output of the text recognition model should be a probability matrix.
+    The shape should be `(T, B, Dim)`, where
+    - `T` is the sequence length
+    - `B` is the batch size (only support `B=1` in inference)
+    - and `Dim` is the length of vocabulary +1('Blank' of CTC is at the index=0 of Dim).
+
+@ref cv::dnn::TextRecognitionModel::recognize() is the main function for text recognition.
+- The input image should be a cropped text image or an image with `roiRects`
+- Other decoding methods may supported in the future
+
+---
+
+#### TextDetectionModel:
+
+@ref cv::dnn::TextDetectionModel API provides these methods for text detection:
+- cv::dnn::TextDetectionModel::detect() returns the results in std::vector<std::vector<Point>> (4-points quadrangles)
+- cv::dnn::TextDetectionModel::detectTextRectangles() returns the results in std::vector<cv::RotatedRect> (RBOX-like)
+
+In the current version, @ref cv::dnn::TextDetectionModel supports these algorithms:
+- use @ref cv::dnn::TextDetectionModel_DB with "DB" models
+- and use @ref cv::dnn::TextDetectionModel_EAST with "EAST" models
+
+The following provided pretrained models are variants of DB (w/o deformable convolution),
+and the performance can be referred to the Table.1 in the [paper]((https://arxiv.org/abs/1911.08947)).
+For more information, please refer to the [official code](https://github.com/MhLiao/DB)
+
+---
+
+You can train your own model with more data, and convert it into ONNX format.
+We encourage you to add new algorithms to these APIs.
+
+
+## Pretrained Models
+
+#### TextRecognitionModel:
+
+```
+crnn.onnx:
+url: https://drive.google.com/uc?export=dowload&id=1ooaLR-rkTl8jdpGy1DoQs0-X0lQsB6Fj
+sha: 270d92c9ccb670ada2459a25977e8deeaf8380d3,
+alphabet_36.txt: https://drive.google.com/uc?export=dowload&id=1oPOYx5rQRp8L6XQciUwmwhMCfX0KyO4b
+parameter setting: -rgb=0;
+description: The classification number of this model is 36 (0~9 + a~z).
+             The training dataset is MJSynth.
+
+crnn_cs.onnx:
+url: https://drive.google.com/uc?export=dowload&id=12diBsVJrS9ZEl6BNUiRp9s0xPALBS7kt
+sha: a641e9c57a5147546f7a2dbea4fd322b47197cd5
+alphabet_94.txt: https://drive.google.com/uc?export=dowload&id=1oKXxXKusquimp7XY1mFvj9nwLzldVgBR
+parameter setting: -rgb=1;
+description: The classification number of this model is 94 (0~9 + a~z + A~Z + punctuations).
+             The training datasets are MJsynth and SynthText.
+
+crnn_cs_CN.onnx:
+url: https://drive.google.com/uc?export=dowload&id=1is4eYEUKH7HR7Gl37Sw4WPXx6Ir8oQEG
+sha: 3940942b85761c7f240494cf662dcbf05dc00d14
+alphabet_3944.txt: https://drive.google.com/uc?export=dowload&id=18IZUUdNzJ44heWTndDO6NNfIpJMmN-ul
+parameter setting: -rgb=1;
+description: The classification number of this model is 3944 (0~9 + a~z + A~Z + Chinese characters + special characters).
+             The training dataset is ReCTS (https://rrc.cvc.uab.es/?ch=12).
+```
+
+More models can be found in [here](https://drive.google.com/drive/folders/1cTbQ3nuZG-EKWak6emD_s8_hHXWz7lAr?usp=sharing),
+which are taken from [clovaai](https://github.com/clovaai/deep-text-recognition-benchmark).
+You can train more models by [CRNN](https://github.com/meijieru/crnn.pytorch), and convert models by `torch.onnx.export`.
+
+#### TextDetectionModel:
+
+```
+- DB_IC15_resnet50.onnx:
+url: https://drive.google.com/uc?export=dowload&id=17_ABp79PlFt9yPCxSaarVc_DKTmrSGGf
+sha: bef233c28947ef6ec8c663d20a2b326302421fa3
+recommended parameter setting: -inputHeight=736, -inputWidth=1280;
+description: This model is trained on ICDAR2015, so it can only detect English text instances.
+
+- DB_IC15_resnet18.onnx:
+url: https://drive.google.com/uc?export=dowload&id=1sZszH3pEt8hliyBlTmB-iulxHP1dCQWV
+sha: 19543ce09b2efd35f49705c235cc46d0e22df30b
+recommended parameter setting: -inputHeight=736, -inputWidth=1280;
+description: This model is trained on ICDAR2015, so it can only detect English text instances.
+
+- DB_TD500_resnet50.onnx:
+url: https://drive.google.com/uc?export=dowload&id=19YWhArrNccaoSza0CfkXlA8im4-lAGsR
+sha: 1b4dd21a6baa5e3523156776970895bd3db6960a
+recommended parameter setting: -inputHeight=736, -inputWidth=736;
+description: This model is trained on MSRA-TD500, so it can detect both English and Chinese text instances.
+
+- DB_TD500_resnet18.onnx:
+url: https://drive.google.com/uc?export=dowload&id=1vY_KsDZZZb_svd5RT6pjyI8BS1nPbBSX
+sha: 8a3700bdc13e00336a815fc7afff5dcc1ce08546
+recommended parameter setting: -inputHeight=736, -inputWidth=736;
+description: This model is trained on MSRA-TD500, so it can detect both English and Chinese text instances.
+
+```
+
+We will release more models of DB [here](https://drive.google.com/drive/folders/1qzNCHfUJOS0NEUOIKn69eCtxdlNPpWbq?usp=sharing) in the future.
+
+```
+- EAST:
+Download link: https://www.dropbox.com/s/r2ingd0l3zt8hxs/frozen_east_text_detection.tar.gz?dl=1
+This model is based on https://github.com/argman/EAST
+```
+
+## Images for Testing
+
+```
+Text Recognition:
+url: https://drive.google.com/uc?export=dowload&id=1nMcEy68zDNpIlqAn6xCk_kYcUTIeSOtN
+sha: 89205612ce8dd2251effa16609342b69bff67ca3
+
+Text Detection:
+url: https://drive.google.com/uc?export=dowload&id=149tAhIcvfCYeyufRoZ9tmc2mZDKE_XrF
+sha: ced3c03fb7f8d9608169a913acf7e7b93e07109b
+```
+
+## Example for Text Recognition
+
+Step1. Loading images and models with a vocabulary
+
+```cpp
+    // Load a cropped text line image
+    // you can find cropped images for testing in "Images for Testing"
+    int rgb = IMREAD_COLOR; // This should be changed according to the model input requirement.
+    Mat image = imread("path/to/text_rec_test.png", rgb);
+
+    // Load models weights
+    TextRecognitionModel model("path/to/crnn_cs.onnx");
+
+    // The decoding method
+    // more methods will be supported in future
+    model.setDecodeType("CTC-greedy");
+
+    // Load vocabulary
+    // vocabulary should be changed according to the text recognition model
+    std::ifstream vocFile;
+    vocFile.open("path/to/alphabet_94.txt");
+    CV_Assert(vocFile.is_open());
+    String vocLine;
+    std::vector<String> vocabulary;
+    while (std::getline(vocFile, vocLine)) {
+        vocabulary.push_back(vocLine);
+    }
+    model.setVocabulary(vocabulary);
+```
+
+Step2. Setting Parameters
+
+```cpp
+    // Normalization parameters
+    double scale = 1.0 / 127.5;
+    Scalar mean = Scalar(127.5, 127.5, 127.5);
+
+    // The input shape
+    Size inputSize = Size(100, 32);
+
+    model.setInputParams(scale, inputSize, mean);
+```
+Step3. Inference
+```cpp
+    std::string recognitionResult = recognizer.recognize(image);
+    std::cout << "'" << recognitionResult << "'" << std::endl;
+```
+
+Input image:
+
+![Picture example](text_rec_test.png)
+
+Output:
+```
+'welcome'
+```
+
+
+## Example for Text Detection
+
+Step1. Loading images and models
+```cpp
+    // Load an image
+    // you can find some images for testing in "Images for Testing"
+    Mat frame = imread("/path/to/text_det_test.png");
+```
+
+Step2.a Setting Parameters (DB)
+```cpp
+    // Load model weights
+    TextDetectionModel_DB model("/path/to/DB_TD500_resnet50.onnx");
+
+    // Post-processing parameters
+    float binThresh = 0.3;
+    float polyThresh = 0.5;
+    uint maxCandidates = 200;
+    double unclipRatio = 2.0;
+    model.setBinaryThreshold(binThresh)
+         .setPolygonThreshold(polyThresh)
+         .setMaxCandidates(maxCandidates)
+         .setUnclipRatio(unclipRatio)
+    ;
+
+    // Normalization parameters
+    double scale = 1.0 / 255.0;
+    Scalar mean = Scalar(122.67891434, 116.66876762, 104.00698793);
+
+    // The input shape
+    Size inputSize = Size(736, 736);
+
+    model.setInputParams(scale, inputSize, mean);
+```
+
+Step2.b Setting Parameters (EAST)
+```cpp
+    TextDetectionModel_EAST model("EAST.pb");
+
+    float confThreshold = 0.5;
+    float nmsThreshold = 0.4;
+    model.setConfidenceThreshold(confThresh)
+         .setNMSThreshold(nmsThresh)
+    ;
+
+    double detScale = 1.0;
+    Size detInputSize = Size(320, 320);
+    Scalar detMean = Scalar(123.68, 116.78, 103.94);
+    bool swapRB = true;
+    model.setInputParams(detScale, detInputSize, detMean, swapRB);
+```
+
+
+Step3. Inference
+```cpp
+    std::vector<std::vector<Point>> detResults;
+    model.detect(detResults);
+
+    // Visualization
+    polylines(frame, results, true, Scalar(0, 255, 0), 2);
+    imshow("Text Detection", image);
+    waitKey();
+```
+
+Output:
+
+![Picture example](text_det_test_results.jpg)
+
+## Example for Text Spotting
+
+After following the steps above, it is easy to get the detection results of an input image.
+Then, you can do transformation and crop text images for recognition.
+For more information, please refer to **Detailed Sample**
+```cpp
+    // Transform and Crop
+    Mat cropped;
+    fourPointsTransform(recInput, vertices, cropped);
+
+    String recResult = recognizer.recognize(cropped);
+```
+
+Output Examples:
+
+![Picture example](detect_test1.jpg)
+
+![Picture example](detect_test2.jpg)
+
+## Source Code
+The [source code](https://github.com/opencv/opencv/blob/master/modules/dnn/src/model.cpp)
+of these APIs can be found in the DNN module.
+
+## Detailed Sample
+For more information, please refer to:
+- [samples/dnn/scene_text_recognition.cpp](https://github.com/opencv/opencv/blob/master/samples/dnn/scene_text_recognition.cpp)
+- [samples/dnn/scene_text_detection.cpp](https://github.com/opencv/opencv/blob/master/samples/dnn/scene_text_detection.cpp)
+- [samples/dnn/text_detection.cpp](https://github.com/opencv/opencv/blob/master/samples/dnn/text_detection.cpp)
+- [samples/dnn/scene_text_spotting.cpp](https://github.com/opencv/opencv/blob/master/samples/dnn/scene_text_spotting.cpp)
+
+#### Test with an image
+Examples:
+```bash
+example_dnn_scene_text_recognition -mp=path/to/crnn_cs.onnx -i=path/to/an/image -rgb=1 -vp=/path/to/alphabet_94.txt
+example_dnn_scene_text_detection -mp=path/to/DB_TD500_resnet50.onnx -i=path/to/an/image -ih=736 -iw=736
+example_dnn_scene_text_spotting -dmp=path/to/DB_IC15_resnet50.onnx -rmp=path/to/crnn_cs.onnx -i=path/to/an/image -iw=1280 -ih=736 -rgb=1 -vp=/path/to/alphabet_94.txt
+example_dnn_text_detection -dmp=path/to/EAST.pb -rmp=path/to/crnn_cs.onnx -i=path/to/an/image -rgb=1 -vp=path/to/alphabet_94.txt
+```
+
+#### Test on public datasets
+Text Recognition:
+
+The download link for testing images can be found in the **Images for Testing**
+
+
+Examples:
+```bash
+example_dnn_scene_text_recognition -mp=path/to/crnn.onnx -e=true -edp=path/to/evaluation_data_rec -vp=/path/to/alphabet_36.txt -rgb=0
+example_dnn_scene_text_recognition -mp=path/to/crnn_cs.onnx -e=true -edp=path/to/evaluation_data_rec -vp=/path/to/alphabet_94.txt -rgb=1
+```
+
+Text Detection:
+
+The download links for testing images can be found in the **Images for Testing**
+
+Examples:
+```bash
+example_dnn_scene_text_detection -mp=path/to/DB_TD500_resnet50.onnx -e=true -edp=path/to/evaluation_data_det/TD500 -ih=736 -iw=736
+example_dnn_scene_text_detection -mp=path/to/DB_IC15_resnet50.onnx -e=true -edp=path/to/evaluation_data_det/IC15 -ih=736 -iw=1280
+```
@@ -1,9 +1,16 @@
 YOLO DNNs  {#tutorial_dnn_yolo}
 ===============================

+@tableofcontents
+
@prev_tutorial{tutorial_dnn_android}
@next_tutorial{tutorial_dnn_javascript}

+|    |    |
+| -: | :- |
+| Original author | Alessandro de Oliveira Faria |
+| Compatibility | OpenCV >= 3.3.1 |
+
 Introduction
 ------------

@@ -2,81 +2,23 @@ Deep Neural Networks (dnn module) {#tutorial_table_of_content_dnn}
 =====================================

 -   @subpage tutorial_dnn_googlenet
-
-    *Languages:* C++
-
-    *Compatibility:* \> OpenCV 3.3
-
-    *Author:* Vitaliy Lyudvichenko
-
-    In this tutorial you will learn how to use opencv_dnn module for image classification by using GoogLeNet trained network from Caffe model zoo.
-
 -   @subpage tutorial_dnn_halide
-
-    *Languages:* Halide
-
-    *Compatibility:* \> OpenCV 3.3
-
-    *Author:* Dmitry Kurtaev
-
-    This tutorial guidelines how to run your models in OpenCV deep learning module using Halide language backend.
-
 -   @subpage tutorial_dnn_halide_scheduling
-
-    *Languages:* Halide
-
-    *Compatibility:* \> OpenCV 3.3
-
-    *Author:* Dmitry Kurtaev
-
-    In this tutorial we describe the ways to schedule your networks using Halide backend in OpenCV deep learning module.
-
 -   @subpage tutorial_dnn_android
-
-    *Languages:* Java
-
-    *Compatibility:* \> OpenCV 3.3
-
-    *Author:* Dmitry Kurtaev
-
-    This tutorial will show you how to run deep learning model using OpenCV on Android device.
-
 -   @subpage tutorial_dnn_yolo
-
-    *Languages:* C++, Python
-
-    *Compatibility:* \> OpenCV 3.3.1
-
-    *Author:* Alessandro de Oliveira Faria
-
-    In this tutorial you will learn how to use opencv_dnn module using yolo_object_detection with device capture, video file or image.
-
 -   @subpage tutorial_dnn_javascript
-
-    *Languages:* JavaScript
-
-    *Compatibility:* \> OpenCV 3.3.1
-
-    *Author:* Dmitry Kurtaev
-
-    In this tutorial we'll run deep learning models in browser using OpenCV.js.
-
 -   @subpage tutorial_dnn_custom_layers
-
-    *Languages:* C++, Python
-
-    *Compatibility:* \> OpenCV 3.4.1
-
-    *Author:* Dmitry Kurtaev
-
-    How to define custom layers to import networks.
-
 -   @subpage tutorial_dnn_OCR
+-   @subpage tutorial_dnn_text_spotting

-    *Languages:* C++
+#### PyTorch models with OpenCV
+In this section you will find the guides, which describe how to run classification, segmentation and detection PyTorch DNN models with OpenCV.
+-   @subpage pytorch_cls_tutorial_dnn_conversion
+-   @subpage pytorch_cls_c_tutorial_dnn_conversion
+-   @subpage pytorch_segm_tutorial_dnn_conversion

-    *Compatibility:* \> OpenCV 4.3
-
-    *Author:* Zihao Mu
-
-    In this tutorial you will learn how to use opencv_dnn module using custom OCR models.
+#### TensorFlow models with OpenCV
+In this section you will find the guides, which describe how to run classification, segmentation and detection TensorFlow DNN models with OpenCV.
+-   @subpage tf_cls_tutorial_dnn_conversion
+-   @subpage tf_det_tutorial_dnn_conversion
+-   @subpage tf_segm_tutorial_dnn_conversion