potok/opencv
1
0
Fork 0
mirror of https://github.com/opencv/opencv.git synced 2026-07-30 07:43:03 +04:00
Code Packages Releases Wiki Activity

Merge pull request #26056 from vpisarev:new_dnn_engine

Browse Source 
New dnn engine #26056

This is the 1st PR with the new engine; CI is green and PR is ready to be merged, I think.
Merge together with https://github.com/opencv/opencv_contrib/pull/3794

---

**Known limitations:**
* [solved] OpenVINO is temporarily disabled, but is probably easy to restore (it's not a deal breaker to merge this PR, I guess)
* The new engine does not support any backends nor any targets except for the default CPU implementation. But it's possible to choose the old engine when loading a model, then all the functionality is available.
* [Caffe patch is here: #26208] The new engine only supports ONNX. When a model is constructed manually or is loaded from a file of different format (.tf, .tflite, .caffe, .darknet), the old engine is used.
* Even in the case of ONNX some layers are not supported by the new engine, such as all quantized layers (including DequantizeLinear, QuantizeLinear, QLinearConv etc.), LSTM, GRU, .... It's planned, of course, to have full support for ONNX by OpenCV 5.0 gold release. When a loaded model contains unsupported layers, we switch to the old engine automatically  (at ONNX parsing time, not at `forward()` time).
* Some layers , e.g. Expat, are only partially supported by the new engine. In the case of unsupported flavours it switches to the old engine automatically (at ONNX parsing time, not at `forward()` time).
* 'Concat' graph optimization is disabled. The optimization eliminates Concat layer and instead makes the layers that generate tensors to be concatenated to write the outputs to the final destination. Of course, it's only possible when `axis=0` or `axis=N=1`. The optimization is not compatible with dynamic shapes since we need to know in advance where to store the tensors. Because some of the layer implementations have been modified to become more compatible with the new engine, the feature appears to be broken even when the old engine is used.
* Some `dnn::Net` API is not available with the new engine. Also, shape inference may return false if some of the output or intermediate tensors' shapes cannot be inferred without running the model. Probably this can be fixed by a dummy run of the model with zero inputs.
* Some overloads of `dnn::Net::getFLOPs()` and `dnn::Net::getMemoryConsumption()` are not exposed any longer in wrapper generators; but the most useful overloads are exposed (and checked by Java tests).
* [in progress] A few Einsum tests related to empty shapes have been disabled due to crashes in the tests and in Einsum implementations. The code and the tests need to be repaired.
* OpenCL implementation of Deconvolution is disabled. It's very bad and very slow anyway; need to be completely revised.
* Deconvolution3D test is now skipped, because it was only supported by CUDA and OpenVINO backends, both of which are not supported by the new engine.
* Some tests, such as FastNeuralStyle, checked that the in the case of CUDA backend there is no fallback to CPU. Currently all layers in the new engine are processed on CPU, so there are many fallbacks. The checks, therefore, have been temporarily disabled.

---

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
This commit is contained in:
Vadim Pisarevsky
2024-10-16 15:28:19 +03:00
committed by GitHub
parent 12738deaef
commit 3cd57ea09e
112 changed files with 11201 additions and 558 deletions
Download Patch File Download Diff File Expand all files Collapse all files
modules/dnn/include/opencv2/dnn/all_layers.hpp
+131
View File
@@ -86,6 +86,15 @@ CV__DNN_INLINE_NS_BEGIN
static Ptr<Layer> create(const LayerParams &params);
};
/**
* Constant layer produces the same data blob at an every forward pass.
*/
class CV_EXPORTS ConstantOfShapeLayer : public Layer
{
public:
static Ptr<ConstantOfShapeLayer> create(const LayerParams &params);
};
//! LSTM recurrent layer
class CV_EXPORTS LSTMLayer : public Layer
{
@@ -360,6 +369,15 @@ CV__DNN_INLINE_NS_BEGIN
static Ptr<GatherLayer> create(const LayerParams& params);
};
// ONNX-compliant implementation of Gather
class CV_EXPORTS Gather2Layer : public Layer
{
public:
int axis;
static Ptr<Gather2Layer> create(const LayerParams& params);
};
/** @brief GatherElements layer
* GatherElements takes two inputs data and indices of the same rank r >= 1 and an optional attribute axis and works such that:
* output[i][j][k] = data[index[i][j][k]][j][k] if axis = 0 and r = 3
@@ -462,6 +480,14 @@ CV__DNN_INLINE_NS_BEGIN
static Ptr<MVNLayer> create(const LayerParams& params);
};
class CV_EXPORTS ShapeLayer : public Layer
{
public:
int start, end;
static Ptr<ShapeLayer> create(const LayerParams& params);
};
/* Reshaping */
class CV_EXPORTS ReshapeLayer : public Layer
@@ -473,12 +499,42 @@ CV__DNN_INLINE_NS_BEGIN
static Ptr<ReshapeLayer> create(const LayerParams& params);
};
class CV_EXPORTS Reshape2Layer : public Layer
{
public:
MatShape newShapeDesc;
static Ptr<Reshape2Layer> create(const LayerParams& params);
};
class CV_EXPORTS FlattenLayer : public Layer
{
public:
static Ptr<FlattenLayer> create(const LayerParams &params);
};
class CV_EXPORTS SqueezeLayer : public Layer
{
public:
std::vector<int> axes;
static Ptr<SqueezeLayer> create(const LayerParams& params);
};
class CV_EXPORTS UnsqueezeLayer : public Layer
{
public:
std::vector<int> axes;
static Ptr<UnsqueezeLayer> create(const LayerParams& params);
};
class CV_EXPORTS RangeLayer : public Layer
{
public:
static Ptr<RangeLayer> create(const LayerParams& params);
};
class CV_EXPORTS QuantizeLayer : public Layer
{
public:
@@ -487,6 +543,17 @@ CV__DNN_INLINE_NS_BEGIN
static Ptr<QuantizeLayer> create(const LayerParams &params);
};
class CV_EXPORTS QuantizeLinearLayer : public Layer
{
public:
int axis;
int block_size;
int output_dtype;
bool saturate;
static Ptr<QuantizeLinearLayer> create(const LayerParams& params);
};
class CV_EXPORTS DequantizeLayer : public Layer
{
public:
@@ -495,6 +562,15 @@ CV__DNN_INLINE_NS_BEGIN
static Ptr<DequantizeLayer> create(const LayerParams &params);
};
class CV_EXPORTS DequantizeLinearLayer : public Layer
{
public:
int axis;
int block_size;
static Ptr<DequantizeLinearLayer> create(const LayerParams& params);
};
class CV_EXPORTS RequantizeLayer : public Layer
{
public:
@@ -518,6 +594,14 @@ CV__DNN_INLINE_NS_BEGIN
static Ptr<ConcatLayer> create(const LayerParams &params);
};
class CV_EXPORTS Concat2Layer : public Layer
{
public:
int axis;
static Ptr<Concat2Layer> create(const LayerParams &params);
};
class CV_EXPORTS SplitLayer : public Layer
{
public:
@@ -526,6 +610,16 @@ CV__DNN_INLINE_NS_BEGIN
static Ptr<SplitLayer> create(const LayerParams &params);
};
// ONNX-compliant version of Split
class CV_EXPORTS Split2Layer : public Layer
{
public:
int axis;
std::vector<int> split;
static Ptr<Split2Layer> create(const LayerParams& params);
};
/**
* Slice layer has several modes:
* 1. Caffe mode
@@ -567,12 +661,31 @@ CV__DNN_INLINE_NS_BEGIN
static Ptr<SliceLayer> create(const LayerParams &params);
};
// ONNX-compliant version of Slice
class CV_EXPORTS Slice2Layer : public Layer
{
public:
std::vector<int> starts, ends, axes;
static Ptr<Slice2Layer> create(const LayerParams &params);
};
class CV_EXPORTS PermuteLayer : public Layer
{
public:
static Ptr<PermuteLayer> create(const LayerParams& params);
};
// ONNX-compliant version of Transpose
// (previously implemented in PermuteLayer)
class CV_EXPORTS TransposeLayer : public Layer
{
public:
std::vector<int> perm;
static Ptr<TransposeLayer> create(const LayerParams& params);
};
/**
* Permute channels of 4-dimensional input blob.
* @param group Number of groups to split input channels and pick in turns
@@ -616,6 +729,12 @@ CV__DNN_INLINE_NS_BEGIN
static Ptr<PaddingLayer> create(const LayerParams& params);
};
class CV_EXPORTS Pad2Layer : public Layer
{
public:
static Ptr<Pad2Layer> create(const LayerParams& params);
};
/* Activations */
class CV_EXPORTS ActivationLayer : public Layer
{
@@ -1157,6 +1276,12 @@ CV__DNN_INLINE_NS_BEGIN
static Ptr<TileLayer> create(const LayerParams& params);
};
class CV_EXPORTS Tile2Layer : public Layer
{
public:
static Ptr<Tile2Layer> create(const LayerParams& params);
};
class CV_EXPORTS LayerNormLayer : public Layer
{
public:
@@ -1188,6 +1313,12 @@ CV__DNN_INLINE_NS_BEGIN
static Ptr<ExpandLayer> create(const LayerParams &params);
};
class CV_EXPORTS Expand2Layer : public Layer
{
public:
static Ptr<Expand2Layer> create(const LayerParams &params);
};
class CV_EXPORTS InstanceNormLayer : public Layer {
public:
float epsilon;
modules/dnn/include/opencv2/dnn/dict.hpp
+4
View File
@@ -138,6 +138,10 @@ public:
template <typename T>
T get(const String &key, const T &defaultValue) const;
//! If the @p key in the dictionary then returns its value, else returns empty vector.
template <typename T>
std::vector<T> getVector(const String &key) const;
//! Sets new @p value for the @p key, or adds new key-value pair into the dictionary.
template<typename T>
const T &set(const String &key, const T &value);
modules/dnn/include/opencv2/dnn/dnn.hpp
+253 -60
View File
@@ -42,6 +42,7 @@
#ifndef OPENCV_DNN_DNN_HPP
#define OPENCV_DNN_DNN_HPP
#include <ostream>
#include <vector>
#include <opencv2/core.hpp>
#include "opencv2/core/async.hpp"
@@ -61,7 +62,6 @@ CV__DNN_INLINE_NS_BEGIN
//! @addtogroup dnn
//! @{
typedef std::vector<int> MatShape;
typedef int MatType;
/**
@@ -107,21 +107,30 @@ CV__DNN_INLINE_NS_BEGIN
DNN_TARGET_CPU_FP16, // Only the ARM platform is supported. Low precision computing, accelerate model inference.
};
/**
* @brief Enum of data layout for model inference.
* @see Image2BlobParams
*/
enum DataLayout
enum TracingMode
{
DNN_LAYOUT_UNKNOWN = 0,
DNN_LAYOUT_ND = 1, //!< OpenCV data layout for 2D data.
DNN_LAYOUT_NCHW = 2, //!< OpenCV data layout for 4D data.
DNN_LAYOUT_NCDHW = 3, //!< OpenCV data layout for 5D data.
DNN_LAYOUT_NHWC = 4, //!< Tensorflow-like data layout for 4D data.
DNN_LAYOUT_NDHWC = 5, //!< Tensorflow-like data layout for 5D data.
DNN_LAYOUT_PLANAR = 6, //!< Tensorflow-like data layout, it should only be used at tf or tflite model parsing.
DNN_TRACE_NONE = 0, //!< Don't trace anything
DNN_TRACE_ALL = 1, //!< Print all executed operations along with the output tensors, more or less compatible with ONNX Runtime
DNN_TRACE_OP = 2 //!< Print all executed operations. Types and shapes of all inputs and outputs are printed, but the content is not.
};
enum ProfilingMode
{
DNN_PROFILE_NONE = 0, //!< Don't do any profiling
DNN_PROFILE_SUMMARY = 1, //!< Collect the summary statistics by layer type (e.g. all "Conv2D" or all "Add") and print it in the end, sorted by the execution time (most expensive layers first). Note that it may introduce some overhead and cause slowdown, especially in the case of non-CPU backends.
DNN_PROFILE_DETAILED = 2 //!< Print execution time of each single layer. Note that it may introduce some overhead and cause slowdown, especially in the case of non-CPU backends.
};
enum ModelFormat {
DNN_MODEL_GENERIC = 0, //!< Some generic model format
DNN_MODEL_ONNX = 1, //!< ONNX model
DNN_MODEL_TF = 2, //!< TF model
DNN_MODEL_TFLITE = 3, //!< TFLite model
DNN_MODEL_CAFFE = 4, //!< Caffe model
};
CV_EXPORTS std::string modelFormatToString(ModelFormat modelFormat);
CV_EXPORTS std::vector< std::pair<Backend, Target> > getAvailableBackends();
CV_EXPORTS_W std::vector<Target> getAvailableTargets(dnn::Backend be);
@@ -218,6 +227,40 @@ CV__DNN_INLINE_NS_BEGIN
int hostMatDepth = -1;
};
struct CV_EXPORTS Arg
{
Arg();
explicit Arg(int idx_);
bool empty() const;
operator bool() const;
// idx > 0: the Arg is input or output argument of some operation inside inference graph
// idx < 0: the Arg is input or output argument of a pattern
// idx == 0: no/empty argument; used in operations where some of the inputs/outputs are optional.
int idx;
};
enum ArgKind {
DNN_ARG_EMPTY=0, //!< valid only for Arg.idx==0. It's "no-arg"
DNN_ARG_CONST=1, //!< a constant argument.
DNN_ARG_INPUT=2, //!< input of the whole model. Before Net::forward() or in Net::forward() all inputs must be set
DNN_ARG_OUTPUT=3, //!< output of the model.
DNN_ARG_TEMP=4, //!< intermediate result, a result of some operation and input to some other operation(s).
DNN_ARG_PATTERN=5 //!< not used for now
};
CV_EXPORTS std::string argKindToString(ArgKind kind);
struct CV_EXPORTS ArgData
{
ArgData();
std::string name;
ArgKind kind;
MatShape shape;
int type;
};
class CV_EXPORTS Net;
class CV_EXPORTS Graph;
class CV_EXPORTS ActivationLayer;
/** @brief This interface class allows to build new Layers - are building blocks of networks.
@@ -231,6 +274,11 @@ CV__DNN_INLINE_NS_BEGIN
//! List of learned parameters must be stored here to allow read them by using Net::getParam().
CV_PROP_RW std::vector<Mat> blobs;
std::vector<Arg> inputs;
std::vector<Arg> outputs;
void* netimpl;
virtual std::vector<Ptr<Graph> >* subgraphs() const;
/** @brief Computes and sets internal parameters according to inputs, outputs and blobs.
* @deprecated Use Layer::finalize(InputArrayOfArrays, OutputArrayOfArrays) instead
@@ -413,10 +461,30 @@ CV__DNN_INLINE_NS_BEGIN
std::vector<MatType>&internals) const;
virtual int64 getFLOPS(const std::vector<MatShape> &inputs,
const std::vector<MatShape> &outputs) const {CV_UNUSED(inputs); CV_UNUSED(outputs); return 0;}
const std::vector<MatShape> &outputs) const;
virtual bool updateMemoryShapes(const std::vector<MatShape> &inputs);
// returns true if the operation takes a single input and can always be performed in-place,
// assuming that the input is contiguous.
// Examples of such operations are: Reshape, Flatten, Squeeze, Unsqueeze,
// as well many unary element-wise operations (ReLU, Tanh, ...)
virtual bool alwaysSupportInplace() const;
// returns false if the shape of Layer outputs is defined only by the shapes of inputs.
// Sometimes the shape depends on the content of the input(s), then the method should return true.
// In such a rare case forward() method should take care of proper allocation of the output tensors.
// On the other hand, when this method returns false, the engine takes care of proper allocation of the outputs,
// so that forward() can assume that the outputs are already allocated.
virtual bool dynamicOutputShapes() const;
// dumps attributes of the layer (e.g. strides, dilations in Convolution, MaxPool)
virtual std::ostream& dumpAttrs(std::ostream& strm, int indent) const;
// dumps information about the layer. The default implementation is usually good enough,
// just override dumpAttrs().
virtual std::ostream& dump(std::ostream& strm, int indent, bool comma) const;
CV_PROP String name; //!< Name of the layer instance, can be used for logging or other internal purposes.
CV_PROP String type; //!< Type name which was used for creating layer by layer factory.
CV_PROP int preferableTarget; //!< prefer target for layer forwarding
@@ -427,6 +495,32 @@ CV__DNN_INLINE_NS_BEGIN
virtual ~Layer();
};
/** @brief Represents graph or subgraph of a model.
* The graph (in mathematical terms it's rather a multigraph) is represented
* as a topologically-sorted linear sequence of operations.
* Each operation is a smart pointer to a Layer (some of its derivative class instance), which
* includes a list of inputs and outputs, as well as an optional list of subgraphs (e.g. 'If' contains 2 subgraphs).
*/
class CV_EXPORTS Graph
{
public:
static Ptr<Graph> create(void* netimpl, const std::string& name,
const std::vector<Arg>& inputs);
virtual ~Graph();
virtual bool empty() const = 0;
virtual void clear() = 0;
virtual std::string name() const = 0;
virtual const std::vector<Arg>& append(Ptr<Layer>& layer,
const std::vector<std::string>& outnames=std::vector<std::string>()) = 0;
virtual Arg append(Ptr<Layer>& layer, const std::string& outname=std::string()) = 0;
virtual std::ostream& dump(std::ostream& strm, int indent, bool comma) = 0;
virtual const std::vector<Arg>& inputs() const = 0;
virtual const std::vector<Arg>& outputs() const = 0;
virtual void setOutputs(const std::vector<Arg>& outputs) = 0;
virtual const std::vector<Ptr<Layer> >& prog() const = 0;
virtual void setProg(const std::vector<Ptr<Layer> >& newprog) = 0;
};
/** @brief This class allows to create and manipulate comprehensive artificial neural networks.
*
* Neural network is presented as directed acyclic graph (DAG), where vertices are Layer instances,
@@ -491,6 +585,10 @@ CV__DNN_INLINE_NS_BEGIN
* Call method after setInput(). To see correct backend, target and fusion run after forward().
*/
CV_WRAP void dumpToPbtxt(CV_WRAP_FILE_PATH const String& path);
/** @brief Dump net structure, hyperparameters, backend, target and fusion to the specified output stream
* @param strm the target stream
*/
void dumpToStream(std::ostream& strm) const;
/** @brief Adds new layer to the net.
* @param name unique name of the adding layer.
@@ -650,6 +748,33 @@ CV__DNN_INLINE_NS_BEGIN
*/
CV_WRAP void setPreferableTarget(int targetId);
/**
* @brief Set the tracing mode
* @param[in] tracingMode the tracing mode, see DNN_TRACE_*
*/
CV_WRAP void setTracingMode(TracingMode tracingMode);
/**
* @brief Retrieve the current tracing mode
*/
CV_WRAP TracingMode getTracingMode() const;
/**
* @brief Set the profiling mode
* @param[in] profilingMode the profiling mode, see DNN_PROFILE_*
*/
CV_WRAP void setProfilingMode(ProfilingMode profilingMode);
/**
* @brief Retrieve the current profiling mode
*/
CV_WRAP ProfilingMode getProfilingMode() const;
/**
* @brief Retrieve the current model format, see DNN_MODEL_*
*/
CV_WRAP ModelFormat getModelFormat() const;
/** @brief Sets the new input value for the network
* @param blob A new blob. Should have CV_32F or CV_8U depth.
* @param name A name of input layer.
@@ -703,20 +828,25 @@ CV__DNN_INLINE_NS_BEGIN
* @param inLayersShapes output parameter for input layers shapes;
* order is the same as in layersIds
* @param outLayersShapes output parameter for output layers shapes;
* order is the same as in layersIds
* order is the same as in layersIds.
*
* This overload should be deprecated
*/
CV_WRAP void getLayersShapes(const std::vector<MatShape>& netInputShapes,
const std::vector<int>& netInputTypes,
CV_OUT std::vector<int>& layersIds,
CV_OUT std::vector<std::vector<MatShape> >& inLayersShapes,
CV_OUT std::vector<std::vector<MatShape> >& outLayersShapes) const;
void getLayersShapes(const std::vector<MatShape>& netInputShapes,
const std::vector<int>& netInputTypes,
CV_OUT std::vector<int>& layersIds,
CV_OUT std::vector<std::vector<MatShape> >& inLayersShapes,
CV_OUT std::vector<std::vector<MatShape> >& outLayersShapes) const;
/** @overload */
CV_WRAP void getLayersShapes(const MatShape& netInputShape,
const int& netInputType,
CV_OUT std::vector<int>& layersIds,
CV_OUT std::vector<std::vector<MatShape> >& inLayersShapes,
CV_OUT std::vector<std::vector<MatShape> >& outLayersShapes) const;
/** @overload
*
* This overload should be deprecated
*/
void getLayersShapes(const MatShape& netInputShape,
const int& netInputType,
CV_OUT std::vector<int>& layersIds,
CV_OUT std::vector<std::vector<MatShape> >& inLayersShapes,
CV_OUT std::vector<std::vector<MatShape> >& outLayersShapes) const;
/** @brief Returns input and output shapes for layer with specified
* id in loaded model; preliminary inferencing isn't necessary.
@@ -727,15 +857,20 @@ CV__DNN_INLINE_NS_BEGIN
* order is the same as in layersIds
* @param outLayerShapes output parameter for output layers shapes;
* order is the same as in layersIds
*/
CV_WRAP void getLayerShapes(const MatShape& netInputShape,
const int& netInputType,
const int layerId,
CV_OUT std::vector<MatShape>& inLayerShapes,
CV_OUT std::vector<MatShape>& outLayerShapes) const; // FIXIT: CV_WRAP
*
* This overload should be deprecated
*/
void getLayerShapes(const MatShape& netInputShape,
const int& netInputType,
const int layerId,
CV_OUT std::vector<MatShape>& inLayerShapes,
CV_OUT std::vector<MatShape>& outLayerShapes) const; // FIXIT: CV_WRAP
/** @overload */
void getLayerShapes(const std::vector<MatShape>& netInputShapes,
/** @overload
*
* The only overload of getLayerShapes that should be kept in 5.x
*/
CV_WRAP void getLayerShapes(const std::vector<MatShape>& netInputShapes,
const std::vector<int>& netInputTypes,
const int layerId,
CV_OUT std::vector<MatShape>& inLayerShapes,
@@ -748,17 +883,19 @@ CV__DNN_INLINE_NS_BEGIN
*/
CV_WRAP int64 getFLOPS(const std::vector<MatShape>& netInputShapes,
const std::vector<int>& netInputTypes) const;
/** @overload
These overloads should be deprecated
*/
int64 getFLOPS(const MatShape& netInputShape,
const int& netInputType) const;
/** @overload */
CV_WRAP int64 getFLOPS(const MatShape& netInputShape,
const int& netInputType) const;
int64 getFLOPS(const int layerId,
const std::vector<MatShape>& netInputShapes,
const std::vector<int>& netInputTypes) const;
/** @overload */
CV_WRAP int64 getFLOPS(const int layerId,
const std::vector<MatShape>& netInputShapes,
const std::vector<int>& netInputTypes) const;
/** @overload */
CV_WRAP int64 getFLOPS(const int layerId,
const MatShape& netInputShape,
const int& netInputType) const;
int64 getFLOPS(const int layerId,
const MatShape& netInputShape,
const int& netInputType) const;
/** @brief Returns list of types for layer used in model.
* @param layersTypes output parameter for returning types.
@@ -778,20 +915,26 @@ CV__DNN_INLINE_NS_BEGIN
* @param weights output parameter to store resulting bytes for weights.
* @param blobs output parameter to store resulting bytes for intermediate blobs.
*/
void getMemoryConsumption(const std::vector<MatShape>& netInputShapes,
CV_WRAP void getMemoryConsumption(const std::vector<MatShape>& netInputShapes,
const std::vector<int>& netInputTypes,
CV_OUT size_t& weights, CV_OUT size_t& blobs) const; // FIXIT: CV_WRAP
/** @overload */
CV_WRAP void getMemoryConsumption(const MatShape& netInputShape,
CV_OUT size_t& weights, CV_OUT size_t& blobs) const;
/** @overload
It should be deprecated
*/
void getMemoryConsumption(const MatShape& netInputShape,
const int& netInputType,
CV_OUT size_t& weights, CV_OUT size_t& blobs) const;
/** @overload */
CV_WRAP void getMemoryConsumption(const int layerId,
/** @overload
It should be deprecated
*/
void getMemoryConsumption(const int layerId,
const std::vector<MatShape>& netInputShapes,
const std::vector<int>& netInputTypes,
CV_OUT size_t& weights, CV_OUT size_t& blobs) const;
/** @overload */
CV_WRAP void getMemoryConsumption(const int layerId,
/** @overload
It should be deprecated
*/
void getMemoryConsumption(const int layerId,
const MatShape& netInputShape,
const int& netInputType,
CV_OUT size_t& weights, CV_OUT size_t& blobs) const;
@@ -803,18 +946,23 @@ CV__DNN_INLINE_NS_BEGIN
* @param layerIds output vector to save layer IDs.
* @param weights output parameter to store resulting bytes for weights.
* @param blobs output parameter to store resulting bytes for intermediate blobs.
*/
*
* It should be deprecated
*/
void getMemoryConsumption(const std::vector<MatShape>& netInputShapes,
const std::vector<int>& netInputTypes,
CV_OUT std::vector<int>& layerIds,
CV_OUT std::vector<size_t>& weights,
CV_OUT std::vector<size_t>& blobs) const; // FIXIT: CV_WRAP
/** @overload */
CV_OUT std::vector<size_t>& blobs) const;
/** @overload
*
* It should be deprecated
*/
void getMemoryConsumption(const MatShape& netInputShape,
const int& netInputType,
CV_OUT std::vector<int>& layerIds,
CV_OUT std::vector<size_t>& weights,
CV_OUT std::vector<size_t>& blobs) const; // FIXIT: CV_WRAP
CV_OUT std::vector<size_t>& blobs) const;
/** @brief Enables or disables layer fusion in the network.
* @param fusion true to enable the fusion, false to disable. The fusion is enabled by default.
@@ -837,6 +985,28 @@ CV__DNN_INLINE_NS_BEGIN
*/
CV_WRAP int64 getPerfProfile(CV_OUT std::vector<double>& timings);
// Get the main model graph
Ptr<Graph> getMainGraph() const;
const ArgData& argData(Arg arg) const;
const std::string& argName(Arg arg) const;
ArgKind argKind(Arg arg) const;
// if the name is empty, always creates a new argument;
// if it's not empty, returns argument with the specific name if it already exists,
// otherwise creates new argument with the specified name
Arg getArg(const std::string& name);
bool haveArg(const std::string& name) const;
bool isConstArg(Arg arg) const;
Mat& argTensor(Arg arg) const;
int argType(Arg arg) const;
int findDim(const std::string& name, bool insert=false);
std::ostream& dumpArg(std::ostream& strm, Arg arg, int indent,
bool comma=true, bool dump_details=false) const;
std::ostream& dumpDim(std::ostream& strm, int value) const;
struct Impl;
inline Impl* getImpl() const { return impl.get(); }
@@ -846,6 +1016,13 @@ CV__DNN_INLINE_NS_BEGIN
Ptr<Impl> impl;
};
enum EngineType
{
ENGINE_CLASSIC=1, //!< Force use the new dnn engine. The engine does not support non CPU back-ends for now.
ENGINE_NEW=2, //!< Force use the old dnn engine similar to 4.x branch
ENGINE_AUTO=3 //!< Try to use the new engine and then fall back to the classic version.
};
/** @brief Reads a network model stored in <a href="https://pjreddie.com/darknet/">Darknet</a> model files.
* @param cfgFile path to the .cfg file with text description of the network architecture.
* @param darknetModel path to the .weights file with learned network.
@@ -962,6 +1139,9 @@ CV__DNN_INLINE_NS_BEGIN
* * `*.cfg` (Darknet, https://pjreddie.com/darknet/)
* * `*.xml` (OpenVINO, https://software.intel.com/openvino-toolkit)
* @param[in] framework Explicit framework name tag to determine a format.
* @param[in] engine select DNN engine to be used. With auto selection the new engine is used first and falls back to classic.
* Please pay attention that the new DNN does not support non-CPU back-ends for now.
* Use ENGINE_CLASSIC if you want to use other back-ends.
* @returns Net object.
*
* This function automatically detects an origin framework of trained model
@@ -969,7 +1149,10 @@ CV__DNN_INLINE_NS_BEGIN
* or @ref readNetFromDarknet. An order of @p model and @p config
* arguments does not matter.
*/
CV_EXPORTS_W Net readNet(CV_WRAP_FILE_PATH const String& model, CV_WRAP_FILE_PATH const String& config = "", const String& framework = "");
CV_EXPORTS_W Net readNet(CV_WRAP_FILE_PATH const String& model,
CV_WRAP_FILE_PATH const String& config = "",
const String& framework = "",
int engine = ENGINE_AUTO);
/**
* @brief Read deep learning network represented in one of the supported formats.
@@ -978,10 +1161,14 @@ CV__DNN_INLINE_NS_BEGIN
* @param[in] framework Name of origin framework.
* @param[in] bufferModel A buffer with a content of binary file with weights
* @param[in] bufferConfig A buffer with a content of text file contains network configuration.
* @param engine select DNN engine to be used. With auto selection the new engine is used first and falls back to classic.
* Please pay attention that the new DNN does not support non-CPU back-ends for now.
* Use ENGINE_CLASSIC if you want to use other back-ends.
* @returns Net object.
*/
CV_EXPORTS_W Net readNet(const String& framework, const std::vector<uchar>& bufferModel,
const std::vector<uchar>& bufferConfig = std::vector<uchar>());
const std::vector<uchar>& bufferConfig = std::vector<uchar>(),
int engine = ENGINE_AUTO);
/** @brief Load a network from Intel's Model Optimizer intermediate representation.
* @param[in] xml XML configuration file with network's topology.
@@ -1016,28 +1203,34 @@ CV__DNN_INLINE_NS_BEGIN
Net readNetFromModelOptimizer(const uchar* bufferModelConfigPtr, size_t bufferModelConfigSize,
const uchar* bufferWeightsPtr, size_t bufferWeightsSize);
/** @brief Reads a network model <a href="https://onnx.ai/">ONNX</a>.
* @param onnxFile path to the .onnx file with text description of the network architecture.
* @param engine select DNN engine to be used. With auto selection the new engine is used first and falls back to classic.
* Please pay attention that the new DNN does not support non-CPU back-ends for now.
* @returns Network object that ready to do forward, throw an exception in failure cases.
*/
CV_EXPORTS_W Net readNetFromONNX(CV_WRAP_FILE_PATH const String &onnxFile);
CV_EXPORTS_W Net readNetFromONNX(CV_WRAP_FILE_PATH const String &onnxFile, int engine=ENGINE_AUTO);
/** @brief Reads a network model from <a href="https://onnx.ai/">ONNX</a>
* in-memory buffer.
* @param buffer memory address of the first byte of the buffer.
* @param sizeBuffer size of the buffer.
* @param engine select DNN engine to be used. With auto selection the new engine is used first and falls back to classic.
* @returns Network object that ready to do forward, throw an exception
* in failure cases.
*/
CV_EXPORTS Net readNetFromONNX(const char* buffer, size_t sizeBuffer);
CV_EXPORTS Net readNetFromONNX(const char* buffer, size_t sizeBuffer, int engine=ENGINE_AUTO);
/** @brief Reads a network model from <a href="https://onnx.ai/">ONNX</a>
* in-memory buffer.
* @param buffer in-memory buffer that stores the ONNX model bytes.
* @param engine select DNN engine to be used. With auto selection the new engine is used first and falls back to classic.
* Please pay attention that the new DNN does not support non-CPU back-ends for now.
* @returns Network object that ready to do forward, throw an exception
* in failure cases.
*/
CV_EXPORTS_W Net readNetFromONNX(const std::vector<uchar>& buffer);
CV_EXPORTS_W Net readNetFromONNX(const std::vector<uchar>& buffer, int engine=ENGINE_AUTO);
/** @brief Creates blob from .pb file.
* @param path to the .pb file with input tensor.
modules/dnn/include/opencv2/dnn/dnn.inl.hpp
+37 -2
View File
@@ -116,12 +116,24 @@ inline int64 DictValue::get<int64>(int idx) const
template<>
inline int DictValue::get<int>(int idx) const
{
return (int)get<int64>(idx);
return saturate_cast<int>(get<int64>(idx));
}
inline int DictValue::getIntValue(int idx) const
{
return (int)get<int64>(idx);
return saturate_cast<int>(get<int64>(idx));
}
template<>
inline std::vector<int> DictValue::get<std::vector<int> >(int idx) const
{
CV_Assert(idx == -1);
int size_ = size();
std::vector<int> values(size_);
for (int i = 0; i < size_; i++)
values[i] = get<int>(i);
return values;
}
template<>
@@ -368,6 +380,17 @@ inline T Dict::get(const String &key, const T &defaultValue) const
return defaultValue;
}
template <typename T>
inline std::vector<T> Dict::getVector(const String &key) const
{
_Dict::const_iterator i = dict.find(key);
if (i != dict.end())
return i->second.get<std::vector<T> >();
else
return std::vector<T>();
}
template<typename T>
inline const T &Dict::set(const String &key, const T &value)
{
@@ -405,6 +428,18 @@ inline std::map<String, DictValue>::const_iterator Dict::end() const
return dict.end();
}
/////////////////////////////////////////////////////////////////
inline Arg::Arg() : idx(0) {}
inline Arg::Arg(int idx_) : idx(idx_) {}
inline bool Arg::empty() const { return idx == 0; }
inline Arg::operator bool() const { return idx != 0; }
inline bool operator == (const Arg& a, const Arg& b) { return a.idx == b.idx; }
CV__DNN_INLINE_NS_END
}
}
modules/dnn/include/opencv2/dnn/shape_utils.hpp
+39 -18
View File
@@ -125,17 +125,12 @@ static inline MatShape shape(const int* dims, const int n)
static inline MatShape shape(const Mat& mat)
{
return shape(mat.size.p, mat.dims);
}
static inline MatShape shape(const MatSize& sz)
{
return shape(sz.p, sz.dims());
return mat.shape();
}
static inline MatShape shape(const UMat& mat)
{
return shape(mat.size.p, mat.dims);
return mat.shape();
}
#if 0 // issues with MatExpr wrapped into InputArray
@@ -152,10 +147,9 @@ namespace {inline bool is_neg(int i) { return i < 0; }}
static inline MatShape shape(int a0, int a1=-1, int a2=-1, int a3=-1)
{
int dims[] = {a0, a1, a2, a3};
MatShape s = shape(dims, 4);
s.erase(std::remove_if(s.begin(), s.end(), is_neg), s.end());
return s;
int shape_[] = {a0, a1, a2, a3};
int dims = 1 + (a1 >= 0) + (a1 >= 0 && a2 >= 0) + (a1 >= 0 && a2 >= 0 && a3 >= 0);
return shape(shape_, dims);
}
static inline int total(const MatShape& shape, int start = -1, int end = -1)
@@ -206,11 +200,41 @@ static inline int total(const Mat& mat, int start = -1, int end = -1)
static inline MatShape concat(const MatShape& a, const MatShape& b)
{
MatShape c = a;
c.insert(c.end(), b.begin(), b.end());
size_t a_size = a.size(), b_size = b.size(), c_size = a_size + b_size;
c.resize(c_size);
for (size_t i = 0; i < b_size; i++) {
c[i+a_size] = b[i];
}
return c;
}
static inline std::ostream& operator << (std::ostream& strm, const MatShape& shape)
{
strm << '[';
if (shape.empty()) {
strm << "<empty>";
} else {
size_t n = shape.size();
if (n == 0) {
strm << "<scalar>";
} else {
for(size_t i = 0; i < n; ++i)
strm << (i > 0 ? " x " : "") << shape[i];
}
}
strm << "]";
return strm;
}
static inline std::string toString(const MatShape& shape, const String& name = "")
{
std::ostringstream ss;
if (!name.empty())
ss << name << ' ';
ss << shape;
return ss.str();
}
template<typename _Tp>
static inline std::string toString(const std::vector<_Tp>& shape, const String& name = "")
{
@@ -269,14 +293,11 @@ Range normalize_axis_range(const Range& r, int axisSize)
static inline
bool isAllOnes(const MatShape &inputShape, int startPos, int endPos)
{
CV_Assert(!inputShape.empty());
CV_CheckGE((int) inputShape.size(), startPos, "");
CV_CheckGE(startPos, 0, "");
CV_CheckLE(startPos, endPos, "");
CV_CheckLE((size_t)endPos, inputShape.size(), "");
CV_CheckLE(endPos, inputShape.dims, "");
for (size_t i = startPos; i < endPos; i++)
for (int i = startPos; i < endPos; i++)
{
if (inputShape[i] != 1)
return false;