Merge pull request #17020 from dbudniko:dbudniko/serialization_backend

G-API Serialization routines

* Serialization backend in tests, initial version

* S11N/00: A Great Rename

- "Serialization" is too long and too error-prone to type,
  so now it is renamed to "s11n" everywhere;
- Same applies to "SRLZ";
- Tests also renamed to start with 'S11N.*' (easier to run);
- Also updated copyright years in new files to 2020.

* S11N/01: Some basic interface segregation

- Moved some details (low-level functions) out of serialization.hpp;
- Introduced I::IStream and I::OStream interfaces;
- Implemented those via the existing [De]SerializationStream classes;
- Moved all operators to use interfaces instead of classes;
- Moved the htonl/ntohl handling out of operators (to the classes).

The implementation didn't change much, it is a subject to the further
refactoring

* S11N/02: Basic operator reorg, basic tests, vector support

- Reorganized operators on atomic types to follow >>/<< model
  (put them closer in the code for the respective types);
- Introduce more operators for basic (scalar) types;
- Drop all vector s11n overloads -- replace with a generic
  (template-based) one;
- Introduced a new test suite where low-level s11n functionality
  is tested (for the basic types).

* S11N/03: Operators reorganization

- Sorted the Opaque types enum by complexity;
- Reorganized the existing operators for basic types, also ordered by
  complexity;
- Organized operators in three groups (Basics, OpenCV, G-API);
- Added a generic serialization for variant<>;
- Reimplemented some of the existing operators (for OpenCV and G-API
  data structures);
- Introduced new operators for cv::gimpl data types. These operators
  (and so, the data structures) are not yet used in the graph
  dump/reconstruction routine, it will be done as a next step.

* S11N/04: The Great Clean-up

- Drop the duplicates of GModel data structures from the
  serialization, serialize the GModel data structures themselve
  instead (hand-written code replaced with operators).
- Also removed usuned code for printing, etc.

* S11N/05: Internal API Clean-up

- Minimize the serialization API to just Streams and Operators;
- Refactor and fix the graph serialization (deconstruction and
  reconstruction) routines, fix data addressing problems there;
- Move the serialization.[ch]pp files to the core G-API library

* S11N/06: Top-level API introduction

- !!!This is likely the most invasive commit in the series!!!
- Introduced a top-level API to serialize and deserialize a GComputation
- Extended the compiler to support both forms of a GComputation:
  an expession based and a deserialized one. This has led to changes in
  the cv::GComputation::Priv and in its dependent components (even the
  transformation tests);
- Had to extend the kernel API (GKernel) with extra information on
  operations (mainly `outMeta`) which was only available for expression
  based graphs. Now the `outMeta` can be taken from kernels too (and for
  the deserialized graphs it is the only way);
- Revisited the internal serialization API, had to expose previously
  hidden entities (like `GSerialized`);
- Extended the serialized graph info with new details (object counter,
  protocol). Added unordered_map generic serialization for that;
- Reworked the very first pipeline test to be "proper"; GREEN now, the rest
  is to be reworked in the next iteration.

* S11N/07: Tests reworked

- Moved the sample pipeline tests w/serialization to
  test the public API (`cv::gapi::serialize`, then
  followed by `cv::gapi::deserialize<>`). All GREEN.
- As a consequence, dropped the "Serialization" test
  backend as no longer necessary.

* S11N/08: Final touches

- Exposed the C++ native data types at Streams level;
- Switched the ByteMemoryIn/OutStreams to store data in `char`
  internally (2x less memory for sample pipelines);
- Fixed and refactored Mat dumping to the stream;
- Renamed S11N pipeline tests to their new meaning.

* linux build fix

* fix RcDesc and int uint warnings

* more Linux build fix

* white space and virtual android error fix (attempt)

* more warnings to be fixed

* android warnings fix attempt

* one more attempt for android build fix

* android warnings one more fix

* return back override

* avoid size_t

* static deserialize

* and how do you like this, elon? anonymous namespace  to fix android warning.

* static inline

* trying to fix standalone build

* mat dims fix

* fix mat r/w for standalone

Co-authored-by: Dmitry Matveev <dmitry.matveev@intel.com>

modules/gapi/include/opencv2/gapi/cpu/gcpukernel.hpp

@@ -479,9 +479,10 @@ class gapi::cpu::GOCVFunctor : public gapi::GFunctor
 {
 public:
     using Impl = std::function<void(GCPUContext &)>;
+    using Meta = cv::GKernel::M;
 
-    GOCVFunctor(const char* id, const Impl& impl)
-        : gapi::GFunctor(id), impl_{GCPUKernel(impl)}
+    GOCVFunctor(const char* id, const Meta &meta, const Impl& impl)
+        : gapi::GFunctor(id), impl_{GCPUKernel(impl), meta}
     {
     }
 
@@ -497,14 +498,20 @@ template<typename K, typename Callable>
 gapi::cpu::GOCVFunctor gapi::cpu::ocv_kernel(Callable& c)
 {
     using P = detail::OCVCallHelper<Callable, typename K::InArgs, typename K::OutArgs>;
-    return GOCVFunctor(K::id(), std::bind(&P::callFunctor, std::placeholders::_1, std::ref(c)));
+    return GOCVFunctor{ K::id()
+                      , &K::getOutMeta
+                      , std::bind(&P::callFunctor, std::placeholders::_1, std::ref(c))
+                      };
 }
 
 template<typename K, typename Callable>
 gapi::cpu::GOCVFunctor gapi::cpu::ocv_kernel(const Callable& c)
 {
     using P = detail::OCVCallHelper<Callable, typename K::InArgs, typename K::OutArgs>;
-    return GOCVFunctor(K::id(), std::bind(&P::callFunctor, std::placeholders::_1, c));
+    return GOCVFunctor{ K::id()
+                      , &K::getOutMeta
+                      , std::bind(&P::callFunctor, std::placeholders::_1, c)
+                      };
 }
 //! @endcond
 

modules/gapi/include/opencv2/gapi/garg.hpp

@@ -46,6 +46,7 @@ public:
     template<typename T, typename std::enable_if<!detail::is_garg<T>::value, int>::type = 0>
     explicit GArg(const T &t)
         : kind(detail::GTypeTraits<T>::kind)
+        , opaque_kind(detail::GOpaqueTraits<T>::kind)
         , value(detail::wrap_gapi_helper<T>::wrap(t))
     {
     }
@@ -53,6 +54,7 @@ public:
     template<typename T, typename std::enable_if<!detail::is_garg<T>::value, int>::type = 0>
     explicit GArg(T &&t)
         : kind(detail::GTypeTraits<typename std::decay<T>::type>::kind)
+        , opaque_kind(detail::GOpaqueTraits<typename std::decay<T>::type>::kind)
         , value(detail::wrap_gapi_helper<T>::wrap(t))
     {
     }
@@ -78,6 +80,7 @@ public:
     }
 
     detail::ArgKind kind = detail::ArgKind::OPAQUE_VAL;
+    detail::OpaqueKind opaque_kind = detail::OpaqueKind::CV_UNKNOWN;
 
 protected:
     util::any value;

modules/gapi/include/opencv2/gapi/gcomputation.hpp

@@ -36,6 +36,16 @@ namespace detail
     using last_type_t = typename last_type<Ts...>::type;
 }
 
+// Forward-declare the serialization objects
+namespace gimpl {
+namespace s11n {
+namespace I {
+    struct IStream;
+    struct OStream;
+} // namespace I
+} // namespace s11n
+} // namespace gimpl
+
 /**
  * \addtogroup gapi_main_classes
  * @{
@@ -495,6 +505,10 @@ public:
     Priv& priv();
     /// @private
     const Priv& priv() const;
+    /// @private
+    explicit GComputation(cv::gimpl::s11n::I::IStream &);
+    /// @private
+    void serialize(cv::gimpl::s11n::I::OStream &) const;
 
 protected:
 

modules/gapi/include/opencv2/gapi/gkernel.hpp

@@ -2,7 +2,7 @@
 // It is subject to the license terms in the LICENSE file found in the top-level directory
 // of this distribution and at http://opencv.org/license.html.
 //
-// Copyright (C) 2018-2019 Intel Corporation
+// Copyright (C) 2018-2020 Intel Corporation
 
 
 #ifndef OPENCV_GAPI_GKERNEL_HPP
@@ -45,6 +45,7 @@ struct GAPI_EXPORTS GKernel
 struct GAPI_EXPORTS GKernelImpl
 {
     util::any         opaque;    // backend-specific opaque info
+    GKernel::M        outMeta;   // for deserialized graphs, the outMeta is taken here
 };
 
 template<typename, typename> class GKernelTypeM;
@@ -456,12 +457,12 @@ namespace gapi {
         /// @private
         // Partial include() specialization for kernels
         template <typename KImpl>
-        typename std::enable_if<(std::is_base_of<detail::KernelTag, KImpl>::value), void>::type
+        typename std::enable_if<(std::is_base_of<cv::detail::KernelTag, KImpl>::value), void>::type
         includeHelper()
         {
             auto backend     = KImpl::backend();
             auto kernel_id   = KImpl::API::id();
-            auto kernel_impl = GKernelImpl{KImpl::kernel()};
+            auto kernel_impl = GKernelImpl{KImpl::kernel(), &KImpl::API::getOutMeta};
             removeAPI(kernel_id);
 
             m_id_kernels[kernel_id] = std::make_pair(backend, kernel_impl);
@@ -470,7 +471,7 @@ namespace gapi {
         /// @private
         // Partial include() specialization for transformations
         template <typename TImpl>
-        typename std::enable_if<(std::is_base_of<detail::TransformTag, TImpl>::value), void>::type
+        typename std::enable_if<(std::is_base_of<cv::detail::TransformTag, TImpl>::value), void>::type
         includeHelper()
         {
             m_transformations.emplace_back(TImpl::transformation());
@@ -509,7 +510,7 @@ namespace gapi {
         template<typename KImpl>
         bool includes() const
         {
-            static_assert(std::is_base_of<detail::KernelTag, KImpl>::value,
+            static_assert(std::is_base_of<cv::detail::KernelTag, KImpl>::value,
                           "includes() can be applied to kernels only");
 
             auto kernel_it = m_id_kernels.find(KImpl::API::id());
@@ -624,7 +625,7 @@ namespace gapi {
     {
         // FIXME: currently there is no check that transformations' signatures are unique
         // and won't be any intersection in graph compilation stage
-        static_assert(detail::all_unique<typename KK::API...>::value, "Kernels API must be unique");
+        static_assert(cv::detail::all_unique<typename KK::API...>::value, "Kernels API must be unique");
 
         GKernelPackage pkg;
 

modules/gapi/include/opencv2/gapi/gtype_traits.hpp

@@ -41,6 +41,30 @@ namespace detail
         GOPAQUE,      // a cv::GOpaqueU (note - exactly GOpaqueU, not GOpaque<T>!)
     };
 
+    enum class OpaqueKind: int
+    {
+        CV_UNKNOWN,    // Unknown, generic, opaque-to-GAPI data type unsupported in graph seriallization
+        CV_BOOL,       // bool user G-API data
+        CV_INT,        // int user G-API data
+        CV_DOUBLE,     // double user G-API data
+        CV_POINT,      // cv::Point user G-API data
+        CV_SIZE,       // cv::Size user G-API data
+        CV_RECT,       // cv::Rect user G-API data
+        CV_SCALAR,     // cv::Scalar user G-API data
+        CV_MAT,        // cv::Mat user G-API data
+    };
+
+    template<typename T> struct GOpaqueTraits;
+    template<typename T> struct GOpaqueTraits    { static constexpr const OpaqueKind kind = OpaqueKind::CV_UNKNOWN; };
+    template<> struct GOpaqueTraits<int>         { static constexpr const OpaqueKind kind = OpaqueKind::CV_INT; };
+    template<> struct GOpaqueTraits<double>      { static constexpr const OpaqueKind kind = OpaqueKind::CV_DOUBLE; };
+    template<> struct GOpaqueTraits<cv::Size>    { static constexpr const OpaqueKind kind = OpaqueKind::CV_SIZE; };
+    template<> struct GOpaqueTraits<bool>        { static constexpr const OpaqueKind kind = OpaqueKind::CV_BOOL; };
+    template<> struct GOpaqueTraits<cv::Scalar>  { static constexpr const OpaqueKind kind = OpaqueKind::CV_SCALAR; };
+    template<> struct GOpaqueTraits<cv::Point>   { static constexpr const OpaqueKind kind = OpaqueKind::CV_POINT; };
+    template<> struct GOpaqueTraits<cv::Mat>     { static constexpr const OpaqueKind kind = OpaqueKind::CV_MAT; };
+    template<> struct GOpaqueTraits<cv::Rect>    { static constexpr const OpaqueKind kind = OpaqueKind::CV_RECT; };
+
     // Describe G-API types (G-types) with traits.  Mostly used by
     // cv::GArg to store meta information about types passed into
     // operation arguments. Please note that cv::GComputation is

modules/gapi/include/opencv2/gapi/s11n.hpp

@@ -0,0 +1,38 @@
+// This file is part of OpenCV project.
+// It is subject to the license terms in the LICENSE file found in the top-level directory
+// of this distribution and at http://opencv.org/license.html.
+//
+// Copyright (C) 2020 Intel Corporation
+
+#ifndef OPENCV_GAPI_S11N_HPP
+#define OPENCV_GAPI_S11N_HPP
+
+#include <vector>
+#include <opencv2/gapi/gcomputation.hpp>
+
+namespace cv {
+namespace gapi {
+
+namespace detail {
+    GAPI_EXPORTS cv::GComputation getGraph(const std::vector<char> &p);
+} // namespace detail
+
+GAPI_EXPORTS std::vector<char> serialize(const cv::GComputation &c);
+//namespace{
+
+template<typename T> static inline
+T deserialize(const std::vector<char> &p);
+
+//} //ananymous namespace
+
+template<> inline
+cv::GComputation deserialize(const std::vector<char> &p) {
+    return detail::getGraph(p);
+}
+
+
+
+} // namespace gapi
+} // namespace cv
+
+#endif // OPENCV_GAPI_S11N_HPP