1
0
mirror of https://github.com/opencv/opencv.git synced 2026-07-31 00:03:03 +04:00

Merge pull request #17020 from dbudniko:dbudniko/serialization_backend

G-API Serialization routines

* Serialization backend in tests, initial version

* S11N/00: A Great Rename

- "Serialization" is too long and too error-prone to type,
  so now it is renamed to "s11n" everywhere;
- Same applies to "SRLZ";
- Tests also renamed to start with 'S11N.*' (easier to run);
- Also updated copyright years in new files to 2020.

* S11N/01: Some basic interface segregation

- Moved some details (low-level functions) out of serialization.hpp;
- Introduced I::IStream and I::OStream interfaces;
- Implemented those via the existing [De]SerializationStream classes;
- Moved all operators to use interfaces instead of classes;
- Moved the htonl/ntohl handling out of operators (to the classes).

The implementation didn't change much, it is a subject to the further
refactoring

* S11N/02: Basic operator reorg, basic tests, vector support

- Reorganized operators on atomic types to follow >>/<< model
  (put them closer in the code for the respective types);
- Introduce more operators for basic (scalar) types;
- Drop all vector s11n overloads -- replace with a generic
  (template-based) one;
- Introduced a new test suite where low-level s11n functionality
  is tested (for the basic types).

* S11N/03: Operators reorganization

- Sorted the Opaque types enum by complexity;
- Reorganized the existing operators for basic types, also ordered by
  complexity;
- Organized operators in three groups (Basics, OpenCV, G-API);
- Added a generic serialization for variant<>;
- Reimplemented some of the existing operators (for OpenCV and G-API
  data structures);
- Introduced new operators for cv::gimpl data types. These operators
  (and so, the data structures) are not yet used in the graph
  dump/reconstruction routine, it will be done as a next step.

* S11N/04: The Great Clean-up

- Drop the duplicates of GModel data structures from the
  serialization, serialize the GModel data structures themselve
  instead (hand-written code replaced with operators).
- Also removed usuned code for printing, etc.

* S11N/05: Internal API Clean-up

- Minimize the serialization API to just Streams and Operators;
- Refactor and fix the graph serialization (deconstruction and
  reconstruction) routines, fix data addressing problems there;
- Move the serialization.[ch]pp files to the core G-API library

* S11N/06: Top-level API introduction

- !!!This is likely the most invasive commit in the series!!!
- Introduced a top-level API to serialize and deserialize a GComputation
- Extended the compiler to support both forms of a GComputation:
  an expession based and a deserialized one. This has led to changes in
  the cv::GComputation::Priv and in its dependent components (even the
  transformation tests);
- Had to extend the kernel API (GKernel) with extra information on
  operations (mainly `outMeta`) which was only available for expression
  based graphs. Now the `outMeta` can be taken from kernels too (and for
  the deserialized graphs it is the only way);
- Revisited the internal serialization API, had to expose previously
  hidden entities (like `GSerialized`);
- Extended the serialized graph info with new details (object counter,
  protocol). Added unordered_map generic serialization for that;
- Reworked the very first pipeline test to be "proper"; GREEN now, the rest
  is to be reworked in the next iteration.

* S11N/07: Tests reworked

- Moved the sample pipeline tests w/serialization to
  test the public API (`cv::gapi::serialize`, then
  followed by `cv::gapi::deserialize<>`). All GREEN.
- As a consequence, dropped the "Serialization" test
  backend as no longer necessary.

* S11N/08: Final touches

- Exposed the C++ native data types at Streams level;
- Switched the ByteMemoryIn/OutStreams to store data in `char`
  internally (2x less memory for sample pipelines);
- Fixed and refactored Mat dumping to the stream;
- Renamed S11N pipeline tests to their new meaning.

* linux build fix

* fix RcDesc and int uint warnings

* more Linux build fix

* white space and virtual android error fix (attempt)

* more warnings to be fixed

* android warnings fix attempt

* one more attempt for android build fix

* android warnings one more fix

* return back override

* avoid size_t

* static deserialize

* and how do you like this, elon? anonymous namespace  to fix android warning.

* static inline

* trying to fix standalone build

* mat dims fix

* fix mat r/w for standalone

Co-authored-by: Dmitry Matveev <dmitry.matveev@intel.com>
This commit is contained in:
Dmitry Budnikov
2020-06-26 22:41:29 +03:00
committed by GitHub
parent 085bd5f55f
commit 7566921364
20 changed files with 1592 additions and 53 deletions
@@ -0,0 +1,611 @@
// This file is part of OpenCV project.
// It is subject to the license terms in the LICENSE file found in the top-level directory
// of this distribution and at http://opencv.org/license.html.
//
// Copyright (C) 2020 Intel Corporation
#include <set> // set
#include <map> // map
#include <ade/util/zip_range.hpp> // indexed
#ifdef _WIN32
#include <winsock.h> // htonl, ntohl
#else
#include <netinet/in.h> // htonl, ntohl
#endif
#include <opencv2/gapi/gtype_traits.hpp>
#include "backends/common/serialization.hpp"
namespace cv {
namespace gimpl {
namespace s11n {
namespace {
void putData(GSerialized& s, const GModel::ConstGraph& cg, const ade::NodeHandle &nh) {
const auto gdata = cg.metadata(nh).get<gimpl::Data>();
const auto it = ade::util::find_if(s.m_datas, [&gdata](const cv::gimpl::Data &cd) {
return cd.rc == gdata.rc && cd.shape == gdata.shape;
});
if (s.m_datas.end() == it) {
s.m_datas.push_back(gdata);
}
}
void putOp(GSerialized& s, const GModel::ConstGraph& cg, const ade::NodeHandle &nh) {
const auto& op = cg.metadata(nh).get<gimpl::Op>();
for (const auto &in_nh : nh->inNodes()) { putData(s, cg, in_nh); }
for (const auto &out_nh : nh->outNodes()) { putData(s, cg, out_nh); }
s.m_ops.push_back(op);
}
void mkDataNode(ade::Graph& g, const cv::gimpl::Data& data) {
GModel::Graph gm(g);
auto nh = gm.createNode();
gm.metadata(nh).set(NodeType{NodeType::DATA});
gm.metadata(nh).set(data);
}
void mkOpNode(ade::Graph& g, const cv::gimpl::Op& op) {
GModel::Graph gm(g);
auto nh = gm.createNode();
gm.metadata(nh).set(NodeType{NodeType::OP});
gm.metadata(nh).set(op);
}
void linkNodes(ade::Graph& g) {
std::map<cv::gimpl::RcDesc, ade::NodeHandle> dataNodes;
GModel::Graph gm(g);
for (const auto& nh : g.nodes()) {
if (gm.metadata(nh).get<NodeType>().t == NodeType::DATA) {
const auto &d = gm.metadata(nh).get<gimpl::Data>();
const auto rc = cv::gimpl::RcDesc{d.rc, d.shape, d.ctor};
dataNodes[rc] = nh;
}
}
for (const auto& nh : g.nodes()) {
if (gm.metadata(nh).get<NodeType>().t == NodeType::OP) {
const auto& op = gm.metadata(nh).get<gimpl::Op>();
for (const auto& in : ade::util::indexed(op.args)) {
const auto& arg = ade::util::value(in);
if (arg.kind == cv::detail::ArgKind::GOBJREF) {
const auto idx = ade::util::index(in);
const auto rc = arg.get<gimpl::RcDesc>();
const auto& in_nh = dataNodes.at(rc);
const auto& in_eh = g.link(in_nh, nh);
gm.metadata(in_eh).set(Input{idx});
}
}
for (const auto& out : ade::util::indexed(op.outs)) {
const auto idx = ade::util::index(out);
const auto rc = ade::util::value(out);
const auto& out_nh = dataNodes.at(rc);
const auto& out_eh = g.link(nh, out_nh);
gm.metadata(out_eh).set(Output{idx});
}
}
}
}
void relinkProto(ade::Graph& g) {
// identify which node handles map to the protocol
// input/output object in the reconstructed graph
using S = std::set<cv::gimpl::RcDesc>; // FIXME: use ...
using M = std::map<cv::gimpl::RcDesc, ade::NodeHandle>; // FIXME: unordered!
cv::gimpl::GModel::Graph gm(g);
auto &proto = gm.metadata().get<Protocol>();
const S set_in(proto.inputs.begin(), proto.inputs.end());
const S set_out(proto.outputs.begin(), proto.outputs.end());
M map_in, map_out;
// Associate the protocol node handles with their resource identifiers
for (auto &&nh : gm.nodes()) {
if (gm.metadata(nh).get<cv::gimpl::NodeType>().t == cv::gimpl::NodeType::DATA) {
const auto &d = gm.metadata(nh).get<cv::gimpl::Data>();
const auto rc = cv::gimpl::RcDesc{d.rc, d.shape, d.ctor};
if (set_in.count(rc) > 0) {
GAPI_DbgAssert(set_out.count(rc) == 0);
map_in[rc] = nh;
} else if (set_out.count(rc) > 0) {
GAPI_DbgAssert(set_in.count(rc) == 0);
map_out[rc] = nh;
}
}
}
// Reconstruct the protocol vectors, ordered
proto.in_nhs.reserve(proto.inputs.size());
proto.in_nhs.clear();
proto.out_nhs.reserve(proto.outputs.size());
proto.out_nhs.clear();
for (auto &rc : proto.inputs) { proto.in_nhs .push_back(map_in .at(rc)); }
for (auto &rc : proto.outputs) { proto.out_nhs.push_back(map_out.at(rc)); }
}
} // anonymous namespace
////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////
// Graph dump operators
// OpenCV types ////////////////////////////////////////////////////////////////
I::OStream& operator<< (I::OStream& os, const cv::Point &pt) {
return os << pt.x << pt.y;
}
I::IStream& operator>> (I::IStream& is, cv::Point& pt) {
return is >> pt.x >> pt.y;
}
I::OStream& operator<< (I::OStream& os, const cv::Size &sz) {
return os << sz.width << sz.height;
}
I::IStream& operator>> (I::IStream& is, cv::Size& sz) {
return is >> sz.width >> sz.height;
}
I::OStream& operator<< (I::OStream& os, const cv::Rect &rc) {
return os << rc.x << rc.y << rc.width << rc.height;
}
I::IStream& operator>> (I::IStream& is, cv::Rect& rc) {
return is >> rc.x >> rc.y >> rc.width >> rc.height;
}
I::OStream& operator<< (I::OStream& os, const cv::Scalar &s) {
return os << s.val[0] << s.val[1] << s.val[2] << s.val[3];
}
I::IStream& operator>> (I::IStream& is, cv::Scalar& s) {
return is >> s.val[0] >> s.val[1] >> s.val[2] >> s.val[3];
}
namespace
{
#if !defined(GAPI_STANDALONE)
template<typename T>
void write_plain(I::OStream &os, const T *arr, std::size_t sz) {
for (auto &&it : ade::util::iota(sz)) os << arr[it];
}
template<typename T>
void read_plain(I::IStream &is, T *arr, std::size_t sz) {
for (auto &&it : ade::util::iota(sz)) is >> arr[it];
}
template<typename T>
void write_mat_data(I::OStream &os, const cv::Mat &m) {
// Write every row individually (handles the case when Mat is a view)
for (auto &&r : ade::util::iota(m.rows)) {
write_plain(os, m.ptr<T>(r), m.cols*m.channels());
}
}
template<typename T>
void read_mat_data(I::IStream &is, cv::Mat &m) {
// Write every row individually (handles the case when Mat is aligned)
for (auto &&r : ade::util::iota(m.rows)) {
read_plain(is, m.ptr<T>(r), m.cols*m.channels());
}
}
#else
void write_plain(I::OStream &os, const uchar *arr, std::size_t sz) {
for (auto &&it : ade::util::iota(sz)) os << arr[it];
}
void read_plain(I::IStream &is, uchar *arr, std::size_t sz) {
for (auto &&it : ade::util::iota(sz)) is >> arr[it];
}
template<typename T>
void write_mat_data(I::OStream &os, const cv::Mat &m) {
// Write every row individually (handles the case when Mat is a view)
for (auto &&r : ade::util::iota(m.rows)) {
write_plain(os, m.ptr(r), m.cols*m.channels()*sizeof(T));
}
}
template<typename T>
void read_mat_data(I::IStream &is, cv::Mat &m) {
// Write every row individually (handles the case when Mat is aligned)
for (auto &&r : ade::util::iota(m.rows)) {
read_plain(is, m.ptr(r), m.cols*m.channels()*sizeof(T));
}
}
#endif
} // namespace
I::OStream& operator<< (I::OStream& os, const cv::Mat &m) {
#if !defined(GAPI_STANDALONE)
GAPI_Assert(m.size.dims() == 2 && "Only 2D images are supported now");
#else
GAPI_Assert(m.dims.size() == 2 && "Only 2D images are supported now");
#endif
os << m.rows << m.cols << m.type();
switch (m.depth()) {
case CV_8U: write_mat_data< uint8_t>(os, m); break;
case CV_8S: write_mat_data< char>(os, m); break;
case CV_16U: write_mat_data<uint16_t>(os, m); break;
case CV_16S: write_mat_data< int16_t>(os, m); break;
case CV_32S: write_mat_data< int32_t>(os, m); break;
case CV_32F: write_mat_data< float>(os, m); break;
case CV_64F: write_mat_data< double>(os, m); break;
default: GAPI_Assert(false && "Unsupported Mat depth");
}
return os;
}
I::IStream& operator>> (I::IStream& is, cv::Mat& m) {
int rows = -1, cols = -1, type = 0;
is >> rows >> cols >> type;
m.create(cv::Size(cols, rows), type);
switch (m.depth()) {
case CV_8U: read_mat_data< uint8_t>(is, m); break;
case CV_8S: read_mat_data< char>(is, m); break;
case CV_16U: read_mat_data<uint16_t>(is, m); break;
case CV_16S: read_mat_data< int16_t>(is, m); break;
case CV_32S: read_mat_data< int32_t>(is, m); break;
case CV_32F: read_mat_data< float>(is, m); break;
case CV_64F: read_mat_data< double>(is, m); break;
default: GAPI_Assert(false && "Unsupported Mat depth");
}
return is;
}
// G-API types /////////////////////////////////////////////////////////////////
// Stubs (empty types)
I::OStream& operator<< (I::OStream& os, cv::util::monostate ) {return os;}
I::IStream& operator>> (I::IStream& is, cv::util::monostate &) {return is;}
I::OStream& operator<< (I::OStream& os, const cv::GScalarDesc &) {return os;}
I::IStream& operator>> (I::IStream& is, cv::GScalarDesc &) {return is;}
I::OStream& operator<< (I::OStream& os, const cv::GOpaqueDesc &) {return os;}
I::IStream& operator>> (I::IStream& is, cv::GOpaqueDesc &) {return is;}
I::OStream& operator<< (I::OStream& os, const cv::GArrayDesc &) {return os;}
I::IStream& operator>> (I::IStream& is, cv::GArrayDesc &) {return is;}
// Enums and structures
namespace {
template<typename E> I::OStream& put_enum(I::OStream& os, E e) {
return os << static_cast<int>(e);
}
template<typename E> I::IStream& get_enum(I::IStream& is, E &e) {
int x{}; is >> x; e = static_cast<E>(x);
return is;
}
} // anonymous namespace
I::OStream& operator<< (I::OStream& os, cv::GShape sh) {
return put_enum(os, sh);
}
I::IStream& operator>> (I::IStream& is, cv::GShape &sh) {
return get_enum<cv::GShape>(is, sh);
}
I::OStream& operator<< (I::OStream& os, cv::detail::ArgKind k) {
return put_enum(os, k);
}
I::IStream& operator>> (I::IStream& is, cv::detail::ArgKind &k) {
return get_enum<cv::detail::ArgKind>(is, k);
}
I::OStream& operator<< (I::OStream& os, cv::detail::OpaqueKind k) {
return put_enum(os, k);
}
I::IStream& operator>> (I::IStream& is, cv::detail::OpaqueKind &k) {
return get_enum<cv::detail::OpaqueKind>(is, k);
}
I::OStream& operator<< (I::OStream& os, cv::gimpl::Data::Storage s) {
return put_enum(os, s);
}
I::IStream& operator>> (I::IStream& is, cv::gimpl::Data::Storage &s) {
return get_enum<cv::gimpl::Data::Storage>(is, s);
}
I::OStream& operator<< (I::OStream& os, const cv::GArg &arg) {
// Only GOBJREF and OPAQUE_VAL kinds can be serialized/deserialized
GAPI_Assert( arg.kind == cv::detail::ArgKind::OPAQUE_VAL
|| arg.kind == cv::detail::ArgKind::GOBJREF);
os << arg.kind << arg.opaque_kind;
if (arg.kind == cv::detail::ArgKind::GOBJREF) {
os << arg.get<cv::gimpl::RcDesc>();
} else {
GAPI_Assert(arg.kind == cv::detail::ArgKind::OPAQUE_VAL);
GAPI_Assert(arg.opaque_kind != cv::detail::OpaqueKind::CV_UNKNOWN);
switch (arg.opaque_kind) {
case cv::detail::OpaqueKind::CV_BOOL: os << arg.get<bool>(); break;
case cv::detail::OpaqueKind::CV_INT: os << arg.get<int>(); break;
case cv::detail::OpaqueKind::CV_DOUBLE: os << arg.get<double>(); break;
case cv::detail::OpaqueKind::CV_POINT: os << arg.get<cv::Point>(); break;
case cv::detail::OpaqueKind::CV_SIZE: os << arg.get<cv::Size>(); break;
case cv::detail::OpaqueKind::CV_RECT: os << arg.get<cv::Rect>(); break;
case cv::detail::OpaqueKind::CV_SCALAR: os << arg.get<cv::Scalar>(); break;
case cv::detail::OpaqueKind::CV_MAT: os << arg.get<cv::Mat>(); break;
default: GAPI_Assert(false && "GArg: Unsupported (unknown?) opaque value type");
}
}
return os;
}
I::IStream& operator>> (I::IStream& is, cv::GArg &arg) {
is >> arg.kind >> arg.opaque_kind;
// Only GOBJREF and OPAQUE_VAL kinds can be serialized/deserialized
GAPI_Assert( arg.kind == cv::detail::ArgKind::OPAQUE_VAL
|| arg.kind == cv::detail::ArgKind::GOBJREF);
if (arg.kind == cv::detail::ArgKind::GOBJREF) {
cv::gimpl::RcDesc rc;
is >> rc;
arg = (GArg(rc));
} else {
GAPI_Assert(arg.kind == cv::detail::ArgKind::OPAQUE_VAL);
GAPI_Assert(arg.opaque_kind != cv::detail::OpaqueKind::CV_UNKNOWN);
switch (arg.opaque_kind) {
#define HANDLE_CASE(E,T) case cv::detail::OpaqueKind::CV_##E: \
{ T t{}; is >> t; arg = (cv::GArg(t)); } break
HANDLE_CASE(BOOL , bool);
HANDLE_CASE(INT , int);
HANDLE_CASE(DOUBLE , double);
HANDLE_CASE(POINT , cv::Point);
HANDLE_CASE(SIZE , cv::Size);
HANDLE_CASE(RECT , cv::Rect);
HANDLE_CASE(SCALAR , cv::Scalar);
HANDLE_CASE(MAT , cv::Mat);
#undef HANDLE_CASE
default: GAPI_Assert(false && "GArg: Unsupported (unknown?) opaque value type");
}
}
return is;
}
I::OStream& operator<< (I::OStream& os, const cv::GKernel &k) {
return os << k.name << k.tag << k.outShapes;
}
I::IStream& operator>> (I::IStream& is, cv::GKernel &k) {
return is >> const_cast<std::string&>(k.name)
>> const_cast<std::string&>(k.tag)
>> const_cast<cv::GShapes&>(k.outShapes);
}
I::OStream& operator<< (I::OStream& os, const cv::GMatDesc &d) {
return os << d.depth << d.chan << d.size << d.planar << d.dims;
}
I::IStream& operator>> (I::IStream& is, cv::GMatDesc &d) {
return is >> d.depth >> d.chan >> d.size >> d.planar >> d.dims;
}
I::OStream& operator<< (I::OStream& os, const cv::gimpl::RcDesc &rc) {
// FIXME: HostCtor is not serialized!
return os << rc.id << rc.shape;
}
I::IStream& operator>> (I::IStream& is, cv::gimpl::RcDesc &rc) {
// FIXME: HostCtor is not deserialized!
return is >> rc.id >> rc.shape;
}
I::OStream& operator<< (I::OStream& os, const cv::gimpl::Op &op) {
return os << op.k << op.args << op.outs;
}
I::IStream& operator>> (I::IStream& is, cv::gimpl::Op &op) {
return is >> op.k >> op.args >> op.outs;
}
I::OStream& operator<< (I::OStream& os, const cv::gimpl::Data &d) {
// FIXME: HostCtor is not stored here!!
// FIXME: Storage may be incorrect for subgraph-to-graph process
return os << d.shape << d.rc << d.meta << d.storage;
}
I::IStream& operator>> (I::IStream& is, cv::gimpl::Data &d) {
// FIXME: HostCtor is not stored here!!
// FIXME: Storage may be incorrect for subgraph-to-graph process
return is >> d.shape >> d.rc >> d.meta >> d.storage;
}
I::OStream& operator<< (I::OStream& os, const cv::gimpl::DataObjectCounter &c) {
return os << c.m_next_data_id;
}
I::IStream& operator>> (I::IStream& is, cv::gimpl::DataObjectCounter &c) {
return is >> c.m_next_data_id;
}
I::OStream& operator<< (I::OStream& os, const cv::gimpl::Protocol &p) {
// NB: in_nhs/out_nhs are not written!
return os << p.inputs << p.outputs;
}
I::IStream& operator>> (I::IStream& is, cv::gimpl::Protocol &p) {
// NB: in_nhs/out_nhs are reconstructed at a later phase
return is >> p.inputs >> p.outputs;
}
void serialize( I::OStream& os
, const ade::Graph &g
, const std::vector<ade::NodeHandle> &nodes) {
cv::gimpl::GModel::ConstGraph cg(g);
GSerialized s;
for (auto &nh : nodes) {
switch (cg.metadata(nh).get<NodeType>().t)
{
case NodeType::OP: putOp (s, cg, nh); break;
case NodeType::DATA: putData(s, cg, nh); break;
default: util::throw_error(std::logic_error("Unknown NodeType"));
}
}
s.m_counter = cg.metadata().get<cv::gimpl::DataObjectCounter>();
s.m_proto = cg.metadata().get<cv::gimpl::Protocol>();
os << s.m_ops << s.m_datas << s.m_counter << s.m_proto;
}
GSerialized deserialize(I::IStream &is) {
GSerialized s;
is >> s.m_ops >> s.m_datas >> s.m_counter >> s.m_proto;
return s;
}
void reconstruct(const GSerialized &s, ade::Graph &g) {
GAPI_Assert(g.nodes().empty());
for (const auto& d : s.m_datas) cv::gimpl::s11n::mkDataNode(g, d);
for (const auto& op : s.m_ops) cv::gimpl::s11n::mkOpNode(g, op);
cv::gimpl::s11n::linkNodes(g);
cv::gimpl::GModel::Graph gm(g);
gm.metadata().set(s.m_counter);
gm.metadata().set(s.m_proto);
cv::gimpl::s11n::relinkProto(g);
gm.metadata().set(cv::gimpl::Deserialized{});
}
////////////////////////////////////////////////////////////////////////////////
// Streams /////////////////////////////////////////////////////////////////////
const std::vector<char>& ByteMemoryOutStream::data() const {
return m_storage;
}
I::OStream& ByteMemoryOutStream::operator<< (uint32_t atom) {
m_storage.push_back(0xFF & (atom));
m_storage.push_back(0xFF & (atom >> 8));
m_storage.push_back(0xFF & (atom >> 16));
m_storage.push_back(0xFF & (atom >> 24));
return *this;
}
I::OStream& ByteMemoryOutStream::operator<< (bool atom) {
m_storage.push_back(atom ? 1 : 0);
return *this;
}
I::OStream& ByteMemoryOutStream::operator<< (char atom) {
m_storage.push_back(atom);
return *this;
}
I::OStream& ByteMemoryOutStream::operator<< (unsigned char atom) {
return *this << static_cast<char>(atom);
}
I::OStream& ByteMemoryOutStream::operator<< (short atom) {
static_assert(sizeof(short) == 2, "Expecting sizeof(short) == 2");
m_storage.push_back(0xFF & (atom));
m_storage.push_back(0xFF & (atom >> 8));
return *this;
}
I::OStream& ByteMemoryOutStream::operator<< (unsigned short atom) {
return *this << static_cast<short>(atom);
}
I::OStream& ByteMemoryOutStream::operator<< (int atom) {
static_assert(sizeof(int) == 4, "Expecting sizeof(int) == 4");
return *this << static_cast<uint32_t>(atom);
}
//I::OStream& ByteMemoryOutStream::operator<< (std::size_t atom) {
// // NB: type truncated!
// return *this << static_cast<uint32_t>(atom);
//}
I::OStream& ByteMemoryOutStream::operator<< (float atom) {
static_assert(sizeof(float) == 4, "Expecting sizeof(float) == 4");
uint32_t tmp = 0u;
memcpy(&tmp, &atom, sizeof(float));
return *this << static_cast<uint32_t>(htonl(tmp));
}
I::OStream& ByteMemoryOutStream::operator<< (double atom) {
static_assert(sizeof(double) == 8, "Expecting sizeof(double) == 8");
uint32_t tmp[2] = {0u};
memcpy(tmp, &atom, sizeof(double));
*this << static_cast<uint32_t>(htonl(tmp[0]));
*this << static_cast<uint32_t>(htonl(tmp[1]));
return *this;
}
I::OStream& ByteMemoryOutStream::operator<< (const std::string &str) {
//*this << static_cast<std::size_t>(str.size()); // N.B. Put type explicitly
*this << static_cast<uint32_t>(str.size()); // N.B. Put type explicitly
for (auto c : str) *this << c;
return *this;
}
ByteMemoryInStream::ByteMemoryInStream(const std::vector<char> &data)
: m_storage(data) {
}
I::IStream& ByteMemoryInStream::operator>> (uint32_t &atom) {
check(sizeof(uint32_t));
uint8_t x[4];
x[0] = static_cast<uint8_t>(m_storage[m_idx++]);
x[1] = static_cast<uint8_t>(m_storage[m_idx++]);
x[2] = static_cast<uint8_t>(m_storage[m_idx++]);
x[3] = static_cast<uint8_t>(m_storage[m_idx++]);
atom = ((x[0]) | (x[1] << 8) | (x[2] << 16) | (x[3] << 24));
return *this;
}
I::IStream& ByteMemoryInStream::operator>> (bool& atom) {
check(sizeof(char));
atom = (m_storage[m_idx++] == 0) ? false : true;
return *this;
}
I::IStream& ByteMemoryInStream::operator>> (char &atom) {
check(sizeof(char));
atom = m_storage[m_idx++];
return *this;
}
I::IStream& ByteMemoryInStream::operator>> (unsigned char &atom) {
char c{};
*this >> c;
atom = static_cast<unsigned char>(c);
return *this;
}
I::IStream& ByteMemoryInStream::operator>> (short &atom) {
static_assert(sizeof(short) == 2, "Expecting sizeof(short) == 2");
check(sizeof(short));
uint8_t x[2];
x[0] = static_cast<uint8_t>(m_storage[m_idx++]);
x[1] = static_cast<uint8_t>(m_storage[m_idx++]);
atom = ((x[0]) | (x[1] << 8));
return *this;
}
I::IStream& ByteMemoryInStream::operator>> (unsigned short &atom) {
short s{};
*this >> s;
atom = static_cast<unsigned short>(s);
return *this;
}
I::IStream& ByteMemoryInStream::operator>> (int& atom) {
static_assert(sizeof(int) == 4, "Expecting sizeof(int) == 4");
atom = static_cast<int>(getU32());
return *this;
}
//I::IStream& ByteMemoryInStream::operator>> (std::size_t& atom) {
// // NB. Type was truncated!
// atom = static_cast<std::size_t>(getU32());
// return *this;
//}
I::IStream& ByteMemoryInStream::operator>> (float& atom) {
static_assert(sizeof(float) == 4, "Expecting sizeof(float) == 4");
uint32_t tmp = ntohl(getU32());
memcpy(&atom, &tmp, sizeof(float));
return *this;
}
I::IStream& ByteMemoryInStream::operator>> (double& atom) {
static_assert(sizeof(double) == 8, "Expecting sizeof(double) == 8");
uint32_t tmp[2] = {ntohl(getU32()), ntohl(getU32())};
memcpy(&atom, tmp, sizeof(double));
return *this;
}
I::IStream& ByteMemoryInStream::operator>> (std::string& str) {
//std::size_t sz = 0u;
uint32_t sz = 0u;
*this >> sz;
if (sz == 0u) {
str.clear();
} else {
str.resize(sz);
for (auto &&i : ade::util::iota(sz)) { *this >> str[i]; }
}
return *this;
}
} // namespace s11n
} // namespace gimpl
} // namespace cv
@@ -0,0 +1,313 @@
#ifndef OPENCV_GAPI_COMMON_SERIALIZATION_HPP
#define OPENCV_GAPI_COMMON_SERIALIZATION_HPP
// This file is part of OpenCV project.
// It is subject to the license terms in the LICENSE file found in the top-level directory
// of this distribution and at http://opencv.org/license.html.
//
// Copyright (C) 2020 Intel Corporation
#include <iostream>
#include <fstream>
#include <string.h>
#include <ade/util/iota_range.hpp> // used in the vector<</>>
#include "compiler/gmodel.hpp"
namespace cv {
namespace gimpl {
namespace s11n {
struct GSerialized {
std::vector<cv::gimpl::Op> m_ops;
std::vector<cv::gimpl::Data> m_datas;
cv::gimpl::DataObjectCounter m_counter;
cv::gimpl::Protocol m_proto;
};
////////////////////////////////////////////////////////////////////////////////
// Stream interfaces, so far temporary
namespace I {
struct GAPI_EXPORTS OStream {
virtual ~OStream() = default;
// Define the native support for basic C++ types at the API level:
virtual OStream& operator<< (bool) = 0;
virtual OStream& operator<< (char) = 0;
virtual OStream& operator<< (unsigned char) = 0;
virtual OStream& operator<< (short) = 0;
virtual OStream& operator<< (unsigned short) = 0;
virtual OStream& operator<< (int) = 0;
//virtual OStream& operator<< (std::size_t) = 0;
virtual OStream& operator<< (uint32_t) = 0;
virtual OStream& operator<< (float) = 0;
virtual OStream& operator<< (double) = 0;
virtual OStream& operator<< (const std::string&) = 0;
};
struct GAPI_EXPORTS IStream {
virtual ~IStream() = default;
virtual IStream& operator>> (bool &) = 0;
virtual IStream& operator>> (char &) = 0;
virtual IStream& operator>> (unsigned char &) = 0;
virtual IStream& operator>> (short &) = 0;
virtual IStream& operator>> (unsigned short &) = 0;
virtual IStream& operator>> (int &) = 0;
virtual IStream& operator>> (float &) = 0;
virtual IStream& operator>> (double &) = 0;
//virtual IStream& operator>> (std::size_t &) = 0;
virtual IStream& operator >> (uint32_t &) = 0;
virtual IStream& operator>> (std::string &) = 0;
};
} // namespace I
////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////
// S11N operators
// Note: operators for basic types are defined in IStream/OStream
// OpenCV types ////////////////////////////////////////////////////////////////
GAPI_EXPORTS I::OStream& operator<< (I::OStream& os, const cv::Point &pt);
GAPI_EXPORTS I::IStream& operator>> (I::IStream& is, cv::Point &pt);
GAPI_EXPORTS I::OStream& operator<< (I::OStream& os, const cv::Size &sz);
GAPI_EXPORTS I::IStream& operator>> (I::IStream& is, cv::Size &sz);
GAPI_EXPORTS I::OStream& operator<< (I::OStream& os, const cv::Rect &rc);
GAPI_EXPORTS I::IStream& operator>> (I::IStream& is, cv::Rect &rc);
GAPI_EXPORTS I::OStream& operator<< (I::OStream& os, const cv::Scalar &s);
GAPI_EXPORTS I::IStream& operator>> (I::IStream& is, cv::Scalar &s);
GAPI_EXPORTS I::OStream& operator<< (I::OStream& os, const cv::Mat &m);
GAPI_EXPORTS I::IStream& operator>> (I::IStream& is, cv::Mat &m);
// G-API types /////////////////////////////////////////////////////////////////
GAPI_EXPORTS I::OStream& operator<< (I::OStream& os, cv::util::monostate );
GAPI_EXPORTS I::IStream& operator>> (I::IStream& is, cv::util::monostate &);
GAPI_EXPORTS I::OStream& operator<< (I::OStream& os, cv::GShape shape);
GAPI_EXPORTS I::IStream& operator>> (I::IStream& is, cv::GShape &shape);
GAPI_EXPORTS I::OStream& operator<< (I::OStream& os, cv::detail::ArgKind k);
GAPI_EXPORTS I::IStream& operator>> (I::IStream& is, cv::detail::ArgKind &k);
GAPI_EXPORTS I::OStream& operator<< (I::OStream& os, cv::detail::OpaqueKind k);
GAPI_EXPORTS I::IStream& operator>> (I::IStream& is, cv::detail::OpaqueKind &k);
GAPI_EXPORTS I::OStream& operator<< (I::OStream& os, cv::gimpl::Data::Storage s);
GAPI_EXPORTS I::IStream& operator>> (I::IStream& is, cv::gimpl::Data::Storage &s);
GAPI_EXPORTS I::OStream& operator<< (I::OStream& os, const cv::gimpl::DataObjectCounter &c);
GAPI_EXPORTS I::IStream& operator>> (I::IStream& is, cv::gimpl::DataObjectCounter &c);
GAPI_EXPORTS I::OStream& operator<< (I::OStream& os, const cv::gimpl::Protocol &p);
GAPI_EXPORTS I::IStream& operator>> (I::IStream& is, cv::gimpl::Protocol &p);
GAPI_EXPORTS I::OStream& operator<< (I::OStream& os, const cv::GArg &arg);
GAPI_EXPORTS I::IStream& operator>> (I::IStream& is, cv::GArg &arg);
GAPI_EXPORTS I::OStream& operator<< (I::OStream& os, const cv::GKernel &k);
GAPI_EXPORTS I::IStream& operator>> (I::IStream& is, cv::GKernel &k);
GAPI_EXPORTS I::OStream& operator<< (I::OStream& os, const cv::GMatDesc &d);
GAPI_EXPORTS I::IStream& operator>> (I::IStream& is, cv::GMatDesc &d);
GAPI_EXPORTS I::OStream& operator<< (I::OStream& os, const cv::GScalarDesc &);
GAPI_EXPORTS I::IStream& operator>> (I::IStream& is, cv::GScalarDesc &);
GAPI_EXPORTS I::OStream& operator<< (I::OStream& os, const cv::GOpaqueDesc &);
GAPI_EXPORTS I::IStream& operator>> (I::IStream& is, cv::GOpaqueDesc &);
GAPI_EXPORTS I::OStream& operator<< (I::OStream& os, const cv::GArrayDesc &);
GAPI_EXPORTS I::IStream& operator>> (I::IStream& is, cv::GArrayDesc &);
GAPI_EXPORTS I::OStream& operator<< (I::OStream& os, const cv::gimpl::RcDesc &rc);
GAPI_EXPORTS I::IStream& operator>> (I::IStream& is, cv::gimpl::RcDesc &rc);
GAPI_EXPORTS I::OStream& operator<< (I::OStream& os, const cv::gimpl::Op &op);
GAPI_EXPORTS I::IStream& operator>> (I::IStream& is, cv::gimpl::Op &op);
GAPI_EXPORTS I::OStream& operator<< (I::OStream& os, const cv::gimpl::Data &op);
GAPI_EXPORTS I::IStream& operator>> (I::IStream& is, cv::gimpl::Data &op);
// The top-level serialization routine.
// Note it is just a single function which takes a GModel and a list of nodes
// and writes the data to the stream (recursively)
GAPI_EXPORTS void serialize( I::OStream& os
, const ade::Graph &g
, const std::vector<ade::NodeHandle> &nodes);
// The top-level deserialization routineS.
// Unfortunately the deserialization is a two-step process:
// 1. First we decode a stream into some intermediate representation
// (called "GSerialized");
// 2. Then we produce an ade::Graph from this intermediate representation.
//
// An ade::Graph can't be produced from the stream immediately
// since every GCompiled object has its own unique ade::Graph, so
// we can't do it once and for all since every compilation process
// is individual and _is_ altering the ade::Graph state (structure and metadata).
// At the same time, we can't hold the reference to "is" within the GComputation
// forever since this input stream may be associated with an external resource
// and have side effects.
//
// Summarizing, the `deserialize()` happens *once per GComputation* immediately
// during the cv::gapi::deserialize<GComputation>(), and `reconstruct()` happens
// on every compilation process issued for this GComputation.
GAPI_EXPORTS GSerialized deserialize(I::IStream& is);
GAPI_EXPORTS void reconstruct(const GSerialized &s, ade::Graph &g);
// Legacy //////////////////////////////////////////////////////////////////////
// Generic: vector serialization ///////////////////////////////////////////////
template<typename T>
I::OStream& operator<< (I::OStream& os, const std::vector<T> &ts) {
//const std::size_t sz = ts.size(); // explicitly specify type
const uint32_t sz = (uint32_t)ts.size(); // explicitly specify type
os << sz;
for (auto &&v : ts) os << v;
return os;
}
template<typename T>
I::IStream& operator>> (I::IStream& is, std::vector<T> &ts) {
//std::size_t sz = 0u;
uint32_t sz = 0u;
is >> sz;
if (sz == 0u) {
ts.clear();
} else {
ts.resize(sz);
for (auto &&i : ade::util::iota(sz)) is >> ts[i];
}
return is;
}
// Generic: unordered_map serialization ////////////////////////////////////////
template<typename K, typename V>
I::OStream& operator<< (I::OStream& os, const std::unordered_map<K, V> &m) {
//const std::size_t sz = m.size(); // explicitly specify type
const uint32_t sz = (uint32_t)m.size(); // explicitly specify type
os << sz;
for (auto &&it : m) os << it.first << it.second;
return os;
}
template<typename K, typename V>
I::IStream& operator>> (I::IStream& is, std::unordered_map<K, V> &m) {
m.clear();
//std::size_t sz = 0u;
uint32_t sz = 0u;
is >> sz;
if (sz != 0u) {
for (auto &&i : ade::util::iota(sz)) {
(void) i;
K k{};
V v{};
is >> k >> v;
m.insert({k,v});
}
GAPI_Assert(sz == m.size());
}
return is;
}
// Generic: variant serialization //////////////////////////////////////////////
namespace detail { // FIXME: breaks old code
template<typename V>
I::OStream& put_v(I::OStream&, const V&, std::size_t) {
GAPI_Assert(false && "variant>>: requested index is invalid");
};
template<typename V, typename X, typename... Xs>
I::OStream& put_v(I::OStream& os, const V& v, std::size_t x) {
return (x == 0u)
? os << cv::util::get<X>(v)
: put_v<V, Xs...>(os, v, x-1);
}
template<typename V>
I::IStream& get_v(I::IStream&, V&, std::size_t, std::size_t) {
GAPI_Assert(false && "variant<<: requested index is invalid");
}
template<typename V, typename X, typename... Xs>
I::IStream& get_v(I::IStream& is, V& v, std::size_t i, std::size_t gi) {
if (i == gi) {
X x{};
is >> x;
v = std::move(x);
return is;
} else return get_v<V, Xs...>(is, v, i+1, gi);
}
} // namespace detail FIXME: breaks old code
template<typename... Ts>
I::OStream& operator<< (I::OStream& os, const cv::util::variant<Ts...> &v) {
os << (uint32_t)v.index();
return detail::put_v<cv::util::variant<Ts...>, Ts...>(os, v, v.index());
}
template<typename... Ts>
I::IStream& operator>> (I::IStream& is, cv::util::variant<Ts...> &v) {
int idx = -1;
is >> idx;
GAPI_Assert(idx >= 0 && idx < (int)sizeof...(Ts));
return detail::get_v<cv::util::variant<Ts...>, Ts...>(is, v, 0u, idx);
}
// FIXME: Basic Stream implementaions //////////////////////////////////////////
// Basic in-memory stream implementations.
class GAPI_EXPORTS ByteMemoryOutStream final: public I::OStream {
std::vector<char> m_storage;
//virtual I::OStream& operator << (uint32_t) override;
//virtual I::OStream& operator<< (uint32_t) final;
public:
const std::vector<char>& data() const;
virtual I::OStream& operator<< (bool) override;
virtual I::OStream& operator<< (char) override;
virtual I::OStream& operator<< (unsigned char) override;
virtual I::OStream& operator<< (short) override;
virtual I::OStream& operator<< (unsigned short) override;
virtual I::OStream& operator<< (int) override;
//virtual I::OStream& operator<< (std::size_t) override;
virtual I::OStream& operator<< (float) override;
virtual I::OStream& operator<< (double) override;
virtual I::OStream& operator<< (const std::string&) override;
virtual I::OStream& operator<< (uint32_t) override;
};
class GAPI_EXPORTS ByteMemoryInStream final: public I::IStream {
const std::vector<char>& m_storage;
size_t m_idx = 0u;
void check(std::size_t n) { (void) n; GAPI_DbgAssert(m_idx+n-1 < m_storage.size()); }
uint32_t getU32() { uint32_t v{}; *this >> v; return v; };
//virtual I::IStream& operator>> (uint32_t &) final;
public:
explicit ByteMemoryInStream(const std::vector<char> &data);
virtual I::IStream& operator>> (bool &) override;
virtual I::IStream& operator>> (char &) override;
virtual I::IStream& operator>> (unsigned char &) override;
virtual I::IStream& operator>> (short &) override;
virtual I::IStream& operator>> (unsigned short &) override;
virtual I::IStream& operator>> (int &) override;
virtual I::IStream& operator>> (float &) override;
virtual I::IStream& operator>> (double &) override;
//virtual I::IStream& operator>> (std::size_t &) override;
virtual I::IStream& operator >> (uint32_t &) override;
virtual I::IStream& operator>> (std::string &) override;
};
} // namespace s11n
} // namespace gimpl
} // namespace cv
#endif // OPENCV_GAPI_COMMON_SERIALIZATION_HPP