felixguendling/cista

cista::variant serialization of cista::unique_ptr with T=cista::container of the variant itself

Closed this issue · 7 comments

Hi!
I know, I know... how this issue summary sounds :) so let's go to an example.
I have cista representation of JSON structure like this (cista::mode is NONE default):

namespace CISTA = cista::offset;

using Null = std::monostate;

struct MetaArray;
struct MetaObject;

using MetaValue = CISTA::variant
	<
	Null,
	CISTA::string,
	std::int64_t,
	std::uint64_t,
	double,
	bool,
	CISTA::unique_ptr<MetaArray>,
	CISTA::unique_ptr<MetaObject>
	>;

struct MetaArray : public CISTA::vector<MetaValue> {};

struct MetaObject : public CISTA::hash_map<CISTA::string, MetaValue> {};

Building it works (starting at MetaObject level) with no problem -> after this silly change #96
Serialization of MetaObject or directly MetaValue breaks at:


with (VS2019/VS2022prev output):
"Error C3448 the number of identifiers must match the number of array elements or members in a structured binding declaration"

Same for both std::vector<>/void cista::serialize(...) approaches

OK, so it is a missing serializer for MetaArray, MetaObject - poor me

Stripping down a bit to use cista::unique_ptr with T-inheritance like below and serialization of value breaks:

struct TestObject;		
using TestValue = CISTA::variant<std::int64_t, CISTA::unique_ptr<TestObject>>;
struct TestObject : public CISTA::vector<TestValue> {};

TestValue value{ std::int64_t(42) };

...but this time with direct static-error: "Please implement custom serializer"

I am looking for making this simple as possible, with no pass-through/dummy inheritance de-/serialization methods.

If you want to use Cista's implemented serializer for cista::offset::vector<T>, you can just replace inheritance with a typedef / using TestObject = CISTA::vector<TestValue>;. Same for meta object as well as meta array. Don't use inheritance unless you really really need a new (distinct) type. If you need a new type, you also need to implement a custom serialization function (which would probably just cast to the parent type and call cista::serialize() again.

Saddly: "nope" - so I reopened this issue.
It is not obvious to me how to actually atact this... could please help me here?
I have this guy:

typedef struct TestObject;

using TestValue = CISTA::variant<std::int64_t, CISTA::unique_ptr<TestObject>>;
using TestObject = CISTA::vector<TestValue>;
TestValue value { std::int64_t(42) };

..and it gets (during serialization) an: "Error C2027 use of undefined type 'Alpha::Core::Config::TestObject"

reopen - in the context of my JSON-to-cista serialization attac (first comment)

Sorry for the confusion. My statement regarding using vs. inheritance is basically correct. However, here, I didn't see how to accomplish what you need with the using approach as this is a recursive data structure. You could of course use a vector<unique_ptr<variant<...>>> instead of vector<variant<...>> but this is not as efficient as it could be.

So I just implemented forwarding serialization / deserialazation functions:
https://wandbox.org/permlink/CPKeifMsVV35qiyf

#include <cassert>
#include "cista.h"


namespace CISTA = cista::offset;

struct Null {};

struct MetaArray;
struct MetaObject;

using MetaValue = CISTA::variant
	<
	Null,
	CISTA::string,
	std::int64_t,
	std::uint64_t,
	double,
	bool,
	CISTA::unique_ptr<MetaArray>,
	CISTA::unique_ptr<MetaObject>
	>;

struct MetaArray : public CISTA::vector<MetaValue> {};
struct MetaObject : public CISTA::hash_map<CISTA::string, MetaValue> {};

template <typename Ctx>
inline void serialize(Ctx& c, MetaArray const* origin,
                      cista::offset_t const offset) {
  cista::serialize(c, static_cast<CISTA::vector<MetaValue> const*>(origin), offset);
}

template <typename Ctx>
inline void deserialize(Ctx const& c, MetaArray* el) {
  cista::deserialize(c, static_cast<CISTA::vector<MetaValue>*>(el));
}

template <typename Ctx>
inline void serialize(Ctx& c, MetaObject const* origin,
                      cista::offset_t const offset) {
  cista::serialize(c, static_cast<CISTA::hash_map<CISTA::string, MetaValue> const*>(origin), offset);
}

template <typename Ctx>
inline void deserialize(Ctx const& c, MetaObject* el) {
  cista::deserialize(c, static_cast<CISTA::hash_map<CISTA::string, MetaValue>*>(el));
}

int main() {
  namespace data = cista::raw;

  std::vector<unsigned char> buf;
  {  // Serialize.
    MetaValue obj{Null{}};
    buf = cista::serialize(obj);
  }

  // Deserialize.
  auto deserialized = cista::deserialize<MetaValue>(buf);
  (void) deserialized;
}

Maybe a simple JSON <-> cista converter could also be integrated into cista itself. But this would probably require something like RapidJSON / Boost.JSON as a dependency for Cista to use this functionality.

Thank you for providing an example on this - I was stuck somehow on the cista::serialize/cista::deserialize, I found that only union_derive_test.cc shows something related...

Regarding JSON, I am using simdjson since I am reading only, I could share some stuff here (lic-free), on the weekend perhaps.