Supporting STL containers of types without public default constructor

[arekfu]

I am opening this issue on behalf of my colleague @Teloze and myself.

Currently cereal supports serialization of types without a public default constructor via friend cereal::access declarations. This mechanism however fails when a type without a public default constructor is used in an STL container, as in:

#include <cereal/access.hpp>
#include <cereal/archives/json.hpp>
#include <cereal/types/vector.hpp>
#include <sstream>
#include <vector>

class A {
 private:
  A() = default;
  int x_;
  friend class cereal::access;
  template <class Archive>
  void serialize(Archive& ar) {
    ar(this->x_);
  }

 public:
  A(int x) : x_(x) {}
};

int main() {
  std::vector<A> v;
  std::string serialized = R"({
    "value0": [
        {
            "value0": 1
        },
        {
            "value0": 2
        },
        {
            "value0": 3
        },
        {
            "value0": 4
        }
    ]
})";
  std::stringstream stream(serialized);
  cereal::JSONInputArchive archive(stream);
  archive(v);
  return 0;
}

Compiling this with gcc 5.4.0 yields

$ g++ -std=c++14 main.cc -o main -I/home/dm232107/src/cereal/include
[snip]
main.cc:41:12:   required from here
main.cc:9:3: error: ‘A::A()’ is private
   A() = default;
   ^
In file included from /usr/include/c++/5/memory:65:0,
                 from /home/dm232107/src/cereal/include/cereal/details/helpers.hpp:36,
                 from /home/dm232107/src/cereal/include/cereal/access.hpp:39,
                 from main.cc:1:
/usr/include/c++/5/bits/stl_uninitialized.h:540:22: error: within this context
    return std::fill_n(__first, __n, _ValueType());
                      ^

This was discussed in issue #425, where it was correctly identified that the problem stems from the fact that cereal tries to resize the vector before filling it, for efficiency reasons, and this operation fails if the default constructor for the type is not accessible from the STL (because it is private, for instance).

We find that this issue is very restrictive and we have come up with two possible solutions.

Solution 1

There is an overload of std::vector::resize that takes (in our example) an A const & as an argument and copies it in the empty slots. Now, cereal can instantiate a dummy A object, because cereal::access is friend with A; so it seems that the restriction can at least be lifted for copy-insertable types, at virtually no cost.

For types that are not copy-insertable and that have no public default constructor, we have come up with a workaround that forfeits the call to std::vector::resize. However, the user needs to opt in voluntarily in the workaround. That gives us the chance to warn them about the performance loss in the documentation.

Solution 2

Alternatively, it is possible to support vectors of types without public constructors by replacing the calls to std::vector::resize with std::vector::reserve. As long as cereal::access is able to call the default constructor for A, then it will work. In principle calling reserve is slightly less efficient, because push_back needs to bump up the logical size of the vector. However, we have made some performance measurements on fairly large vectors (10⁸) and we have not been able to identify any difference between calling resize and reserve.

Would you be willing to review a PR based on any of the ideas above?

Thanks for reading!
Davide @arekfu and Daniele @Teloze

[Teloze]

To complement the issue. We ran the sandbox/performance test with "Release" and gcc 4.8.5 with the standard implementation using resize, and using the reserve/push_back pair (solution 2). The modification has been done only for non-arithmetic/non-bool types.

I attach here the relevant results for two runs each;

benchCerealStandard.txt
benchCerealReservePushBack.txt

[Teloze]

Another complementary information.

We have implemented Solution 2 for all relevant standard containers and update tests accordingly.

The "cereal::access" has a new method to default-construct objects on stack. All attempts to default-construct objects within cereal are now substituted by a call to this new method.