serialization of 3rd party datastructures

[voidRaptor]

Is it possible to serialize e.g. glm datastructures (glm::vec2, glm::vec3, ...)? The examples show only serialization of classes and struct defined by the user, and simply defining

#include <glm/vec3.hpp>

template<typename S>
void serialize(S& s, glm::vec3& vec) {
    s.value4b(vec.x);
    s.value4b(vec.y);
    s.value4b(vec.z);
}

somewhere where the serialization is done was apparently not the correct way.

[fraillt]

It is correct way. The problem is with ADL. Please define serialize function in same namespace as object, or in bitsery namespace.

[voidRaptor]

Thanks for fast reply! Oh, it was that easy to fix. Well then my next problem: suppose I wanted to enable bit packing, since size of single vec3 is 128B (length of std::vector of uint8_ts), which is quite large. How could I achieve that? I had something like this in mind:

namespace glm {

template<typename S>
void serialize(S& s, glm::vec3& vec) {

    s.enableBitPacking([&vec](typename S::BPEnabledType& sbp) {
        constexpr bitsery::ext::ValueRange<float> range{-100000.0f, 100000.0f, 0.0001f};
        sbp.ext(vec.x, range);
        sbp.ext(vec.y, range);
        sbp.ext(vec.z, range);
    });
}


}

but only thing that has changed is precision, which doesn't seem to be correct, since (96.4234f, 50.213f, -11.123f) becomes (96.4141f, 50.2109f, -11.125f).

[fraillt]

Ok, there are several things going on here.
First, vec3 should fit in 3*4B=12B=96bits.
Second, bit packing in your example is used correctly, but the range you specify is the real problem.
You provide a range [-1e5 .. 1e5] with precision 1e-4. Which means that there is 2e5/1e-4=2e9 possible values. Which requires 31bits (2^31 = 2.147e9). The other problem is with float numbers. In IEEE754 standard floats has 24 significant bits and 8 exponent bits, which means that there are between 7-8 significant digits, e.g. float allows to specify a number such as 123456700000000000000000000000000... or 0.0000000000000000000001234567... with 7 digits precision.

Conversions between fixed and floating point always loose some precision, but in your provided example it looks too much of precision loss.. so algorithm for floating numbers in ValueRange should be definitely improved.

So if you really want to save space you should specify the least tolerable precision and and range, so -10000 .. 10000 with 0.001 would require 25bits per number. So 3 floats would be 75bits rounded to a byte= 80bits=10B that would save you 12-10=2B per vec3 :)

Other optimization would be to serialize multiple vec3 in the same enableBitPacking scope. E.g. say you have 1000 values of vec3. Using previous example it would require 75bit×1000=75000/8=9375+1B instead of 10000B.

Hope that helped:)

[voidRaptor]

Yeah that helped a lot! It seems the real size of the data after compression is not size of std::vector used as buffer, but the number of bytes quickSerialization() gives. I have no idea why, but the vector had 128 elements in it and only the first 8 contained other than just zeros.

[fraillt]

Yes... this is a common issue that users, new to the bitsery, find confusing...
I wish I had a better approach to it, but I don't. So you always need to call writtenBytesCount on the adapter, after serialization.
If you're interested there was a discussion related to this problem, in here #60.

[voidRaptor]

Oh using resize is the reason, well that explains. Now that I know only the size returned by bitsery functions is reliable everything is easier. One thing I noticed that floats seemed to lose some precision (about 4th decimal and beyond) even without compression.

In any case, thanks for explaning, it really helped :)