fraillt/bitsery

Why does the buffer uses resize instead of reserve?

IceflowRE opened this issue · 6 comments

https://github.com/fraillt/bitsery/blob/master/include/bitsery/traits/core/std_defaults.h#L85

Wouldn't be the reserve method a better solution instead of resize?
Wasn't that possible?

I see your point, but I do resize for performance reasons mainly.
I resize once and before writing I only check if I still have large enough buffer, instead of resizing on every write which would hurt performance A LOT.
It also provides consistent behaviour independent if target buffer is resizable or not, you just always query for writtenBytesCount. Hope that helped.

It also provides consistent behaviour independent if target buffer is resizable or not, you just always query for writtenBytesCount.

Hope that helped.
But opting for reserve if available will improve the performance a lot. Also shouldn't all dynamic sized container support reserve?

Whole point of reserve is to avoid memory allocation on resize.
So you can do it yourself, if you know how much space you need :)
Since bitsery itself has no way of knowing how much data it will need, there is no need to call reserve. it immediately resizes by allocating bigger chunks, instead of resizing on each write to buffer.

Yes, but my point is resize will initialize that space with default values, reserve not. You will save that time.

Also using reserve will not increase the used size (only the capacity) of the buffer, you don't need to return how many bytes are written into the buffer.

https://stackoverflow.com/questions/13029299/stdvectorresize-vs-stdvectorreserve

Reserve is not about unitialized memory, its about preallocating memory when you know in advance how many space you might need.
Most important thing is that writing to end iterator (and past it) is UB (undefined behaviour) and you simply get crash on Visual Studio debug version, and there is a good reason for this:
I think there is one imporant aspect that you're missing, the way reserve works.
In pseodo code it look something like this:

fn reserve(new_size) {
  if capacity < new_size {
    reallocate(new_size)
  }
}

fn reallocate(new_size) {
  new_begin = allocate(new_size)
  new_end = copy_aligned(old_begin, old_end, new_begin)
  deallocate old memory
  reset iterators and capacity to new values
}

The most important aspect of this is copy_aligned(old_begin, old_end, new_begin) it copies only range [begin.. end), not [begin..capacity), and this is the key.
This means that if I do not resize a container and instead write past end iterator, after reserve is called all elements that was written past the end will be affectively gone, because they will not be copied to new location.

So if you really care about performance, and still want that things would work, use reserve before passing buffer to bitsery, or better yet, use fixed-size buffer. That way bitsery will not check for end of the buffer and it's up to you to make sure that it will not write pass the end, but that would be as fast as it possible could.
https://github.com/fraillt/cpp_serializers_benchmark look at bitsery fixed_buffer to see performance gains.

Reserve is not about unitialized memory, its about preallocating memory when you know in advance how many space you might need.
Most important thing is that writing to end iterator (and past it) is UB (undefined behaviour) and you simply get crash on Visual Studio debug version, and there is a good reason for this:
I think there is one imporant aspect that you're missing, the way reserve works.
In pseodo code it look something like this:

fn reserve(new_size) {
  if capacity < new_size {
    reallocate(new_size)
  }
}

fn reallocate(new_size) {
  new_begin = allocate(new_size)
  new_end = copy_aligned(old_begin, old_end, new_begin)
  deallocate old memory
  reset iterators and capacity to new values
}

The most important aspect of this is copy_aligned(old_begin, old_end, new_begin) it copies only range [begin.. end), not [begin..capacity), and this is the key.
This means that if I do not resize a container and instead write past end iterator, after reserve is called all elements that was written past the end will be affectively gone, because they will not be copied to new location.

So if you really care about performance, and still want that things would work, use reserve before passing buffer to bitsery, or better yet, use fixed-size buffer. That way bitsery will not check for end of the buffer and it's up to you to make sure that it will not write pass the end, but that would be as fast as it possible could.
https://github.com/fraillt/cpp_serializers_benchmark look at bitsery fixed_buffer to see performance gains.

Ah right, if you cannot or do not use something like emplace_back, push_back reserve won't work.