scn::owning_file + scn::getline() 1000x slower than STL in release and 3000x slower in debug
mark-99 opened this issue · 1 comments
I am reading 962 lines of text from a file, as 3 comma separated strings.
Source code (note I elided some error checking for clarity):
STL:
while (true)
{
std::getline(playbackFile, timestampStr, ',');
if (playbackFile.eof())
break;
std::getline(playbackFile, instrumentName, ',');
std::getline(playbackFile, str);
playbackStrings_.emplace_back(timestampStr, instrumentName, str);
}
scn:
while (true)
{
result = scn::getline(result.range(), timestampStr, ',');
if (result.error() == scn::error::end_of_range)
break;
result = scn::getline(result.range(), instrumentName, ',');
result = scn::getline(result.range(), str);
playbackStrings_.emplace_back(timestampStr, instrumentName, str);
}
std::istream + std::getline, debug:
Load took 13ms202us
std::istream + std::getline, release:
Load took 2ms457us
scn::owning_file + scn::getline, debug:
Load took 47s636ms
scn::owning_file + scn::getline, release:
Load took 3s223ms
So that's a pretty massive difference.
I tried to use scn::mapped_file
but that failed because of CRLF Windows vs Linux, scn::getline() returns something like "foo\r". There doesn't seem to be an easy fix for this as all the "separator" optional params are all char rather than strings, so you can't set "\r\n".
If I comment out enough checks I got it to run, and that is indeed fast:
scn::mapped_file + scn::getline, debug:
Load took 12ms843us
scn::mapped_file + scn::getline, release:
Load took 864us
So it's a choice between unusable due to CRLF or unusable due to being 1000x-3000x slower than std.
For now I'm sticking with STL (I don't actually need scnlib for this particular thing, I just wanted to try it out as I may need more sophisticated parsing in future). Please let me know if I'm doing anything dumb here, or there's a solution I've missed - thanks.
clang-cl v13.0.0 (msvcrt runtime). H/w: Core i9 12900K @ 4.9GHz + PCI4 M2 SSD @7GB/s.
That's unfortunate. It's unlikely I'll really do anything about it, though. I've removed file
and getline
from v2, for the reason that I'm unable to maintain that many interfaces and implementations, which is quite apparent by the performance of some of these APIs.
Thank you for your detailed report, anyway!