bug: Requested Allocation Size too big
eknoes opened this issue · 2 comments
eknoes commented
When compiled with AddressSanitizer, I could catch the following issue when parsing
InvalidPNGColumns.pdf
SUMMARY: AddressSanitizer: allocation-size-too-big
Backtrace:
#0 InputPredictorPNGOptimumStream::Assign (this=0x506000000080, inSourceStream=0x504000000cd0, inColors=1,
inBitsPerComponent=8 '\b', inColumns=9223372036854775807)
at PDF-Writer/PDFWriter/InputPredictorPNGOptimumStream.cpp:145
#1 0x00005555557c6a09 in InputPredictorPNGOptimumStream::InputPredictorPNGOptimumStream (this=0x506000000080,
inSourceStream=0x504000000cd0, inColors=1, inBitsPerComponent=8 '\b', inColumns=9223372036854775807)
at PDF-Writer/PDFWriter/InputPredictorPNGOptimumStream.cpp:61
#2 0x0000555555761728 in PDFParser::WrapWithPredictorStream (this=0x7ffff5d09060, inputStream=0x504000000cd0,
inDecodeParams=0x50c000000100) at PDF-Writer/PDFWriter/PDFParser.cpp:1989
#3 0x000055555575e85c in PDFParser::CreateFilterForStream (this=0x7ffff5d09060, inStream=0x503000000250,
inFilterName=0x50b000001220, inDecodeParams=0x50c000000100, inPDFStream=0x508000000220)
at PDF-Writer/PDFWriter/PDFParser.cpp:2025
#4 0x000055555575b187 in PDFParser::CreateInputStreamReader (this=0x7ffff5d09060, inStream=0x508000000220)
at PDF-Writer/PDFWriter/PDFParser.cpp:1909
#5 0x0000555555755602 in PDFParser::ParseXrefFromXrefStream (this=0x7ffff5d09060, inXrefTable=std::vector of length 0, capacity 0,
inXrefSize=41, inXrefStream=0x508000000220, outReadTableSize=0x7ffff5b09b50)
at PDF-Writer/PDFWriter/PDFParser.cpp:1461
#6 0x000055555575900b in PDFParser::BuildXrefTableAndTrailerFromXrefStream (this=0x7ffff5d09060, inXrefStreamObjectID=40)
at PDF-Writer/PDFWriter/PDFParser.cpp:1368
#7 0x000055555573ece7 in PDFParser::ParseFileDirectory (this=0x7ffff5d09060)
at PDF-Writer/PDFWriter/PDFParser.cpp:1276
#8 0x000055555573b437 in PDFParser::StartPDFParsing (this=0x7ffff5d09060, inSourceStream=0x7ffff5d09020, inOptions=...)
at PDF-Writer/PDFWriter/PDFParser.cpp:120
The problem occurs at PDFWriter/InputPredictorPNGOptimumStream.cpp:146:
140
141 delete[] mBuffer;
142 delete[] mUpValues;
143 mBytesPerPixel = inColors * inBitsPerComponent / 8;
144 // Rows may contain empty bits at end
145 mBufferSize = (inColumns * inColors * inBitsPerComponent + 7) / 8 + 1;
146 mBuffer = new Byte[mBufferSize];
when inColumns is huge.
eknoes commented
The same can happen here with a high inColumns value.
void InputPredictorTIFFSubStream::Assign(IByteReader* inSourceStream,
LongBufferSizeType inColors,
Byte inBitsPerComponent,
LongBufferSizeType inColumns)
{
mSourceStream = inSourceStream;
mColors = inColors;
mBitsPerComponent = inBitsPerComponent;
mColumns = inColumns;
delete mRowBuffer;
mRowBuffer = new Byte[(inColumns*inColors*inBitsPerComponent)/8];