long lines cause memory use to balloon

[joeyh]

perl -e 'use Text::Markdown "markdown"; markdown("a" x 6000000)'

This command feeds 6 mb of data to markdown, which takes 200 mb of memory to process it.

Most of the problem seems to be caused by the very long line. The same amount of data in short lines needs only 45 mb of memory.

perl -e 'use Text::Markdown "markdown"; markdown("a\n" x 3000000)'

While doubling the length of the single line nearly doubled the memory needed, to 385 mb.

perl -e 'use Text::Markdown "markdown"; markdown("a" x 12000000)'

This was observed in the wild, where a spammer DOSed a wiki by entering a single 6 mb long line into it.