lonekorean/wordpress-export-to-markdown

Image download delay

Closed this issue · 2 comments

In writer.js, function writeImageFilesPromise(posts, config), line 81, the delay increments by 100ms. For large WP sites on slow/overburdened servers, this can result in what amounts to a DoS attack, with exceptions (RequestError: Error: read ECONNRESET) proliferating as the server becomes overloaded.

After modifying the delay to increment by 1000ms instead of 100ms, I successfully downloaded each file, although of course it was slow. Luckily most users won't need to run this software frequently.

Suggest allowing user to specify image download delay or at least raise the default delay to something higher, perhaps 500ms.

Otherwise software worked a charm.

Hey thanks, yeah that's something I've wrestled with. Want it to be fast... but not too fast. :)

Long term I want to update the code to not exceed X pending requests, but for now I'll just increase the delay.

To give me a better idea, how many images did you have to download?

Increased the default delay to 500ms. Also moved those variables into a settings file.

d8c513f

Thanks again!