Report crashes - Allocation failed - JavaScript heap out of memory

[mgifford]

If I crawl this sitemap.xml file https://cnib.ca/en/sitemap.xml?region=on

The page fails. There are over 1800 pages in it, but had hoped it would be able to manage sites larger than this.

INFO: BasicCrawler: All the requests from request list and/or request queue have been processed, the crawler will shut down.
INFO: Crawler final request statistics: {"avgDurationMillis":8803,"perMinute":34,"finished":1813,"failed":3,"retryHistogram":[1811,2,null,3]}
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

<--- Last few GCs --->

[9364:0x102679000]  3255724 ms: Mark-sweep 1306.0 (1388.6) -> 1305.9 (1357.1) MB, 68.6 / 0.0 ms  (average mu = 0.719, current mu = 0.000) last resort GC in old space requested
[9364:0x102679000]  3255797 ms: Mark-sweep 1305.9 (1357.1) -> 1305.9 (1357.1) MB, 72.7 / 0.0 ms  (average mu = 0.556, current mu = 0.000) last resort GC in old space requested


<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 0x200b585878a1]
Security context: 0x1c9e2dd1e6e1 <JSObject>
    1: byteLength(aka byteLength) [0x1c9eb2305ef1] [buffer.js:~510] [pc=0x200b58ca5d2c](this=0x1c9e693826f1 <undefined>,string=0x1c9e2920a331 <Very long string[460960089]>,encoding=0x1c9e2dd3d4f1 <String[4]: utf8>)
    2: arguments adaptor frame: 3->2
    3: fromString(aka fromString) [0x1c9eb231c639] [buffer.js:~335] [pc=0x200b58cec790](this=0x1c9e693826f1 ...

 1: 0x10003ae75 node::Abort() [/usr/local/bin/node]
 2: 0x10003b07f node::OnFatalError(char const*, char const*) [/usr/local/bin/node]
 3: 0x1001a7ae5 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node]
 4: 0x100572ef2 v8::internal::Heap::FatalProcessOutOfMemory(char const*) [/usr/local/bin/node]
 5: 0x10057c3f4 v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/usr/local/bin/node]
 6: 0x10054e1e4 v8::internal::Factory::NewRawTwoByteString(int, v8::internal::PretenureFlag) [/usr/local/bin/node]
 7: 0x10067fd99 v8::internal::String::SlowFlatten(v8::internal::Handle<v8::internal::ConsString>, v8::internal::PretenureFlag) [/usr/local/bin/node]
 8: 0x1001c587d v8::String::Utf8Length() const [/usr/local/bin/node]
 9: 0x10004e7b6 node::Buffer::(anonymous namespace)::ByteLengthUtf8(v8::FunctionCallbackInfo<v8::Value> const&) [/usr/local/bin/node]
10: 0x200b585878a1 
The file /Users/mikegifford/purple-hats/results/2020-9-24/160096044136e10467e2/reports/report.html does not exist.

What's the best way to deal with this memory error

[younglim]

No longer replicated on current PlaywrightCrawler / Playwright used in Purple A11y