Running out of memory during large quickCheck' runs

[seagreen]

Is quickCheck' supposed to run in constant space? I'm running out of memory if I tell it to run a large number of tests (over 5000).

Also, I'm curious why stdout includes 10000/10000 test(s) passed. right before the tests crash. It seems weird for the program to run out of memory if the actual testing is complete.

Code:

module Test.Main where

import Prelude

import Control.Monad.Eff (Eff())
import Data.Either
import Data.Maybe

import Data.Argonaut.Core
import Data.Argonaut.Parser
import Data.Argonaut.Printer
import Test.StrongCheck ((<?>), quickCheck', Result())

f :: Number -> Result
f num =
  case jsonParser (printJson (fromNumber num)) of
    Left e -> false <?> ("jsonParser problem: " <> e)
    Right a ->
      case foldJsonNumber Nothing Just a of
        Nothing -> false <?> "not a number"
        Just b -> b == num <?> "nonequal"

main :: Eff _ Unit
main = quickCheck' 10000 f

Results:

vagrant@mytrusty:/var/apps/purescript-halogen-template$ pulp test
* Building project in /var/apps/purescript-halogen-template
psc: No files found using pattern: src/**/*.js
psc: No files found using pattern: test/**/*.js
Warning found:
in module Test.Main
at /var/apps/purescript-halogen-template/test/Main.purs line 23, column 1 - line 24, column 1

  Wildcard type definition has the inferred type

    ( err :: EXCEPTION
    , random :: RANDOM
    , console :: CONSOLE
    | _14
    )


in value declaration main

See https://github.com/purescript/purescript/wiki/Error-Code-WildcardInferredType for more information,
or to contribute content related to this warning.

* Build successful.
* Running tests...
10000/10000 test(s) passed.

<--- Last few GCs --->

   21704 ms: Scavenge 1388.7 (1435.0) -> 1387.1 (1435.0) MB, 27.2 / 0 ms (+ 7.2 ms in 31 steps since last GC) [allocation failure].
   21740 ms: Scavenge 1388.7 (1435.0) -> 1388.3 (1435.0) MB, 28.6 / 0 ms (+ 5.9 ms in 25 steps since last GC) [allocation failure].
   21771 ms: Scavenge 1388.7 (1435.0) -> 1388.7 (1435.0) MB, 29.4 / 0 ms (+ 1.5 ms in 6 steps since last GC) [allocation failure].
   21801 ms: Scavenge 1388.7 (1435.0) -> 1388.7 (1435.0) MB, 29.6 / 0 ms [allocation failure].


<--- JS stacktrace --->
Cannot get stack trace in GC.
FATAL ERROR: MarkCompactCollector: semi-space copy, fallback in old gen Allocation failed - JavaScript heap out of memory
* ERROR: Subcommand terminated as a result of receiving SIGABRT

[seagreen]

Oh, maybe the problem is that quickCheck' just eventually generates an enormous test case and that's what causes the crash, instead of a memory leak. I'll look into it...

[garyb]

We've had this problem when running a large number of iterations of a test too, but not investigated enough to figure out what the real cause is. Thanks for taking a look, it would be great if we could figure it out as this sorta spoils one of the reasons to use SC over QC - SC is stack safe, just apparently it'll consume too much memory instead.

[seagreen]

Sorry I never got around to taking an in-depth look at this: I've moved on to a different project, so feel free to close this/continue research as you wish, just don't wait around for me.

[syaiful6]

look like this cause by Node.js known issue: nodejs/node#6456