Stackoverflow on bigger jsons

[mcenkar]

Hi,

if you run following snippet parsing json you will get stackoverflow error:

Exception in thread "main" java.lang.StackOverflowError
    at spray.json.lenses.Ops$$anon$3.spray$json$lenses$Ops$$anon$$inner$1(Ops.scala:63)
    at spray.json.lenses.Ops$$anon$3$$anonfun$spray$json$lenses$Ops$$anon$$inner$1$1.apply(Ops.scala:67)
    at spray.json.lenses.Ops$$anon$3$$anonfun$spray$json$lenses$Ops$$anon$$inner$1$1.apply(Ops.scala:66)
    at scala.util.Either$RightProjection.flatMap(Either.scala:522)
    at spray.json.lenses.Ops$$anon$3.spray$json$lenses$Ops$$anon$$inner$1(Ops.scala:66)
    at spray.json.lenses.Ops$$anon$3$$anonfun$spray$json$lenses$Ops$$anon$$inner$1$1.apply(Ops.scala:67)
    at spray.json.lenses.Ops$$anon$3$$anonfun$spray$json$lenses$Ops$$anon$$inner$1$1.apply(Ops.scala:66)
    at scala.util.Either$RightProjection.flatMap(Either.scala:522)
    at spray.json.lenses.Ops$$anon$3.spray$json$lenses$Ops$$anon$$inner$1(Ops.scala:66)
    at spray.json.lenses.Ops$$anon$3$$anonfun$spray$json$lenses$Ops$$anon$$inner$1$1.apply(Ops.scala:67)
    at spray.json.lenses.Ops$$anon$3$$anonfun$spray$json$lenses$Ops$$anon$$inner$1$1.apply(Ops.scala:66)
    at scala.util.Either$RightProjection.flatMap(Either.scala:522)

object SO extends App {
  import spray.json.lenses.JsonLenses._
  import spray.json.DefaultJsonProtocol._

  val lens = 'aggregations / 'unique_values / 'buckets / * / 'key

  val bucket = List.fill(3000)("""{"key": 123456789,"doc_count": 1}""")
  val json =
    s"""
       |{
       |  "took": 53,
       |  "timed_out": false,
       |  "_shards": {
       |    "total": 5,
       |    "successful": 5,
       |    "failed": 0
       |  },
       |  "hits": {
       |    "total": 11581,
       |    "max_score": 0.0,
       |    "hits": [
       |
       |    ]
       |  },
       |  "aggregations": {
       |    "unique_values": {
       |      "doc_count_error_upper_bound": 0,
       |      "sum_other_doc_count": 0,
       |      "buckets": [
       |        ${bucket.mkString(",")}
       |      ]
       |    }
       |  }
       |}
    """.stripMargin
  println(json.extract[Long](lens))

}

For me it fails on 'bucket containing about 2100 elements, which is not actually that big value I suppose. Everything below works quite well.

Cheers,
Maciek

[jrudolph]

Thanks, @mcenkar. Indeed, the code in question uses a recursive algorithm which is both slower and less reliable. This needs to be fixed.

[jrudolph]

I just released 0.6.1 with this fix included.