Stackoverflow in typechecker.Typers$Typer in scala 2.12.3/2.12.4
fanf opened this issue · 25 comments
I'm observing a stackoverflow in typechecker with ScalaIDE 4.7.0 when using scalac 2.12.3 or 2.12.4, which is not present with scala 2.12.2. The stackoverflow does not happen when I'm compiling from the command line using maven.
I saw a lot of variation of the problem (call site initialization exception and java.lang.UnsupportedOperationException: Position.point on NoPosition) but these other problem seems to be consequences of the stackoverflow.
Of course I tried to set a bigger Xss, but it does not change anything (even with 256m), and the fact that the maven compilation works or that it works on 2.12.2 with normal Xss tends to let me think that it is a genuine bug.
I don't have at all a minimal reproducer, and the stacktraces are not very telling, but the problem happen with an open source project: https://github.com/Normation/rudder/ (on branch master).
It may or not may be related to Daniel "infinite macro expansion in scala 2.12.3/2.12.4" #10584 or perhaps both have same root cause (rudder-core uses doobie and so macro expansion).
Nonetheless, here are the SOE and other exception I saw:
2017-10-26 18:33:47,710 ERROR [main] - org.scala-ide.sdt.core - org.scala-ide.sdt.core - org.scala-ide.sdt.core - 0 - Error in Scala compiler
java.lang.StackOverflowError
at scala.tools.nsc.typechecker.Contexts$Context.make(Contexts.scala:463)
at scala.tools.nsc.typechecker.Contexts$Context.makeNewScope(Contexts.scala:516)
at scala.tools.nsc.typechecker.Typers$Typer.typedOutsidePatternMode$1(Typers.scala:5511)
at scala.tools.nsc.typechecker.Typers$Typer.typedInAnyMode$1(Typers.scala:5546)
at scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:5553)
at scala.tools.nsc.typechecker.Typers$Typer.runTyper$1(Typers.scala:5589)
at scala.tools.nsc.typechecker.Typers$Typer.typedInternal(Typers.scala:5619)
at scala.tools.nsc.typechecker.Typers$Typer.body$2(Typers.scala:5563)
at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5567)
at scala.tools.nsc.typechecker.Typers$Typer.typedQualifier(Typers.scala:5670)
at scala.tools.nsc.typechecker.Typers$Typer.typedQualifier(Typers.scala:5676)
at scala.tools.nsc.typechecker.Typers$Typer.typedSelectOrSuperCall$1(Typers.scala:4999)
at scala.tools.nsc.typechecker.Typers$Typer.typedInAnyMode$1(Typers.scala:5537)
at scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:5553)
at scala.tools.nsc.typechecker.Typers$Typer.runTyper$1(Typers.scala:5589)
at scala.tools.nsc.typechecker.Typers$Typer.typedInternal(Typers.scala:5619)
at scala.tools.nsc.typechecker.Typers$Typer.body$2(Typers.scala:5563)
at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5567)
at scala.tools.nsc.typechecker.Typers$Typer.$anonfun$typed1$38(Typers.scala:4697)
at scala.tools.nsc.typechecker.Typers$Typer.silent(Typers.scala:699)
at scala.tools.nsc.typechecker.Typers$Typer.normalTypedApply$1(Typers.scala:4699)
at scala.tools.nsc.typechecker.Typers$Typer.typedApply$1(Typers.scala:4746)
at scala.tools.nsc.typechecker.Typers$Typer.typedInAnyMode$1(Typers.scala:5536)
at scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:5553)
at scala.tools.nsc.typechecker.Typers$Typer.runTyper$1(Typers.scala:5589)
at scala.tools.nsc.typechecker.Typers$Typer.typedInternal(Typers.scala:5619)
at scala.tools.nsc.typechecker.Typers$Typer.body$2(Typers.scala:5563)
.... and on and on...
It may, perhaps, not sure anymore at anything, that the problem might be link with the use of the "-verbose" option. Removing that flag seems to make it disapear (but as I don't have a clean way to trigger it in the first place, I'm not sure of anything).
Yeah we're seeing java.lang.UnsupportedOperationException: Position.point on NoPosition.
Upgrading from 2.12.3 to 2.12.4 introduced a StackOverflowError in inductive shapeless derivation in the scalafix repo scalacenter/scalafix#477 (comment) Increasing -Xss8m fixed the problem. This regression took a while to track down since the SO was caught by the compiler during implicit expansion and only reported under -Xlog-implicits. I'm not sure if it's directly related to this PR, but wanted to leave a note somewhere. At least, I'm not the only one hitting on SO while upgrading in 2.12.x
I took the liberty to open another issues about the fact that the SO was getting swallowed and not reported without -Xlog-implicits #10649
I've a variant of that one on IntelliJ (I believe I also got it from time to time on scala IDE) with a NPE:
Error:scala: ## Exception when compiling 55 sources to /home/fanf/java/workspaces/rudder-project/rudder/rudder-core/target/test-classes
null
scala.tools.nsc.typechecker.Typers$Typer.typedBlock(Typers.scala:2486)
scala.tools.nsc.typechecker.Typers$Typer.$anonfun$typed1$96(Typers.scala:5505)
scala.tools.nsc.typechecker.Typers$Typer.$anonfun$typerWithLocalContext$1(Typers.scala:486)
scala.tools.nsc.typechecker.Typers$Typer.typedOutsidePatternMode$1(Typers.scala:486)
scala.tools.nsc.typechecker.Typers$Typer.typedInAnyMode$1(Typers.scala:5540)
scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:5547)
scala.tools.nsc.typechecker.Typers$Typer.runTyper$1(Typers.scala:5584)
scala.tools.nsc.typechecker.Typers$Typer.typedInternal(Typers.scala:5616)
scala.tools.nsc.typechecker.Typers$Typer.body$2(Typers.scala:5557)
scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5562)
scala.tools.nsc.typechecker.Typers$Typer.typedQualifier(Typers.scala:5667)
scala.tools.nsc.typechecker.Typers$Typer.typedQualifier(Typers.scala:5673)
scala.tools.nsc.typechecker.Typers$Typer.typedSelectOrSuperCall$1(Typers.scala:5008)
scala.tools.nsc.typechecker.Typers$Typer.typedInAnyMode$1(Typers.scala:5531)
scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:5547)
scala.tools.nsc.typechecker.Typers$Typer.runTyper$1(Typers.scala:5584)
scala.tools.nsc.typechecker.Typers$Typer.typedInternal(Typers.scala:5616)
scala.tools.nsc.typechecker.Typers$Typer.body$2(Typers.scala:5557)
scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5562)
scala.tools.nsc.typechecker.Typers$Typer.$anonfun$typed1$38(Typers.scala:4708)
scala.tools.nsc.typechecker.Typers$Typer.silent(Typers.scala:698)
scala.tools.nsc.typechecker.Typers$Typer.normalTypedApply$1(Typers.scala:4710)
scala.tools.nsc.typechecker.Typers$Typer.typedApply$1(Typers.scala:4757)
scala.tools.nsc.typechecker.Typers$Typer.typedInAnyMode$1(Typers.scala:5530)
scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:5547)
scala.tools.nsc.typechecker.Typers$Typer.runTyper$1(Typers.scala:5584)
scala.tools.nsc.typechecker.Typers$Typer.typedInternal(Typers.scala:5616)
scala.tools.nsc.typechecker.Typers$Typer.body$2(Typers.scala:5557)
scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5562)
scala.tools.nsc.typechecker.Typers$Typer.typedBlock(Typers.scala:2457)
scala.tools.nsc.typechecker.Typers$Typer.$anonfun$typed1$96(Typers.scala:5505)
scala.tools.nsc.typechecker.Typers$Typer.$anonfun$typerWithLocalContext$1(Typers.scala:486)
scala.tools.nsc.typechecker.Typers$Typer.typedOutsidePatternMode$1(Typers.scala:486)
scala.tools.nsc.typechecker.Typers$Typer.typedInAnyMode$1(Typers.scala:5540)
scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:5547)
scala.tools.nsc.typechecker.Typers$Typer.runTyper$1(Typers.scala:5584)
scala.tools.nsc.typechecker.Typers$Typer.typedInternal(Typers.scala:5616)
...... and again and again and again
Is this issue being actively worked on by anyone? We suddenly started getting the same problem on my project at work. We bumped the stack size which has fixed the issue but, that seems more like a band-aid rather than an actual fix.
We are seeing this issue when compiling with Maven on the command line instead of just the IDE.
Same here with 2.12.4
bump
fwiw, Im having this issue w/ shapeless / slickless in Scala 2.11.11
some of y'all know this already, but maybe not everyone does: it's considered normal in Scala that you may have to increase JVM stack size in order to compile some kinds of especially involved or nested Scala code.
does anyone know a way to reproduce the problem they're having on Scala 2.12.5 (yes, I mean 2.12.5 specifically, now that it's out), with a minimal example and without needing access to your project's entire source tree? without that, it's really hard to know whether this ticket even identifies a specific issue, or whether it's just a miscellaneous collection of people reporting a miscellaneous collection of StackOverflowErrors in miscellaneous versions of Scala, which could have miscellaneous causes.
(I'm not criticizing anyone who's reported or commented here, but in order to actually do anything about this, we need to converge on something more specific and actionable.)
note that 2.12.5 fixes #10552, so trying again in 2.12.5 may produce different error logs that may continue additional clues that we didn't have before.
I had the same issue and I can confirm that increasing of the stack size from default one (1MB to 5MB) helped.
closing since there isn't currently anything actionable here, unless new information comes to light.
Hello,
We had the same kind of problem today at work.
We wanted to write a very big case class but encountered these compilation errors.
The errors were fixed with the magical -Xss2048K flag.
case class VeryBigRecord(
x001: Int,
x002: Int,
x003: Int,
x004: Int,
x005: Int,
x006: Int,
...
x178: Int
)
I have reproduced the bug in a minimal reproducible way.
https://github.com/mycaule/scalabug10604-testcase
It might be the same stuff as the 22 fields case class limit earlier.
@mycaule unrelated to the 22 limit. it's:
it's considered normal in Scala that you may have to increase JVM stack size in order to compile some kinds of especially involved or nested Scala code.
I'm getting this same error with 2.12.7 and I bumped the memory from 1GB to 4GB to no effect:
$ sbt -v ✔ 8106 07:48:28
[process_args] java_version = '11'
[copyRt] java9_rt = '/Users/eric/.sbt/0.13/java9-rt-ext-oracle_corporation_11_0_1/rt.jar'
# Executing command line:
java
-Xms16384m
-Xmx16384m
-XX:ReservedCodeCacheSize=512m
-XX:MaxMetaspaceSize=1024m
-XX:+UseParallelGC
-Dscala.ext.dirs=/Users/eric/.sbt/0.13/java9-rt-ext-oracle_corporation_11_0_1
-jar
/usr/local/Cellar/sbt/1.2.6/libexec/bin/sbt-launch.jar
[error] ## Exception when compiling 22 sources to /Users/eric/Work/ta-webservice-api/ws-generator/target/scala-2.12/classes
[error] null
[error] scala.tools.nsc.transform.Erasure$Eraser.adaptMember(Erasure.scala:670)
[error] scala.tools.nsc.transform.Erasure$Eraser.typed1(Erasure.scala:789)
[error] scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5617)
[error] scala.tools.nsc.typechecker.Typers$Typer.$anonfun$typed1$38(Typers.scala:4746)
[error] scala.tools.nsc.typechecker.Typers$Typer.silent(Typers.scala:693)
[error] scala.tools.nsc.typechecker.Typers$Typer.normalTypedApply$1(Typers.scala:4748)
[error] scala.tools.nsc.typechecker.Typers$Typer.typedApply$1(Typers.scala:4776)
[error] scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:5571)
[error] scala.tools.nsc.transform.Erasure$Eraser.typed1(Erasure.scala:789)
[error] scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5617)
[error] scala.tools.nsc.transform.Erasure$Eraser.adaptMember(Erasure.scala:714)
[error] scala.tools.nsc.transform.Erasure$Eraser.typed1(Erasure.scala:789)
[error] scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5617)
[error] scala.tools.nsc.typechecker.Typers$Typer.$anonfun$typed1$38(Typers.scala:4746)
[error] scala.tools.nsc.typechecker.Typers$Typer.silent(Typers.scala:693)
Java 11 issue?
$ java -version
java version "11.0.1" 2018-10-16 LTS
Java(TM) SE Runtime Environment 18.9 (build 11.0.1+13-LTS)
Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11.0.1+13-LTS, mixed mode)
I also have the same issue in scala 2.12.2, but it has a much more interesting stack trace:
java.lang.AssertionError: assertion failed: class Any: class Definitions
at scala.tools.nsc.symtab.classfile.ClassfileParser.parseMethod(ClassfileParser.scala:570)
at scala.tools.nsc.symtab.classfile.ClassfileParser.$anonfun$parseClass$4(ClassfileParser.scala:470)
at scala.tools.nsc.symtab.classfile.ClassfileParser.queueLoad$1(ClassfileParser.scala:470)
at scala.tools.nsc.symtab.classfile.ClassfileParser.$anonfun$parseClass$5(ClassfileParser.scala:480)
at scala.tools.nsc.symtab.classfile.ClassfileParser.parseClass(ClassfileParser.scala:485)
at scala.tools.nsc.symtab.classfile.ClassfileParser.$anonfun$parse$1(ClassfileParser.scala:156)
at scala.tools.nsc.symtab.classfile.ClassfileParser.parse(ClassfileParser.scala:127)
at scala.tools.nsc.symtab.SymbolLoaders$ClassfileLoader.doComplete(SymbolLoaders.scala:316)
at scala.tools.nsc.symtab.SymbolLoaders$SymbolLoader.complete(SymbolLoaders.scala:218)
at scala.reflect.internal.Symbols$Symbol.info(Symbols.scala:1521)
at scala.tools.nsc.backend.jvm.BCodeHelpers.completeSilentlyAndCheckErroneous(BCodeHelpers.scala:235)
at scala.tools.nsc.backend.jvm.BTypesFromSymbols.$anonfun$classBTypeFromSymbol$5(BTypesFromSymbols.scala:122)
at scala.collection.MapLike.getOrElse(MapLike.scala:128)
at scala.collection.MapLike.getOrElse$(MapLike.scala:126)
at scala.collection.concurrent.TrieMap.getOrElse(TrieMap.scala:631)
at scala.tools.nsc.backend.jvm.BTypesFromSymbols.classBTypeFromSymbol(BTypesFromSymbols.scala:118)
at scala.tools.nsc.backend.jvm.BTypesFromSymbols.$anonfun$setClassInfo$17(BTypesFromSymbols.scala:440)
at scala.tools.nsc.backend.jvm.BTypesFromSymbols.setClassInfo(BTypesFromSymbols.scala:440)
at scala.tools.nsc.backend.jvm.BTypesFromSymbols.$anonfun$classBTypeFromSymbol$5(BTypesFromSymbols.scala:126)
at scala.collection.MapLike.getOrElse(MapLike.scala:128)
at scala.collection.MapLike.getOrElse$(MapLike.scala:126)
at scala.collection.concurrent.TrieMap.getOrElse(TrieMap.scala:631)
at scala.tools.nsc.backend.jvm.BTypesFromSymbols.classBTypeFromSymbol(BTypesFromSymbols.scala
Full log: https://gist.github.com/er1c/4650e34b02c92a1a4c00bdaf1889f598
It works successfully on 2.12.3
@Jasper-M yeah I was basically playing with multiple options if you see above, sbt is running with 16GB, are you simply talking about javaOptions in (Compile, run) ++= Seq("-Xms4G", "-Xmx4G"), sbt launcher memory via /usr/local/etc/sbtopts, or .jvmopts? What's interesting is it compiles in 2.12.3 but not 2.12.7 or 2.12.2
Ahh, I missed/overlooked the -Xms vs -Xss, and that isn't part of the /usr/local/etc/sbtopts mem options.
.jvmopts file with:
-Xss4m
Does work fine with 2.12.7
👍 also note that the scala compiler runs in the same JVM as sbt, so you need to start sbt with enough Xss
@lrytz the .jvmopts file gets picked up by the sbt launcher:
$ sbt -v
[process_args] java_version = '11'
[copyRt] java9_rt = '/Users/eric/.sbt/0.13/java9-rt-ext-oracle_corporation_11_0_1/rt.jar'
# Executing command line:
java
-Xms1024m
-Xmx1024m
-XX:ReservedCodeCacheSize=128m
-XX:MaxMetaspaceSize=256m
-XX:+UseParallelGC
-Xss4m
-Dscala.ext.dirs=/Users/eric/.sbt/0.13/java9-rt-ext-oracle_corporation_11_0_1
-jar
/usr/local/Cellar/sbt/1.2.6/libexec/bin/sbt-launch.jar
你好，
我们今天在工作中遇到了同样的问题。 我们想写一个非常大的
case class但是遇到了这些编译错误。 错误已通过魔法-Xss2048K标志修复。case class VeryBigRecord( x001: Int, x002: Int, x003: Int, x004: Int, x005: Int, x006: Int, ... x178: Int )我以最小的可重现方式重现了该错误。 https://github.com/mycaule/scalabug10604-testcase
它可能与之前的 22 个字段案例类限制相同。
Hello, I have the same problem as you. I created a very large case class and encountered these errors while compiling. I solved the problem by increasing stack memory (-Xss2048m), but I don't understand the cause, do you know the cause?
@Mr-HHP是“-Xss5m”可以解决问题
This solves the problem, but I'm not sure why. What does "Scala" do with compiling "case Class"
@Mr-HHP I don't know in particular, but I was just looking at how it typechecks a function application, and a case class's synthetic unapply will involve two apply similar to
Some.apply[(Int, Int)](scala.Tuple2.apply[Int, Int](x$0.x, x$0.y))
Each apply requires checking first the Some.apply itself (typedQualifier in the stack trace) and then the application of the args. Possibly it consumes stack inferring those type args, if they are not specified in the synthetic code. The other stack trace is in erasure, which I don't know about.
Edit: see #12397