Another example of an unplannable graph

[johnynek]

- all optimization rules do not increase steps 

arg0 = WithDescriptionTypedPipe(Mapped(WithDescriptionTypedPipe(CrossPipe(WithDescriptionTypedPipe(Filter(IterablePipe(List(1406023175)),<function1>),List((org.scalacheck.Gen$R$class.map(Gen.scala:237),true))),WithDescriptionTypedPipe(TrappedPipe(WithDescriptionTypedPipe(ForceToDisk(WithDescriptionTypedPipe(Mapped(WithDescriptionTypedPipe(FilterKeys(WithDescriptionTypedPipe(Mapped(WithDescriptionTypedPipe(ForceToDisk(WithDescriptionTypedPipe(MergedTypedPipe(IterablePipe(List(1)),WithDescriptionTypedPipe(Fork(IterablePipe(List(1))),List((org.scalacheck.Gen$R$class.map(Gen.scala:237),true)))),List((org.scalacheck.Gen$R$class.map(Gen.scala:237),true)))),List((org.scalacheck.Gen$R$class.map(Gen.scala:237),true))),<function1>),List((org.scalacheck.Gen$R$class.map(Gen.scala:237),true))),<function1>),List((org.scalacheck.Gen$R$class.map(Gen.scala:237),true))),<function1>),List((org.scalacheck.Gen$R$class.map(Gen.scala:237),true)))),List((org.scalacheck.Gen$R$class.map(Gen.scala:237),true))),com.twitter.scalding.source.FixedTypedText(frco8uwemnb3cyHuwd9Feqeqrsc6ceqhsEnNOUmhbnk1apqnjs2IhzxMcg),Single(com.twitter.scalding.TupleGetter$IntGetter$@20d3d9e3)),List((org.scalacheck.Gen$R$class.map(Gen.scala:237),true)))),List((org.scalacheck.Gen$R$class.map(Gen.scala:237),true))),<function1>),List((org.scalacheck.Gen$R$class.map(Gen.scala:237),true))),
[info]       arg1 = com.twitter.scalding.typed.OptimizationRules$ComposeFlatMap$@6091c00e.orElse(com.twitter.scalding.typed.OptimizationRules$ComposeMap$@66a991c1).orElse(com.twitter.scalding.typed.OptimizationRules$ComposeFilter$@6c399402).orElse(com.twitter.scalding.typed.OptimizationRules$ComposeWithOnComplete$@3f0e4827).orElse(com.twitter.scalding.typed.OptimizationRules$ComposeMapFlatMap$@51717e78).orElse(com.twitter.scalding.typed.OptimizationRules$ComposeFilterFlatMap$@7b7cca0e).orElse(com.twitter.scalding.typed.OptimizationRules$EmptyIterableIsEmpty$@5bc5d8e7).orElse(com.twitter.scalding.typed.OptimizationRules$DescribeLater$@1647ecc8).orElse(com.twitter.scalding.typed.OptimizationRules$DiamondToFlatMap$@4c1fc2c6).orElse(com.twitter.scalding.typed.OptimizationRules$RemoveDuplicateForceFork$@51ae5df1).orElse(com.twitter.scalding.typed.OptimizationRules$IgnoreNoOpGroup$@24d9a5c3).orElse(com.twitter.scalding.typed.OptimizationRules$DeferMerge$@d513c3c).orElse(com.twitter.scalding.typed.OptimizationRules$FilterKeysEarly$@3feac6a2).orElse(com.twitter.scalding.typed.OptimizationRules$FilterLocally$@1c08457b).orElse(com.twitter.scalding.typed.OptimizationRules$EmptyIsOftenNoOp$@75960c84)

giving the cascading stack:

[info]   Cause: java.lang.NullPointerException:
[info]   at java.util.Objects.requireNonNull(Objects.java:203)
[info]   at java.util.Arrays$ArrayList.<init>(Arrays.java:3813)
[info]   at java.util.Arrays.asList(Arrays.java:3800)
[info]   at cascading.pipe.Splice.<init>(Splice.java:280)
[info]   at cascading.pipe.Splice.<init>(Splice.java:232)
[info]   at cascading.pipe.Splice.<init>(Splice.java:192)
[info]   at cascading.pipe.Splice.<init>(Splice.java:181)
[info]   at cascading.pipe.Splice.<init>(Splice.java:428)
[info]   at cascading.pipe.Merge.<init>(Merge.java:49)

I don't think this is a problem in production because we are testing unoptimized vs optimized graphs. The unoptimized graphs sometimes fail because cascading either has bugs or some other kinds of rules we don't follow about how Merge works. It would be nice to add a test for this graph so we can make sure even unoptimized scalding graphs will plan correctly.