typelead/eta-examples

Get error when try to use PrefixSpanModel in spark

JulianZhang opened this issue · 4 comments

JulianZhang commented

Dear, I'm trying use PrefixSpanModel in spark mllib. I start with the spark-examples in this repo.

I had add blow code in the Types.hs

data {-# CLASS "org.apache.spark.mllib.fpm.PrefixSpan" #-} PrefixSpan = 
  PrefixSpan (Object# PrefixSpan)
  deriving Class

data {-# CLASS "org.apache.spark.mllib.fpm.PrefixSpanModel" #-} PrefixSpanModel  a= 
  PrefixSpanModel(Object# (PrefixSpanModel a))

and blow in Methods.hs

foreign import java safe run :: JArray e c => Java c (PrefixSpanModel a)

foreign import java safe freqSequences ::  Java () (JavaRDD a)

But I will get error when run etlas build


Preprocessing executable 'eta-spark' for eta-spark-0.1.0.0..
Building executable 'eta-spark' for eta-spark-0.1.0.0..
zhangjuns-MacBook-Pro:eta-spark zhangjun$ etlas build
Preprocessing executable 'eta-spark' for eta-spark-0.1.0.0..
Building executable 'eta-spark' for eta-spark-0.1.0.0..
[2 of 3] Compiling Spark.Methods    ( src/Spark/Methods.hs, dist/build/eta-spark/eta-spark-tmp/Spark/Methods.jar )

<no location info>:
    eta: panic! (the 'impossible' happened)
  (Eta version 0.0.9b2):
	rawTagTypeToText: You should annotate  ()

Please report this as a Eta bug: http://github.com/typelead/eta/issues

If I remove the line about freqSequences the error will change to

Preprocessing executable 'eta-spark' for eta-spark-0.1.0.0..
Building executable 'eta-spark' for eta-spark-0.1.0.0..
[2 of 3] Compiling Spark.Methods    ( src/Spark/Methods.hs, dist/build/eta-spark/eta-spark-tmp/Spark/Methods.jar )

<no location info>: expectJust tcSplitExtendsType
rahulmutt commented

Hi Julian,

foreign import java safe run :: JArray e c => Java c (PrefixSpanModel a)

JArray e c is not supported in the FFI right now, you'll have to do manual conversions yourself. There seems to be something off about this import, can you share the origin method that you're trying to import?

foreign import java safe freqSequences ::  Java () (JavaRDD a)

The first parameter in Java should be a JWT, in particular the class for which the method should run (if an instance method), and () is not a JWT. Perhaps you meant

foreign import java safe freqSequences ::  Java (PrefixSpanModel a) (JavaRDD (FreqSequence Item))

?

rahulmutt commented 
public <Item,Itemset extends java.lang.Iterable<Item>,Sequence extends java.lang.Iterable<Itemset>> PrefixSpanModel<Item> run(JavaRDD<Sequence> data)

This maps to

foreign import java unsafe run ::
  (item <: Object, itemset <: Iterable item, sequence <: Iterable itemset) =>
  JavaRDD sequence -> Java PrefixSpan (PrefixSpanModel item)

Make sure you do import Java.Collections so you get the Iterable JWT in scope.

JulianZhang commented

@rahulmutt Thanks，I had pass the build