Seq creation function
Jolanrensen opened this issue · 5 comments
For creating custom SparkSQL Expressions that use org.apache.spark.sql.catalyst.expressions.LambaFunction
and possibly in other places in the API a Scala Seq.
Note that the Java API almost never uses Seqs, so this function would only have to be used in edge cases.
Implementation could look like:
fun <T> seqOf(vararg elements: T): scala.collection.Seq<T> = scala.collection.JavaConverters.asScalaBuffer(elements.asList())
If I may add a few requests:
seqOf()
should return theNil
singleton- There should be functions to prepend and append to a
Seq
. - There should be a function to concatenate
Seq
s - It would be nice if you can destructure a
Seq
into its "head" and "rest" elements:val (head: T, rest: Seq<T>) = mySeq
Hmm, I'm doubting whether all those extra features would be in the scope of the API...
The goal is to make Spark work great from Kotlin, not to improve Scala-Kotlin interoperability per se.
We decided to add all the helper stuff for Scala Tuples because the Java Spark API explicitly requests Tuples and there are speed benefits for using Tuples all the way through instead of converting other classes to them. However, Seqs are not needed for the Java API apart from some very specific edge cases. So while a seqOf()
function would be in the scope, all those extras might be better placed in a separate library.
@tianyu I'm sorry, but it's definitely not on the plate right now. Thing is our goal is to provide idiomatic Kotlin API, not Scala API. Destructuring this way is alien to Kotlin, for Kotlin destructuring of any list-like structure returns elements.
However everything you want is definitely possible to implement as extension methods, for example
object Nil
fun seqOf() = Nil
Or for destructuring
fun Sequence.component0() = head()
fun Sequence.component1() = tail()
And so on.
I did btw find a better "direct" translation of Scala's Seq(a, b, c)
function which includes Nil
in the empty()
function:
fun <T> seqOf(vararg elements: T): Seq<T> = with(`Seq$`.`MODULE$`) {
if (elements.isEmpty()) {
empty<T>() as Seq<T>
} else {
newBuilder<T>().apply {
for (elem in elements)
`$plus$eq`(elem)
}.result()
}
}
added for 1.2.0