Python or Java support?
Closed this issue · 3 comments
Hello,
I was wondering if the HyperLogLog functions can be used in Java or Python (through Pyspark). I would assume so for Java. If there is support, I was wondering if examples can be posted on the Wiki.
Thank you!
Hi,
Based on these:
https://github.com/swoop-inc/spark-alchemy/wiki/Spark-Native-Functions
To use the functions from SparkSQL, you have to register them by calling the equivalent of HLLFunctionRegistration.register(spark)
It seems like you can use all the functions here through SparkSQL:
https://github.com/swoop-inc/spark-alchemy/blob/master/alchemy/src/main/scala/com/swoop/alchemy/spark/expressions/hll/HLLFunctionRegistration.scala#L9-L16
Here is how to register the Spark native functions for HLL from Python: #9 (comment)
Java can use the Scala registration code directly.