Build new Spark-Tensorflow-Connector for scala 2.12
wuxianxingkong opened this issue · 5 comments
Spark has released 3.x which built with scala 2.12, but th newest version of Spark-Tensorflow-Connector is built with scala 2.11 which means not compatiable with spark 3.x. Maven repository is still 2.11.
Spark has released 3.x which built with scala 2.12, but th newest version of Spark-Tensorflow-Connector is built with scala 2.11 which means not compatiable with spark 3.x. Maven repository is still 2.11.
You can build 2.12 by yourself!!!
Spark has released 3.x which built with scala 2.12, but th newest version of Spark-Tensorflow-Connector is built with scala 2.11 which means not compatiable with spark 3.x. Maven repository is still 2.11.
Also, you use spark-tfrecord https://github.com/linkedin/spark-tfrecord, that support scala 2.12.
It would be great to have an example of how to build this for Spark 3.x and Tensorflow 2.x
I tried a number of combinations with no success. I can build the jar, but it writes an empty TFRecord file.
If others have successfully built the jar for Spark 3.x and Tensorflow 2.x I'd love to hear the details.
@dennisobrien Hi, try this: https://github.com/wuxianxingkong/spark-tfrecord (Built with spark 3.1.1 and scala 2.12)
It would be super helpful if TF team could kindly help releasing spark-tensorflow-connector built with scala 2.12 to maven artifact 🙌 Right now the best workaround seems to be spark-tfrecord but there'll be migration effort across products using it ..