Apache Beam provides Java and Python SDKs for writing the pipelines. For writing beam pipelines in scala, there are two options available at this time.
-
Use Scio, a scala API for Beam, close to that of Spark and Scalding core APIs. This API is still not part of Apache Beam, and often lags behind the official Beam releases.
-
Use the Beam Java SDK in scala applications. This repo shows you how to use the Java API in your scala code.
The examples shown in this repo are same as that of the Java wordcount examples in Beam Example Pipelines. In order to understand the concepts, first go through the code & documentation here and then refer to the scala code for the same examples in this repo.
Licensed under the Apache License, Version 2.0: http://www.apache.org/licenses/LICENSE-2.0