- Hadoop 2.7.7
- Spark 2.4.5
- Scala 2.12.10
- Avro file format
- BigQuery connector
- Cloud Storage connector
Base64's encoded Account Service File
Spark context
docker build -t felipeortega/spark-2.4 .
docker run -p 4040:4040 -e GOOGLE_APPLICATION_CREDENTIALS=$(base64 -w0 /path/to/json) -it --rm --name spark24 felipeortega/spark-2.4
Executes /app/app.jar
docker run -p 4040:4040 -e GOOGLE_APPLICATION_CREDENTIALS=$(base64 -w0 /path/to/json) -v $PWD/app:/app -it --rm --name spark24 felipeortega/spark-2.4