/spark-ipservices

Example spark application using scala pattern matching to injest /etc/services

Primary LanguageScala

spark-ipservices

An example spark application to demonstrate the exercise of importing a file and saving to a parquet table. This demonstrates using Scala Pattern Matching for efficient and effective parsing.

For this example, we use the IP Services file as defined on all unix systems in '/etc/services'.

This file serves as a more complicated parsing example as fields are tab-delimited, with four (sparse) columns, with two of the four columns being optional.