/PythonSparkLogParser

Primary LanguagePythonApache License 2.0Apache-2.0

PythonSparkLogParser

A python script that operates a simple parse into csv files of a Spark log. To use the parser:

  1. Type python parser.py [PATH_TO_JSON_FILE_TO_PARSE] [ID_TO_BE_PUT_INTO_CSV_FILENAMES]
  2. You will find the csv files resulting from the parsing process under the output folder

Note that the parser.py contains default parsing fields that can be changed inside the code. Note that Python 2.7 is required.