/sparkr-tutorials

Code snippets and tutorials for working with SparkR.

Primary LanguageROtherNOASSERTION

sparkr-tutorials

Description: Code snippets and tutorials for working with SparkR.

Spark Social Science Manual

The tutorials included in this repository are geared towards social scientists and policy researchers that want to undertake research using "big data" sets. A manual to accompany these tutorials is linked below. The objective of the manual is to provide social scientists with a brief overview of the distributed computing solution developed by The Urban Institute's Research Programming Team, and of the changes in how researchers manage and analyze data required by this computing environment.

Spark Social Science Manual

Getting Started with SparkR Tutorials

Last Updated: May 23, 2017

In order to begin working with SparkR, users must first:

  1. Make sure that SPARK_HOME is set in environment (using Sys.getenv)
  2. Load the SparkR library
  3. Initiate a sparkR.session
if (nchar(Sys.getenv("SPARK_HOME")) < 1) {
  Sys.setenv(SPARK_HOME = "/home/spark")
}

# Load the SparkR library
library(SparkR)

# Initiate a SparkR session
sparkR.session()

❗ The expressions given above must be evaluated in SparkR before beginning any of the tutorials hosted here. Example data loading is included in each tutorial.

Users can end a SparkR session with the following expression:

sparkR.session.stop()

Note: the data visualization tutorial, linked to below, is currently not updated for SparkR 2.0, but does still function with SparkR 1.6.

Table of Contents:

Created by gh-md-toc