hooman650/spark-nlp-workshop

Public runnable examples of using John Snow Labs' NLP for Apache Spark.

Spark NLP Workshop

Showcasing notebooks and codes of how to use Spark NLP in Python and Scala.

Table of contents

• Jupyter Notebooks
  • annotation
  • evalulation
  • training
• Tutorial Notebooks
  • Jupyter
  • Colab (for Google Colab)
• Databricks Notebooks

Python Setup

$ java -version
# should be Java 8 (Oracle or OpenJDK)
$ conda create -n sparknlp python=3.6 -y
$ conda activate sparknlp

# Install Spark NLP and PySpark 2.4.x
$ pip install spark-nlp pyspark==2.4.7

Colab setup

import os

# Install JDK 8
! apt-get update -qq
! apt-get install -y openjdk-8-jdk-headless -qq > /dev/null

os.environ["JAVA_HOME"] = "/usr/lib/jvm/java-8-openjdk-amd64"
os.environ["PATH"] = os.environ["JAVA_HOME"] + "/bin:" + os.environ["PATH"]
! java -version

# Install PySpark 2.4.x
! pip install -q pyspark==2.4.7
! pip install -q spark-nlp

Main repository

https://github.com/JohnSnowLabs/spark-nlp

Project's website

Take a look at our official spark-nlp page: http://nlp.johnsnowlabs.com/ for user documentation and examples

Slack community channel

Join Slack

Contributing

If you find any example that is no longer working, please create an issue.

License

Apache Licence 2.0