/APACHE-SPARK-COM-DELTA-LAKE-E-APACHE-ICEBERG

Apache Spark com Delta Lake e Apache Iceberg

Primary LanguageJupyter Notebook

Integrantes

GuilhermeVolpato
GuilhermeMSilveira
luizotavio-vieira

Links Uteis

https://www.youtube.com/watch?v=eOrWEsZIfKU - Como sair do ZERO no DELTA LAKE e PYSPARK em apenas 1H

https://www.youtube.com/watch?v=WwrX1YVmOyA - Como sair do ZERO no PySpark em 2024!

Configuração do ambiente:

  • Windows
    • Download python - https://www.python.org/downloads/
    • Download poetry - (Invoke-WebRequest -Uri https://install.python-poetry.org -UseBasicParsing).Content | py -
    • Download dependencias - poetry add pyspark==3.4.2 delta-spark==2.4.0 jupyterlab
  • Mac
    • Download python - https://www.python.org/downloads/
    • Download poetry - brew install poetry
    • Download dependencias - poetry add pyspark==3.4.2 delta-spark==2.4.0 jupyterlab

Rodar projeto:

  • poetry shell
  • jupyter-labs

Link DataFrame