scalytics/SDE

Scalytics Connect development environment, pre-build

Scalytics Connect Development Environment (SDE)

Read our Wiki how to use BDE, with code examples and how-to's

SDE comes with:

• Java 11
• Maven 3.6.1
• Apache Wayang 0.6.1
• Apache Hadoop 3.1.2
• Apache Spark 3.1.2
• iJava 1.3.0
• Jupyter lab 3.3.0

Starting the SDE

You can bring the SDE up and running by executing the command

docker compose up

Using custom input files

If you need the BDE to process your custom files, you can do this by altering the docker-compose.yml.

Change this part

  jupyter:
    build: jupyter
    ports:
      - "8888:8888"
    extra_hosts:
      - "host.docker.internal:host-gateway"  

to this:

  jupyter:
    build: jupyter
    ports:
      - "8888:8888"
    extra_hosts:
      - "host.docker.internal:host-gateway"
    volumes:
      - /path/to/custom/input/files:/home/jovyan/files

Your files should be located in a known location on your machine (referred to as /path/to/custom/input/files). These files will be accessible within your BDE under the path files (for example, files/my.custom.file).

Connecting to a database on localhost

Use host.docker.internal instead of localhost.