This repository provides a "startup template" of specific Docker Dev Container. Developers can easily create, compile and run Hadoop MapReduce jobs with a "pseudo-distributed" deployment of Apache Hadoop v3.4.1.
Just follow these steps for happy development 🎉
-
Select "Use this template" button, a brand new project under your account based on this template.
-
Clone this repository to your local device.
-
Make sure you have Docker and Visual Studio Code already installed.
- Dev Container feature should be supported by your Docker installation.
- Dev Containers plugin should also be installed inside Visual Studio Code.
[!NOTE]
This is only manually verified in Windows 11 Professional Workstation with Docker Desktop for Windows, using WSL2 backend.
-
Open local repository with Visual Studio Code, it will automatically suggest you relaunch the project inside Dev Containers.
-
Follow its guide to build the container. You will be connected to the container soon after the success.
For convenience, developers will be connected to the container as user hadoop. The already-configured password is also hadoop.
Caution
This is only a Docker Dev Container. It's target for temporary tests.
DO NOT USE IN PRODUCTION!!!
# Start OpenSSH Server
sudo service ssh start
# Start HDFS Service
/usr/local/hadoop-3.4.1/sbin/start-dfs.sh
# Stop HDFS Service
/usr/local/hadoop-3.4.1/sbin/stop-dfs.sh-
Docker image is based on latest Ubuntu, currently
v24.04 Noble Numbat. -
aptof this image uses Huawei Cloud Mirror instead of Ubuntu official mirror. -
Dockerfileis provided as.devcontainer/Dockerfile. It might be able to use as a standalone recipe for building Docker images.[!CAUTION]
Use it DIRECTLY at your own risk! Use as Dev Containers (as mentioned above) is recommended.