spancer/bigdata-docker-builds

WebHDFS 无法上传文件

Closed this issue · 2 comments

heriec commented

只部署了hadoop,使用WebHDFS上传文件报错无法上传,根据浏览器的nerwork请求显示,响应的ip为容器id,主机无法解析导致无法上传文件。
在容器中是可以操作文件的,不知道这个问题能否解决?
image

heriec commented

可以配置域名解析解决这个问题,但是总觉得不是很方便。
作为学习,我就如此使用,对hadoop的理解不够到位

heriec commented

这次我又看了下日志,发现datanode的9866端口没有开放导致使用Api无法上传文件
下列指定了容器主机名,在做域名映射即可,并暴露了9866端口即可解决问题

version: "3"
services:
  namenode:
    image: coomia/hadoop3.2-namenode:1.0
    container_name: namenode
    hostname: namenode
    ports:
      - 9870:9870
      - 9000:9000
    environment:
      - CLUSTER_NAME=coomia-hdp-cluster
    env_file:
      - ./hadoop.env

  datanode:
    image: coomia/hadoop3.2-datanode:1.0
    container_name: datanode
    hostname: datanode
    environment:
      SERVICE_PRECONDITION: "namenode:9870"
    ports:
      - 9864:9864
      - 9866:9866
    env_file:
      - ./hadoop.env
  
  resourcemanager:
    image: coomia/hadoop3.2-resourcemanager:1.0
    container_name: resourcemanager
    hostname: resourcemanager
    ports:
      - 8088:8088
    environment:
      SERVICE_PRECONDITION: "namenode:9000 namenode:9870 datanode:9864"
    env_file:
      - ./hadoop.env

  nodemanager:
    image: coomia/hadoop3.2-nodemanager:1.0
    container_name: nodemanager
    hostname: nodemanager
    environment:
      SERVICE_PRECONDITION: "namenode:9000 namenode:9870 datanode:9864 resourcemanager:8088"
    env_file:
      - ./hadoop.env
  
  historyserver:
    image: coomia/hadoop3.2-historyserver:1.0
    container_name: historyserver
    hostname: historyserver
    environment:
      SERVICE_PRECONDITION: "namenode:9000 namenode:9870 datanode:9864 resourcemanager:8088"
    env_file:
      - ./hadoop.env