big-data-europe/docker-flink

Network address of flink master should be configurable for each flink worker

earthquakesan opened this issue · 1 comments

At the moment the config is hardcoded as follows:

flinkmaster:
    image: bde2020/flink-master:0.10.1-hadoop2.7
    hostname: flink-master
    environment:
      - INIT_DAEMON_STEP=setup_flink
#    networks:
#      - flink
flinkworker:
    image: bde2020/flink-worker:0.10.1-hadoop2.7
    hostname: flink-worker
#    networks:
#     - flink
    links:
      - "flinkmaster:localhost"

The worker is connected to flinkmaster using hardcoded "localhost" host name (in master/slaves file?). The proper docker-compose.yml should look like:

flinkmaster:
    image: bde2020/flink-master:0.10.1-hadoop2.7
    environment:
      - INIT_DAEMON_STEP=setup_flink
flinkworker:
    image: bde2020/flink-worker:0.10.1-hadoop2.7
    links:
      - "flinkmaster"
    environment:
       - FLINK_MASTER=flinkmaster

Done via:

sed -i -e "s/%jobmanager%/`hostname -i`/g" /usr/local/flink/conf/flink-conf.yaml 

into master, and

sed -i -e "s/%jobmanager%/$FLINK_MASTER_PORT_6123_TCP_ADDR/g" /usr/local/flink/conf/flink-conf.yaml

into every worker instance.

Many thanks.