Logs should be available through docker logs command
Closed this issue · 3 comments
earthquakesan commented
Master and worker docker containers should dump the logs into stdout.
At the moment, logs are not available. For instance, in a case of failing flink-worker, there will be empty output in docker logs:
docker logs dockerflink_flinkworker_1
Missing streaming mode (batch|streaming). Using 'batch'.
Starting taskmanager daemon on host flink-worker.
GezimSejdiu commented
Hi @earthquakesan , logs are now available through:
tail -f `ls /usr/local/flink/log/*.log | head -n1`
and by doing docker logs you would be able to see them.
earthquakesan commented
Logs are located in /usr/local/flink/log. docker logs -f flink-master should provide output such as:
11:54:01,697 WARN org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
11:54:01,808 INFO org.apache.flink.runtime.jobmanager.JobManager - --------------------------------------------------------------------------------
11:54:01,808 INFO org.apache.flink.runtime.jobmanager.JobManager - Starting JobManager (Version: 0.10.1, Rev:2e9b231, Date:22.11.2015 @ 12:41:12 CET)
11:54:01,808 INFO org.apache.flink.runtime.jobmanager.JobManager - Current user: root
11:54:01,808 INFO org.apache.flink.runtime.jobmanager.JobManager - JVM: OpenJDK 64-Bit Server VM - Oracle Corporation - 1.8/25.91-b14
11:54:01,808 INFO org.apache.flink.runtime.jobmanager.JobManager - Maximum heap size: 245 MiBytes
11:54:01,808 INFO org.apache.flink.runtime.jobmanager.JobManager - JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
11:54:01,812 INFO org.apache.flink.runtime.jobmanager.JobManager - Hadoop version: 2.7.0
11:54:01,812 INFO org.apache.flink.runtime.jobmanager.JobManager - JVM Options:
11:54:01,812 INFO org.apache.flink.runtime.jobmanager.JobManager - -Xms256m
11:54:01,812 INFO org.apache.flink.runtime.jobmanager.JobManager - -Xmx256m
11:54:01,812 INFO org.apache.flink.runtime.jobmanager.JobManager - -Dlog.file=/usr/local/flink/log/flink--jobmanager-1-flink-master.log
11:54:01,812 INFO org.apache.flink.runtime.jobmanager.JobManager - -Dlog4j.configuration=file:/usr/local/flink/conf/log4j.properties
11:54:01,813 INFO org.apache.flink.runtime.jobmanager.JobManager - -Dlogback.configurationFile=file:/usr/local/flink/conf/logback.xml
11:54:01,813 INFO org.apache.flink.runtime.jobmanager.JobManager - Program Arguments:
11:54:01,813 INFO org.apache.flink.runtime.jobmanager.JobManager - --configDir
11:54:01,813 INFO org.apache.flink.runtime.jobmanager.JobManager - /usr/local/flink/conf
11:54:01,813 INFO org.apache.flink.runtime.jobmanager.JobManager - --executionMode
11:54:01,813 INFO org.apache.flink.runtime.jobmanager.JobManager - cluster
11:54:01,813 INFO org.apache.flink.runtime.jobmanager.JobManager - --streamingMode
11:54:01,813 INFO org.apache.flink.runtime.jobmanager.JobManager - batch
11:54:01,814 INFO org.apache.flink.runtime.jobmanager.JobManager - Classpath: /usr/local/flink/lib/flink-dist_2.11-0.10.1.jar:/usr/local/flink/lib/flink-python_2.11-0.10.1.jar:/usr/local/flink/lib/log4j-1.2.17.jar:/usr/local/flink/lib/slf4j-log4j12-1.7.7.jar:::
11:54:01,814 INFO org.apache.flink.runtime.jobmanager.JobManager - --------------------------------------------------------------------------------
11:54:01,970 INFO org.apache.flink.runtime.jobmanager.JobManager - Loading configuration from /usr/local/flink/conf
11:54:01,990 INFO org.apache.flink.runtime.jobmanager.JobManager - Staring JobManager without high-availability
11:54:01,998 INFO org.apache.flink.runtime.jobmanager.JobManager - Starting JobManager on 172.17.0.4:6123 with execution mode CLUSTER and streaming mode BATCH_ONLY
11:54:02,036 INFO org.apache.flink.runtime.jobmanager.JobManager - Security is not enabled. Starting non-authenticated JobManager.
11:54:02,038 INFO org.apache.flink.runtime.jobmanager.JobManager - Starting JobManager
11:54:02,039 INFO org.apache.flink.runtime.jobmanager.JobManager - Starting JobManager actor system at 172.17.0.4:6123
11:54:02,474 INFO akka.event.slf4j.Slf4jLogger - Slf4jLogger started
11:54:02,523 INFO Remoting - Starting remoting
11:54:02,667 INFO Remoting - Remoting started; listening on addresses :[akka.tcp://flink@172.17.0.4:6123]
11:54:02,677 INFO org.apache.flink.runtime.jobmanager.JobManager - Starting JobManger web frontend
11:54:02,701 INFO org.apache.flink.runtime.webmonitor.WebRuntimeMonitor - Using directory /tmp/flink-web-db77abaa-ef30-4486-b772-18e5230be591 for the web interface files
11:54:02,702 INFO org.apache.flink.runtime.webmonitor.WebRuntimeMonitor - Serving job manager log from /usr/local/flink/log/flink--jobmanager-1-flink-master.log
11:54:02,703 INFO org.apache.flink.runtime.webmonitor.WebRuntimeMonitor - Serving job manager stdout from /usr/local/flink/log/flink--jobmanager-1-flink-master.out
11:54:03,078 INFO org.apache.flink.runtime.webmonitor.WebRuntimeMonitor - Web frontend listening at 0:0:0:0:0:0:0:0:8081
11:54:03,079 INFO org.apache.flink.runtime.jobmanager.JobManager - Starting JobManager actor
11:54:03,087 INFO org.apache.flink.runtime.blob.BlobServer - Created BLOB server storage directory /tmp/blobStore-53d332da-7a55-4141-9965-40fd7243ccd3
11:54:03,088 INFO org.apache.flink.runtime.blob.BlobServer - Started BLOB server at 0.0.0.0:34389 - max concurrent requests: 50 - max backlog: 1000
11:54:03,104 INFO org.apache.flink.runtime.webmonitor.WebRuntimeMonitor - Starting with JobManager akka.tcp://flink@172.17.0.4:6123/user/jobmanager on port 8081
11:54:03,105 INFO org.apache.flink.runtime.webmonitor.JobManagerRetriever - New leader reachable under akka.tcp://flink@172.17.0.4:6123/user/jobmanager:null.
11:54:03,108 INFO org.apache.flink.runtime.jobmanager.JobManager - Starting JobManager at akka.tcp://flink@172.17.0.4:6123/user/jobmanager.
11:54:03,108 INFO org.apache.flink.runtime.jobmanager.MemoryArchivist - Started memory archivist akka://flink/user/archive
11:54:03,114 INFO org.apache.flink.runtime.jobmanager.JobManager - JobManager akka.tcp://flink@172.17.0.4:6123/user/jobmanager was granted leadership with leader session ID None.
11:54:04,555 INFO org.apache.flink.runtime.instance.InstanceManager - Registered TaskManager at flink-worker (akka.tcp://flink@172.17.0.5:53439/user/taskmanager) as e38082a305e24ba71fe90f938c261bdc. Current number of registered hosts is 1. Current number of alive task slots is 1.
11:54:01,333 WARN org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
11:54:01,435 INFO org.apache.flink.client.WebFrontend - --------------------------------------------------------------------------------
11:54:01,435 INFO org.apache.flink.client.WebFrontend - Starting Web Client (Version: 0.10.1, Rev:2e9b231, Date:22.11.2015 @ 12:41:12 CET)
11:54:01,435 INFO org.apache.flink.client.WebFrontend - Current user: root
11:54:01,435 INFO org.apache.flink.client.WebFrontend - JVM: OpenJDK 64-Bit Server VM - Oracle Corporation - 1.8/25.91-b14
11:54:01,435 INFO org.apache.flink.client.WebFrontend - Maximum heap size: 491 MiBytes
11:54:01,435 INFO org.apache.flink.client.WebFrontend - JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
11:54:01,439 INFO org.apache.flink.client.WebFrontend - Hadoop version: 2.7.0
11:54:01,439 INFO org.apache.flink.client.WebFrontend - JVM Options:
11:54:01,439 INFO org.apache.flink.client.WebFrontend - -Xmx512m
11:54:01,439 INFO org.apache.flink.client.WebFrontend - -Dlog.file=/usr/local/flink/log/flink--webclient-flink-master.log
11:54:01,439 INFO org.apache.flink.client.WebFrontend - -Dlog4j.configuration=file:/usr/local/flink/conf/log4j.properties
11:54:01,439 INFO org.apache.flink.client.WebFrontend - -Dlogback.configurationFile=file:/usr/local/flink/conf/logback.xml
11:54:01,439 INFO org.apache.flink.client.WebFrontend - Program Arguments:
11:54:01,440 INFO org.apache.flink.client.WebFrontend - --configDir
11:54:01,440 INFO org.apache.flink.client.WebFrontend - /usr/local/flink/conf
11:54:01,440 INFO org.apache.flink.client.WebFrontend - Classpath: /usr/local/flink/lib/flink-dist_2.11-0.10.1.jar:/usr/local/flink/lib/flink-python_2.11-0.10.1.jar:/usr/local/flink/lib/log4j-1.2.17.jar:/usr/local/flink/lib/slf4j-log4j12-1.7.7.jar:::
11:54:01,440 INFO org.apache.flink.client.WebFrontend - --------------------------------------------------------------------------------
11:54:01,466 INFO org.apache.flink.client.web.WebInterfaceServer - Setting up web client server, using web-root directory 'jar:file:/usr/local/flink/lib/flink-dist_2.11-0.10.1.jar!/web-docs'.
11:54:01,466 INFO org.apache.flink.client.web.WebInterfaceServer - Web frontend server will store temporary files in '/tmp', uploaded jobs in '/tmp/webclient-jobs', plan-json-dumps in '/tmp/webclient-plans'.
11:54:01,466 INFO org.apache.flink.client.web.WebInterfaceServer - Web client will submit jobs to JobManager at flink-master, port 6123.
11:54:01,481 INFO org.apache.flink.client.CliFrontend - Using configuration directory /usr/local/flink/conf
11:54:01,481 INFO org.apache.flink.client.CliFrontend - Trying to load configuration file
11:54:01,713 INFO org.apache.flink.client.WebFrontend - Starting web frontend server on port 8080.
11:54:01,715 INFO org.eclipse.jetty.util.log - jetty-0.10.1
11:54:01,771 INFO org.eclipse.jetty.util.log - Started SelectChannelConnector@0.0.0.0:8080
earthquakesan commented