Java 7 FileSystem wrapper for debugging PROBLEMS. It logs all major destructive filesystem operations (not individual writes, seriously major things such as overwriting a file, deleting a file, renaming a file) to stdout
or to a file along with timestamp, thread name, and stacktrace.
Log messages look like this:
FS 0 [2016-06-08T05:28:10.829Z; elasticsearch[Rock Python][clusterService#updateTask][T#1]]: newFileChannel[WRITE]: /home/rmuir/Downloads/elasticsearch-2.3.1/data/elasticsearch/nodes/0/_state/global-12.st.tmp
at rmuir.VerboseFS.newFileChannel(VerboseFS.java:209)
at java.nio.channels.FileChannel.open(FileChannel.java:287)
at java.nio.channels.FileChannel.open(FileChannel.java:335)
at org.apache.lucene.util.IOUtils.fsync(IOUtils.java:391)
at org.elasticsearch.gateway.MetaDataStateFormat.write(MetaDataStateFormat.java:131)
at org.elasticsearch.gateway.MetaStateService.writeGlobalState(MetaStateService.java:149)
at org.elasticsearch.gateway.GatewayMetaState.clusterChanged(GatewayMetaState.java:148)
at org.elasticsearch.gateway.Gateway.clusterChanged(Gateway.java:185)
at org.elasticsearch.cluster.service.InternalClusterService.runTasksForExecutor(InternalClusterService.java:610)
at org.elasticsearch.cluster.service.InternalClusterService$UpdateTask.run(InternalClusterService.java:772)
at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:231)
at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:194)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
- Download the jar: https://github.com/rmuir/verbosefs/releases/download/1.1/verbosefs.jar
- Put this jar in the application classpath
- You must pass this argument to the JVM:
-Djava.nio.file.spi.DefaultFileSystemProvider=rmuir.VerboseFS
To log to a file instead, also pass this argument to the JVM -Dverbosefs.logfile=/somewhere/file.txt
- Download the jar: https://github.com/rmuir/verbosefs/releases/download/1.1/verbosefs.jar
- Put jar in elasticsearch
lib
folder (alongside elasticsearch-2.3.1.jar or whatever) - Start elasticsearch, specifying options in
ES_JAVA_OPTS
environment variable.
Example:
ES_JAVA_OPTS="-Djava.nio.file.spi.DefaultFileSystemProvider=rmuir.VerboseFS" bin/elasticsearch
Example 2:
ES_JAVA_OPTS="-Djava.nio.file.spi.DefaultFileSystemProvider=rmuir.VerboseFS -Dverbosefs.logfile=/somewhere/file.txt" bin/elasticsearch