flume-source-plugin flume ng reliable tailSource it can monitor a directory and get log info in real time.