Logstash Grok 导入自定义日志文件

[JasonWu73]

Logstash Grok 导入自定义日志文件

测试数据

sample.log

2020-01-02T14:58:40Z INFO initializing the bootup
2020-03-14T22:50:34Z ERROR cannot find the requested resource
2020-05-07T03:07:11Z INFO variable server value is tomcat
2020-06-04T06:56:04Z DEBUG initializing checksum
2020-10-11T09:49:35Z INFO variable server value is tomcat
55.12.32.134 GET /user/id/properties

Logstash 配置文件

logstash.conf

input {
  file {
    path => "/Users/jasonwu/WorkSpace/learn/Elasticsearch/sample.log"
    start_position => "beginning"
    sincedb_path => "/Users/jasonwu/.Trash/sincedb.trash"
  }
}

filter {
  grok {
    match => {
      "message" => [
        "%{TIMESTAMP_ISO8601:time} %{LOGLEVEL:log_level} %{GREEDYDATA:log_message}",
        "%{IP:client_ip} %{WORD:http_method} %{URIPATH:url}"
      ]
    }
  }

  mutate {
    remove_field => ["path", "@version", "message", "@timestamp", "host"]
  }
}

output {
  stdout {}

  elasticsearch {
    hosts => ["localhost:9200"]
    index => "logs"
  }
}

执行 Logstash 导入 Elasticsearch

$ bin/logstash -f /Users/jasonwu/WorkSpace/learn/Elasticsearch/logstash.conf