Logstash Ipinfo Filter

This is a filter plugin for Logstash.

It is fully free and fully open source. The license is Apache 2.0, meaning you are pretty much free to use it however you want in whatever way.

Documentation

The Ipinfo filter adds geolocation information for IP addresses from logstash events via the Ipinfo API.

Usage

1. Installation

You can use the built-in plugin tool of Logstash to install the filter:

$LS_HOME/bin/logstash-plugin install logstash-filter-ipinfo

Or you can build it yourself:

git clone https://github.com/nsherron90/logstash-filter-ipinfo.git
bundle install
gem build logstash-filter-ipinfo.gemspec
$LS_HOME/bin/logstash-plugin install logstash-filter-ipinfo-1.1.0.gem

2. Filter Configuration

Add the following inside the filter section of your logstash configuration:

filter {
  ipinfo {
    ip => "ip_value"                 # string (required, reference to ip address field)
    token => "your_ipinfo_token"      # string (optional, no default)
    target => "ipinfo"            # string (optional, default = ipinfo)
  }
}

Print plugin version:

bin/logstash-plugin list --verbose | grep ipinfo

Example for running logstash from cli:

bin/logstash --debug -e \
'input {
    stdin {}
}


filter {
  ipinfo {
    ip => "%{message}"
   }
}

output {
   stdout {
      codec => rubydebug {
          metadata => true
          }
      }
}'

Example configuration file for adding geolocation data to access logs collected by the nginx Filebeat module.

# logstash.conf

input {
  beats {
    port => 5044
    host => "0.0.0.0"
  }
}
filter {
  if [fileset][module] == "nginx" {
    if [fileset][name] == "access" {
      grok {
        match => { "message" => ["%{IPORHOST:[nginx][access][remote_ip]} - %{DATA:[nginx][access][user_name]} \[%{HTTPDATE:[nginx][access][time]}\] \"%{WORD:[nginx][access][method]} %{DATA:[nginx][access][url]} HTTP/%{NUMBER:[nginx][access][http_version]}\" %{NUMBER:[nginx][access][response_code]} %{NUMBER:[nginx][access][body_sent][bytes]} \"%{DATA:[nginx][access][referrer]}\" \"%{DATA:[nginx][access][agent]}\""] }
        remove_field => "message"
      }
      mutate {
        add_field => { "read_timestamp" => "%{@timestamp}" }
      }
      date {
        match => [ "[nginx][access][time]", "dd/MMM/YYYY:H:m:s Z" ]
        remove_field => "[nginx][access][time]"
      }
      useragent {
        source => "[nginx][access][agent]"
        target => "[nginx][access][user_agent]"
        remove_field => "[nginx][access][agent]"
      }
      ipinfo {
        ip => "[nginx][access][remote_ip]"
        target => "[nginx][access][ipinfo]"
      }
    }
  }
}
output {
  elasticsearch {
    hosts => localhost
    manage_template => false
    index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
  }
}

Contributing

All contributions are welcome: ideas, patches, documentation, bug reports, complaints, and even something you drew up on a napkin.

Programming is not a required skill. Whatever you've seen about open source and maintainers or community members saying "send patches or die" - you will not see that here.

It is more important to the community that you are able to contribute.