netflow codec geoip lookup not working

[sgreszcz]

• Version: 6.4
• Operating System: Ubuntu
• Config File (if you have sensitive info, please remove it):

# Refer to this: https://www.elastic.co/guide/en/logstash/current/plugins-codecs-netflow.html
input{
	udp{
		port => 2055
		codec => netflow
	    tags => ["netflow"]
		id => "netflow_input"
	}
}

filter{

# Enhance with ISP geolocation information using free GeoIP2 ISP Database

    geoip {
        database => "/usr/share/logstash/geolite2/test.mmdb"
        source => "ipv4_src_addr"
    }
}

output {
    if "netflow" in [tags]{
    	elasticsearch{
		hosts => "${ELASTICSEARCH_HOST}:${ELASTICSEARCH_PORT}"
		index => "netflow-%{+YYYY.MM.dd}"
		id => "netflow_es_output"
	}
	#stdout{ }
    }
}

Also tried:

filter{
    geoip {
        database => "/usr/share/logstash/geolite2/test.mmdb"
        source => "netflow.ipv4_src_addr"
    }
}

The test.mmdb is a MaxMind DB with internal private IP geo mappings.

• Sample Data:

Using Logstash Netflow codec for example, and trying to source off of "ipv4_src_addr" or "netflow.ipv4_src_addr":

{
  "_index": "netflow-2018.11.13",
  "_type": "doc",
  "_id": "q4thD2cBb3n6NizXdq3b",
  "_version": 1,
  "_score": null,
  "_source": {
    "netflow": {
      "l4_dst_port": 56335,
      "src_tos": 0,
      "ipv4_src_addr": "10.230.91.4",
      "input_snmp": 7,
      "flow_sampler_id": 0,
      "protocol": 17,
      "src_mask": 25,
      "in_pkts": 80,
      "tcp_flags": 0,
      "version": 9,
      "ipv4_dst_addr": "173.38.200.85",
      "l4_src_port": 161,
      "flow_seq_num": 208343981,
      "first_switched": "2018-11-13T23:21:41.999Z",
      "dst_as": 0,
      "ipv4_next_hop": "10.230.2.98",
      "flowset_id": 257,
      "dst_mask": 21,
      "output_snmp": 8,
      "last_switched": "2018-11-13T23:22:08.999Z",
      "in_bytes": 68403,
      "src_as": 0
    },
    "tags": [
      "netflow",
      "_geoip_lookup_failure"
    ],
    "@version": "1",
    "host": "172.19.0.1",
    "@timestamp": "2018-11-13T23:22:23.000Z"
  },
  "fields": {
    "netflow.first_switched": [
      "2018-11-13T23:21:41.999Z"
    ],
    "@timestamp": [
      "2018-11-13T23:22:23.000Z"
    ],
    "netflow.last_switched": [
      "2018-11-13T23:22:08.999Z"
    ]
  },
  "sort": [
    1542151343000
  ]
}

• Steps to Reproduce:

Debugs always tag output with _geoip_lookup_failure

The DB is valid and works OK with the python geoip2 library:

import geoip2.database
reader = geoip2.database.Reader('/Users/me/Downloads/test.mmdb')
response = reader.city('10.50.172.174')

response.location.latitude
51.445324
response.location.longitude
-0.443962

response
geoip2.models.City({'city': {'names': {'en': ' FELTHAM'}}, 'continent': {'code': 'something'}, 'country': {'iso_code': ' UNITED KINGDOM', 'names': {'en': ' UNITED KINGDOM'}}, 'location': {'accuracy_radius': 1000, 'latitude': 51.445324, 'longitude': -0.443962}, 'traits': {'ip_address': '10.50.172.174'}}, ['en'])

[hkelley]

Any updates here? I have the same scenario. We have inserted our RFC1918 network into the GeoLite2-City.mmdb DB and would like to use these with geoip .