allinurl/goaccess

Custom MMDB IP data DB not working with Goaccess

Opened this issue · 6 comments

konkhra commented

Is custom mmdb going to work with goaccess? My current mmdb file is structured like this:

mmdblookup --file /usr/share/GeoIP/country_asn.mmdb --ip 1.1.1.1

{
"as_domain":
"cloudflare.com" <utf8_string>
"as_name":
"Cloudflare, Inc." <utf8_string>
"asn":
"AS13335" <utf8_string>
"continent":
"Oceania" <utf8_string>
"continent_code":
"OC" <utf8_string>
"country":
"Australia" <utf8_string>
"country_code":
"AU" <utf8_string>
"network":
"1.1.1.0/24" <utf8_string>
}

I set the location of mmdb in goaccess.conf like this geoip-database /usr/share/GeoIP/country_asn.mmdb but it's not working.

allinurl commented

Can you share the differences between your MMDB format and the working one? Or, can you post a comparison highlighting the specific discrepancies between the two?

allinurl commented

Any updates on this?

konkhra commented

Hi!

Sorry for delay on my reply.

$mmdblookup --file GeoLite2-Country.mmdb --ip 1.1.1.1

  {
    "registered_country":
      {
        "geoname_id":
          2077456 <uint32>
        "iso_code":
          "AU" <utf8_string>
        "names":
          {
            "de":
              "Australien" <utf8_string>
            "en":
              "Australia" <utf8_string>
            "es":
              "Australia" <utf8_string>
            "fr":
              "Australie" <utf8_string>
            "ja":
              "オーストラリア" <utf8_string>
            "pt-BR":
              "Austrália" <utf8_string>
            "ru":
              "Австралия" <utf8_string>
            "zh-CN":
              "澳大利亚" <utf8_string>
          }
      }
  }

and custom mmdb file

mmdblookup --file country_asn.mmdb --ip 1.1.1.1

{
"as_domain":
"cloudflare.com" <utf8_string>
"as_name":
"Cloudflare, Inc." <utf8_string>
"asn":
"AS13335" <utf8_string>
"continent":
"Oceania" <utf8_string>
"continent_code":
"OC" <utf8_string>
"country":
"Australia" <utf8_string>
"country_code":
"AU" <utf8_string>
"network":
"1.1.1.0/24" <utf8_string>
}

I am suspecting goaccess has a hardcoded mmdb structure that is looking at in order to work. GEO IP is different than my custom mmdb file. If that's the case I will just get rid of the GEO IP function and that's it.

allinurl commented

Thanks for sharing. GoAccess uses the MaxMindDB API (source), and it may be possible to have it parse custom fields. Would you be comfortable writing some C code to implement this?

konkhra commented

A viable solution in the future would be to let the user define those custom fields so that any mmdb file can be parsed without problems.
I am not a programmer but what would be a temporary workaround?

allinurl commented

What fields do you need not in the mmdb? I'd use the https://db-ip.com/db/download/ip-to-asn-lite