GeoLite Legacy databases are now discontinued

Question

GeoLite Legacy databases are now discontinued

toddmazierski opened this issue 6 years ago · 13 comments

toddmazierski commented 6 years ago

As of today, MaxMind's GeoLite Legacy databases are discontinued:

https://support.maxmind.com/geolite-legacy-discontinuation-notice/

Two of the locations are now returning 404:

You may want to consider updating the README to indicate this. Thanks!

Answer 1 · 2019-01-03T00:55:55.000Z

Thank you for the heads-up. Readme updated and pushed to Github. I don't think it's necessary to re-release the gem for this change.

Answer 2 · 2019-01-03T15:32:36.000Z

Many thanks, @cjheath!

Answer 3 · 2019-01-04T16:14:36.000Z

@cjheath I'm a bit confused by the update - the readme tells people they can download a Geolite2 database from https://dev.maxmind.com/geoip/geoip2/geolite2, but those aren't supported with GeoIP, right?

Answer 4 · 2019-01-05T23:35:15.000Z

Unfortunately correct, this gem has no support for the mmdb database format. Brain fart - I haven't touched this code in a long time. There are two options:

implement support for the new indexing and database format
implement a compiler from the CSV data to the legacy format.

It might sound bogus, but the latter is a much more well-defined problem.
I don't want to resort to a C extension (this gem was written to replace one!).
I don't know when I'll have time to read Maxmind's V2 code and re-implement it, or even if enough information is provided to make that possible. A contributor might offer to help, of course.

Any other options?

Answer 5 · 2019-01-06T14:57:28.000Z

I’ve not tried it yet, but https://github.com/yhirose/maxminddb is pure ruby and handles geoip2. If it’s any good, we could point users to that rather than maintaining a competing gem

Answer 6 · 2019-01-06T15:38:12.000Z

We could. But this gem has been in continuous use since 2005 (rubyforge days, before rubygems existed) and a lot of people still use it. maxminddb is MIT-licensed, so we could borrow the code with attribution. A quick scan of that code shows that the basic index structure and search algorithm is unchanged in any case, so it would also be possible to enhance the current implementation to support V2 databases.

Answer 7 · 2019-01-06T16:49:05.000Z

If you are thinking about implementing support, there is a public spec for the MaxMind DB format. There is also a relatively recent official pure Ruby implementation.

Answer 8 · 2019-01-06T22:50:44.000Z

Finally - it only took them 14 years to document and implement in Ruby! I don't feel too bad about letting mine die now, but my users might want something different.

Answer 9 · 2019-04-09T01:12:38.000Z

@cjheath I think it's worth wrapping the maxmind-db gem with the existing geoip API just to make it nice and easy for everybody to continue to without needing code changes.

I started to look into doing this, but I cannot figure out for the life of me how to make these tests run since the test Rake task doesn't quite work with files like test_file.rb

Answer 10 · 2019-04-09T02:03:54.000Z

There never has been an automated test for this gem, just a stub.
test_file.rb is a manual test, used for dumping a GeoIP file in IP order - it needs to be given the pathname to a data file.
I'd welcome a new implementation, but some tests are obviously needed first.

Answer 11 · 2019-04-18T13:36:30.000Z

@cjheath Am I understanding correctly, that you'd accept a new implementation based on maxminddb-reader that implemented the current GeoIP interfaces?

Answer 12 · 2019-04-18T21:31:45.000Z

Yes, I'd accept that. Perhaps the old code could be loaded to handle legacy files in case the maxminddb-reader doesn't?

Answer 13 · 2020-05-30T18:03:39.000Z

I haven't tried this out yet, but it seems that there's been some progress on the conversion of geo-ip2 to legacy formats.

There's a python script https://github.com/sherpya/geolite2legacy

And a site distributing dat files generated, presumably with this script: https://www.miyuru.lk/geoiplegacy