Re-indexing still works with a large number of modules
Closed this issue · 1 comments
Right now we cannot easily rebuild a minicpan with a lot of modules because the MetaCPAN lookup fails.
The problem is here: https://metacpan.org/source/OALDERS/OrePAN2-0.44/lib/OrePAN2/Indexer.pm#L148
This code needs to break up the query after X releases have been pushed onto the @file_search stack. I don't have number handy, but trying to rebuild the minicpan will yield it fairly quickly.
AC:
- OrePAN2 can accommodate lookups for an arbitrary number of modules
For testing the MetaCPAN behaviour, see the use MetaCPAN subtest in t/06_inject_live.t.
The logic which needs to be tweaked is in OrePAN2::Indexer::do_metacpan_lookup(). We could create an accessor that sets a threshold on how many modules to search on in @search_by_archives. If the number of files we need to look up exceeds the threshold, then we need to loop over the MetaCPAN search logic in order to get everything we need. The accompanying test could inject 2 files into the $tmpdir and then use a very low threshold (like 1 archive) in order to force the looping behaviour.
If both releases are found in $orepan->_metacpan_lookup then we have a green light.