duckduckgo/community-platform

Add a 'lazy' interface to slow packages

Opened this issue · 0 comments

Rationale: It takes approximately 2~3 seconds to just load DDGC due to all the DB setup it has to perform (I had to hunt this down in NYTProf!); some scripts require DDGC for certain operations, but not all; for example a --help tag shouldn't have to take the time to load DDGC.

Example:

script/ddgc_generate_traffic.pl --help takes approx. real 0m2.401s, which is an awful long time to wait for just the help!

Using a lazy-loading approach, script/ddgc_generate_traffic.pl --help takes approx. real 0m0.208s - a much more acceptable time (IMO).

There are various solutions to this, such as only requiring the package in the script when you need to (this has some issues, such as forcing variables to be lexically bound); providing a Role for scripts that provides the necessary functions, or the approach I suggest, which is to provide a package purely for lazy references to other packages.

Example:

package DDGC::Lazy;

use Moo;

has ddgc => (
    is => 'ro',
    lazy => 1,
    builder => 1,
);

sub _build_ddgc {
    require DDGC;
    return DDGC->new;
}

Then in the script you could do:

use DDGC::Lazy; # Instead of DDGC

my $lazy = DDGC::Lazy->new;

...

# We actually need it now (take the 2~3 second hit)
my $d = $lazy->ddgc;
# Now we can use $d, or just use $lazy->ddgc if we want (depending on scope).

/cc @jdorweiler @jbarrett