qiime2/docs

Plugin Directory Extension is Suddenly Slow

Closed this issue · 4 comments

thermokarst commented

Bug Description

Generating action help text is suddenly slower. This old build on busywork (2019.07.28) ran in ~38s. A newer build on busywork (2019.08.08) ran in ~8m. The main thing that has changed is the addition of the deprecation API in the framework. I am filing this issue here on the docs repo since this is the main pain-point for this slow-down at present.

Steps to reproduce the behavior

  1. run make dummy on the docs using 2019.7 or 2019.10.dev0

Expected behavior
Generating the help text should be fast (~60s).

Comments

  1. We need to confirm that the slow-down is actually caused by the deprecation API before deciding how to proceed.
thermokarst commented

Benchmark environments

"Good" env (not slow)

# change to the macos yml for darwin testing
wget https://data.qiime2.org/distro/core/qiime2-2019.4-py36-linux-conda.yml -O 2019.4.yml
conda env create -n slow-2019.4 --file 2019.4.yml
conda activate slow-2019.4

"Bad" env (slow)

# change to the macos yml for darwin testing
wget https://raw.githubusercontent.com/qiime2/environment-files/f606efa321d443c93aa686c173a294562f557683/latest/staging/qiime2-latest-py36-linux-conda.yml -O 2019.10.yml
conda env create -n slow-2019.10 --file 2019.10.yml
conda activate slow-2019.10

Benchmarks

make dummy doc

cd docs
pip install -r requirements.txt
/usr/bin/time make dummy

Artifact API

/usr/bin/time python -c "from qiime2.plugins.cutadapt.methods import demux_paired; print(demux_paired.__call__.__doc__);

q2cli

qiime dev refresh-cache
/usr/bin/time qiime cutadapt demux-paired --help

Results

Release make dummy Artifact API q2cli
2019.4 39.8s 3.6s 0.2s
2019.10.dev 429.3s 3.0s 3.1s

Preliminary Conclusion

Looks like q2cli is taking ~10x longer in 2019.10.dev vs 2019.4.

@David-Rod - can you replicate these results on a Mac? Thanks!

David-Rod commented

Hey there @thermokarst,

Here are the results for my replication of the issue:

2019.4
make dummy: 1 m 9 s
Artifact API: 5.46 s
q2cli: 0.36 s

2019.10.dev
make dummy: 11 m 22 s
Artifact API: 5.25 s
q2cli: 4.70 s

  • 👍1
David-Rod commented

Hello @thermokarst,

I just opened a PR for this issue: qiime2/q2cli#228

I tried adding it to Project 2019.10, but I cannot edit that field.

Thanks.

  • 👍1