shap/shap

ENH: Faster import performance

connortann opened this issue · 1 comments

connortann commented

Problem Description

The package import time is rather slow- over 2s on my machine. We could profile the import (e.g. with tuna) to identify any bottlenecks, and potentially improve performance.

Alternative Solutions

We could copy scipy's approach and import subpackages lazily, with some getattr magic. This would probably be a bit more complex and nonstandard.

https://peps.python.org/pep-0562/

Additional Context

No response

Feature request checklist

  • I have checked the issue tracker for duplicate issues.
  • I'd be interested in making a PR to implement this feature
connortann commented

Here's a profile of import shap in an environment than includes the test dependencies. It seems the vast majority of time is spent importing dependencies:

  1. torch (0.94s)
  2. IPython (0.25s)
  3. pandas (0.19s)
  4. matplotlib (0.18s)
  5. sklearn (0.17s)
  6. numba (0.12s)

image

Tensorflow has no effect here as it is imported lazily. I think an easy way to speedup import shap would be to import torch lazily in the same manner.