Does dtreeviz lib update of API for 2.0 release affect explainerdashboard?

Hi @oegedijk, @tlapusan and I are creating a big change to the API for dtreeviz, although we hope it is backwards compatible. Regardless, you will start to get deprecation warnings if you pull in dtreeviz 2.0 (coming out shortly). The current dev branch is the following if you'd like to take a look:

https://github.com/parrt/dtreeviz/tree/dev

Happy to make tweaks to smooth any transitions! The functionality should be the same.

Hey @parrt,
Encountered the following error when trying to import ClassifierExplainer and ExplainerDashboard

@parrt thanks for the headshup! Already noticed some failing tests, but with two sick kids at home and myself out with a sinus infection, haven't had any time yet to look at it.

@haizadtarik thanks for the PR, will have a quick look at it!

@oegedijk I hope you and your family will be well soon ! :)

The error about dtreeviz import function is because we moved it from trees.py into another module (compatibility.py).

To have backward compatibility with the old API, you will have to adjust to
from dtreeviz import * from dtreeviz.trees import ShadowDecTree

hope that this will help. Tudor

@tlapusan is this more specific/safer?

from dtreeviz.models.shadow_decision_tree import ShadowDecTree

yes, definitely. that's the real path to ShadowDecTree

In future though they should avoid as it's our internal impl. Folks should use the API object coming back from model(). :)

Just made the release should be fixed now. Thanks for the headsupp @parrt and thanks for the PR @haizadtarik !

Ow, and happy new year :)

Yay!

Hi @oegedijk
I am importing the latest version of explainer dashboard but still getting error due to dtreeviz imports.

from dtreeviz.trees import ShadowDecTree, dtreeviz
ImportError: cannot import name 'dtreeviz' from 'dtreeviz.trees'

Also tried using older version of explainer dashboard explainerdashboard==0.3.8 and downgrading Python to 3.7 but it didn't help.

Please help me with this and let me know if I am missing something.

@suraj-gade strange that the same old import code appear .. it should be changed/fixed based on this message from @oegedijk #243 (comment)

@suraj-gade what's your latest version number of the explainerdashboard ?

Hi @tlapusan , Thanks for response.

My latest version for explainerdashboard is 0.4.1.1

@oegedijk do you have any ideas ?

@suraj-gade could you give a complete stacktrace of the error ?

This issue was opened for the same reason/error, but looking into the code

explainerdashboard/explainerdashboard/explainers.py

Line 28 in 6b21dc1

from dtreeviz import *

, we can see that the changes was made accordingly.

Hi @tlapusan ,

Now, I tried the same version explainerdashboard ==0.4.1.1 with Python 3.9.13. So import is working now.

But Now I am getting another error in later stage of my code, and I guess its a compatibility issue. PFA the code and error below.

I have saved my explainer dashboard object as (dashboard.yaml, explainer.joblib).
Then I want to integrate this dashboard with my Flask app so ,I am loading this saved dashboard from .yaml and .joblib file using ExplainerDashboard.from_config method as below.

from flask import Flask
from explainerdashboard import ClassifierExplainer, ExplainerDashboard
import pickle
import yaml
from yaml.loader import SafeLoader

app = Flask(__name__)

@app.route('/dashboard/')
def load_dashboard():
    return dashboard .app.index()


if __name__ == '__main__':
    dashboard = ExplainerDashboard.from_config("dashboard.yaml",server=app,url_base_pathname="/dashboard/")
    app.run(host="0.0.0.0")

Now, I am getting error on ExplainerDashboard.from_config line

UserWarning: Trying to unpickle estimator DecisionTreeClassifier from version 1.0.2 when using version 1.2.1. This might lead to breaking code or invalid results. Use at your own risk. For more info please refer to:
https://scikit-learn.org/stable/model_persistence.html#security-maintainability-limitations
C:\Users\suraj.gade\Anaconda3\lib\site-packages\sklearn\base.py:299: UserWarning: Trying to unpickle estimator RandomForestClassifier from version 1.0.2 when using version 1.2.1. This might lead to breaking code or invalid results. Use at your own risk. For more info please refer to:
https://scikit-learn.org/stable/model_persistence.html#security-maintainability-limitations
Traceback (most recent call last):
  File "C:\Users\suraj.gade\Desktop\explainer_db\temp.py", line 15, in <module>
    dashboard = ExplainerDashboard.from_config("dashboard.yaml",server=app,url_base_pathname="/dashboard/")
  File "C:\Users\suraj.gade\Anaconda3\lib\site-packages\explainerdashboard\dashboards.py", line 680, in from_config
    explainer = BaseExplainer.from_file(config['dashboard']['explainerfile'])
  File "C:\Users\suraj.gade\Anaconda3\lib\site-packages\explainerdashboard\explainers.py", line 338, in from_file
    return joblib.load(filepath)
  File "C:\Users\suraj.gade\Anaconda3\lib\site-packages\joblib\numpy_pickle.py", line 658, in load
    obj = _unpickle(fobj, filename, mmap_mode)
  File "C:\Users\suraj.gade\Anaconda3\lib\site-packages\joblib\numpy_pickle.py", line 577, in _unpickle
    obj = unpickler.load()
  File "C:\Users\suraj.gade\Anaconda3\lib\pickle.py", line 1212, in load
    dispatch[key[0]](self)
  File "C:\Users\suraj.gade\Anaconda3\lib\pickle.py", line 1589, in load_reduce
    stack[-1] = func(*args)
  File "C:\Users\suraj.gade\Anaconda3\lib\site-packages\numba\core\serialize.py", line 97, in _unpickle__CustomPickled
    ctor, states = loads(serialized)
TypeError: an integer is required (got type bytes)

I am using latest version of scikit-learn but still its giving warning about the same.

Please have a look and let me know what is the issue or if I am missing anything

@suraj-gade I assume this error is based on your environments :)
Please be sure that the initial environment and flask environement are using the same library versions.

For example, you could try "ExplainerDashboard.from_config", right after you saved the joblib object (in the same environment/notebook) to check if you have the same issue.

@tlapusan @suraj-gade I will try to find some time to look at this this weekend. I think we need a bit more work to make the dashboard dtreeviz v2 compatible after all...

@tlapusan @suraj-gade I will try to find some time to look at this this weekend. I think we need a bit more work to make the dashboard dtreeviz v2 compatible after all...

@oegedijk sorry for that :(

Hi! @oegedijk @suraj-gade

Just to check with @oegedijk, did you updated the code base to fix the issue encountered by @suraj-gade?
On my end, I managed to integrate the explainerdashboard v0.4.1.1 with flask app using the following code:

from explainerdashboard import ExplainerDashboard

db = ExplainerDashboard.from_config("dashboard.yaml", url_base_pathname="/dashboard/")
app = db.flask_server()

@app.route("/")
def root():
    return "<p>Hello!</p>"

if __name__ == '__main__':
    app.run(host="0.0.0.0")

Yes, just did the release as well. V0.4.2 should fix the issue!