Baseline is ignored?

[mstaniak]

I'm running the following code:

set.seed(17)
x1 <- runif(1000, -10, 10)
x2 <- runif(1000, -10, 10)
y <- 0.05*x1^2 + 0.05*x2^2

v2_df <- data.frame(x = x1,
                    y = x2,
                    z = y)

true_model <- function(model, newdata) {
  0.05*newdata[, 1]^2 + 0.05*newdata[, 2]^2
}

library(DALEX)
v2_explainer <- explain(list(), data = v2_df[, -3], predict_function = true_model,
                        label = "double_quadratic")

library(iBreakDown)
ibd_expl_1 <- local_attributions(v2_explainer, data.frame(x = -6, y = -6))
ibd_expl_2 <- local_attributions(v2_explainer, data.frame(x = -6, y = -6), baseline = 0)

ibd_plot_1 <- plot(ibd_expl_1, baseline = 0)
ibd_plot_2 <- plot(ibd_expl_2, baseline = 0)

ibd_plot_1
ibd_plot_2

And I'm getting same plots in both cases, neither of them starts in 0.

[pbiecek]

@mstaniak
baseline defines where plots shall start. In this case both plots starts in 0
(first green and last violet bars start in 0)

[mstaniak]

So it's not possible to decompose f(x) - baseline rather than f(x) - mean(f)?

[pbiecek]

Currently only f(x) - mean(f) is decomposed.
What would be the usecase for baselines other than mean(f)?

[mstaniak]

I'm playing with the example
f(x, y) = x^2 + y^2
at (x, y) = (0, 0) both variables contribute nothing (and the model prediction is 0, of course)
but iBreakDown explanation shows two negative contributions
which is counterintuitive for me.
But I guess it's a matter of what we expect from explanations in particular cases.

Though, I'd say that using other baselines can be helpful in cases like above:

• we know something more about the f function (for example that it's a regression function that returns nonnegative values) and we can find more reasonable baselines.

[pbiecek]

contributions are negative, because in the point (0,0) the x^2 + y^2 has minimum. So both x and y moves from the average to the minimum

it is easier to see once you plot distribution of f()

set.seed(1)
df <- data.frame(x = rnorm(100), y = rnorm(100))

expl <- DALEX::explain(1, df, 
                       predict_function = function(m,x) x[,1]^2 + x[,2]^2)

library(iBreakDown)
bd <- break_down(expl, data.frame(x=0, y=0), keep_distributions=TRUE)
plot(bd, plot_distributions = TRUE)

[mstaniak]

I know why they're negative, I'm just saying that I'm not sure that it's the expected/best description. But it's a material for a whole other discussion, the issue itself can be closed.

[mstaniak]

On a loosely related note, I think that localModel "suffers" from the same problem.
A point on a monotonic curve can be explained with a negative and positive weight depending on a reference point. Similarly here: the sign of the explanation depends on the reference point aka the mean.

[pbiecek]

True,
yet since (0,0) minimises x^2+y^2 then I would expect that in any explanation effects of both 0's should be negative.

I can image that in LIME/live/localModel and similar solutions the derivative is zero in the (0,0) but for me it is counterintuitive (again - point for discussion and maybe a survery)