LRP Relevance Values

[Yohei-u]

Using the LRP analyzers, like alpa1beta0, is there a way to extract the relevance scores calculated per Node or "connection", instead of the relevance per input pixel?

Thanks in advance.

[adrhill]

Hi @Yohei-u,

all LRP analyzers take the optional parameter reverse_keep_tensors=True, which will store all intermediate tensors on the backward pass.

An example is shown in the last cell of this notebook: Developing with iNNvestigate.

[Yohei-u]

Thank you very much!

[Yohei-u]

First of all, thank you for the lightning fast response. Sadly, I've run into new problems. I'm trying to implement pruning on the basis of LRP scores. You've kindly provided my with these, but im struggling with the "pruning" part.

I use the model described in one of the tutorials on the MNIST dataset.

model = keras.models.Sequential([
    keras.layers.Conv2D(32, (3, 3), activation="relu", input_shape=input_shape),
    keras.layers.Conv2D(64, (3, 3), activation="relu"),
    keras.layers.MaxPooling2D((2, 2)),
    keras.layers.Flatten(),
    keras.layers.Dense(512, activation="relu"),
    keras.layers.Dense(10, activation="softmax"),
])

I'm using your beatiful utils to remove the softmax and passing it through the LRP analyzer (alpha1beta0 rule). Originally I planned on just looking at a low LRP score to then set the corresponding weight to 0. But I've noticed that the dimensions of the _reversed_tensors don't link up with the weights of the model.

Above the ==== are the tensors, below are the weights. (node/ layer id, linebreak, shape, size)

0
(1, 1) 1
1
(1, 28, 28, 1) 784
2
(1, 26, 26, 32) 21632
3
(1, 24, 24, 64) 36864
4
(1, 12, 12, 64) 9216
5
(1, 9216) 9216
6
(1, 512) 512
7
(1, 10) 10
=============================
0
(3, 3, 1, 32) 288
1
(3, 3, 32, 64) 18432
2
3
4
(9216, 512) 4718592
5
(512, 10) 5120

weights: model.get_layer(index=x).get_weights()
tensors: lrp_analyzer._reversed_tensors
Is my idea just flat wrong or am I mising something?
Thanks in advance

[adrhill]

Hi Yohei-u,

you can ignore the singleton tensor _reversed_tensors[0].

The other tensors correspond to the layer-wise relevances. The last relevance _reversed_tensors[7] corresponds to the masked output of your model_wo_softmax, that's why there is one more than your layer-count.

[Yohei-u]

Ah, ok thank you thank brings them closer together. As a follow up, how do the values in the tensors corelate to the weights of the model? As there are waay more weights then tensor values. Initiall i thought that the arrays would be shaped alike on both sides, just filled with different values. How do i infer anything from my lrp values onto the weights?
Thank you again in advance.

[adrhill]

The relevances have the same shape as the inputs to each layer, not the weights. LRP tells you which part of the input to a layer contributed most towards the activation of an output neuron.

[Yohei-u]

Ohh, my bad. Thank you again very much. Thanks!

[sebastian-lapuschkin]

hey @Yohei-u , what is your end goal?

if you want to prune whole filters, you can use the knowledge that the relevance of a neuron corresponds to the relevance of its associated filter operating on this neuron's input space. for dense layers, this is a 1:1 relationship (wrt the sample you computed the relevance for). for conv layers, you can obtain the relevance of the filter by aggregating the relevances of this filter's output channel over all locations of the filters application (ie, sum or mean or whatever the relevances of the neurons in output channel c to obtain the total/average/whatever relevance of filter c connected to output channel c)

in general, iNNvestigate works by modifiying the backward gradient from layer l+1 to layer l in order to transport attribution scores through the net towards the input. if you would require in layer l the attributions not for the inputs but for the weights, you just need to compute the modified gradient wrt to the weights instead of the inputs.

in its explicit form, LRP first computes "relevance messages" which are actually aligned to the weights, which then are aggregated in a certain way; e.g. each input neuron collects all incoming relevance messages. the use of a modified backward gradient approach allows us to do this efficiently and fast without blowing up your RAM. changing the gradient target to the weights correspondingly changes the aggregation strategy, e.g. the collection of incoming relevance messages per weight.

if pruning individual weights however is your end goal, I doubt there is any feasible practical applicability within keras/tensorflow, since you would require dedicated compute backends and hardware to benefit from sparse filters. If you just want to remove some filters as a whole, you might find a suitable workaround in the correspondence of neuron relevance to the relevance of the connected filters.