Lack of documentation for steps in interpreters.py
Closed this issue · 4 comments
Hello, thank you for your work.
I have been trying to run the attack on YOLOv3 using python3 yolov3.py 0.
However, I have not been able to get it to work.
After running, I encounter this error:
Traceback (most recent call last):
File "yolov3.py", line 44, in <module>
rad_coco(load_model, get_index, group_dimension=9, attack_dimension=4, transfer_enhance=['SI'])
File "/content/RAD/rad.py", line 194, in rad_coco
**kwargs,
File "/content/RAD/rad.py", line 73, in __init__
self.analysis = build_lrp(self.model.model_attack, self.index_place)
File "/content/RAD/utils.py", line 165, in build_lrp
analyzer = SGLRP(partial_model, target_id=target_id, multi=multi, relu=True, low=tf.reduce_min(inp), high=tf.reduce_max(inp), allow_lambda_layers=True)
File "/content/RAD/interpreters.py", line 288, in __init__
super(SGLRP, self).__init__(model, target_id=target_id, low=low, high=high, **kwargs)
File "/content/RAD/interpreters.py", line 126, in __init__
self.target_analyzer = self._get_target_analyzer(**kwargs)
File "/content/RAD/interpreters.py", line 292, in _get_target_analyzer
return _SGLRPTarget(self.model, target_id=self.target_id, multi=self.multi, **kwargs)
File "/content/RAD/interpreters.py", line 221, in __init__
super(_SGLRPBase, self).__init__(model, neuron_selection_mode="all", **kwargs)
File "/content/RAD/innvestigate/analyzer/deeptaylor.py", line 184, in __init__
model, **kwargs)
File "/content/RAD/innvestigate/analyzer/deeptaylor.py", line 49, in __init__
super(DeepTaylor, self).__init__(model, *args, **kwargs)
File "/content/RAD/innvestigate/analyzer/base.py", line 592, in __init__
super(ReverseAnalyzerBase, self).__init__(model, **kwargs)
File "/content/RAD/innvestigate/analyzer/base.py", line 341, in __init__
super(AnalyzerNetworkBase, self).__init__(model, **kwargs)
File "/content/RAD/innvestigate/analyzer/base.py", line 73, in __init__
self._do_model_checks()
File "/content/RAD/innvestigate/analyzer/base.py", line 98, in _do_model_checks
checked = kgraph.model_contains(self._model, check)
File "/content/RAD/innvestigate/utils/keras/graph.py", line 419, in model_contains
tmp = [layer for layer in layers if condition(layer)]
File "/content/RAD/innvestigate/utils/keras/graph.py", line 419, in <listcomp>
tmp = [layer for layer in layers if condition(layer)]
File "/content/RAD/innvestigate/analyzer/base.py", line 349, in <lambda>
layer, activation="softmax"),
File "/content/RAD/innvestigate/utils/keras/checks.py", line 219, in contains_activation
raise Exception("Cannot detect activation type.")
Exception: Cannot detect activation type.
I think it is related to the comment in interpreters.py, where the following is mentioned
"""rewrite from https://github.com/uchidalab/softmaxgradient-lrp"""
"""enable attention gradient in tensorflow and Multi-Node SGLRP if specify multi=True"""
""" !!! need to modify source code in innvestigate to run"""
Could a step by step guide be provided on how to do that?
The specific modification on the iNNvestigate library could be seen in https://github.com/Sizhe-Chen/DAmageNet readme/generation part
The specific modification on the iNNvestigate library could be seen in https://github.com/Sizhe-Chen/DAmageNet readme/generation part
Thank you for that, however I am still getting the error in the original post.
The error could be related to this instruction, since I realised that multi is true when using yolov3.
"""enable attention gradient in tensorflow and Multi-Node SGLRP if specify multi=True"""
Could you guide me in how to "enable attention gradient"? Your help will be greatly appreciated, thank you!
Update: The error was solved with modifying checks.py for LeakyRelu.
However there is a new error:
File "yolov3.py", line 44, in <module>
rad_coco(load_model, get_index, group_dimension=9, attack_dimension=4, transfer_enhance=['SI'])
File "/content/RAD/rad.py", line 194, in rad_coco
**kwargs,
File "/content/RAD/rad.py", line 80, in __init__
self.direction = build_direction(loss, self.model.model_attack.input, TI=('TI' in self.transfer_enhance), transform='1norm')
File "/content/RAD/utils.py", line 38, in build_direction
grad = tf.gradients(loss, input_place)[0]
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/gradients_impl.py", line 664, in gradients
unconnected_gradients)
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/gradients_impl.py", line 923, in _GradientsHelper
(op.name, op.type))
LookupError: No gradient defined for operation 'model_7/gradient_wrt_54/gradients/model_7/up_sampling2d_1/ResizeNearestNeighbor_grad/ResizeNearestNeighborGrad' (op type: ResizeNearestNeighborGrad)
Is there any advice on how to solve this error?
Update 2:
It seems like the issue is UpSampling2D in yolo_body and tiny_yolo_body. The UpSampling2D uses ResizeNearestNeighbor which is what is causing the error.
A possible solution is to write your own gradient, but I am not sure how to do it.
Thanks for pointing out the missing of instructions. Besides replacing base.py in iNNvestigate, it is also necessary to replace deeptaylor.py (which has been uploaded in this repo) in the same folder. The problem is indeed about UpSampling2D.
Thank you, that solved the issue.