Occlusion sensitivity works for only one class, but shows error for other class in cat_vs_dogs dataset from tensorflow
rao208 opened this issue · 3 comments
I am trying to apply occlusion sensitivity on the "cats_vs_dog" dataset. First, the model is trained as given in the link https://www.tensorflow.org/tutorials/images/transfer_learning and then the trained model and the sample image is loaded to carry out occlusion sensitivity.
In an amateur attempt, the following lines were added in the code for tf-explain.
img, lbl = format_example(image, label) #img and lbl are tensor. Please see the above link for format_example funciton
img = img.numpy()
data = ([img], None)
print("Data", data)
classindex = lbl.numpy()
print("classindex", type(classindex))
# Instantiation of the explainer
explainer = OcclusionSensitivity()
# Call to explain() method
output = explainer.explain(data, new_model, class_index = classindex, patch_size = 4)
# Save output
explainer.save(grid = output, output_dir = ".",
output_name = "os-" + datetime.now().strftime("%Y%m%d-%H%M%S") + ".png")
It works fine for class 0 i.e. cats but fails to produce results for class 1 i.e. dogs.
Traceback (most recent call last):
File "C:\Users\Vanditha Rao.spyder-py3\os_tf_explain_cats_vs_dogs.py", line 107, in
output = explainer.explain(data, new_model, class_index = classindex, patch_size = 4)File "D:\Anaconda\envs\master_thesis\lib\site-packages\tf_explain\core\occlusion_sensitivity.py", line 49, in explain
for image in imagesFile "D:\Anaconda\envs\master_thesis\lib\site-packages\tf_explain\core\occlusion_sensitivity.py", line 49, in
for image in imagesFile "D:\Anaconda\envs\master_thesis\lib\site-packages\tf_explain\core\occlusion_sensitivity.py", line 102, in get_sensitivity_map
prediction[class_index] for prediction in predictionsFile "D:\Anaconda\envs\master_thesis\lib\site-packages\tf_explain\core\occlusion_sensitivity.py", line 102, in
prediction[class_index] for prediction in predictionsIndexError: index 1 is out of bounds for axis 0 with size 1
The complete code is:
import matplotlib.pyplot as plt
import tensorflow as tf
import tensorflow_datasets as tfds
import urllib3
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
from tensorflow.keras.models import load_model
from tensorflow.keras.models import model_from_json
from tf_explain.core.occlusion_sensitivity import OcclusionSensitivity
from datetime import datetime
tfds.disable_progress_bar()
(raw_train, raw_validation, raw_test), metadata = tfds.load('cats_vs_dogs',
split=[
tfds.Split.TRAIN.subsplit(tfds.percent[:80]),
tfds.Split.TRAIN.subsplit(tfds.percent[80:90]),
tfds.Split.TRAIN.subsplit(tfds.percent[90:]),
],
with_info=True,
shuffle_files=True,
as_supervised=True,)
get_label_name = metadata.features['label'].int2str
classes = metadata.features["label"].names
print("done")
for image, label in raw_train.take(1):
print("Label: %d" % label.numpy())
plt.figure()
plt.imshow(image)
plt.title(get_label_name(label))
IMG_SIZE = 160 # All images will be resized to 160x160
def format_example(image, label):
image = tf.cast(image, tf.float32)
image = (image/127.5) - 1
image = tf.image.resize(image, (IMG_SIZE, IMG_SIZE))
return image, label
# Apply this function to each item in the dataset using the map method:
train = raw_train.map(format_example)
validation = raw_validation.map(format_example)
test = raw_test.map(format_example)
BATCH_SIZE = 32
SHUFFLE_BUFFER_SIZE = 1000
train_batches = train.shuffle(SHUFFLE_BUFFER_SIZE).batch(BATCH_SIZE)
validation_batches = validation.batch(BATCH_SIZE)
test_batches = test.batch(BATCH_SIZE)
for image_batch, label_batch in train_batches.take(10):
pass
print("image_batch.shape", image_batch.shape)
print("label_batch.shape", label_batch.shape)
# Loads the weights
# Model reconstruction from JSON file
with open('./model/model_cats_vs_dogs.json', 'r') as f:
new_model = model_from_json(f.read())
new_model = load_model('./model/model_cats_vs_dogs.h5')
checkpoint_path = "./training_2_weights/cp.ckpt"
new_model.load_weights(checkpoint_path)
new_model.summary()
# Re-evaluate the model
test_steps=20
loss,acc = new_model.evaluate(test_batches, verbose=1, steps = test_steps)
print("Restored model, accuracy: {:5.2f}%".format(100*acc))
img, lbl = format_example(image, label)
print(img.shape, type(img))
img = img.numpy()
print(type(img))
data = ([img], None)
print("Data", data)
classindex = lbl.numpy()
print("classindex", type(classindex))
# Instantiation of the explainer
explainer = OcclusionSensitivity()
# Call to explain() method
output = explainer.explain(data, new_model, class_index = classindex, patch_size = 4)
# Save output
explainer.save(grid = output, output_dir = ".",
output_name = "os-" + datetime.now().strftime("%Y%m%d-%H%M%S") + ".png")
Also, the occlusion sensitivity method works fine on the pre-trained model like MobileNet or VGG but doesn't work on the own model.
facing the same issue, any solution?
@ShreyashSoni Yes, the problem was I wasn't using the right activation function to get the probability score for both the classes. What I mean is, if you use a softmax activation function in the final layer (even if the dataset has two labels), you will get the probability score for both of them whereas if you use any other activation function then you will just get the probability for one class.
It is important to have the probability score for both the class because when we call tf explain Occlusion Sensitivity, we must specify the class index.
output = explainer.explain(data, new_model, class_index = classindex, patch_size = 4)
I hope it helps!
@rao208 thanks for the explanation. this helps