Trusted-AI/adversarial-robustness-toolbox

Trigger Placement Bugs for Image Poisoning Perturbations

f4str opened this issue · 0 comments

f4str commented

Describe the bug
The insert_image function in art.attacks.poisoning.perturbations has two bugs involving the placement of the trigger image.

  1. Due to Pillow swapping the height and width, a non-square trigger is the wrong dimensions when placed onto the image. This was partially addressed by PR #2046 which fixed the incorrect dimensions for the input image, but did not make the same change for the trigger. The fix is is to apply the same where the height and width are swapped.

  2. When the trigger height or width is the same size as the image height or width, the insert_image function will error due to the np.random.randint function not accepting 0 as the upper bound. This numpy function will sample from the lower bound inclusively and upper bound exclusively. This is an off-by-one error and can be easily fixed by adding one to the difference between the image height/width and trigger height/width.

To Reproduce

  1. The following code snippet will reproduce the first bug:

    import numpy as np
    import matplotlib.pyplot as plt
    from art.attacks.poisoning.perturbations import insert_image
    
    image = np.ones((32, 32, 3))
    image_poisoned = insert_image(
        image,
        backdoor_path='../utils/data/backdoors/htbd.png',
        size=(5, 10),
        random=False,
        x_shift=0,
        y_shift=0,
        mode='RGB'
    )
    plt.imshow(image_poisoned)

    From this, the trigger will be inserted as a 10 x 5 rather than the specified 5 x 10 due to Pillow swapping the height and width order.

  2. The following code snippet will reproduce the second bug:

    import numpy as np
    import matplotlib.pyplot as plt
    from art.attacks.poisoning.perturbations import insert_image
    
    image = np.ones((32, 32, 3))
    image_poisoned = insert_image(
        image,
        backdoor_path='../utils/data/backdoors/htbd.png',
        size=(32, 32),
        random=True,
        mode='RGB'
    )
    plt.imshow(image_poisoned)

    This will raise an exception due to the np.random.randint function not accepting a value of 0 as the trigger and image size are the same.

Expected behavior
The insert_image function should produce the correct behavior and not error when valid inputs are provided.

Screenshots
N/A

System information (please complete the following information):

  • OS
  • Python version
  • ART version or commit number
  • TensorFlow / Keras / PyTorch / MXNet version