Can it run on Mac M1 (Apple silicon?)

[slashdottir]

I love this proggie so much, wish I could run it on my M1 laptop.

Trying your readme example:

    from PIL import Image
    from clip_interrogator import Config, Interrogator
    
    image_path = "./images/art.jpg"
    image = Image.open(image_path).convert('RGB')
    ci = Interrogator(Config(clip_model_name="ViT-L-14/openai"))
    result = ci.interrogate(image)
    print(result)

It fails with this error:

 RuntimeError: "LayerNormKernelImpl" not implemented for 'Half'

stacktrace:

    i File "/Users/foo/src/pharmapsychotic_reverse_prompt/clip-interrogator/clip_interrogator/clip_interrogator.py", line 266, in __init__
        text_features = clip_model.encode_text(text_tokens)
      File "/opt/homebrew/anaconda3/envs/revprompt/lib/python3.10/site-packages/open_clip/model.py", line 235, in encode_text
        x = self.transformer(x, attn_mask=self.attn_mask)
      File "/opt/homebrew/anaconda3/envs/revprompt/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "/opt/homebrew/anaconda3/envs/revprompt/lib/python3.10/site-packages/open_clip/transformer.py", line 321, in forward
        x = r(x, attn_mask=attn_mask)
      File "/opt/homebrew/anaconda3/envs/revprompt/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "/opt/homebrew/anaconda3/envs/revprompt/lib/python3.10/site-packages/open_clip/transformer.py", line 242, in forward
        x = q_x + self.ls_1(self.attention(q_x=self.ln_1(q_x), k_x=k_x, v_x=v_x, attn_mask=attn_mask))
      File "/opt/homebrew/anaconda3/envs/revprompt/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "/opt/homebrew/anaconda3/envs/revprompt/lib/python3.10/site-packages/open_clip/transformer.py", line 27, in forward
        x = F.layer_norm(x, self.normalized_shape, self.weight, self.bias, self.eps)
      File "/opt/homebrew/anaconda3/envs/revprompt/lib/python3.10/site-packages/torch/nn/functional.py", line 2515, in layer_norm
        return torch.layer_norm(input, normalized_shape, weight, bias, eps, torch.backends.cudnn.enabled)
    RuntimeError: "LayerNormKernelImpl" not implemented for 'Half'
    python-BaseException

Tried setting device to 'cpu' instead of 'cuda', same error.

Thanks for any clues

[pharmapsychotic]

hey I just finally got around to playing with it on M1 mac. it looks like it does work with pytorch 2 and fast mode is works well but "best" is very slow. if you haven't yet try updating to latest torch (pip install --upgrade torch)

[slashdottir]

I tried it out with the latest code and set the torch.device to 'mps' and it worked! The 'flavor chain' part is very slow, but it seemed to finish even with flavor chain only at 28% mark? Anyway. If there is a recommended way to set it to maximize use of M1 capabilities, please advise.

Seems like this bit of code here overrides the device setting and sets the torch device back to 'cuda' which of course defaults back to 'cpu'

Looks like there is a recent bug to fix this in pytorch (active 2 days ago): pytorch/pytorch#99272

Thank you! ❤️

[remixer-dec]

There is already a PR suggesting autocast changes for a more recent version, but no fp16 is mentioned. Just wanna add that fp16 also works on mps. It reduces RAM usage in half.
Here is a hardcoded fp16 mps code for clip-interrogator==0.5.4