facebookresearch/xformers

Allowing decoder only definition

nh2liu opened this issue · 2 comments

nh2liu commented

🚀 Feature

Allow only a decoder config to be defined.

Motivation

I want to define only a decoder and pass in a memory vector from another source.

Pitch

I tried this change locally and it allows me to do what I want it to do: main...nh2liu:xformers:patch-1

Not sure if this has extending implications because it seems this code has been around for a while but the comments # If decoder: either use the encoder ouput, or just decode, both options are possible indicate that this may be a bug.

Alternatives

  • NOOP encoder will also allow this functionality.
danthe3rd commented

cc @blefaudeux @fmassa maybe?

  • 👍1
blefaudeux commented

@danthe3rd thanks for the ping ! @nh2liu makes a lot of sense, thanks for the patch ! I would just alter it a tiny bit: only clone the input if there are no encoders, else you'll incur a useless memcopy. Other than that looks great to me, and I would add this configuration (decoder only) to the existing unit test to make sure that this is nailed ? Just my two cents !