Apply new fused rotary embedding
Quentin-Anthony opened this issue · 10 comments
This could really help us: NVIDIA/apex#1746
May also need NVIDIA/apex#1750 applied in order to test FYI
This is naively ported but untested.
This is naively ported but untested.
Huh? Please clarify.
This is naively ported but untested.
Huh? Please clarify.
Sorry, that was a progress update. I've ported the code changes to GPT-NeoX but haven't had a chance to test them yet.
This is naively ported but untested.
Huh? Please clarify.
Sorry, that was a progress update. I've ported the code changes to GPT-NeoX but haven't had a chance to test them yet.
Gotcha.
@StellaAthena -- Can you write up what you have into a draft PR? We can offload testing from you.
@Quentin-Anthony They're not ready for testing yet, but I can still open a draft PR if you'd like. Right now I'm in a place where I think I've copied over all of the core code but the kernels aren't building and I haven't been able to debug why that is yet.
I had hoped to make more progress on this last week, but got swamped with some other stuff. I'm happy to hand it off if someone else wants to take it over.
@Quentin-Anthony They're not ready for testing yet, but I can still open a draft PR if you'd like. Right now I'm in a place where I think I've copied over all of the core code but the kernels aren't building and I haven't been able to debug why that is yet.
I had hoped to make more progress on this last week, but got swamped with some other stuff. I'm happy to hand it off if someone else wants to take it over.
Yes please make a draft PR
@StellaAthena Still wanting help getting this to work / testing this?
@StellaAthena Still wanting help getting this to work / testing this?
That would be excellent, thank you.