[BUG] The argument --no-position-embedding should be fixed
Opened this issue · 0 comments
Hoonly commented
Describe the bug
from the file Megatron-LM/megatron/training/arguments.py
group.add_argument('--no-position-embedding',
action='store_false',
help='Disable position embedding. Deprecated: use --position-embedding-type',
dest='add_position_embedding')
I can see that this argument is Deprecated, but if we only use --position-embedding-type=rope, the add_position_embedding will accept default value True, conflict to the rope
To Reproduce
Expected behavior
Stack trace/logs
Environment (please complete the following information):
- Megatron-LM commit ID 0d983e6
- PyTorch version 2.3.0a0+ebedce2
- CUDA version V12.3.107
- NCCL version 2.19.4
Proposed fix
If you have a proposal for how to fix the issue state it here or link to a PR.
Additional context
Add any other context about the problem here.