NVIDIA/Megatron-LM

[BUG] The argument --no-position-embedding should be fixed

Opened this issue · 0 comments

Describe the bug
from the file Megatron-LM/megatron/training/arguments.py

    group.add_argument('--no-position-embedding',
                       action='store_false',
                       help='Disable position embedding. Deprecated: use --position-embedding-type',
                       dest='add_position_embedding')

I can see that this argument is Deprecated, but if we only use --position-embedding-type=rope, the add_position_embedding will accept default value True, conflict to the rope

To Reproduce

Expected behavior

Stack trace/logs

Environment (please complete the following information):

  • Megatron-LM commit ID 0d983e6
  • PyTorch version 2.3.0a0+ebedce2
  • CUDA version V12.3.107
  • NCCL version 2.19.4

Proposed fix
If you have a proposal for how to fix the issue state it here or link to a PR.

Additional context
Add any other context about the problem here.