AttributeError: 'GPT2Model' object has no attribute '_step'

[iseesaw]

I meet an error while I running the following code to convert the tensorflow checkpoint into pytorch version.

export OPENAI_GPT2_CHECKPOINT_PATH=kogpt2-large-v2-tf/model.ckpt-225000

transformers-cli convert --model_type gpt2 \
  --tf_checkpoint $OPENAI_GPT2_CHECKPOINT_PATH \
  --pytorch_dump_output kogpt2-large-v2-tf/pytorch_model.bin

What's the problem? Could you please help me ? Thank you.

...
Loading TF weight newslm/layer23/query_layer/kernel/adafactor_vc with shape [1536]
Loading TF weight newslm/layer23/query_layer/kernel/adafactor_vr with shape [1536]
Loading TF weight newslm/layer23/value_layer/bias with shape [1536]
Loading TF weight newslm/layer23/value_layer/bias/adafactor_v with shape [1536]
Loading TF weight newslm/layer23/value_layer/kernel with shape [1536, 1536]
Loading TF weight newslm/layer23/value_layer/kernel/adafactor_vc with shape [1536]
Loading TF weight newslm/layer23/value_layer/kernel/adafactor_vr with shape [1536]
Traceback (most recent call last):
  File "/users6/kyzhang/anaconda3/bin/transformers-cli", line 8, in <module>
    sys.exit(main())
  File "/users6/kyzhang/anaconda3/lib/python3.6/site-packages/transformers/commands/transformers_cli.py", line 51, in main
    service.run()
  File "/users6/kyzhang/anaconda3/lib/python3.6/site-packages/transformers/commands/convert.py", line 152, in run
    convert_gpt2_checkpoint_to_pytorch(self._tf_checkpoint, self._config, self._pytorch_dump_output)
  File "/users6/kyzhang/anaconda3/lib/python3.6/site-packages/transformers/models/gpt2/convert_gpt2_original_tf_checkpoint_to_pytorch.py", line 39, in convert_gpt2_checkpoint_to_pytorch
    load_tf_weights_in_gpt2(model, config, gpt2_checkpoint_path)
  File "/users6/kyzhang/anaconda3/lib/python3.6/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 110, in load_tf_weights_in_gpt2
    pointer = getattr(pointer, scope_names[0])
  File "/users6/kyzhang/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 1131, in __getattr__
    type(self).__name__, name))
AttributeError: 'GPT2Model' object has no attribute '_step'

[ksjae]

Referencing this, I think you should write the custom loading script.

I did not use transformers-cli convert and IIRC my models using HF transformers are pytorch files (should be in the earlier versions) so I'm afraid that I can't be of much help. Maybe re-training KoGPT may help.

[iseesaw]

Thanks, I get it.