How to solve the error "NaN loss during training"? thanks

[datar001]

environment: tensorflow -gpu1.10 python3.65 no matlab(only train model)
I comment line 8、11 and 35 in trainer_preid.py, and set do_evaluation=False.
#from evaluation.evaluation_helper import get_evaluation_summary_writer
#from predictor_preid import run_evaluation_conserving_best
#evaluation_summary_writer = get_evaluation_summary_writer(do_evaluation, output_directory)
When i execute the following code:
python trainer_preid.py --output=outputs/ --data=data/market/ --dataset-name=market1501 --batch-size=16 --num-epochs=100 --network-name=resnet_v1_50 --initial-checkpoint=checkpoint/resnet_v1_50.ckpt --checkpoint-exclude-scopes=resnet_v1_50/logits --trainable-scopes=resnet_v1_50/logits --no-evaluation
The error occured:
tensorflow.python.training.basic_session_run_hooks.NanLossDuringTrainingError: NaN loss during training.