Can not find "tokenizer.pkl" of demo.ipynb

Question

Can not find "tokenizer.pkl" of demo.ipynb

Closed this issue 4 years ago · 4 comments

Answer 1 · 2020-08-06T01:52:19.000Z

I try to run the demo.ipynb, I found that there need to load tokenizer.pkl, but I could not found this file in the project.
It will be great if you upload the tokenizer.pkl file, thank you.

with open('tokenizer.pkl', 'rb') as f:
    tokenizer = pickle.load(f)
print(len(tokenizer.word_index))

Error

/Show-and-Tell-keras-master/demo.ipynb", line 36, in <module>
    with open('tokenizer.pkl', 'rb') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'tokenizer.pkl'

Answer 2 · 2020-08-06T06:36:00.000Z

I try to run the demo.ipynb, I found that there need to load tokenizer.pkl, but I could not found this file in the project.
It will be great if you upload the tokenizer.pkl file, thank you.
with open('tokenizer.pkl', 'rb') as f:
    tokenizer = pickle.load(f)
print(len(tokenizer.word_index))
Error
/Show-and-Tell-keras-master/demo.ipynb", line 36, in <module>
    with open('tokenizer.pkl', 'rb') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'tokenizer.pkl'

Hi!
Thanks for your issue raising!
The demo.ipynb is the last step that can be tested.
You should first create tokenizer.pkl with data processing.ipynb. (The 19th execution cell)

with open('tokenizer.pkl', 'wb') as f:
    pickle.dump(tokenizer, f, protocol=pickle.HIGHEST_PROTOCOL)

Answer 3 · 2020-08-14T02:43:53.000Z

Thank you for your advice, I have generated tokenizer.pkl and run inference successfully.
I have a question about this repo: Is this repo based the paper <Show and Tell: Lessons learned from the 2015 MSCOCO Image Captioning Challenge> or ?

I found there are 2 paper about Show and Tell / NIC:

https://arxiv.org/pdf/1609.06647v1.pdf
《Show and Tell: Lessons learned from the 2015
MSCOCO Image Captioning Challenge》 -----------------------Implement NIC_v2
https://arxiv.org/pdf/1411.4555.pdf
《Show and Tell: A Neural Image Caption Generator》 -----------Implement NIC_v1

I noticed that the readme.md of this repo shows Show and Tell: A Neural Image Caption Generator , but I am not sure

Answer 4 · 2020-08-14T02:50:55.000Z

Thank you for your advice, I have generated tokenizer.pkl and run inference successfully.
I found there are 2 paper about Show and Tell / NIC:

https://arxiv.org/pdf/1609.06647v1.pdf
《Show and Tell: Lessons learned from the 2015
MSCOCO Image Captioning Challenge》

https://arxiv.org/pdf/1411.4555.pdf
《Show and Tell: A Neural Image Caption Generator》

I noticed that this repo

I'm glad to be of help!