What is the goal or result of this RNN language model ?

Question

guotong1988 opened this issue 8 years ago · 3 comments

Thank you .

Answer 1 · 2016-09-07T02:39:15.000Z

You can generate text by this RNN language model, more fantastic examples can be found in http://karpathy.github.io/2015/05/21/rnn-effectiveness/.

Answer 2 · 2016-09-07T04:41:04.000Z

多谢大神，问下应该训练多少epoch？我训练了10个epoch之后只能生成一些the，a，I之类的词

Answer 3 · 2016-09-07T07:53:55.000Z

第一，这里的训练集还是太小；第二，只用CPU训练太慢了，10个epoch不一定能收敛；第三，纯RNN对long term的记忆不好，现在主流都是用LSTM等。

PS. 我的代码只是toy，为了理解RNN的机制。

我写了一个用Google Tensorflow训练语言模型的代码，可以参考https://github.com/pangolulu/char-rnnlm-tensorflow

希望对你有帮助。