chiphuyen/stanford-tensorflow-tutorials

Why BOT responses are blanks?

gopinathankm opened this issue · 26 comments

When I run following, Why do I get blank responses from Bot.
(my_env) ubuntu@ip-172-31-16-139:~/chatbot/annbot$ python3 chatbot.py --mode chat
Preparing data to be model-ready ...
Data ready!
Initialize new model
Create placeholders
Create inference
Creating loss...
It might take a couple of minutes depending on how many buckets you have.
WARNING:tensorflow:From /home/ubuntu/anaconda3/envs/my_env/lib/python3.6/site-packages/tensorflow/python/ops/nn_impl.py:1310: softmax_cross_entropy_with_logits (from tensorflow.python.ops.nn_ops) is deprecated and will be removed in a future version.
Instructions for updating:

Future major versions of TensorFlow will allow gradients to flow
into the labels input on backprop by default.

See tf.nn.softmax_cross_entropy_with_logits_v2.

Time: 27.975615978240967
Create optimizer...
It might take a couple of minutes depending on how many buckets you have.
Loading parameters for the Chatbot
Welcome to TensorBro. Say something. Enter to exit. Max length is 60

hi
[[-0.584015 6.833303 -0.7068891 ... -0.23560944 -0.49890977
-0.4948687 ]]
.
hello
[[-0.58008343 7.1182957 -0.69766235 ... -0.22473697 -0.4460981
-0.4717362 ]]
?
Hello
[[-0.58008343 7.1182957 -0.69766235 ... -0.22473697 -0.4460981
-0.4717362 ]]
?
How are you?
[[-0.45013425 6.5066013 -0.71693504 ... -0.17606334 -0.6236066
-0.541961 ]]
.

==================================================================
Also Why output_convo.txt BOT responses are blanks?

(my_env) ubuntu@ip-172-31-16-139:~/chatbot/annbot/processed$ nano output_convo.txt
HUMAN ++++ hi
BOT ++++ .
HUMAN ++++ hello
BOT ++++ ?
HUMAN ++++ Hello
BOT ++++ ?
HUMAN ++++ How are you?
BOT ++++ .
=============================================

I have met the same issue with you.The test loss is high. Did you solve the issue?

No yet.

train it more

How long we need to iterate? I have gone for more than 14000 iterations in 10 hours.
Is not enough?

enough is when you satisfied with output. sometimes something is wrong and training not going well at all and ya can train forewer without successes.

you need to play with your data and settings. chose smaller params first then move to bigger ones to see do it works at all

I trained it for 73000 iterations but still i am getting same results like ? and .

this is settings in my config.py last iteration is 62800

BUCKETS = [(16, 19)]

NUM_LAYERS = 3
HIDDEN_SIZE = 256
BATCH_SIZE = 64

LR = 0.5
MAX_GRAD_NORM = 5.0

and result

> i dont know agent unk
[[ -0.58301872  10.56363678  -0.75994956 ...,  -0.98926491  -1.0604918
   -0.550138  ]]
oh , yeah .
> yeah
[[-0.48157236  9.12944603 -0.45098591 ..., -0.6943714  -0.65536427
  -0.37269545]]
.
> stop doting
[[ -0.68257785  11.46051979  -0.91514087 ...,  -0.51534909  -0.68435407
   -0.14121974]]
- -
> no and stripes i dont like even
[[-0.30254534  7.19619989 -0.30324209 ..., -0.46949279 -0.61815649
  -0.2052134 ]]
you ' re lying .
> no i not
[[ -0.74903452  10.35870457  -0.86198902 ...,  -0.66905749  -0.53297275
   -0.56081152]]
nothing .
> philosophy
[[ -0.57804114  11.67677402  -0.7756232  ...,  -0.4330861   -0.4938038
   -0.44071352]]
.
> how much legs cat hawe ?
[[-0.41352031  8.16804886 -0.73054808 ..., -0.73398674 -0.83470362
  -0.72415113]]
with the <unk> .
> yeah
[[-0.48157236  9.12944603 -0.45098591 ..., -0.6943714  -0.65536427
  -0.37269545]]
.
> tell me 
[[-0.63063455  6.57353258 -0.46259886 ..., -0.70763153 -0.37358218
  -0.24181336]]
don ' t you tell me about it .
> no i dont know it
[[ -0.61053139  10.25772285  -0.99005854 ...,  -0.82599133  -1.04334843
   -0.67073309]]
and you do everything .
> no i dont do everything
[[-0.81464332  8.16336727 -1.04276168 ..., -0.75862557 -1.03167498
  -1.02897918]]
that ' s all .
> ok
[[-0.617365    9.04081535 -0.75897199 ..., -0.75397456 -0.4791148
  -0.37771353]]
don ' t have much .
> bye
[[-0.46397874  8.31635571 -0.79642689 ..., -0.59941679 -0.60873872
  -0.34941944]]
you ' re not in the mood .
> 

What is the test loss? @MartinAbilev

~/dev/tenzorflow-chat-testdrive$ python chatbot.py                                                         
Data ready!                                                                                                                  
Bucketing conversation number 9999                                                                                           
Bucketing conversation number 19999                                                                                          
Bucketing conversation number 9999                                                                                           
Bucketing conversation number 19999                                                                                          
Bucketing conversation number 29999                                                                                          
Bucketing conversation number 39999                                                                                          
Bucketing conversation number 49999                                                                                          
Bucketing conversation number 59999                                                                                          
Bucketing conversation number 69999                                                                                          
Bucketing conversation number 79999                                                                                          
Bucketing conversation number 89999                                                                                          
Bucketing conversation number 99999                                                                                          
Bucketing conversation number 109999                                                                                         
Bucketing conversation number 119999                                                                                         
Bucketing conversation number 129999                                                                                         
Bucketing conversation number 139999                                                                                         
Bucketing conversation number 149999                                                                                         
Bucketing conversation number 159999                                                                                         
Bucketing conversation number 169999                                                                                         
Bucketing conversation number 179999                                                                                         
Bucketing conversation number 189999                                                                                         
Number of samples in each bucket:
 [103198]
Bucket scale:
 [1.0]
Initialize new model
Create placeholders
Create inference
Creating loss... 
It might take a couple of minutes depending on how many buckets you have.
Time: 2.9458863735198975
Create optimizer... 
It might take a couple of minutes depending on how many buckets you have.
Creating opt for bucket 0 took 6.980544805526733 seconds
Running session
Loading parameters for the Chatbot
Iter 62900: loss 1.7389020776748658, time 1.6152856349945068
Iter 63000: loss 1.713471269607544, time 1.4531621932983398
Test bucket 0: loss 3.626643180847168, time 2.8442370891571045

she is fun as ever :D

Running session
Loading parameters for the Chatbot
Iter 62900: loss 1.7389020776748658, time 1.6152856349945068
Iter 63000: loss 1.713471269607544, time 1.4531621932983398
Test bucket 0: loss 3.626643180847168, time 2.8442370891571045
Iter 63100: loss 1.7354186856746674, time 1.5451974868774414
Iter 63200: loss 1.7274975204467773, time 1.4273159503936768
Iter 63300: loss 1.7350391232967377, time 1.4423978328704834
^CTraceback (most recent call last):
  File "chatbot.py", line 262, in <module>
    main()
  File "chatbot.py", line 257, in main
    train()
  File "chatbot.py", line 156, in train
    _, step_loss, _ = run_step(sess, model, encoder_inputs, decoder_inputs, decoder_masks, bucket_id, False)
  File "chatbot.py", line 82, in run_step
    outputs = sess.run(output_feed, input_feed)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py", line 889, in run
    run_metadata_ptr)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py", line 1120, in _run
    feed_dict_tensor, options, run_metadata)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py", line 1317, in _do_run
    options, run_metadata)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py", line 1323, in _do_call
    return fn(*args)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py", line 1302, in _run_fn
    status, run_metadata)
KeyboardInterrupt
~/dev/tenzorflow-chat-testdrive$ python3 chatbot.py --mode chat
Data ready!
Initialize new model
Create placeholders
Create inference
Creating loss... 
It might take a couple of minutes depending on how many buckets you have.
Time: 2.344371795654297
Create optimizer... 
It might take a couple of minutes depending on how many buckets you have.
Loading parameters for the Chatbot
Welcome to TensorBro. Say something. Enter to exit. Max length is 16
> hello
[[ -0.42601886  11.76952934  -0.96129787 ...,  -0.38945305  -0.63972712
   -0.52184105]]
come on .
> why you want me to do this ?
[[-0.84985662  7.54191875 -0.66726327 ..., -0.46486002 -0.69005805
  -0.38926086]]
throw me a little while .
> i trow some chocolate to you
[[-0.63454562  7.31223917 -0.68008763 ..., -0.3172918  -0.62918943
  -0.36875695]]
i ' m serious .
> me to !
[[ -1.19059336  10.17427731  -1.53483987 ...,  -0.87535918  -1.28455663
   -0.89420712]]
way to my room !
> ohh no
[[-0.62382394  8.92223644 -0.51953173 ..., -0.67145741 -0.60165089
  -0.46206495]]
no .
> i don`t want to get to bed with you now .
[[-0.61282045  7.70583582 -0.86621624 ..., -1.14901936 -1.03707504
  -0.30115816]]
where ' s the money ?
> i don`t tell you !
[[-0.9617241   9.48097229 -1.16938531 ..., -0.74119282 -1.03512502
  -0.71569276]]
what ?
> nothing
[[ -0.5954656   11.44416332  -0.8853721  ...,  -0.87170106  -0.29809356
   -0.50806785]]
- -
> bye
[[-0.35947308  7.6860323  -0.81516069 ..., -0.51186019 -0.61428398
  -0.26302782]]
i need to be .
> me too
[[-0.63649333  8.02639294 -1.06983578 ..., -0.93391991 -0.56139541
  -0.53440243]]
don ' t you ' re going to be here .
> no i go away. i go home .
[[-0.90490222  9.08857822 -0.83062935 ..., -0.40304869 -1.07223058
  -1.00504088]]
you will not . . . but you are .
> i am same as you 
[[-1.13782763  8.06291008 -1.10772276 ..., -0.78122187 -1.07012486
  -0.79459012]]
if you please leave .
> yes i leave. bye.    
[[-0.99189955  9.18696594 -0.97211885 ..., -0.34530568 -1.24671197
  -0.69110519]]
good .
> 

i even have the same setting in config.py as your and my last iteration was 117000, but still only exclaimations and dots!!!
@MartinAbilev

something is wrong looks like with code or training data. it is hard to tell what exactly. you can create repo with your code and drop link.

or you can check my copy and compare with you code.. it is public.

I have the same problem. @akhil2910c have you solved it?

No I didn't @EthanPhan!!

Ok. I'm gonna check MartinAbilev's code out and see if it works and see what is the difference. I'll update if I find any.

Unfortunately I have the same blank results. Something isn't working right with the code out of the box. Adjusting parameters made it worse.

I also have that problem. Is there any practical solution?

Any one found a solution?

I'm still getting blank answers

If you're not wed to using this particular model, this one works better than any other I've seen out in the open: https://github.com/pender/chatbot-rnn

It can be insulting though, as it was trained on reddit data, so be forewarned.

@J-Fo-S thank you for sharing the link. Is this your project?

You're welcome but no, it is not my project.

Train your model and check. You can train with your custom small dataset line-by-line dialog:

===
Hi
Hello, How are you?
I am fine. Thanks.
What are you doing?
I am just chatting with you.

wz111 commented

Is it related to the number of buckets?

I think the model is too simple! So its matching competence is not enough to hold so much data.
When I use 189999 samples, the final loss is around 2.2.
When 110000 samples, corresponding loss is around 2.0.
When lower samples, loss is still decreasing!
Hope this can help you.