Everything appears to be installed OK, but no audio is detected and no messages are sent
Opened this issue · 14 comments
Hi there; thanks for this excellent set of scripts.
I have installed everything, asterisk 17.2 works and hears audio and can complete a normal call and record audio, MQTT is up and running (I have tested with MQTTLens), but it never seems to detect any speaking.
I would really appreciate it if you could see if anything stands out as being wrong with the config and logs. I know port 1883 is open OK, and I can send and subscribe to topics on MQTTLens.
But should I not also be able to see/post to the following topics? Nothing ever happens there.
dialogflow-asterisk-newStream
dialogflow-asterisk-streamEnded
Many thanks
asterisk-extensions.conf.txt
dialogflow-rtp-audioserver-config.txt
dialogflow-ari-bridge-config.txt
asterisk-agi-debug-output.txt
asterisk-dialogflow-rtp-audioserver-log.txt
dialogflow-ari-bridge-log.txt
hey @digitaltoast ! so it looks like you are getting the new stream and stream ended topics.... I can see start and end events in the rtp audio server log.
have you enabled the v1beta api stuff in your project in the dialogflow console? while youre there have you enabled audio output? its a setting on the project
Oh, many many thanks for getting back to me so quickly @danjenkins - well, I think this is right, but I'm not sure about "enabled the v1beta api stuff" - did you mean v2? I don't seem to have a choice - the radio box for v2 is checked.
Yup that's it! Interesting.... I'll have a proper dig through the logs in the morning and see if I can see anything else :)
Just wondering if there's a way of turning on more logging, or perhaps intercepting a copy of the audio or messages sent to Dialogflow? I don't want you to have to be doing my debugging for me, but equally, it looks like info and examples for External Media are sparse.
I've found a few reference items and similar (but not as good!) projects
https://wiki.asterisk.org/wiki/display/AST/External+Media+and+ARI
https://wiki.asterisk.org/wiki/display/AST/Getting+Started+with+ARI
https://blogs.asterisk.org/2019/10/09/external-media-a-new-way-to-get-media-in-and-out-of-asterisk/
https://cloud.google.com/dialogflow/docs/detect-intent-stream#detect-intent-stream-nodejs
https://github.com/asterisk/asterisk-external-media
https://github.com/pc-m/transcript-demo/blob/master/transcript_demo/ari_stasis.py
My project is self funded for a test service that that helps blind people who don't use the internet, but your project looks so ideal for what I need that if you can help me get it going or nudge me in the right direction, I promise I will write a complete "newbie FAQ" to help others from what I learnt setting it up, and give you a tip (won't be a huge amount as self-funding, but certainly won't be £2 or something, as I appreciate your efforts)
@digitaltoast if you want and feel comfortable doing it, do you want to give me access to your firebase project and I could do some basic debugging my side. Basically - there are so many different parts that trying to point you in the right direction is difficult :(
Hi @danjenkins - many thanks! Only just saw this - I tried adding the contact address on website but it said "Account contact@xxxx.e is of type "group". Please set the type prefix to be "group:".
Drop me an email to digitaltoast@gmail.com
I've just setup a default demo, so please feel free to have a play, break it etc. Please let me know if you don't have the right permissions or need something else.
Hey @digitaltoast I just created a key for your project so that I could access the dialogflow project, whacked that into the config, and set the project ID and everything works perfectly - it now gives back a welcome event (new code added into master today) so no need to prompt yourself and it hears me and transcribes me. The only thing I can think of is that you're not getting any media from asterisk? Setting it all up can be a real pain - asterisk & the RTP Server component of this project have to be able to see each other with no nat involved - we base the join between a caller and the dialogflow session by the port we receive audio from asterisk - asterisk's source port rather than where it turned up (because it all turns up on that one port)
Thanks @danjenkins - hmmm; I tend to use small "throwaway" AWS EC2 instances for this type of thing, so I know I've got a "fresh-n-clean" Ubunutu minimal. I might dig around and see if it's a firewall issue, but I know I can make a call between 2 microsip softphones and record the audio.
I've turned on every kind of logging I can in Asterisk; might see if I can make MQTT more verbose
Thanks so much for checking this out. At least I know it's something at my end!
I can almost guarantee you its a nat issue :)
AWS doesn't give you a nice public IP on your instance. Instead they attach the public IP elsewhere in their infra.So I can guarantee you that the source port that we get from Asterisk and how we know to listen for traffic etc is different to what you actually get the media from.
Thats actually only if you're using a separate instance than your asterisk box? If theyre on the same box then that won't be it. If theyre both in AWS make sure youre sending media using the private IP and then you wont have the NAT issue. Another candidate is to use Digital Ocean (I can give you a nice referral link if you dont have an account) because they attach the public IP directly to the instance
Ha! You're not going to believe this - I was determined to nail this today, so I tried EVERYTHING from pjsip configuration, to codecs firewall configs to... well, everything I could think of!
In the end, do you know what got audio flowing and everything working perfectly?
Doesn't work:
same => n,Answer()
same => n,Stasis(dialogflow)
Works perfectly (complete chance I just thought to try this)
same => n,Answer()
same => n,Playback(hello-world)
same => n,Stasis(dialogflow)
As to WHY this is the case, is a mystery, but it appears that sending some audio first gets things flowing. Almost certainly an Asterisk weirdness and not a problem with your app, but might be worth dropping into the docs... unless there is something I can diagnose first to narrow it down?
Anyway, seems to be OK now - thanks again!
@digitaltoast try without the answer - you dont want to answer it - the ARI app does that
this is mine
[ITEXPO]
exten = dialogflow,1,Verbose(1, "ITEXPO call, woot")
same = n,Stasis(dialogflow)
same = n,Hangup()
Hi @danjenkins - nope, same with or without answer. The reason I put answer in there in the first place is because that's what the Asterisk docs say :)
https://wiki.asterisk.org/wiki/display/AST/Getting+Started+with+ARI
I updated to your latest release today - I notice that if you hang up while receiving audio, it throws an uncaught exception. No biggy, I'll take a look over the weekend, chuck a "catch" in there, learn to use Github properly and maybe even make myself useful and submit a pull request and "newbie/dummies" guide!
Tell you what, this script of your is amazing - it has opened up a huge world of possibilities for my test project....
[1584132122747] INFO (Dialogflow-AudioServer/11547 on ip-172-XXX-XXX-244): sending audio back to asterisk 332 16412 127.0.0.1
[1584132122780] INFO (Dialogflow-AudioServer/11547 on ip-172-XXX-XXX-244): sending audio back to asterisk 332 16412 127.0.0.1
[1584132122789] INFO (Dialogflow-AudioServer/11547 on ip-172-XXX-XXX-244): Ending stream of audio from Asterisk to send to Dialogflow
payload: {
"name": "1001",
"port": "16412",
"callerName": "tnsff1",
"channelId": "1584132119.60"
}
[1584132122790] INFO (Dialogflow-AudioServer/11547 on ip-172-XXX-XXX-244): Asterisk Stream closed so closing connection to DialogFlow and doing tidy up
id: "1584132119.60"
[1584132122790] INFO (Dialogflow-AudioServer/11547 on ip-172-XXX-XXX-244): Destroying DialogFlow stream
id: "1584132119.60"
events.js:288
throw er; // Unhandled 'error' event
^
Error [ERR_STREAM_WRITE_AFTER_END]: write after end
at writeAfterEnd (_stream_writable.js:264:14)
at PassThrough.Writable.write (_stream_writable.js:313:5)
at Timeout._onTimeout (/var/lib/asterisk/agi-bin/asterisk-dialogflow/asterisk-dialogflow-rtp-audioserver/lib/DialogFlowConnector.js:168:46)
at listOnTimeout (internal/timers.js:549:17)
at processTimers (internal/timers.js:492:7)
Emitted 'error' event on PassThrough instance at:
at errorOrDestroy (internal/streams/destroy.js:108:12)
at writeAfterEnd (_stream_writable.js:266:3)
at PassThrough.Writable.write (_stream_writable.js:313:5)
[... lines matching original stack trace ...]
at processTimers (internal/timers.js:492:7) {
code: 'ERR_STREAM_WRITE_AFTER_END'
}
Done in 9.29s.
Thanks! I've made an issue for the error eriting to a non existant stream #7
Going to close this one now. Not sure why you have to answer first..... I don't!
@digitaltoast I was wondering if you experience issues on AWS with audio playback from Dialogflow