Broadcast request gets send, but not played/broadcasted to the speakers

Question

Broadcast request gets send, but not played/broadcasted to the speakers

AleandroD opened this issue 2 years ago · 27 comments

I was able to get the container up & running, but when I send request via Insomnia/CURL/Home Asisstant I get the following lines in the logs of the container:

But the speakers (Google Nest Audio's) do not broadcast the message.
If I look at the Google Console Platform I see that the request do get send but still they aren't played:

The client_secret is made for my main google account which is also the owner of my google home, so it should have all the rights/persmissions it needs.

My question is did I forget to configure some things?

In your setup do you have to do specific settings to tell it which speakers it could broadcast to? (I have Google Nest speakers which I can broadcast to with Google Assistant via my phone/car)
In the OAuth consent screen do you need to publish the APP? I did not and have it set tot external user type with myself (main google account email address) as a test user.

Answer 1 · 2022-05-28T18:09:17.000Z

I have experienced the same (strange) case as you describe here and it is kind of related to language settings of your Google Assistant and the language you broadcast the message is in.

I have configured language settings in my Google Assistant app (on my phone and logging in with same Google username as I configured with the Oauth token) with two languages: Norwegian and English.

If I try to broadcast the message "Hello world!" I see the same log output as you pasted above, but nothing is played on my speakers. But if I broadcast the message "Hei verden!" (in Norwegian) I see the same log output, but with sound on speakers. I tried to remove Norwegian from selected languages in Google Assistant app, but the same behaviors where still there.

I where able to reproduce the same behavior by typing "Broadcast Hello world!" and "Broadcast Hei verden!" in Google Assistant app, and only the last message where broadcasted to the speakers. Though, in Google Assistant mobile app I received reply "Sorry, I didn't understand." for first message.

Answer 2 · 2022-05-28T18:52:38.000Z

My suspicion on language setting where correct. I changed language settings at https://github.com/ismarslomic/google-assistant-broadcast/blob/main/assistant.js#L15 from en-US to en-GB and now I can broadcast messages in both languages: Norwegian and English. Valid locales are described in Google documentation: https://developers.google.com/assistant/console/languages-locales

I will make it possible to set language property as environment variable in the docker container, so you can override the default value (which I will change to en-GB) so each user can play with this property if they experience the same issue as you.

Answer 3 · 2022-05-28T19:56:23.000Z

I'll test it after your changes.
But I also have two languages Dutch & English and I can let google assistant broadcast in both languages.
I checked in my google assistant logs and both come in the log but doesn't get played.

But i'll check after changes :D

Answer 4 · 2022-05-28T20:59:31.000Z

@AleandroD can you please test again with docker container ismarslomic/google-assistant-broadcast:pr-6 and report back if the issue is resolved?

Answer 5 · 2022-05-28T21:58:12.000Z

Sadly still nothing, still the same results.
Then I was trying to use your new environment variable langue, but got some errors:

google-assistant-broadcast    | [ERROR] Error while broadcasting: Error: 3 INVALID_ARGUMENT: Invalid 'dialog_state_in': bad language_code.
google-assistant-broadcast    |     at Object.callErrorFromStatus (/usr/src/app/node_modules/@grpc/grpc-js/build/src/call.js:31:26)
google-assistant-broadcast    |     at Object.onReceiveStatus (/usr/src/app/node_modules/@grpc/grpc-js/build/src/client.js:391:49)
google-assistant-broadcast    |     at Object.onReceiveStatus (/usr/src/app/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:328:181)
google-assistant-broadcast    |     at /usr/src/app/node_modules/@grpc/grpc-js/build/src/call-stream.js:187:78
google-assistant-broadcast    |     at process.processTicksAndRejections (node:internal/process/task_queues:77:11) {
google-assistant-broadcast    |   code: 3,
google-assistant-broadcast    |   details: "Invalid 'dialog_state_in': bad language_code.",
google-assistant-broadcast    |   metadata: Metadata {
google-assistant-broadcast    |     internalRepr: Map(1) { 'grpc-server-stats-bin' => [Array] },
google-assistant-broadcast    |     options: {}
google-assistant-broadcast    |   }
google-assistant-broadcast    | }

With LANGUAGE='en-GB' of LANGUAGE='nl-BE', weird because those languages/locales should be supported or is that the wrong variable?

Answer 6 · 2022-05-28T21:58:46.000Z

Ow that close was a misclick on my end.

Answer 7 · 2022-05-28T22:06:21.000Z

nl-BE is not supported language, please see list of supported languages. I suggest that you only change to the correct region of English language (GB, US, etc), based on the English selection you have in your Google Assistant app.

For instance, I have following two languages for my Assistant:

Norwegian
English (United Kingdom)

Answer 8 · 2022-05-28T22:09:51.000Z

Ow I see, I was looking at another page.
But still I get the same error with the following compose file:

version: '3.8'
services:
  google-assistant-broadcast:
    container_name: google-assistant-broadcast
    image: ismarslomic/google-assistant-broadcast:pr-6
    restart: unless-stopped
    ports:
      - "8085:8085"
    volumes:
      - ./config:/usr/src/config
    environment:
      - LANGUAGE='en-GB'

Answer 9 · 2022-05-28T22:10:52.000Z

So you should set following languages for your Assistant (in Google Assistant app):

Nederlands (Belgie)
English (United Kingdom)

Answer 10 · 2022-05-28T22:12:56.000Z

Yes this is indeed the case:
The title says: Languages for the Assistant.

Answer 11 · 2022-05-28T22:16:44.000Z

en-GB is already the default language in container so you can omit sending the LANGUAGE env var.

And btw, you must define env variable value without ' ', like this:

LANGUAGE=en-GB

Answer 12 · 2022-05-28T22:21:24.000Z

Yes my bad. Now I don't get the error but still nothing gets played on the speakers.
I'll try to play with my language settings some more tomorrow and see if that helps.
Thanks for all the help.

Answer 13 · 2022-05-28T22:22:02.000Z

Yes, that was my next suggestion. Try broadcasting an message in Nederlands and see how it goes.

Answer 14 · 2022-05-30T15:07:55.000Z

@AleandroD did you do any further tests on this?

Answer 15 · 2022-05-30T20:50:05.000Z

@ismarslomic Sorry I was a little busy:
What I tested:

Default Container settings:
- Google Assistant on Dutch (BE) & English UK
  - Dutch text: Nothing gets played
  - English text: Nothing gets played
- Google Assistant in English UK
  - Dutch text: Nothing gets played
  - English text: Nothing gets played
- Google Assistant in English UK & Norwegian (Your settings)
  - Dutch text: Nothing gets played
  - English text: Nothing gets played
  - Norwegian text: Nothing gets played
Container with English US Language Env variable
- Google Assistant in English UK
  - Dutch text: Nothing gets played
  - English text: Nothing gets played
- Google Assistant in English US
  - Dutch text: Nothing gets played
  - English text: Nothing gets played
- Google Assistant in English US & Dutch (BE)
  - Dutch text: Nothing gets played
  - English text: Nothing gets played
- Google Assistant in English UK & Norwegian (Your settings)
  - Dutch text: Nothing gets played
  - English text: Nothing gets played
  - Norwegian text: Nothing gets played

In my testing all requests went through, but still the same as before nothing gets played sadly.

If I look in my google activity screen of the account I can see all the past "executed" activities:

Even the details says it was executed by the Google Assistant SDK (Sorry for the Dutch screenshot)

I don't know why it won't play it on the speakers, I think I'll make a new account and test it again from fresh.
Maybe one question, do you know which project language and country you chose while making the Cloud Project?

Answer 16 · 2022-06-01T20:02:52.000Z

Hmm, this is very strange.

Lets try broadcasting an message, in Dutch, from your Google Assistant app.

With language settings (in Google Assistant app):

Nederlands (Belgie)
English (United Kingdom)

Use keyboard in Google Assistant app and write "Broadcast Hello world!". Do you hear this message on your speaker?

Answer 17 · 2022-06-01T22:10:37.000Z

Yes that is what's strange, every broadcast via phone/car (which also uses my phone) works like expected (in both languages).

Answer 18 · 2022-06-07T07:40:49.000Z

@AleandroD Im going to merge PR #6, since it did resolve language issues on my side. I'm not sure what else to try to fix these issues for you.

Are you familiar with JavaScript and Node? If so, I could guide you on how to do some more experiments and debug on your machine, directly with the Google Assistant API. Since this issue is hard for me to reproduce on my machine.

Answer 19 · 2022-06-08T21:31:34.000Z

@ismarslomic I'll have some spare time in two weeks or so. Then I'll give it a look.
Thank you for all your support. I hope that I'll be able to get it working.

Answer 20 · 2022-06-08T21:36:59.000Z

Sounds good!

Actually my Google Assistant (with use of the Docker container and from the mobile app) stopped broadcasting the sound on mine Sonos One speakers now. Not sure why, since Google Assistant can successfully broadcast the message (with use of the Docker container) on my Google Home device. Might be some issues on Sonos side..

Will let you know if I found out what has happened.

I feel that these issues are annoying, since you can't see any error messages in response from Google Assistant, nor in the logs in GCP console.

Answer 21 · 2022-06-08T21:39:51.000Z

Yes that wat I find weird as well.
Google shows them in the activity log, but there is no way to see/validate that it actually has done what you asked it to do.
If I don't get you docker container working, I think I'll play with the Assistant SDK itself in like and IDE and get it working on that.

Answer 22 · 2022-06-08T21:52:39.000Z

Yeah, that's probably good idea. That was the reason why I asked you if you where familiar with JavaScript and Node, since you can run index.js directly on your machine, outside Docker. Just clone this repo, run npm install and npm start.

Answer 23 · 2022-06-08T21:56:38.000Z

Yeah I should be able to do that.
I'll keep you posted.

Answer 24 · 2022-06-09T21:06:28.000Z

@AleandroD I fixed similar issue on my side today. No sound where played on my Sonos speakers suddenly. I resolved the issue by going through these steps:

Remove existing registered device by using Google Actions Console

Registering the device back again, by clicking Register Model button. Make sure you choose TV as Device type

Ignore downloading the credentials and just click Save Traits button on last page, without any changes.
Then go to Api & Services / Credentials page at GCP Console and download the OAuth client file

Then create the tokens.json file by using the google-assistant-oauth Docker container.
Stop and delete existing Docker container google-assistant-broadcast by executing docker-compose down
Recreate new Docker container by running docker-compose up -d (make sure you have copied the client_secret.json and tokens.json to mounted path first).

Answer 25 · 2022-06-09T22:15:01.000Z

Ow that sounds promissing I will try it in the evening.

Answer 26 · 2022-06-11T09:34:19.000Z

I have followed your steps:

An used the client secret of this newly registered device, created a new token.

Updated to the latest version (v1.0.6) of the image, but still nothing was broadcasted.
It is still logged in my google assistant activity log. So it must be a problem that het doesn't know to which device he needs to send it.

I'll try it locally, in an IDE, after my exams (when I have more time).

Answer 27 · 2022-07-24T17:33:36.000Z

Hi, same here, messages are not being announced in the speakers, tried by deleteing the device and modifying the langage, same issue:
$ docker run --name google-assistant-broadcast \ -p 8085:8085 \ -v /softw/app/docker/google-assistant-oauth:/usr/src/config \ -e LANGUAGE=es-MX \ ismarslomic/google-assistant-broadcast:latest [2022-07-24T17:12:49.555Z] [OK] Google Assistant Broadcast started at: http://localhost:8085 [2022-07-24T17:13:35.929Z] Sending message (es-MX): Broadcast Hola!! [2022-07-24T17:13:38.348Z] [OK] Conversation Response: empty [2022-07-24T17:13:38.494Z] [OK] Conversation Completed [2022-07-24T17:22:30.128Z] Sending message (es-MX): Broadcast Hola!! [2022-07-24T17:22:32.342Z] [OK] Conversation Response: empty [2022-07-24T17:22:32.541Z] [OK] Conversation Completed