Inaccurate Response - Prompt Template not being sent?

Question

Inaccurate Response - Prompt Template not being sent?

rwelsh09 opened this issue 15 days ago · 4 comments

The problem

Hello,

The response I got is completely inaccurate. I think the prompt template from the Options is not being sent to the AI.
Or am I missing something?

Here are the options:

Here’s the prompt and response:

More info in the forums

Let me know if you need any more information.

Thanks

What version of Home Assistant Core has the issue?

core-2024.5.3

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

Google Generative AI Conversation

Link to integration documentation on our website

https://www.home-assistant.io/integrations/google_generative_ai_conversation/

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

Answer 1 · 2024-05-17T22:38:52.000Z

Hey there @tronikos, mind taking a look at this issue as it has been labeled with an integration (google_generative_ai_conversation) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of google_generative_ai_conversation can trigger bot actions by commenting:

@home-assistant close Closes the issue.
@home-assistant rename Awesome new title Renames the issue.
@home-assistant reopen Reopen the issue.
@home-assistant unassign google_generative_ai_conversation Removes the current integration label and assignees on the issue, add the integration domain after the command.
@home-assistant add-label needs-more-information Add a label (needs-more-information, problem in dependency, problem in custom component) to the issue.
@home-assistant remove-label needs-more-information Remove a label (needs-more-information, problem in dependency, problem in custom component) on the issue.

_{^{(message by CodeOwnersMention)}}

google_generative_ai_conversation documentation
google_generative_ai_conversation source
_{^{(message by IssueLinks)}}

Answer 2 · 2024-05-18T02:03:30.000Z

The google_generative_ai_conversation.generate_content isn't tied to a config entry so not to any prompt. I need to clarify this in the docs. You should use conversation.process if you want to use the prompt in the configure page.

Answer 3 · 2024-05-18T02:25:51.000Z

Oh! Thanks for the clarification.

I just tried this:

And got this error:

Logger: homeassistant.helpers.script.websocket_api_script
Source: helpers/script.py:501
First occurred: 8:19:25 PM (1 occurrences)
Last logged: 8:19:25 PM

websocket_api script: Error executing script. Unexpected error for call_service at pos 1: list index out of range
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 501, in _async_step
    await getattr(self, handler)()
  File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 736, in _async_call_service_step
    response_data = await self._async_run_long_action(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 699, in _async_run_long_action
    return await long_task
           ^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/core.py", line 2738, in async_call
    response_data = await coro
                    ^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/core.py", line 2779, in _execute_service
    return await target(service_call)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/conversation/__init__.py", line 206, in handle_process
    result = await async_converse(
             ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/conversation/agent_manager.py", line 87, in async_converse
    return await method(
           ^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/google_generative_ai_conversation/__init__.py", line 226, in async_process
    intent_response.async_set_speech(chat_response.text)
                                     ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/google/generativeai/types/generation_types.py", line 313, in text
    if len(parts) > 1 or "text" not in parts[0]:
                                       ~~~~~^^^
  File "/usr/local/lib/python3.12/site-packages/proto/marshal/collections/repeated.py", line 125, in __getitem__
    return self._marshal.to_python(self._pb_type, self.pb[key])
                                                  ~~~~~~~^^^^^
IndexError: list index out of range

Answer 4 · 2024-05-18T06:03:14.000Z

There was a fix in the latest patch release for this exception. For your prompt I get "Sorry, I had a problem talking to Google Generative AI. Likely blocked". Not sure why Google blocks this prompt. I want to add a setting in the options flow to disable all safety checks.