Cannot load Zyxel data:

Question

Cannot load Zyxel data:

quizzical86 opened this issue 2 years ago · 12 comments

Dear Lukas,
I only saw your messages about updating the code to work with firmware v.2.70 today. I was very excited (especially with the high electricity prices nowadays your code could make a big difference given how many PoE devices I have that don't need to run all day).
I of course tested it immediately.
The problem I have is that I always get the error 'Cannot load Zyxel data:'.
I see in your code it should in theory give some more explanations about the error but in my case it doesn't.
I've tried many different possible configurations (e.g 192.168.0.3 with an without "quotes" around it). But always get this error. Even when I put a wrong IP address or wrong password I still get that same error.
Any tips on what I could try to do to get it working?
Thanks again!

Answer 1 · 2022-10-19T18:55:38.000Z

I encounter exactly the same issue. Alwas the error Cannot load Zyxel data: Cannot connect to host switch1.:80
Try to connect to a 1900-24EP running on v2.70

Answer 2 · 2022-10-20T20:12:11.000Z

@quizzical86 do you see additional details in Home-Assistant under Settings - System - Logs? Can you click on that error there, and if you do, do you get a Traceback?

@fired757 could you copy the snippet you used for the configuration? (if it contains passwords, please redact them)

Answer 3 · 2022-10-20T21:27:14.000Z

Thanks Lukas!

What I mentioned before is coming from the logs.
To be more precise, when I restart Home Assistant this is what the logs say exactly:

2022-10-20 23:22:21.558 WARNING (MainThread) [homeassistant.components.switch] Setup of switch platform zyxel_poe is taking over 10 seconds.
2022-10-20 23:22:22.066 ERROR (MainThread) [custom_components.zyxel_poe.switch] Cannot load Zyxel data:

And here's what I have in configuration.yaml:

switch:
  - platform: zyxel_poe
    devices:
    - host: 192.168.0.3
      username: REDACTED
      password: REDACTED

Answer 4 · 2022-10-21T21:56:53.000Z

I had just copied your example and interpreted that every port needs a seperate entry. Now as i read quizzical86 config, I tried without the leeding "switch1." and it works absolutely fine. Also swithing on and off PoE Power for each port works for the 1900-24EP


switch:
  - platform: zyxel_poe
    devices:
      - host: switch1.10.0.0.5
        username: REDACTED
        password: REDACTED

Answer 5 · 2022-10-22T07:56:39.000Z

@fired757 glad my example code helped you.

@lukas-hetzenecker I think I now understand what a traceback is. The logs do not provide it :(
Perhaps this more complete extract of the logs (from the start until the end) can give any clues?

2022-10-20 23:38:54.335 INFO (MainThread) [homeassistant.components.switch] Setting up switch.zyxel_poe
2022-10-20 23:38:54.337 INFO (MainThread) [homeassistant.components.switch] Setting up switch.template
2022-10-20 23:38:54.343 INFO (MainThread) [homeassistant.components.switch] Setting up switch.template
2022-10-20 23:38:54.344 INFO (MainThread) [homeassistant.components.switch] Setting up switch.template
2022-10-20 23:39:11.292 INFO (MainThread) [homeassistant.setup] Setting up alarmo
2022-10-20 23:39:11.293 INFO (MainThread) [homeassistant.setup] Setup of domain alarmo took 0.0 seconds
2022-10-20 23:39:11.326 INFO (SyncWorker_4) [homeassistant.loader] Loaded weather from homeassistant.components.weather
2022-10-20 23:39:11.364 WARNING (MainThread) [homeassistant.components.sensor] Setup of sensor platform hacs is taking over 10 seconds.
2022-10-20 23:39:11.366 WARNING (MainThread) [homeassistant.components.switch] Setup of switch platform zyxel_poe is taking over 10 seconds.
2022-10-20 23:39:11.367 WARNING (MainThread) [homeassistant.components.switch] Setup of switch platform template is taking over 10 seconds.
2022-10-20 23:39:11.369 WARNING (MainThread) [homeassistant.components.switch] Setup of switch platform template is taking over 10 seconds.
2022-10-20 23:39:11.370 WARNING (MainThread) [homeassistant.components.switch] Setup of switch platform template is taking over 10 seconds.
2022-10-20 23:39:11.405 WARNING (MainThread) [homeassistant.config_entries] Config entry 'Z-Wave JS' for zwave_js integration not ready yet: Failed to connect: ; Retrying in background
2022-10-20 23:39:11.409 WARNING (MainThread) [homeassistant.config_entries] Config entry redacted@gmail.com' for life360 integration not ready yet; Retrying in background
2022-10-20 23:39:11.428 ERROR (MainThread) [custom_components.zyxel_poe.switch] Cannot load Zyxel data:

As mentioned it doesn't really matter what I put in my config, I always get that error, whetter I put the real IP of my switch or jiberish like 999999999999.
My switch is reachable from Home Assistant, using the terminal I can ping it just fine.

Answer 6 · 2022-10-23T14:21:29.000Z

@fired757 happy to hear you got it working!

@quizzical86 to further troubleshoot your issue, I created PR #4

Could you use the component from that branch, and add the following section to your configuration.yaml file:

logger:
  default: info
  logs:
    custom_components.zyxel_poe: debug

This should then give your a more verbose logging output, which should help to narrow down the issue.

Answer 7 · 2022-10-23T19:23:02.000Z

@lukas-hetzenecker the logs provide a lot more information with all the changes you've made.

I've restarted Home Assistant multiple times and it consistently gives the below result where it fails to login the first time but succeeds the second time.
Could it be that the code is looking for an entry with 13 elements but not finding any?
As you might remember from the previous issue I have the same model as yours but the 8-ports version (and @fired757 seems to be using a somewhat different model as yours is called 10HP, mine 8HP and his is called 24EP). In my logs I see that there are 8 entries with 12-elements. Perhaps the 8HP somehow has 1 element less than the 10HP and 24EP?
Is my understanding then that the next step is to somehow figure out what this should look like for the 8-port model?
_, _, port, state, pd_class, pd_priority, power_up, wide_range_detection, consuming_power_mw, max_power_mw, time_range_name, time_range_status, _ = map(lambda a: a.text.strip(), cols)

Here are the logs:

2022-10-23 20:51:31.472 INFO (MainThread) [homeassistant.components.switch] Setting up switch.zyxel_poe
2022-10-23 20:51:36.804 DEBUG (MainThread) [custom_components.zyxel_poe.switch] No credential info stored, performing a login
2022-10-23 20:51:36.805 DEBUG (MainThread) [custom_components.zyxel_poe.switch] Logging in to: http://192.168.0.3/cgi-bin/dispatcher.cgi
2022-10-23 20:51:53.599 WARNING (MainThread) [homeassistant.components.switch] Setup of switch platform zyxel_poe is taking over 10 seconds.
2022-10-23 20:51:54.142 WARNING (MainThread) [homeassistant.components.switch] Platform zyxel_poe not ready yet: Connection error while connecting to http://192.168.0.3/cgi-bin/dispatcher.cgi: ; Retrying in background in 30 seconds
022-10-23 20:52:12.855 INFO (MainThread) [homeassistant.components.switch] Setting up switch.zyxel_poe
2022-10-23 20:52:12.936 INFO (MainThread) [custom_components.hacs] Loading removed repositories
2022-10-23 20:52:12.985 DEBUG (MainThread) [custom_components.zyxel_poe.switch] No credential info stored, performing a login
2022-10-23 20:52:12.985 DEBUG (MainThread) [custom_components.zyxel_poe.switch] Logging in to: http://192.168.0.3/cgi-bin/dispatcher.cgi
2022-10-23 20:52:18.038 DEBUG (MainThread) [custom_components.zyxel_poe.switch] Step 1 returned status code: 200, text:
by8oGWkk7Vr6M9dhZYc0I4R5pfUpf2dG
2022-10-23 20:52:18.800 INFO (MainThread) [custom_components.hacs] Loading known repositories
2022-10-23 20:52:20.074 DEBUG (MainThread) [custom_components.zyxel_poe.switch] Step 2 returned status code: 200, text: OK,
2022-10-23 20:52:20.077 DEBUG (MainThread) [custom_components.zyxel_poe.switch] Logged in successfully
2022-10-23 20:52:22.452 DEBUG (MainThread) [custom_components.zyxel_poe.switch] CMD 773 returned status code: 200
2022-10-23 20:52:22.777 DEBUG (MainThread) [custom_components.zyxel_poe.switch] Found table with 14 elements
2022-10-23 20:52:22.777 DEBUG (MainThread) [custom_components.zyxel_poe.switch] Found table with 1 elements
2022-10-23 20:52:22.777 DEBUG (MainThread) [custom_components.zyxel_poe.switch] Found table with 2 elements
2022-10-23 20:52:22.778 DEBUG (MainThread) [custom_components.zyxel_poe.switch] Found table with 0 elements
2022-10-23 20:52:22.778 DEBUG (MainThread) [custom_components.zyxel_poe.switch] Found table with 12 elements
2022-10-23 20:52:22.778 DEBUG (MainThread) [custom_components.zyxel_poe.switch] Found table with 1 elements
2022-10-23 20:52:22.778 DEBUG (MainThread) [custom_components.zyxel_poe.switch] Found table with 12 elements
2022-10-23 20:52:22.778 DEBUG (MainThread) [custom_components.zyxel_poe.switch] Found table with 1 elements
2022-10-23 20:52:22.779 DEBUG (MainThread) [custom_components.zyxel_poe.switch] Found table with 12 elements
2022-10-23 20:52:22.779 DEBUG (MainThread) [custom_components.zyxel_poe.switch] Found table with 1 elements
2022-10-23 20:52:22.779 DEBUG (MainThread) [custom_components.zyxel_poe.switch] Found table with 12 elements
2022-10-23 20:52:22.779 DEBUG (MainThread) [custom_components.zyxel_poe.switch] Found table with 1 elements
2022-10-23 20:52:22.780 DEBUG (MainThread) [custom_components.zyxel_poe.switch] Found table with 12 elements
2022-10-23 20:52:22.780 DEBUG (MainThread) [custom_components.zyxel_poe.switch] Found table with 1 elements
2022-10-23 20:52:22.793 DEBUG (MainThread) [custom_components.zyxel_poe.switch] Found table with 12 elements
2022-10-23 20:52:22.793 DEBUG (MainThread) [custom_components.zyxel_poe.switch] Found table with 1 elements
2022-10-23 20:52:22.794 DEBUG (MainThread) [custom_components.zyxel_poe.switch] Found table with 12 elements
2022-10-23 20:52:22.794 DEBUG (MainThread) [custom_components.zyxel_poe.switch] Found table with 1 elements
2022-10-23 20:52:22.794 DEBUG (MainThread) [custom_components.zyxel_poe.switch] Found table with 12 elements
2022-10-23 20:52:22.794 DEBUG (MainThread) [custom_components.zyxel_poe.switch] Found table with 1 elements
2022-10-23 20:52:22.794 DEBUG (MainThread) [custom_components.zyxel_poe.switch] Found table with 1 elements
2022-10-23 20:52:23.275 WARNING (MainThread) [homeassistant.components.switch] Setup of switch platform zyxel_poe is taking over 10 seconds.

To try and test a bit further I tried to set this up on my other zyxel switch which is a 24E (so doesn't have POE) but that didn't lead to any interesting insights. It also fails the first attempt and then succeeds but only finds 1 single entry with 1 element:

2022-10-23 21:05:33.179 DEBUG (MainThread) [custom_components.zyxel_poe.switch] Found table with 1 elements

Answer 8 · 2022-10-23T21:21:59.000Z

That looks good, I feel we're getting close to the solution here

As you might remember from the previous issue I have the same model as yours but the 8-ports version (and @fired757 seems to be using a somewhat different model as yours is called 10HP, mine 8HP and his is called 24EP). In my logs I see that there are 8 entries with 12-elements. Perhaps the 8HP somehow has 1 element less than the 10HP and 24EP?

That is right, somehow your 8HP seems to omit some kind of information.

Can you add the following statement

_LOGGER.debug(list(map(lambda a: a.text.strip(), cols)))
after printing "Found table", like here?

I expected an output like this:

2022-10-23 23:15:08.925 DEBUG (MainThread) [custom_components.zyxel_poe.switch] Found table with 15 elements
2022-10-23 23:15:08.925 DEBUG (MainThread) [custom_components.zyxel_poe.switch] ['', '', 'Port', 'State', 'Class', 'PD Priority', 'Power-Up', 'Wide Range Detection', 'Consuming Power (mW)', 'Max Power (mW)', 'Time Range\n\nName\nStatus', 'Time Range', 'Name', 'Status', '']
2022-10-23 23:15:08.925 DEBUG (MainThread) [custom_components.zyxel_poe.switch] Found table with 1 elements
2022-10-23 23:15:08.925 DEBUG (MainThread) [custom_components.zyxel_poe.switch] ['Time Range']
2022-10-23 23:15:08.925 DEBUG (MainThread) [custom_components.zyxel_poe.switch] Found table with 2 elements
2022-10-23 23:15:08.925 DEBUG (MainThread) [custom_components.zyxel_poe.switch] ['Name', 'Status']
[...]
2022-10-23 23:16:09.462 DEBUG (MainThread) [custom_components.zyxel_poe.switch] ['', '', '4', 'Enable', 'class4', 'Low', '802.3at', 'Disable', '3800', '31200', '', '-', '']

But I suspect in your output one of those fields will be missing? Let's figure out what Zyxel made different in the 8HP model :)

on my other zyxel switch which is a 24E (so doesn't have POE)

This queries specifically the web page for the PoE settings, so I would assume this does not work at all for models that don't support PoE

Answer 9 · 2022-10-24T11:01:25.000Z

Thanks @lukas-hetzenecker
With your added piece of code, it shows exactly what's different with the 8HP.
Your 10HP shows the "Wide Range Detection" which for your switch has the value 'Disable' on port 4. My 8HP doesn't have that:

2022-10-24 12:04:56.490 DEBUG (MainThread) [custom_components.zyxel_poe.switch] Found table with 14 elements
2022-10-24 12:04:56.491 DEBUG (MainThread) [custom_components.zyxel_poe.switch] ['', '', 'Port', 'State', 'Class', 'PD Priority', 'Power-Up', 'Consuming Power (mW)', 'Max Power (mW)', 'Time Range\n\nName\nStatus', 'Time Range', 'Name', 'Status', '']
2022-10-24 12:04:56.495 DEBUG (MainThread) [custom_components.zyxel_poe.switch] Found table with 12 elements
2022-10-24 12:04:56.495 DEBUG (MainThread) [custom_components.zyxel_poe.switch] ['', '', '3', 'Enable', 'class2', 'Low', '802.3at', '3200', '7400', '', '-', '']

Now even a dummy like me was able to figure out what to change and indeed now I'm getting the ports in home assistant.
When I try to turn a port on/off it does throw an error "failed to call service switch/turn_on" but despite this error I have the impression that it's working . Does that happen to you as well?


[140209244646864] Connection error while connecting to http://192.168.0.3/cgi-bin/dispatcher.cgi: 400, message='Invalid header token', url=URL('http://192.168.0.3/cgi-bin/dispatcher.cgi')
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/aiohttp/client_reqrep.py", line 898, in start
    message, payload = await protocol.read()  # type: ignore[union-attr]
  File "/usr/local/lib/python3.10/site-packages/aiohttp/streams.py", line 616, in read
    await self._waiter
  File "/usr/local/lib/python3.10/site-packages/aiohttp/client_proto.py", line 213, in data_received
    messages, upgraded, tail = self._parser.feed_data(data)
  File "aiohttp/_http_parser.pyx", line 551, in aiohttp._http_parser.HttpParser.feed_data
aiohttp.http_exceptions.BadHttpMessage: 400, message='Invalid header token'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/config/custom_components/zyxel_poe/switch.py", line 202, in change_state
    res = await self._session.post(self._url, data=command_data)
  File "/usr/local/lib/python3.10/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/usr/local/lib/python3.10/site-packages/aiohttp/client_reqrep.py", line 900, in start
    raise ClientResponseError(
aiohttp.client_exceptions.ClientResponseError: 400, message='Invalid header token', url=URL('http://192.168.0.3/cgi-bin/dispatcher.cgi')

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/websocket_api/commands.py", line 202, in handle_call_service
    await hass.services.async_call(
  File "/usr/src/homeassistant/homeassistant/core.py", line 1738, in async_call
    task.result()
  File "/usr/src/homeassistant/homeassistant/core.py", line 1775, in _execute_service
    await cast(Callable[[ServiceCall], Awaitable[None]], handler.job.target)(
  File "/usr/src/homeassistant/homeassistant/helpers/entity_component.py", line 207, in handle_service
    await service.entity_service_call(
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 678, in entity_service_call
    future.result()  # pop exception if have
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 931, in async_request_call
    await coro
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 715, in _handle_entity_call
    await result
  File "/config/custom_components/zyxel_poe/switch.py", line 104, in async_turn_off
    return await self._poe_data.change_state(self._port, 0)
  File "/config/custom_components/zyxel_poe/switch.py", line 212, in change_state
    raise PlatformNotReady(f"Connection error while connecting to {self._url}: {ex}") from ex
homeassistant.exceptions.PlatformNotReady: Connection error while connecting to http://192.168.0.3/cgi-bin/dispatcher.cgi: 400, message='Invalid header token', url=URL('http://192.168.0.3/cgi-bin/dispatcher.cgi')

Answer 10 · 2022-10-25T16:29:18.000Z

@quizzical86 Thanks for the repo! I also included a workaround for the missing "Wide Range Detection" field into the repo, so your model should now be supported as well.

When I try to turn a port on/off it does throw an error "failed to call service switch/turn_on" but despite this error I have the impression that it's working

This should not be happening, and also does not happen for me.

Also your other observation

consistently gives the below result where it fails to login the first time but succeeds the second time

is not normal behaviour, and does not occur on my device.

I would be really interested in whether https://github.com/jonbulica99/zyxel-poe-manager works for you out-of-the-box, without any errors thrown? Is there any chance you could try that out?

Answer 11 · 2022-10-26T07:12:21.000Z

@lukas-hetzenecker I was thinking that to make my model compatible you'd have to add functionality where in configuration.yaml the user specifies what model he is using, but your solution is much simpler :) Thanks!

I looked a little bit into https://github.com/jonbulica99/zyxel-poe-manager back in April when I first contacted you, but I didn't understand how I could install that in Home Assistant (I guess something like this https://www.home-assistant.io/integrations/python_script/ but then it still requires to somehow BeautifulSoup4 & other things) and even less how I could use that code to see the state of the ports in Home Assistant and change them. So I have the impression it's over my skill level? (which I considered as confirmed when I saw your code which does exactly what I need but is very different).

After a few days of testing I can by the way confirm that despite those strange issues, My camera's now turn off when I'm home (saving energy and being 100% sure that no hacker is watching me) and my outdoor wifi access point is only on when I need it (sorry for the insects that use it as their personal heater)