Exception while attempting to get latest data
Opened this issue · 25 comments
Noticed my integration stopped working on 3/16. HA logs show the following:
Logger: custom_components.uhoo
Source: custom_components/uhoo/__init__.py:85
Integration: uHoo ([documentation](https://github.com/csacca/uhoo-homeassistant), [issues](https://github.com/csacca/uhoo-homeassistant/issues))
First occurred: 9:42:19 AM (382 occurrences)
Last logged: 4:11:48 PM
Error: an exception occurred while attempting to get latest data: Error requesting data from https://auth.uhooinc.com/renewusertoken: 410, message='Gone', url=URL('https://auth.uhooinc.com/renewusertoken')
I tried deleting integration and re-authenticating. Looks like it got one data point but none after.
Running HASS: 2022.3.5 and latest integration version from HACS.
I am thinking uhooair changed their link. I tried that URL manually and it is a blank page.
Did some debugging this weekend, the issue is within pyuhooair package. I wrote a comparable python package that can be used for this home assistant integration here: https://pypi.org/project/pyuhooairq/0.0.1 I am happy to help integrate to get this integration up and running again. If not, I can create a comparable home assistant integration to maintain. Thanks!
@s28400 Sorry for not responding earlier. This is a real issue, mostly due to using a reverse engineered API. I just wrote uHoo again and I'm hoping they'll give me access to their API now that they're supporting more third party integrations that just IFTTT. This integration uses the pyuhoo package, not pyuhooair. I'm a little busy right now with life, but if you submit a pull request to pyuhoo air that fixes the problem I'll take a look and if it works I'll push it and update both pyuhoo and this package. Thank you!
Did some debugging this weekend, the issue is within pyuhooair package. I wrote a comparable python package that can be used for this home assistant integration here: https://pypi.org/project/pyuhooairq/0.0.1 I am happy to help integrate to get this integration up and running again. If not, I can create a comparable home assistant integration to maintain. Thanks!
Any chance you'd be willing to teach others how to use your package to fix our uHoo intergrations?
Hello, I’m interrested to know how to solve this issue also please:-) thank you
@startledhighlandhippo startledhighlandhippo do you have the latest pypi package installed? Version 1.1.0 of pyhuooairq had some bug fixes around this issue and support for python 3.10. I simply re-authenticated by logging in once the token was determined to be expired. I have been trying to make an integration in my free time with little progress but hopefully this is helpful to get this one fixed.
Any update on this, Does it still work via the updated pyuhoo library?
I'd much prefer to access Uhoo through a free method vs their 120 dollar a year "premium" serivce.
or anything I can do to help?
+1
+1
+1... also very interested in the fix.
Can someone fork with the fix in it?
@startledhighlandhippo would you happen to have the logs of the http traffic still? which version of the app did you capture these from? I'm inspecting the code in v5.1 and can't find reference to millis in the refresh url's, I might have to try to find a copy of v5.2 to look into.
However it looks like there could be other token information in the headers, a REFRESH: Token header and RefreshToken: header - perhaps the millis number it's referring to one of these from the previous refresh?
@startledhighlandhippo and @s28400 , if I am reading this right, you both got this to work by swapping pyuhoo to pyuhooairq ? Do you have any instructions on how to swap them? Are you using pyuhooairq inside of uhoo-homeassistant ?
I am thinking uhooair changed their link. I tried that URL manually and it is a blank page.
@s28400 I did some snooping in uHoo's premium web dashboard. I don't think they changed their link, because it is still being hit. I did notice that everytime it is called, it is called twice. First as an OPTIONS request, then as a POST request (with the Token and userDeviceId keys)
@davidcsally I don't know that pyuhooair is needed, this patch in pyuhoo adds a workaround to just login every time rather than refreshing the auth: https://github.com/csacca/pyuhoo/pull/57/files
This is not an ideal solution by any means, but unless we can figure out the added details needed to make refresh work it's possibly the best option.
I did notice that everytime it is called, it is called twice. First as an
OPTIONSrequest, then as aPOSTrequest (with theTokenanduserDeviceIdkeys)
Great idea checking the web interface, I might be able to use that to figure out the refresh! Much easier than inspecting a phone app.
Update: yep I've got some logged traffic from this that follows the login / refresh workflow, hopefully that'll have enough detail to help me fix it properly!
Ok so the logged traffic was very interesting... in that it really did look identical to the flow used here / in pyuhoo! I didn't see any sign of the url arg millis mentioned earlier or anything else particularly different.
I also extended the test script in pyuhoo to run a number of user refresh / get data with a 5 second delay between each one.
Strangely, usually the first free refreshes work perfectly! Sometimes they all work. Other times though it works a few times, then fails in the same way seen here.
It took me quite a few looks back and first, and lots of extra logging being added to the pyuhoo test script, before I really realised the glaring difference; with online logged data the client_id looks like a token, a long (48char) hex string, whereas in pyuhoo it's a fixed string of mostly 0.
I believe this is the issue!
It makes sense that if the login / refresh tokens are generated against the client id, as soon as a second person logs in with the same client id the first users token will be invalidated.
I've raised the PR csacca/pyuhoo#75 which generates a client id in a relatively unique fashion, though I don't know how it compares to the official app. It's a long enough string the chance of user class should be essentially zero now.
@andrewleech That's really interesting, good find. I'm a front end guy, but I forked this and pyuhoo to make my own package with your changes, unfortunately I'm still seeing the same error. Were you able to get it to work consistently? I can try digging into the web api a little more as well. It's crazy, this code and the web api look like they are doing exactly the same thing.
@davidcsally I've now got my development fork of this repo set up to depend on my pyuhoo PR branch so it pulls it in directly from GitHub automatically.
I did a clean install of that a few hours ago and it seems to be still working properly; the sensors have full history during this time and I can't see any errors in the HA logs.
If you want to test it, you can remove this repo from hacs and add mine instead then re-install uhoo integration. https://github.com/andrewleech/uhoo-homeassistant
No guarantees I'll keep this stable long term, it's still where I do development, but does let the PR get easily tested for now!
@davidcsally I've now got my development fork of this repo set up to depend on my pyuhoo PR branch so it pulls it in directly from GitHub automatically.
I did a clean install of that a few hours ago and it seems to be still working properly; the sensors have full history during this time and I can't see any errors in the HA logs.
If you want to test it, you can remove this repo from hacs and add mine instead then re-install uhoo integration. https://github.com/andrewleech/uhoo-homeassistant No guarantees I'll keep this stable long term, it's still where I do development, but does let the PR get easily tested for now!
@andrewleech thank you for your efforts on this. I've installed your integration and am getting this following error:
This error originated from a custom integration.
Logger: custom_components.uhoo
Source: custom_components/uhoo/init.py:55
Integration: uHoo (documentation, issues)
First occurred: 2:02:01 PM (10 occurrences)
Last logged: 2:10:15 PM
Error: an exception occurred while attempting to get latest data: 'temp'
Note that I've never had the uHoo integration working. I attempted to get it setup in HA after it was no longer working so perhaps there is another issue.
@dvhub I think there should be more details of the error / exception further down on either that log entry, or perhaps a second log entry? That snippet doesn't tell me enough unfortunately.
@dvhub I think there should be more details of the error / exception further down on either that log entry, or perhaps a second log entry? That snippet doesn't tell me enough unfortunately.
I'll dig in deeper. I believe I need to enable debug logs in HACS.
@dvhub I think there should be more details of the error / exception further down on either that log entry, or perhaps a second log entry? That snippet doesn't tell me enough unfortunately.
This is all that I see in the home-assistant.log:
2022-12-01 15:35:27.531 ERROR (MainThread) [custom_components.uhoo] Error: an exception occurred while attempting to get latest data: 'temp'
2022-12-01 15:35:27.534 ERROR (MainThread) [custom_components.uhoo] Error fetching uhoo data:
The two errors from Settings > System > Logs are:
This error originated from a custom integration.
Logger: custom_components.uhoo
Source: custom_components/uhoo/init.py:55
Integration: uHoo (documentation, issues)
First occurred: 3:35:27 PM (314 occurrences)
Last logged: 10:41:22 PM
Error: an exception occurred while attempting to get latest data: 'temp'
This error originated from a custom integration.
Logger: custom_components.uhoo
Source: helpers/update_coordinator.py:182
Integration: uHoo (documentation, issues)
First occurred: 3:35:27 PM (314 occurrences)
Last logged: 10:41:22 PM
Error fetching uhoo data:
@dvhub I think there should be more details of the error / exception further down on either that log entry, or perhaps a second log entry? That snippet doesn't tell me enough unfortunately.
This is all that I see in the home-assistant.log:
2022-12-01 15:35:27.531 ERROR (MainThread) [custom_components.uhoo] Error: an exception occurred while attempting to get latest data: 'temp' 2022-12-01 15:35:27.534 ERROR (MainThread) [custom_components.uhoo] Error fetching uhoo data:
The two errors from Settings > System > Logs are:
This error originated from a custom integration.
Logger: custom_components.uhoo Source: custom_components/uhoo/init.py:55 Integration: uHoo (documentation, issues) First occurred: 3:35:27 PM (314 occurrences) Last logged: 10:41:22 PM
Error: an exception occurred while attempting to get latest data: 'temp'
This error originated from a custom integration.
Logger: custom_components.uhoo Source: helpers/update_coordinator.py:182 Integration: uHoo (documentation, issues) First occurred: 3:35:27 PM (314 occurrences) Last logged: 10:41:22 PM
Error fetching uhoo data:
I was unable to determine the root cause, but I did come up with a work around to my issue. I transferred the device to a new uHoo account and it worked without any issues. There must be something wrong with my original account. Thanks @andrewleech .
If you want to test it, you can remove this repo from hacs and add mine instead then re-install uhoo integration. https://github.com/andrewleech/uhoo-homeassistant No guarantees I'll keep this stable long term, it's still where I do development, but does let the PR get easily tested for now!
thank you, i just got my first uHoo and this worked perfectly (at least for initial setup i just did :-) )