Control of Settop Box no longer possible since ca 10 January 2024
Opened this issue · 77 comments
In Switzerland and the UK the settop box can no longer be controlled as of ca 10 Jan 2024
It looks like the MQTT session is no longer connecting
UPDATE 25 Jan 2024: Solved for many countries, please use the latest v2.3.0 (or greater).
Still working on the issue for GB
Working on it...
Issue may also impact other countries, I need user feedback to find out
Might be related - but I’m in the UK and also unable to connect and control set top box but logs indicate 401 error:
Failed to create session - Step 3 of 7:
Unable to login: 401 Unauthorized
I have checked credentials, they work. Account has no biometrics attached. Worked Dec last year (2023).
Hi @jsiegenthaler, Can you keep me up-to-date about your findings?
@Sholofly fixed in v2.2.15-beta.2, MQTT endpoint changed for CH, not sure about other countries yet
Fixed in v2.2.15, released 14.01.2024
@Sholofly I'll send you an email.
@jsiegenthaler Same issue in GB. v2.2.15 has not fixed this here. Let me know if you need any more details.
Hi @absreda and @Flatsphere
I am working on a new version v2.3.0 and will release a beta today for testing. I would love to have some GB and IE users provide some test feedback. v2.3.0 contains adaptations that should fix the changed endpoints for all countries, but I need confirmation of the authentication sequence
Hi @absreda and @Flatsphere
I just published 2.3.0-beta.1 with fixes for the endpoints. Please test. If the authentication fails, please send me a full debug log with eosstb plugin debugging enabled at level 9 (ensure
"debugLevel": 9 exists in the eosstb config).
Send the logs to me by email. My email is my github username with a b between j and s, at that well known email service starting with g
@jsiegenthaler - no problem. I’ll be able to test in a couple of hours and will let you know. Many thanks.
@jsiegenthaler - sorry for the delay. The latest beta has solved the connection issue, but not if I use Method C (GB), only Method A works.
Unfortunately I then get the following error:
[1/15/2024, 9:03:40 PM] [Virgin Media] refreshMasterChannelList: Master channel list refreshed with 421 channels, val id until 1/15/2024, 9:33:40 PM [1/15/2024, 9:03:40 PM] [Virgin Media] Discovering devices... [1/15/2024, 9:03:40 PM] [Virgin Media] Failed to discover devices - No devices found. The backend systems may be down , or you have no supported devices on y our customer account [1/15/2024, 9:03:40 PM] [Virgin Media] sessionWatchdog (1): Exiting sessionWatchdog
@Flatsphere that's great news! Can you send me the log showing the connection occurring? I'd like to see it with full debugging turned on, as @absreda cannot connect.
Send the logs to me by email. My email is my github username with a b between j and s, at that well known email service starting with g
@jsiegenthaler - will do. I might not be able to email the log tonight, but will send them asap,
@jsiegenthaler Where do i find the changed code? Don't see it in your repo...
@Sholofly Sorry, I didn't publish the branch. Here it is:
https://github.com/jsiegenthaler/homebridge-eosstb/tree/Add-auto-endpoint-detection
@jsiegenthaler - I have emailed you the logs. Let me know if there is anything else I can do to help.
Thanks @jsiegenthaler !
To all watching and reading this: the change to the GB logon sequence (and maybe also other countries...) is a change to using OAuth 2.0 with PKCE (Pixie) using a code challenge method of S256. This is an authentication method that I am not familiar with. I thus at the bottom of a very steep learning curve and really need the help of volunteers. If you have experience with use of OAuth and OAuth 2.0 - in any programming language - and are able to help, please get in touch via email. Read the comments above for my email address.
Hi All. 2.3.0-beta.3 has been published. Please try new Auth Method D with your valid credentials. Turn off the watchdog to prevent the watchdog from trying to restart the session and thus flooding the logs. Send me the logs by email - for logon fail and success please, remember to remove your real password.
Hi! For PL it also stopped working. The device is discovered correctly, but sending power key doesn’t turn on set-top box.
@wojtul2323 hi. I guess that means it did work in PL. I'll put PL in the confirmed list :)
Have you tried the latest beta version? If it doesn't work, please send me logs with max debugging enabled
Beta version works lile a charm :)
@wojtul2323 Good to hear. I'll prepare a release version shortly for all users
I am following this thread as I am struggling to get it working. I have tried all of the above. I just keep getting the following…
Failed to create session - Step 4 of 7: Unable to oauth authorize: 404 Not Found
Any advice would be appreciated.
Hi @Thornton69 Can you tell me which country you are in?
@jsiegenthaler I am in the uk.
@Thornton69 the GB authentication method changed in the last couple of weeks and I am trying to make the plugin work with the new authentication method. I appreciate all help I can get. Do you have any experience with authentication methods?
@jsiegenthaler i don't I'm afraid. I am a complete novice. I was reading your feed thinking you may have fixed the issue and I was the only one with the problem.
I hope you get to the bottom of it. Your service is one of the best on here. It has worked impeccably for me.
@Thornton69 I'll do my best. Not sure if my best will be enough. It's difficult. We'll see.
Hello
I am getting this same error. I am also a novice unfortunately.
UK - GB: Virgin Media TV 360
[EOSSTB] Step 4 of 7: Failed to create session - Unable to oauth authorize: 404 Not Found
Found the issue - (For GB)
- Use new servers - messagebroker-prod-gb.gnp.cloud.dmdsdp.com
- In the WebSocket connection request, add the following header:
Sec-WebSocket-Protocol: mqtt, mqttv3.1, mqttv3.11
Thanks. I'll look at it in a few days. But the core issue is not the MQTT session, but rather the authentication method. So I'm now confused. What authentication method did you use?
Found the issue - (For GB)
- Use new servers - messagebroker-prod-gb.gnp.cloud.dmdsdp.com
- In the WebSocket connection request, add the following header:
Sec-WebSocket-Protocol: mqtt, mqttv3.1, mqttv3.11
In my home assistant plugin this comment helped me much. Not for GB but for other countries where they couldn't connect.
Thank you @OrenPo
Just a quick 'me too'. Ironically I've just started using my Virgin Media box again in the UK and had enabled the plugin again, as a result. Let me know if I can help with debugging.
Hi all. I've been unable to work on this due to illness over the last week. I hope to get back into it towards the middle or end of this week, so by Friday 26 January 2023.
The Sec-WebSocket-Protocol headers that @OrenPo mentioned seemed to fix things for NL too! Same server still.
Cool thanks. I will be able to spend some time on this at the end of the week. I'll post results here.
I have a prototype for the GB version, with Challenge-Verifier impl, will post this in the next day or too, after some testing
Wow that's fantastic, I really appreciate the help. You can also send your changes me email as well if you wish.
@SightAndSound-Roger: how did you set the mqtt wss headers? I've looked and seen no obvious way. Can you share a code snippet?
@jsiegenthaler I've made a C# implementation of our own which I added it in so I can't say for sure how it'd fit in the node code. My best guess would be like this:
const createWebsocket = createWebsocket((url, websocketSubProtocols, options) => {
const subProtocols = [
'mqtt',
'mqttv3.1',
'mqttv3.11'
];
return new WebSocket(url, subProtocols)
});
mqttClient = mqtt.connect(mqttUrl, {
createWebsocket,
connectTimeout: 10 * 1000, // 10s
clientId: mqttClientId,
username: mqttUsername,
password: mqttPassword
});In C# I'm using MQTTNet and had to add this to my MqttClientOptionsBuilder.
.WithWebSocketServer(_baseMqttUrl, new MqttClientOptionsBuilderWebSocketParameters()
{
RequestHeaders = new Dictionary<string, string> { ["Sec-WebSocket-Protocol"] = "mqtt, mqttv3.1, mqttv3.11" }
})```@SightAndSound-Roger thanks. Your code snippet is similar to what can be found here: https://github.com/mqttjs/MQTT.js/tree/main#about-reconnection
But in your snippet the const subProtocols is never being used :)
I have tried to add it in but my javascript is still very weak and js complains of a syntax error:
const customWebsocket = createWebsocket(url, websocketSubProtocols, options) => {
const subProtocols = [
'mqtt',
'mqttv3.1',
'mqttv3.11'
]
return new WebSocket(url, subProtocols);
};
The => is highlighted complaining of "';' expected.ts(1005)"
I'm using CommonJS, not es6. I don't understand - anyone with node commonjs experience that can help?
@jsiegenthaler I have updated my comment, the code was a bit off indeed.
@SightAndSound-Roger : thanks! You solved my syntax issue. And I fixed the use of the const for you :)
Hi ... The Challenge-verifier is easy,
But I can't find the replacement URL for 'https://id.virginmedia.com/rest/v40/session/start?protocol=oidc&rememberMe=true'
(gives me 401)
:(
@OrenPo : I get 200 for the same command. Are you on discord? Let's chat there...
Since yesterday, I have had the same problem here in Switzerland as before the update.The box again has no connection to HomeKit. commands are no longer transmitted.
thank you for your help.
@mhary269 Please try v2.3.0-beta.7. This has fixed it for me.
I'm in the UK. I have tried all methods in the current Beta and I have personally not been able to authenticate. The work you guys are doing is to get it sorted is appreciated. This plugin is one of the best.
Hi @Thornton69 : Sorry, I'm still working on this. Delayed due to illness this week I'm better now. Keep watching this thread and I'll post all progress here reports.
Progress report:
GB: none yet (delayed due to illness).
CH (and all other countries): yesterday I fixed the issue with the mqtt client connection that stopped working 2 days ago. Fixed in v2.3.0-beta.7.
@jsiegenthaler I'm also in the UK and really appreciate all your hard work with creating this plugin, and agree with @Thornton69 it really is one of the best. If you require any logs or info to help with the UK problem please let me know.
Hi All, v2.3.0 published to solve logon issues for many countries. GB is still being worked on.
Sure, I'm orenpo
Invite sent
@OrenPo: I sent you some comments on Discord but did not get any response.
Any progress on the GB login credentials yet? Many thanks
@Thornton69 Not much apart from me having learnt a bit about PKCE and OpenId and SSO. I'd dearly love some help from someone with experience in OpenId...
Not sure if it is possible here, but if we can pop up a browser, it can be used for passing the first part of the authentication, and get back the needed tokens.
Same issue with Belgian Telenet.
[27/02/2024, 13:39:21] [EOSSTB] sessionWatchdog(9): Started watchdog instance 9
[27/02/2024, 13:39:21] [EOSSTB] sessionWatchdog(9): Session connected but mqtt not connected, sessionWatchdog(9) will try to reconnect mqtt now...
Not sure if it is possible here, but if we can pop up a browser, it can be used for passing the first part of the authentication, and get back the needed tokens.
You would have to do that every time you restart Homebridge or every time the MQTT connection is lost for any reason. However it is still an interesting thought.
Not sure if it is possible here, but if we can pop up a browser, it can be used for passing the first part of the authentication, and get back the needed tokens.
I respectfully suggest trying this yourself - you will see how complex it is. The logon process is not simple and straight-forward. It's more than just a single cookie or token
It's painful now Virgin media have switched to Ping Identity and use Ping One Protect (browser profiling). Might be worthwhile having a look at the android app, see if there's something helpful in there.
As a fallback, is there anything that can be done with local network access only (e.g. power the box on and off), or does everything require authentication / network access?
It's painful now Virgin media have switched to Ping Identity and use Ping One Protect (browser profiling). Might be worthwhile having a look at the android app, see if there's something helpful in there.
Indeed. Very painful. Unfortunately I do not have any Android devices. But if you do, and you can help, I would welcome the help.
As a fallback, is there anything that can be done with local network access only (e.g. power the box on and off), or does everything require authentication / network access?
There's nothing to stop you powering the box down with a smart plug.
The physical remote control communicates via wireless signal (maybe Bluetooth) to the box. I cannot intercept or emulate the wireless signals.
This is why I was using network control of the box via the backend servers, and the network control requires authentication.
There's nothing to stop you powering the box down with a smart plug. The physical remote control communicates via wireless signal (maybe Bluetooth) to the box. I cannot intercept or emulate the wireless signals.
This is why I was using network control of the box via the backend servers, and the network control requires authentication.
Yep, I think the remote uses bluetooth. I just wondered if the box itself also ran a web server which allowed you to perform certain commands à la Sky Q. I guess not though.
Worst case scenario is to use IR, there are IR codes out there for the 'Ziggo mediabox next' that work with the VM 360 box, just the info and control available via the MQTT connection for integrations is quite appealing
Worst case scenario is to use IR, there are IR codes out there for the 'Ziggo mediabox next' that work with the VM 360 box, just the info and control available via the MQTT connection for integrations is quite appealing
Too bad the newer mini boxes don't have IR :')
Worst case scenario is to use IR, there are IR codes out there for the 'Ziggo mediabox next' that work with the VM 360 box, just the info and control available via the MQTT connection for integrations is quite appealing
Too bad the newer mini boxes don't have IR :')
That's frustrating!
Yep, I think the remote uses bluetooth. I just wondered if the box itself also ran a web server which allowed you to perform certain commands à la Sky Q. I guess not though.
Honestly, I do not know. I've never seen any info on this functionality.
Hi All, I made good progress tonight and the results look positive. Keep watching...
Any updates? Need help?
For me the login works (CH: Sunrise TV / Authentication Method A) but whenever it starts, it is not able to find any devices. When logging in to sunrisetv, I can see the device.
What I saw in the logs in homebridge: getPersonalizationData: GET https://spark-prod-ch.gnp.cloud.sunrisetv.ch/eng/web/personalization-service/v1/customer/anonymous?with=profiles%2Cdevices
But the customer should not be anonymous. As I was able to see with the web-inspector, the url is correct, but the customerId is the correct one (not anonymous). I guess there is the issue. Where does that "anonymous" come from? Was the login not successful at all?
Hi @realmuster: the fact that it shows your customer as anonymous shows it did not work. The login was rejected and it defaulted to anonymous. Without a proper customer number, it cannot know who you are, what boxes you have and what channels you subscribe to. Thus control of the box is not possible. If you have experience working with oauth2.0 and are willing to help, please get in touch! This project is not dead yet, it is just stuck....
Great to see the plugin working again in BE! Succes getting other countries to work! Thx!!!