connection to ATEM crashing after reboot (Companion Pi version 3.1 and latest beta)
dickmann opened this issue · 10 comments
Hi, I have also an ATEM connection issue.
Latest Blackmagic Switcher Software 9.2.1, latest stable Companion 3.1 running on a PI 4 with fast SD card and I also tried latest beta 3.2.
Issue:
I get a connection with ATEM Extreme ISO, when I setup a new connection. But if I want to use an ATEM connection from the backup, it works only until I reboot. After the reboot the module tries to connect and reconnect and reconnect and then crashes (error message: crashed, connection config unavailable).
It worked totally fine with companion 2.4.2. Also using ATEM Software Controll.app on Mac works like a charm, so probably no network configuration or firewall issue. I am running Companion on Companion Pi via Ethernet, Streamdeck XL is directly connected with this PI. Latest Mac OS.
EDIT: If I add another ATEM connection to the same ATEM Switcher, this one comes up stable after a reboot (not used on any buttons). Seems one of the existing button actions or macros or triggers is responsible for this module crash!!!
I want to re-use my existing profiles from @davidjoshuaford...
Logfile:
Here is my logfile (companion 3.1 or 3.2):
Date,Module,Type,Log
2023-10-05T15:27:11.153Z,Instance/ModuleHost,info,"Starting connection: ""atem""(vskApziC1WMESHmLjLvPC)"
2023-10-05T15:27:15.753Z,Instance/ModuleHost,debug,"Connection ""atem"" started"
2023-10-05T15:27:15.786Z,Instance/Wrapper/atem,info,Connected to a ATEM Mini Extreme ISO
2023-10-05T15:27:15.902Z,Instance/ModuleHost,debug,"Connection ""atem"" stopped"
2023-10-05T15:27:21.296Z,Instance/atem,verbose,"stdout: Starting up module class: la
Sentry disabled
"
2023-10-05T15:27:21.297Z,Instance/ModuleHost,debug,"Registered module client ""vskApziC1WMESHmLjLvPC"""
2023-10-05T15:27:28.354Z,Instance/ModuleHost,warn,"Instance ""atem"" failed to init: Error: Call timed out Error: Call timed out
at t.IpcWrapper.sendWithCb (/opt/companion/main.js:2:2165692)
at Object.init (/opt/companion/main.js:2:7901847)
at l. (/opt/companion/main.js:2:7910834)
at l.emit (node:events:514:28)
at l.emit (node:domain:489:12)
at ChildProcess. (/opt/companion/main.js:2:2153197)
at ChildProcess.emit (node:events:514:28)
at ChildProcess.emit (node:domain:489:12)
at emit (node:internal/child_process:937:14)
at process.processTicksAndRejections (node:internal/process/task_queues:83:21)"
2023-10-05T15:27:28.591Z,Instance/atem,verbose,"stdout: Module-host accepted registration
ThreadedClass (6660) Skipping exit handler registration as no exit handler is registered
"
2023-10-05T15:27:32.339Z,Instance/ModuleHost,info,"Starting connection: ""atem""(vskApziC1WMESHmLjLvPC)"
2023-10-05T15:27:39.319Z,Instance/ModuleHost,debug,"Connection ""atem"" started"
2023-10-05T15:27:39.444Z,Instance/Wrapper/atem,info,Connected to a ATEM Mini Extreme ISO
Any idea?
Can you (or @davidjoshuaford) email me a copy of the profile to import? git@julusian.co.uk
This isnt happening with a blank config, so it is probably in some way related to the amount of actions/feedbacks in use in that profile? Trying to recreate that myself will be rather time consuming and not guaranteed to reproduce the issue
Hi Julian, I'll email you. I'm not aware of an atem problem on import. The vlc connection does crash since the latest updates (the work around is to generate a new vlc connection, then import the vlc buttons and patch them to the new connection). I know vlc may be beyond your purview though.
thanks. Im suspecting its some combination of the lower performance of the pi causing things to take slightly too long. not sure though. with the profile it should be a quick and easy test to try and reproduce it though
it looks like a vlc fix was pushed to beta ~2 weeks ago for some upgrade script issues. perhaps that could be what you were seeing? I can take a quick look at it, its probably a small fix
ok, I've just tried importing that config onto a pi4.
My first thought was how slow it was doing that import. That took much longer than I would expect.
Once it finished, I could see it stuck as crashed. Disabling and reenabling it 'fixed' it. I am pretty sure we do retry crashed modules eventually, so it would probably recover by itself eventually.
But even enabling the module was slow, with higher cpu than I would expect for longer than I would expect.
Im going to see if I can figure out why it is taking so long and so much cpu on the companion side
and fyi: The workaround David describes above worked here too. I was able to create a new ATEM connection, then import the atem buttons and patch them to the new connection (quite some work on 50 pages, a mapping for the full import would be nice too). Then everything works fine UNTIL I reboot the companion pi, then also this "new" atem connection keeps crashing... maybe this helps finding the root cause...
I'm having the same issue.
I have pushed some changes to beta, but I have also pushed some new features which may counteract the improvement and could have some instability