Handle ginormous workspaces better
Closed this issue · 9 comments
Is your feature request related to a problem? Please describe.
It takes a long time to get all the data from a big QLab workspace with thousands of cues. Could affect performance on the the device. Similarly for Eos, etc.
Describe the solution you'd like
Slow down or optimize how data is collected.
I'll do some R&D on this!
Branch rate-limiting implements a max message rate of 1 message per 20ms per plugin. Curious what you think @jwetzell
I've noticed interesting behavior in the QLab plugin as a result, but I think that's a separate existing issue made more evident.
Looks like 1 message every 50ms (20 messages per second). Which in my quick testing slows most devices I tried down to some terrible loading times. What is the motivation behind rate limiting?
Getting too much info at once from QLab or EOS can cause temporary performance drops in both. Previously it was sending multiple messages per ms (super super fast) so maybe it just need to be tuned up faster? One per 5 ms?
I lean towards being on the slower side so there's confidence that loading Cue View won't affect the show.
I would tend towards optimizing the plugins that are requiring a ton of messages to be exchanged (loading that demo Hamilton show required 300ish messages to get off the ground (averaging 1 message per ms) that doesn't seem like that much to me. QLab should in a best case world require 1 message to load the general outline of the workspace + 1 message per cue to load all that info that annoyingly isn't included in that first large message, and then a couple more for workspace info to get off the ground and then you'd have the messages that poll for state/respond to update messages.
I don't think I've looked to much at the number of messages being sent around to see if there are places that could be slimmed down but that is where I would start.
I've programmed many QLab shows with thousands of cues that push the computer to the limit. Any hitch in video playback because of Cue View is unacceptable imo. There's QLab situations, like adding a cue nested within many groups, that can require a lot of mostly unchanged data to be transacted, and I don't see a way around it. I don't think there's unnecessary messages.
What would be the benefit of building a message queueing system into the QLab plugin instead of into Cue View as a whole?
I don't have any real world experience with large QLab shows to know that it will actually lag when processing OSC. Like I said I've never really looked to see how the QLab stuff fetches data. I know changes in group cues or cue lists in general (adding/removing/rearranging cues) means you gotta do a couple more but I still wouldn't have though Cue View would have been sending that much that often. I'll give it more of a look this weekend and run it against some more plugins to see if it is really that annoying.
It doesn't seem to be that bad if I lower it to 20ms interval I think it could even go lower like 10 or so. That would still be loading things 15-20x slower then no queue/interval. I would split it from the heartbeat method though.
Yeah I'm totally happy to go down to 10ms, that's like 10x slower than it used to be but I can't notice a difference