Video causes MPE frames to go out of sync

[grauwald]

Been attempting to integrate the Processing native video library with a sketch built on MPE. Apparently (to your point Mr. Shiffman) the video is running on a different thread and so is probably causing the sync problem. The error that gets thrown is always something like this (which frames this happens on is inconsistent):

Client: Extra message, frameCount: 8 received from server: 9

Looking at the Java source it seems that the server just stops if there's a loss of sync.

A likely solution for the video problem might be to use a sequence of still image frames, trouble with that method though is running out of RAM to load images into, particularly on Windows.

Thinking of implementing a custom application instead that keeps track of input controller data (in this case a Kinect via TSPS), calculates things like when to start/stop videos and x,y positions of graphics over time, and broadcasts that raw data to the client applications via OSC asynchronously to tell them what to draw when and where. This could be the basis of another variant of MPE that doesn't use frame sync but rather uses a central app to broadcast control data when it's available. If the data is broadcast frequently enough it could still provide the appearance of sync.