Movie Fluid optical flow not working

[kramser]

Hi,

I'm trying to get the Pixelflow Movie Fluid sketch working on a 2013 Mac Pro. Whenever I run the sketch, my video plays, but the optical flow doesn't appear to be affecting the fluid. There's some slight blurring and smoothing going on in density mode, but the temperature, velocity, and pressure modes are either black or just the regular video. However, I am able to affect the fluid by clicking and dragging the mouse. When I run the sketch I get this in the console: "WARNING: Output of vertex shader 'line_domain' not read by fragment shader"

Here are the specs of the machine:
Mac Pro (Late 2013)
macOS Sierra v10.12.5
3.5 GHz 6-Core Intel Xeon ES
32 GB 1866 MHz DDR3
AMD FirePro D700 6144 MB

There's also this info that prints in the console when I run the sketch, if that's helpful:
OPENGL_VENDOR: ATI Technologies Inc.
OPENGL_RENDERER: AMD Radeon HD - FirePro D700 OpenGL Engine
OPENGL_VERSION: 4.1 ATI-1.51.8
GLSLVersionString: #version 410 core
GLSLVersionNumber: 4.10.0
GLVersion: 4.1 (Core profile, arb, compat[ES2, ES3], FBO, hardware) - 4.1 ATI-1.51.8
GLVendorVersionNumber: 1.51.8 (ATI-1.51.8)

Any ideas? Thanks!

[diwi]

What version of pixelflow are you using?
Just in case, make sure you update to the latest v0.60, and also have the latest version of your graphics-card driver installed. (#7)

Does the Exampe "OpticalFlow_Basic" work? ... to rule out any other potential error-sources.
When you run it, you should get something like this on the screen.
You can ignore the warning, it shouldnt cause any real issues.

[kramser]

Thanks for the quick response! After updating to Pixelflow v0.60 and restarting, OpticalFlow_Basic still isn't working.

I have the latest update for my graphics drivers. I just updated them two days ago.

[diwi]

Thats weird, no errors printed and unfortunately very difficult to debug for me since i cant reproduce this error in any way.

There is another example called ImageProcessing_Filter.
I built it for testing various filters, ... like gaussian blur and sobel egde, both used for the optical flow. So if you run this example and test both filters, it would narrow the source of errors a bit.

By any chance, are there other examples that fail to run as well? like the fluid simulations, skylight renderer?

[kramser]

When I run ImageProcessing_Filter and select Distance Transform / Voronoi the console continuously prints "DistanceTransform.create update | GL_ERROR: invalid operation." Nothing else returns an error or appears to now work.

Most other examples I've tried--including fluid simulation and skylight renderer sketches--appear to run fine. The ones that give me trouble are the Capture and Optical flow sketches. When I ran OpticalFlow_CaptureVerletParticles and was dragging the mouse around I got an error "ArrayIndexOutOfBoundsException: - 530" and then the sketch froze. Seems to happen when I drag the mouse beyond the bottom of the window. Not sure if that's in any way related to the other issues I'm having.

Another curious issue is that when I run OpticalFlow_Capture, the webcam feed comes on for a moment, then the webcam turns off. After that, if I go into another application like Photobooth, the webcam doesn't work until I unplug it and plug it back in again. I tried another webcam and had a similar thing happen. With the second camera the feed didn't even come on for a second. The camera just turned off immediately. It couldn't be seen by Photobooth until unplugged and replugged.

Sorry if including these other issues is confusing rather than helpful. I know relatively little about Javascript and even less about GLSL so I'm not sure how these problems might be related. Thanks!

[diwi]

the ArrayIndexOutOfBoundsException will be fixed in the next release.
For now you can manually fix your local example like i did here, adding these 2 lines:
https://github.com/diwi/PixelFlow/blob/master/examples/OpticalFlow_CaptureVerletParticles/OpticalFlow_CaptureVerletParticles.java#L228

Webcam issues on OpticalFlow_Capture:
Its probably best to check if capturing itself is the problem and test it in a separate very simple sketch, ... again to minimize the possible source for this error.
You could try to test some of the capture examples here:
https://processing.org/reference/libraries/video/index.html
https://processing.org/reference/libraries/video/Capture.html

Other than that, I'm afraid, the optical flow and distance transform problems you encounter can only be fixed by debugging more in depth (especially since no meaningful errors are thrown) on a similar machine to yours.

[kramser]

Thanks so much for your help. I understand there isn't much you can do without error information or access to my machine. I'll keep working at it and let you know if I find anything useful.

[knupel]

Same issue on my computer
OS Sierra / iMac 2012 / Processing 3.3.5

[kramser]

I'm still having issues with the Capture Fluid sketch, but I did get the Movie Fluid sketch working! I figured it out on accident. I dragged the Temporal Smoothing slider all the way down to zero and suddenly the velocity vectors started up. From there I could drag the value up and down and it would continue working. It just needed that kick in the pants to get started for some reason. Again, this only worked with the Movie Fluid sketch. The Capture Fluid sketch on my machine still won't hold onto a webcam feed longer than half a second.

[diwi]

thanks for this info, thats actually quite helpful!
there could be a bug in the library as the Optical flow textures are not cleared at program start.

until there is a new release available a workaround is to call reset once in the beginning:

    opticalflow = new DwOpticalFlow(context, pg_movie_w, pg_movie_h);
    opticalflow.reset();

i cant say yet if this fixes some problem for you, but its worth trying.
I will further check for errors.

[diwi]

this should be fixed now in release v0.62 (32).