Build UI hangs when building Android app with processing video library

[kjhollen]

Description

The build UI freezes when it encounters an error building an Android app that uses the Processing Video Library.

I think the underlying issues is either in Android Mode or Processing Video, but there's a UI portion in the PDE that feels like its own error so I thought I'd start here and ping @codeanticode. Please move this elsewhere if appropriate!

Expected Behavior

I am working on an art project that will play some videos on an Android device and let the viewer select which video to play. I would like to use Android Mode and the Processing Video library for this project. I have a simple prototype working in Java mode, which I expected to be able to flip to Android mode with minimal changes. If a build error occurs that prevents the project from running, I would expect to see and error message and for the PDE build-in-progress UI to disappear.

Current Behavior

My sketch fails to build if I include the video library, though otherwise Android mode is installed and works on my device. The error message is:

Exception in thread "Thread-23" java.lang.NullPointerException: Cannot read the array length because "list" is null
	at processing.app.Library.wrapFiles(Library.java:426)
	at processing.app.Library.getApplicationExports(Library.java:436)
	at processing.mode.android.AndroidBuild.copyImportedLibs(AndroidBuild.java:838)
	at processing.mode.android.AndroidBuild.createAppModule(AndroidBuild.java:475)
	at processing.mode.android.AndroidBuild.createProject(AndroidBuild.java:262)
	at processing.mode.android.AndroidBuild.build(AndroidBuild.java:223)
	at processing.mode.android.AndroidMode.handleRunDevice(AndroidMode.java:294)
	at processing.mode.android.AndroidEditor$17.run(AndroidEditor.java:421)

The build UI then remains until I press the stop button, so it looks like the project is still trying to build despite the error, but nothing happens. So I think there are two things here: 1) fixing either the Android or video library to address what causes the build error and 2) updating the PDE UI so it doesn't look like it's trying to build the project if it's stopped.

Steps to Reproduce

Here is some minimal code that reproduces the error:

import processing.video.*;

void setup() {
  
}

void draw() {
  
}

Your Environment

• Processing version: 4.1.2
• Operating System and OS version: macos 10.15.7 / Intel Graphics
• Other information: Android mode version: 4.5.0b5

[codeanticode]

@kjhollen the video library currently does not support the Android mode, although it is based on GStreamer, and GStreamer is available for Android, so at least in principle it should be possible (although not easy) to make the video library to work on Android.

It's probably easier to play videos using the MediaPlayer API that's part of the Android SDK (you should be able to import all necessary packages from the Processing sketch):

https://developer.android.com/reference/android/media/MediaPlayer

[codeanticode]

Actually, just remembered that a GSOC contributor created a processing-video library for Android:

https://github.com/omerjerk/processing-video-android

It has not been updated for quite a long time, so not sure if it works with the latest version of the mode.

[victordiaz]

@kjhollen In case it helps, I do maintain another Open Source project called PHONK that could help with what you want to achieve since it has a very simplified way to play videos. (it comes with an example exactly for that)

It also contains Processing Android as a Library so you can mix lots of Android stuff with Processing.

@codeanticode I hope you don't mind that I mentioned it.

[codeanticode]

Hey @victordiaz not problem at all, thanks for mentioning your project. PHONK looks really neat, I will give it a try!

[kjhollen]

Thanks @codeanticode, I'll see if I can manually link this library with Processing 4 as a workaround (it's no longer showing up the Contribution Manager) or try Processing 3. If that doesn't work, I'll do a little research to find out if I can get MediaPlayer to give me a GL-friendly texture. Will report back with how it goes.

@victordiaz this looks like a great project, thanks for suggesting!

[codeanticode]

Hi Kate, I think getting the video frames into Processing's PImage object as a GL texture is the tricky part, the video library for desktop does that under the hood, and in principle it should be possible to use it on Android since gstreamer is available for Android... one complication is probably making sure that the gstreamer-java bindings work on ARM with enough performance, but the GL Video library suggests that's is feasible. Let me know how it goes!

[kjhollen]

I was able to get something working by updating the Processing Android Video library & incorporating it into the project instead of the regular Processing video library. Sharing my fork of the project here, in case it's useful to anyone reading this: https://github.com/kjhollen/processing-video-android :) Planning to make a few more updates in the next few weeks, then hope we can get it back in the contributions manager.

Relating to this bug report—maybe the right thing to do in Android mode is to issue an error if the Processing Video Library is imported, suggesting to use the Android version instead?

[codeanticode]

@kjhollen should we close this issue?

[kjhollen]

@codeanticode yes, sounds good to me!