deltacv/EOCV-Sim

Selecting camera input causes the simulator to lock up

e-holder opened this issue · 8 comments

e-holder commented

When I select camera input, the simulator app locks up, and I have to kill the Java Platform SE task using TaskManager.

To Reproduce
Start running a pipeline
Select File->New->Input Source->Camera

Expected behavior
I expected my camera input to be displayed, or at least for it to allow me to choose a camera.
I have a webcam plugged in, and it seems to be working (per the "Camera" app on my Windows PC).

I created a workspace which seems to be working for a static image. I can see my image processing alterations and see telemetry updates when I click on the image (via onViewportTapped).

Desktop

  • OS: Windows 10 Pro
  • Version 19041.1348
  • Java version: jdk-17.0.1

By the way, this setup is completely awesome. As is, it will work great for individual images for the barcode reading. Thanks for providing this. I'm now hoping to get this bug resolved so I can develop a "freight seeking" pipeline using active video from the webcam.

Is there a step I am missing in the code to attach to a camera? I didn't see anything in the documentation pages. It doesn't look like the sim workspace should need that, but I may be misunderstanding.

serivesmejia commented

Hi ! Thanks for the bug report and apologies for the issue you are having, it's actually something I'm aware of and have been trying to find ways to fix it. You can try to use the latest dev version: in this github release. It's still sort of unstable at times but it at least hopefully fixes the main issue with the freezing. Let me know how it works!

I will keep this issue opened until I make a full release that fixes it.

e-holder commented

Thanks for the quick response! The dev version almost got me there. At first, I thought it also locked up, but it just took a long time (30-45 seconds) for the camera selection/test dialog to come up. I have a couple of external cameras on this laptop in addition to its internal camera. I was able to select each external camera and was able to test one of them. Unfortunately, the LogiTech webcam we are using for robotics failed to test/create. The other external camera seemed to connect fine, but I chose a really wide resolution and my image was doubled/wrapped around.

I tried deleting that source so I could try again with the LogiTech webcam. But now it seems to just be locking up again.

Where does the program store its data (e.g., the list of Sources)? I may try deleting all that data so it can start fresh.

serivesmejia commented

Thanks for the quick response! The dev version almost got me there. At first, I thought it also locked up, but it just took a long time (30-45 seconds) for the camera selection/test dialog to come up. I have a couple of external cameras on this laptop in addition to its internal camera. I was able to select each external camera and was able to test one of them. Unfortunately, the LogiTech webcam we are using for robotics failed to test/create. The other external camera seemed to connect fine, but I chose a really wide resolution and my image was doubled/wrapped around.

That's sort of a weird behavior, it shouldn't be taking that long. Just to confirm, does the list of cameras go by "Webcam 1", "Webcam 2", etc. or does it display the actual names of the camera devices? (this helps me determine if it's using the OpenCV camera driver or the OpenIMAJ one).

It would also be great if you could send me some logs, if possible. You might see some crashreport-etc-eocvsim.log or lasterror-eocvsim.log files, or in case you don't see them, would it be possible to run eocv sim from the console and copy paste the output here? Thanks!

I tried deleting that source so I could try again with the LogiTech webcam. But now it seems to just be locking up again.
Where does the program store its data (e.g., the list of Sources)? I may try deleting all that data so it can start fresh.

Depending on your operating system,
C:/Users/<name>/.eocvsim on Windows
/home/<name>/.eocvsim on Linux
/Users/<name>/.eocvsim MacOS (iirc, i never used mac)

You will find an eocvsim_sources.json file there which you can delete to return to the initial state. I do agree that I need to add a better way for deleting sources though, I will try to add a right-click menu to do so and remove the need of having to select the source.

Once again, thanks so much for your help! It's really appreciated

e-holder commented

I can't seem to get the camera selection to work again, even if I delete the entire .eovsim directory to start from scratch. If I recall correctly, I was getting camera names like the internal one was named according to the laptop, and the webcam of interest was named something like "USB 2.0 Webcam" or something like that. I am not seeing any log files. But I will paste the output when running from command line (in the next post).

e-holder commented

C:\Users\joela\Documents\Robotics\OpenCV>java -jar EOCV-SIM-3.3.3-dev-4c244c1-all.jar
00:00 INFO: [LockFile] Probably locked file C:\Users\joela.eocvsim.lock
00:00 INFO: [EOCVSim] Confirmed claiming of the lock file in C:\Users\joela.eocvsim

00:00 INFO: [EOCVSim] Initializing EasyOpenCV Simulator v3.3.3-dev-4c244c1 (2d9d4f9d)

00:00 INFO: [EOCVSim] Loading native lib...
00:00 INFO: [Icons] Loading future icon ico_eocvsim
00:01 INFO: [EOCVSim] Successfully loaded the OpenCV native lib

00:01 INFO: [ConfigManager] Initializing...
00:01 INFO: [ConfigManager] Loaded config from file successfully

00:01 INFO: [ClasspathScan] Starting to scan classpath...
00:01 INFO: [WorkspaceManager] Set current workspace to C:\Users\joela.eocvsim\default_workspace
00:01 INFO: [WorkspaceManager] Loaded workspace config successfully
00:01 INFO: [FileWatcher-Workspace] Starting to watch directories in:
C:\Users\joela.eocvsim\default_workspace
C:\Users\joela.eocvsim\default_workspace

00:01 INFO: [WorkspaceManager] Loaded workspace config successfully

00:01 INFO: [InputSourceManager] Initializing...
00:01 WARN: [Visualizer] Setting the Taskbar icon image is not supported on this platform
00:01 INFO: [InputSourceManager] Adding InputSource ImageSource("C:\Users\joela\AppData\Local\Temp\ug_eocvsim_4.jpg", 320x240) (ImageSource)
00:01 INFO: [InputSourceManager] Adding InputSource ImageSource("C:\Users\joela\AppData\Local\Temp\ug_eocvsim_1.jpg", 320x240) (ImageSource)
00:01 INFO: [InputSourceManager] Adding InputSource ImageSource("C:\Users\joela\AppData\Local\Temp\ug_eocvsim_0.jpg", 320x240) (ImageSource)
00:01 INFO: [Icons] Loading future icon ico_colorpick_pointer
00:01 INFO: [InputSourceManager] Set InputSource to ImageSource("C:\Users\joela\AppData\Local\Temp\ug_eocvsim_4.jpg", 320x240) (ImageSource)
00:02 INFO: [InputSourceLoader] InputSources file version is DOS

00:02 INFO: [PipelineManager] Initializing...
00:02 INFO: [CompiledPipelineManager] Initializing...
00:02 INFO: [WorkspaceManager] Loaded workspace config successfully
00:02 INFO: [CompiledPipelineManager] Building java files in workspace, at C:\Users\joela.eocvsim\default_workspace
00:02 INFO: [PipelineStandardFileManager] Scanning classpath files...
00:02 INFO: [PipelineStandardFileManager] Found classpath file C:\Users\joela\Documents\Robotics\OpenCV\EOCV-SIM-3.3.3-dev-4c244c1-all.jar in classpath

00:03 INFO: [ClasspathScan] ClassGraph finished scanning (took 2.167291701s)

00:03 INFO: [ClasspathScan] Found 0 pipelines

00:03 INFO: [ClasspathScan] Found tunable field com.github.serivesmejia.eocvsim.tuner.field.BooleanField
00:03 INFO: [ClasspathScan] Found tunable field com.github.serivesmejia.eocvsim.tuner.field.EnumField
00:03 INFO: [ClasspathScan] Found acceptor for this tunable field, com.github.serivesmejia.eocvsim.tuner.field.EnumField
00:03 INFO: [ClasspathScan] Found tunable field com.github.serivesmejia.eocvsim.tuner.field.StringField
00:03 INFO: [ClasspathScan] Found tunable field com.github.serivesmejia.eocvsim.tuner.field.cv.PointField
00:03 INFO: [ClasspathScan] Found tunable field com.github.serivesmejia.eocvsim.tuner.field.cv.RectField
00:03 INFO: [ClasspathScan] Found tunable field com.github.serivesmejia.eocvsim.tuner.field.cv.ScalarField
00:03 INFO: [ClasspathScan] Found tunable field com.github.serivesmejia.eocvsim.tuner.field.numeric.DoubleField
00:03 INFO: [ClasspathScan] Found tunable field com.github.serivesmejia.eocvsim.tuner.field.numeric.FloatField
00:03 INFO: [ClasspathScan] Found tunable field com.github.serivesmejia.eocvsim.tuner.field.numeric.IntegerField
00:03 INFO: [ClasspathScan] Found tunable field com.github.serivesmejia.eocvsim.tuner.field.numeric.LongField

00:03 INFO: [ClasspathScan] Found 10 tunable fields and 1 acceptors

00:03 INFO: [ClasspathScan] Finished scanning (took 2.215545001s)
00:03 INFO: [PipelineManager] Found 1 pipeline(s)

00:03 INFO: [Icons] Loading future icon ico_img
00:03 INFO: [EOCVSim] Begin EOCVSim loop

00:03 INFO: [CompiledPipelineManager] Looking for pipelines in jar file C:\Users\joela.eocvsim\compiler\out_jars\pipelines.jar
00:03 INFO: [ClasspathScan] Starting to scan for classes in C:\Users\joela.eocvsim\compiler\out_jars\pipelines.jar...
00:03 INFO: [ClasspathScan] ClassGraph finished scanning (took 0.0200082s)
00:03 INFO: [ClasspathScan] Found pipeline AprilTagDetectionPipeline

00:03 INFO: [ClasspathScan] Found 1 pipelines

00:03 INFO: [ClasspathScan] Found 0 tunable fields and 0 acceptors

00:03 INFO: [ClasspathScan] Finished scanning (took 0.0357709s)
00:03 INFO: [CompiledPipelineManager] Added AprilTagDetectionPipeline from jar
00:03 INFO: [CompiledPipelineManager] Build successful (took 1.68 seconds)

Using the JDK compiler

AprilTagDetectionPipeline.java
(95:17): MANDATORY_WARNING: finalize() in java.lang.Object has been deprecated

00:03 INFO: [Icons] Loading future icon ico_hammer
00:03 INFO: [Icons] Loading future icon ico_gears
00:04 INFO: [PipelineManager] Changing to pipeline com.github.serivesmejia.eocvsim.pipeline.DefaultPipeline
00:04 INFO: [PipelineManager] Instantiated pipeline class com.github.serivesmejia.eocvsim.pipeline.DefaultPipeline
00:04 INFO: [PipelineSnapshot] Taken snapshot of pipeline com.github.serivesmejia.eocvsim.pipeline.DefaultPipeline
00:04 INFO: [Icons] Loading future icon ico_slider
00:04 INFO: [Icons] Loading future icon ico_config
00:04 INFO: [Icons] Loading future icon ico_colorpick
00:04 INFO: [PipelineManager] Initialized pipeline DefaultPipeline

[ WARN:2] global C:\build\master_winpack-bindings-win64-vc14-static\opencv\modules\videoio\src\cap_msmf.cpp (434) `anonymous-namespace'::SourceReaderCB::~SourceReaderCB terminating async callback

serivesmejia commented

Apologies for the delayed response, I have been developing a fix for this by writing a driver with more direct and fast access to the internal apis, also fixing the deadlock in the create camera source dialog. It seems to be working reliably now (in my machine that is), if you still need the software or you are willing to test, the latest dev 3.4.0 release (https://github.com/deltacv/EOCV-Sim/releases/tag/Dev) contains everything i mentioned.

I have found some instances of jvm crashes but hopefully all those edge cases should be properly handled now. Let me know how it goes, thanks!

e-holder commented
serivesmejia commented

Closing this issue due to lack of updates. If further tracking is needed please make a new one. Thanks!