Tracker Crashes

[elijaelt]

Hello,
I am using the 'com.quickbirdstudios:opencv:4.1.0-contrib' version.
I wrote a simple program that uses the TrackerCSRT, but i can't get it working right.

It crashes with:
E/cv::error(): OpenCV(4.1.0) Error: Assertion failed (0 <= roi.x && 0 <= roi.width && roi.x + roi.width <= m.cols && 0 <= roi.y && 0 <= roi.height && roi.y + roi.height <= m.rows) in Mat, file /Users/sellmair/Sources/sellmair/opencv/modules/core/src/matrix.cpp, line 466
E/org.opencv.tracking: tracking::init_10() caught cv::Exception: OpenCV(4.1.0) /Users/sellmair/Sources/sellmair/opencv/modules/core/src/matrix.cpp:466: error: (-215:Assertion failed) 0 <= roi.x && 0 <= roi.width && roi.x + roi.width <= m.cols && 0 <= roi.y && 0 <= roi.height && roi.y + roi.height <= m.rows in function 'Mat'
E/AndroidRuntime: FATAL EXCEPTION: Thread-4
Process: elijaelt.totah.opencvtracker, PID: 23854
CvException [org.opencv.core.CvException: cv::Exception: OpenCV(4.1.0) /Users/sellmair/Sources/sellmair/opencv/modules/core/src/matrix.cpp:466: error: (-215:Assertion failed) 0 <= roi.x && 0 <= roi.width && roi.x + roi.width <= m.cols && 0 <= roi.y && 0 <= roi.height && roi.y + roi.height <= m.rows in function 'Mat'
]
at org.opencv.tracking.Tracker.init_0(Native Method)
at org.opencv.tracking.Tracker.init(Tracker.java:28)
at elijaelt.totah.opencvtracker.MainActivity$3.onCameraFrame(MainActivity.java:265)
at org.opencv.android.CameraBridgeViewBase.deliverAndDrawFrame(CameraBridgeViewBase.java:392)
at org.opencv.android.JavaCameraView$CameraWorker.run(JavaCameraView.java:373)
at java.lang.Thread.run(Thread.java:919)

The code regarding the tracker:
in the Deceleration:

private static final Scalar SELECT_RECT_COLOR = new Scalar(252, 3, 148, 255);
 private static final Scalar ALGORITHM_RECT_COLOR = new Scalar(255, 102, 0, 255);
 private Mat mRgba;
 private Mat mGray;
 private JavaCameraViewEliExtend mOpenCvCameraView;
 PointF initialTouch = new PointF();
 Rect2d screenSelectionArea = new Rect2d();
 Rect2d algorithmArea = new Rect2d();
 int algorithmTrackerStage = -1; // -1 don't show, 0 first point pressed, 1 on drag, 2 finished, 3 track
 TrackerCSRT tracker;

in onManagerConnected function :

// todo start the tracker
  tracker = TrackerCSRT.create();
  mOpenCvCameraView.enableView();

The video listener function (video works great):

    CameraBridgeViewBase.CvCameraViewListener2 cvCameraViewListener2 = new CameraBridgeViewBase.CvCameraViewListener2()
    {
        @Override
        public void onCameraViewStarted(int width, int height)
        {
            mGray = new Mat();
            mRgba = new Mat();
        }

        @Override
        public void onCameraViewStopped()
        {
            mGray.release();
            mRgba.release();
        }

        @Override
        public Mat onCameraFrame(CameraBridgeViewBase.CvCameraViewFrame inputFrame)
        {
            mRgba = inputFrame.rgba();
            mGray = inputFrame.gray();

           if (algorithmTrackerStage == 1)
            {
                Imgproc.rectangle(mRgba, screenSelectionArea.tl(), screenSelectionArea.br(), SELECT_RECT_COLOR, 1); // works great and shows the selected area on top of the video
            }
            else if (algorithmTrackerStage == 2)
            {
                if (screenSelectionArea.width > 0 && screenSelectionArea.height > 0)
                {
                    tracker.setInitialMask(mRgba); // added this method because without it i had a previous crash, don't know what it does. no documentation

                    tracker.init(mRgba, screenSelectionArea); // **<- crashes here**

                    algorithmTrackerStage = 3;
                }
                else
                {
                    algorithmTrackerStage = 0;
                }

            }
            else if (algorithmTrackerStage == 3)    // show tracker result
            {
                tracker.update(mRgba, algorithmArea);// **didn't get to this stage, hope that it will work after you will help me :-)**

                Imgproc.rectangle(mRgba, screenSelectionArea.tl(), screenSelectionArea.br(), ALGORITHM_RECT_COLOR, 1); // without using the tracker (init, update) this part works 
            }

            return mRgba;
        }
    };

Please Help

[nasirky]

Hi @elijaelt. Thank you for posting your questions/issue.

0 <= roi.x && 0 <= roi.width && roi.x + roi.width <= m.cols && 0 <= roi.y && 0 <= roi.height && roi.y + roi.height <= m.rows

The above error that you are getting suggests that (at least 1 point) of the area of interest is outside the image. As you have mentioned in the code that the following line causes the crash:

tracker.init(mRgba, screenSelectionArea);

Can you share the bounds of mRgba as well as the screenSelectionArea. There might be a single pixel that is causing this crash.

[elijaelt]

Hi,

1. This is strange, because as i mentioned in my post, i can see the selected area on top of the image.

2. As you requested for the example mRgba and the screenSelectionArea before the app crashes:
   Coded in onCameraFrame function:

int cols = mRgba.cols();
int rows = mRgba.rows();

Get values:

cols: 2160
rows: 1080

for screenSelectionArea values:

screenSelectionArea.x: 810.0
screenSelectionArea.y: 219.0
screenSelectionArea.width: 281.0
screenSelectionArea.height: 327.0

If you have any experience with this tracking method, it will be great if you could post a simple working usage example.

[nasirky]

@elijaelt This issue might occur even if the calculations are off by a pixel. Maybe in the case of the crash, 1 pixel is outside the ROI?

[nasirky]

@elijaelt Can you try with the latest version of openCV and let me know if it works for you?

[elijaelt]

Hi @nasirky
The pixel is not out of the ROI.
When i shifted to MOSSE algorithm, it works great, so i stayed with it . . .

What changes did you do on the latest build to fix it?
I'll try the new version in the upcoming days and will let you know...

[nasirky]

@elijaelt You can find the change log for v4.3.0 here

[nasirky]

@elijaelt Please note that we are in no way associated with the official OpenCV (as well as contribution) projects. This repository (and the maven based distributions) are created to assist other developers like us to easily use OpenCV (with contributions) in Android projects.

[elijaelt]

@nasirky
I tried the latest versions 4.2.0-contrib and 4.3.0-contrib and the preview screen is not working...
I get a black screen.
I tried :
JavaCameraView
CameraBridgeViewBase

Something went wrong on the last builds.

[nasirky]

@elijaelt are you sure you are asking the right permissions (especially in Android >= M)?

[elijaelt]

@nasirky
What permissions i need to check ?
The camera preview works great on 4.1.0-contrib build.
Did you try the newer versions ?

[nasirky]

You need to check the runtime permissions. Referring one of my comments on another issue:

Which version of Android are you using on the test phone? In case you are using 6.0 (API Level 22) or above, you would need to ask for permissions at run time). You can find more information here.

We do not use the preview in our projects as we are using OpenCV in a different way and it works fine in 4.3.0-contrib. Can you share a minimal version of your project so that I can try it out on my end.

[elijaelt]

Hi,
I use permissions.
Try this code as a minimal version of the project:
https://drive.google.com/file/d/1uIG1y-gmQRjK0HI0EQK-foJgZcQU2E-M/view?usp=sharing

It works fine with 4.1.0-contrib and with the TrackerMOSSE class.
I have marked out irrelevant code for this demo.

Good luck :-)

[nasirky]

Hi @elijaelt
Sorry I couldn't find time earlier. I will try to check it this week.

[nasirky]

Hi @elijaelt, Sorry it has been quite some time that I didn't get back to you. Were you able to resolve the issue? I can try to help if the issue still persists.

[nasirky]

Closing this issue. Please create a new one if the issue persists.