Breakthrough/DVR-Scan

OpenCV won't build properly when using Debian's old pip

bigtonylewis opened this issue · 4 comments

bigtonylewis commented

When running on a Debian-based LXC container, dvr-scan fails:

$ dvr-scan -i record_2021-08-05-14.47.32.mp4
[DVR-Scan] Initializing scan context...
[DVR-Scan] Error: Couldn't load video record_2021-08-05-14.47.32.mp4.
[DVR-Scan] Check that the given file is a valid video clip, and ensure all required software dependencies are installed and configured properly.
[DVR-Scan] Failed to load input, see above output for details.

However running the same command on another machine works:

~/src/env-dvr-scan/bin/dvr-scan -i record_2021-08-05-14.47.32.mp4
[DVR-Scan] Initializing scan context...
[DVR-Scan] Opened video record_2021-08-05-14.47.32.mp4 (1280 x 720 at 14.930 FPS).
[DVR-Scan] Appended video record_2021-08-05-14.47.32.mp4.
[DVR-Scan] Processed:   0%|                                                                                    | 0/1188 [00:00<?, ? frames/s][DVR-Scan] Scanning input video for motion events...
[DVR-Scan] Processed:  83%|█████████████████████████████████████████████████████████████▏            | 983/1188 [00:28<00:04, 42.47 frames/s]

The dependencies seem to be installed properly:

root@recordcameras:~# pip3 install dvr-scan opencv-python
Collecting dvr-scan
  Downloading https://files.pythonhosted.org/packages/48/b3/77940e1da1f82bdee1d0e9cdc08058a9646440c49b1d7ad9c24c95890a3f/dvr_scan-1.3-py2.py3-none-any.whl
Collecting opencv-python
  Downloading https://files.pythonhosted.org/packages/01/9b/be08992293fb21faf35ab98e06924d7407fcfca89d89c5de65442631556a/opencv-python-4.5.3.56.tar.gz (89.2MB)
    100% |████████████████████████████████| 89.2MB 8.1kB/s 
  Installing build dependencies ... done
Collecting tqdm (from dvr-scan)
  Downloading https://files.pythonhosted.org/packages/0b/e8/d6f4db0886dbba2fc87b5314f2d5127acdc782e4b51e6f86972a2e45ffd6/tqdm-4.62.0-py2.py3-none-any.whl (76kB)
    100% |████████████████████████████████| 81kB 2.0MB/s 
Collecting numpy (from dvr-scan)
  Downloading https://files.pythonhosted.org/packages/b8/46/3f1a1a1f5fa3c0325a407a98396b8ac90e4eaa058420668b4b640729c784/numpy-1.21.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (14.1MB)
    100% |████████████████████████████████| 14.1MB 48kB/s 
Building wheels for collected packages: opencv-python
  Running setup.py bdist_wheel for opencv-python ... done
  Stored in directory: /root/.cache/pip/wheels/5e/f7/5c/55741ad3642e2e72934b934476ce67460097b494c207210e1c
Successfully built opencv-python
Installing collected packages: tqdm, numpy, dvr-scan, opencv-python
Successfully installed dvr-scan-1.3 numpy-1.21.1 opencv-python-4.5.3.56 tqdm-4.62.0

# pip3 install dvr-scan
Requirement already satisfied: dvr-scan in /usr/local/lib/python3.7/dist-packages (1.3)
Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from dvr-scan) (1.21.1)
Requirement already satisfied: tqdm in /usr/local/lib/python3.7/dist-packages (from dvr-scan) (4.62.0)
# pip3 install opencv-python
Requirement already satisfied: opencv-python in /usr/local/lib/python3.7/dist-packages (4.5.3.56)
Requirement already satisfied: numpy>=1.14.5 in /usr/local/lib/python3.7/dist-packages (from opencv-python) (1.21.1)
# dvr-scan
usage: dvr-scan [-h] [-v] -i VIDEO_FILE [-o OUTPUT_VIDEO.avi] [-b TYPE] [-so]
                [-c FOURCC] [-t value] [-k N] [-l T] [-tp T] [-tb T] [-q]
                [-st time] [-dt time] [-et time] [-df factor] [-fs num_frames]
                [-tc] [-roi [x0 y0 w h [x0 y0 w h ...]]]
dvr-scan: error: the following arguments are required: -i/--input

The differences in environments are:

  • CPU: AMD (fail) vs Intel (works)
  • Virtual-env: not used (fails)) vs used (works)
  • Containers: LXC used (fails) vs not sued (works)

I tried using strace to see if I could see any missing libraries, but I got too lost.

Breakthrough commented

Can you provide the debug logs by specifying -v debug? Thank you!

bigtonylewis commented

Can you provide the debug logs by specifying -v debug? Thank you!

On dvr-scan? There appears to be no debug option. -v is for version:

dvr-scan -h | grep -- -v
usage: dvr-scan [-h] [-v] -i VIDEO_FILE [-o OUTPUT_VIDEO.avi] [-b TYPE] [-so]
  -v, --version         show version number and license/copyright information

And as best I can tell, there is nothing in the help about debugging:

dvr-scan -h | grep -i debug | wc -l
0
bigtonylewis commented

Solved. If I used virtualenv to build dvr-scan, the resulting install via pip works:

cam@recordcameras:~/src$ virtualenv -p python3 dvr-scan-env
Already using interpreter /usr/bin/python3
Using base prefix '/usr'
New python executable in /home/cam/src/dvr-scan-env/bin/python3
Also creating executable in /home/cam/src/dvr-scan-env/bin/python
Installing setuptools, pkg_resources, pip, wheel...done.
cam@recordcameras:~/src$ . ~/src/dvr-scan-env/bin/activate
(dvr-scan-env) cam@recordcameras:~/src$ pip install opencv-python dvr-scan
Collecting opencv-python
  Downloading opencv_python-4.5.3.56-cp37-cp37m-manylinux2014_x86_64.whl (49.9 MB)
     |████████████████████████████████| 49.9 MB 34 kB/s 
Collecting dvr-scan
  Downloading dvr_scan-1.3-py2.py3-none-any.whl (29 kB)
Collecting numpy>=1.14.5
  Downloading numpy-1.21.1-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (15.7 MB)
     |████████████████████████████████| 15.7 MB 1.6 MB/s 
Collecting tqdm
  Downloading tqdm-4.62.0-py2.py3-none-any.whl (76 kB)
     |████████████████████████████████| 76 kB 267 kB/s 
Installing collected packages: tqdm, numpy, opencv-python, dvr-scan
Successfully installed dvr-scan-1.3 numpy-1.21.1 opencv-python-4.5.3.56 tqdm-4.62.0

(dvr-scan-env) cam@recordcameras:~/tmp$ dvr-scan -i record_2021-08-05-14.47.32.mp4 
[DVR-Scan] Initializing scan context...
[DVR-Scan] Opened video record_2021-08-05-14.47.32.mp4 (1280 x 720 at 14.930 FPS).
[DVR-Scan] Appended video record_2021-08-05-14.47.32.mp4.
[DVR-Scan] Processed:   0%|                                                                           | 0/1188 [00:00<?, ? frames/s][DVR-Scan] Scanning input video for motion events...
[DVR-Scan] Processed: 100%|████████████████████████████████████████████████████████████████| 1188/1188 [00:34<00:00, 34.52 frames/s]
[DVR-Scan] Processed 1188 frames read in 34.5 secs (avg 34.4 FPS).
[DVR-Scan] Detected 2 motion events in input.
[DVR-Scan] Scan-only mode specified, list of motion events:
-------------------------------------------------------------
|   Event #    |  Start Time  |   Duration   |   End Time   |
-------------------------------------------------------------
|  Event    1  |  00:00:28.9  |  00:00:13.0  |  00:00:41.9  |
|  Event    2  |  00:00:48.6  |  00:00:07.2  |  00:00:55.8  |
-------------------------------------------------------------
[DVR-Scan] Comma-separated timecode values:
00:00:28.935,00:00:41.930,00:00:48.628,00:00:55.795
[DVR-Scan] Motion events written to disk.

The issue might be Debian's use of an old pip version (18.1) vs the latest one coming through virtualenv (21.2.3). I found the clue through this StackOverflow answer:

Since opencv-python version 4.3.0.*, manylinux1 wheels were replaced by manylinux2014 wheels. If your pip is too old, it will try to use the new source distribution introduced in 4.3.0.38 to manually build OpenCV because it does not know how to install manylinux2014 wheels. However, source build will also fail because of too old pip because it does not understand build dependencies in pyproject.toml. To use the new manylinux2014 pre-built wheels (or to build from source), your pip version must be >= 19.3.

Breakthrough commented

Sorry about the -v option, I was thinking of another project - my apologies. Thanks for digging into the cause of this and providing an update!