Installing vrn in Fedora. /lib64/libmatio.so.2: undefined symbol: H5T_NATIVE_SCHAR_g

Question

Installing vrn in Fedora. /lib64/libmatio.so.2: undefined symbol: H5T_NATIVE_SCHAR_g

fpradom opened this issue 6 years ago · 18 comments

Hello, Aaron

I desisted from use Fedora 29, while F29 and F28 have python3 preinstalled as default and internal use so installing python 2.7 was inutil.

I have made a clean installation from Fedora 27 that have python 2.7.15 preinstalled.

The full installation elements are as follow:
Prerequisites:

sudo -i
dnf install kernel-devel kernel-headers gcc gcc-c++ make dkms acpid libglvnd-glx libglvnd-opengl libglvnd-devel pkgconfig
dnf install cmake wget git ImageMagick-devel
dnf install openssl-devel bzip2-devel glog-devel boost-devel

NVidia driver for GeForce GTX 750 Ti

dnf install mesa-libGLU-devel libX11-devel libXi-devel libXmu-devel
cuda 8.0 and cudnn 5.1 for coda 8

For python:
dnf install python-devel

pip install visvis
pip install imageio
pip install numpy
pip install dlib
pip install matplotlib

Added gcc53-c++-5.3.0 for Torch installation

Installed Torch 7 from https://github.com/torch/distro.git

Installed THPP from https://github.com/1adrianb/thpp.git

Installed fb.python from https://github.com/facebook/fblualib.git

Installed VRN from https://github.com/AaronJackson/vrn.git

On running run.sh I had the
Fatal Python error: ceval: tstate mix-up github

I surfed your Issues and finded in #59 your recomendation to remove from main.lua:

from mpl_toolkits.mplot3d import Axes3D
import matplotlib.pyplot as plt
import matplotlib.patches as patches

and replace on vrn/face-alignment/facedetection_dlib.lua:

local detections = py.reval('[np.asarray([d.left(), d.top(), d.right(), d.bottom()]) for i, d in enumerate(dets)]',{dets=dets})
with
local detections = py.reval('[np.asarray([d.left(), d.top(), d.right(), d.bottom()],dtype=float) for i, d in enumerate(dets)]',{dets=dets})

main.lua needs npy4th and matio so,

---- installing npy4th ----
git clone https://github.com/htwaijry/npy4th.git
cd npy4th
luarocks make

---- installing matio ----
downloaded matio-1.5.2-7.fc23.x86_64.rpm from https://rpm.pbone.net/
sudo rpm -ihv --nodeps --force matio-1.5.2-7.fc23.x86_64.rpm
luarocks install matio

With this done I tryed the run.sh again with following result:

[f_prado@localhost vrn]$ ./run.sh
...rado/usr/local/torch/install/share/lua/5.1/matio/ffi.lua:20: /lib64/libmatio.so.2: undefined symbol: H5T_NATIVE_SCHAR_g
stack traceback:
[C]: in function 'load'
...rado/usr/local/torch/install/share/lua/5.1/matio/ffi.lua:20: in function <...rado/usr/local/torch/install/share/lua/5.1/matio/ffi.lua:20>
[C]: in function 'xpcall'
...rado/usr/local/torch/install/share/lua/5.1/matio/ffi.lua:20: in main chunk
[C]: at 0x0044e490
[C]: in function 'pcall'
...ado/usr/local/torch/install/share/lua/5.1/trepl/init.lua:385: in function 'require'
...ado/usr/local/torch/install/share/lua/5.1/matio/init.lua:3: in main chunk
[C]: at 0x0044e490
[C]: in function 'pcall'
...ado/usr/local/torch/install/share/lua/5.1/trepl/init.lua:385: in function 'require'
...rado/usr/local/torch/install/share/lua/5.1/xlua/init.lua:360: in function <...rado/usr/local/torch/install/share/lua/5.1/xlua/init.lua:360>
[C]: in function 'pcall'
...rado/usr/local/torch/install/share/lua/5.1/xlua/init.lua:361: in function 'xrequire'
main.lua:16: in main chunk
[C]: in function 'dofile'
...ocal/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
[C]: at 0x004059e0
...ado/usr/local/torch/install/share/lua/5.1/trepl/init.lua:389: ...ado/usr/local/torch/install/share/lua/5.1/trepl/init.lua:389: ...rado/usr/local/torch/install/share/lua/5.1/matio/ffi.lua:24: Could not find libmatio. Please make sure that you installd MatIO and you have the shared libraries (libmatio.so or libmatio.dylib) in your library path
warning: could not be loaded (is it installed?)
Scanning directory for data...
Found 5 images
5 images require a face detector
Initialising python libs...
Initialising detector...
/home/f_prado/usr/local/torch/install/bin/luajit: getrf : Lapack library not found in compile time
at /home/f_prado/usr/local/torch/pkg/torch/lib/TH/generic/THLapack.c:140
stack traceback:
[C]: at 0x7f8a18f8e350
[C]: in function 'inverse'
./utils.lua:51: in function 'transform'
./utils.lua:59: in function 'crop'
main.lua:58: in main chunk
[C]: in function 'dofile'
...ocal/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
[C]: at 0x004059e0
ls: no se puede acceder a '.txt': No such file or directory
ls: no se puede acceder a '.raw': No such file or directory

I have libmatio.so.2 in /lib64/

Can you please help me?

Answer 1 · 2018-12-17T16:35:09.000Z

Hmm, interesting. Thanks for your detailed report. Instead of fixing the issue, it might be easier to remove any dependency on matio from the face-alignment code. VRN doesn't depend need to access any MAT files.

Answer 2 · 2018-12-18T19:06:31.000Z

Hi Aaron
eliminated line 16 in main.lua:
-- xrequire('matio') -- matlab

now the run.sh execution gives:
[f_prado@localhost vrn]$ ./run.sh
Scanning directory for data...
Found 5 images
5 images require a face detector
Initialising python libs...
Initialising detector...
/home/f_prado/usr/local/torch/install/bin/luajit: getrf : Lapack library not found in compile time
at /home/f_prado/usr/local/torch/pkg/torch/lib/TH/generic/THLapack.c:140
stack traceback:
[C]: at 0x7f47635b9350
[C]: in function 'inverse'
./utils.lua:51: in function 'transform'
./utils.lua:59: in function 'crop'
main.lua:58: in main chunk
[C]: in function 'dofile'
...ocal/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
[C]: at 0x004059e0
ls: no se puede acceder a '.txt': No such file or directory
ls: no se puede acceder a '.raw': No such file or directory

Any idea how to progress?
Thanks in advance.

Answer 3 · 2018-12-18T19:12:42.000Z

Not sure about that one. I have not seen it under CentOS. Maybe try making sure *blas-devel is installed and trying to reinstall Torch7? fpradom writes:

…

Hi Aaron eliminated line 16 in main.lua: -- xrequire('matio') -- matlab now the run.sh execution gives: ***@***.*** vrn]$ ./run.sh Scanning directory for data... Found 5 images 5 images require a face detector Initialising python libs... Initialising detector... /home/f_prado/usr/local/torch/install/bin/luajit: getrf : Lapack library not found in compile time at /home/f_prado/usr/local/torch/pkg/torch/lib/TH/generic/THLapack.c:140 stack traceback: [C]: at 0x7f47635b9350 [C]: in function 'inverse' ./utils.lua:51: in function 'transform' ./utils.lua:59: in function 'crop' main.lua:58: in main chunk [C]: in function 'dofile' ...ocal/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk [C]: at 0x004059e0 ls: no se puede acceder a '*.txt': No such file or directory ls: no se puede acceder a '*.raw': No such file or directory Any idea how to progress? Thanks in advance.

-- Aaron Jackson - M6PIU http://aaronsplace.co.uk/

Answer 4 · 2018-12-18T19:15:50.000Z

Will try,
Thanks

Answer 5 · 2018-12-19T15:49:37.000Z

Hi Aaron:

Solved the previous error by executing:
CMAKE_LIBRARY_PATH=/opt/OpenBLAS/include:/opt/OpenBLAS/lib:$CMAKE_LIBRARY_PATH luarocks install torch

now a new error appears:

f_prado@localhost vrn]$ ./run.sh
Scanning directory for data...
/home/f_prado/usr/local/torch/install/bin/luajit: ...o/usr/local/torch/install/share/lua/5.1/torch/Tensor.lua:466: Wrong size for view. Input size: 1x68x2. Output size: 2
stack traceback:
[C]: in function 'error'
...o/usr/local/torch/install/share/lua/5.1/torch/Tensor.lua:466: in function 'view'
./utils.lua:178: in function 'bounding_box'
./utils.lua:300: in function 'getFileList'
main.lua:25: in main chunk
[C]: in function 'dofile'
...ocal/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
[C]: at 0x004059e0
ls: no se puede acceder a '.txt': No such file or directory
ls: no se puede acceder a '.raw': No such file or directory
Any suggestion?
Thanks in advance

Answer 6 · 2018-12-19T15:56:05.000Z

Can you make sure nothing else changed in utils.lua? other than the matio stuff

Answer 7 · 2018-12-19T19:34:01.000Z

I have restored the original utils.lua and removed the above indicated lines.
The error is the same

Answer 8 · 2018-12-19T19:49:51.000Z

Hmm, I can't think why that would happen.
Check that run.sh still contains

th main.lua -model 2D-FAN-300W.t7 \
   -input ../$INPUT/ \
   -detectFaces true \
   -mode generate \
   -output ../$INPUT/ \
   -device gpu \
-outputFormat txt

and that you didn't remove or change any of the lines there.

Are you using the included example images? Or are you using a different folder, perhaps one that contains some txt files?

Answer 9 · 2018-12-21T18:58:32.000Z

Hi Aaron:
All lines are present in run.sh
I am using your images in the examples directory and there are not .txt files but .t7, one for each .jpg image.

Answer 10 · 2018-12-21T19:08:32.000Z

in the file ../usr/local/torch/install/share/lua/5.1/torch/Tensor.lua the line that gives the error is in a function that has as comment "-- TODO : This should be implemented in TH and and wrapped."
Can this have an influence in the process?

Answer 11 · 2018-12-21T22:09:08.000Z

It shouldn't be calling it anyway. The t7 files shouldn't be there so I am wondering if you had run it outside of the script in the past for testing or something. Anyway, delete the t7 files and try again. The face-alignment code thinks they are facial landmarks and is trying to use them as a bounding box.

Answer 12 · 2018-12-22T13:16:17.000Z

Hi Aaron:
You are right. I have made many tries. After deleting the t7 files the program has continue until an out of memory error appeared.
I have changed the gpu device with cpu, and now the output is:
[f_prado@localhost vrn]$ ./run.sh
Scanning directory for data...
Found 5 images
5 images require a face detector
Initialising python libs...
Initialising detector...
Cropped and scaled AFLW_image00046.jpg
Cropped and scaled AFLW_image00095.jpg
Cropped and scaled AFLW_image00190.jpg
Cropped and scaled AFLW_image00656.jpg
Cropped and scaled asj.jpg
Processed AFLW_image00190.
Processed asj.
Processed AFLW_image00046.
Processed AFLW_image00656.
Processed AFLW_image00095.
Traceback (most recent call last):
File "../vis.py", line 20, in
t = vv.imshow(im)
File "/usr/lib/python2.7/site-packages/visvis/functions/imshow.py", line 57, in imshow
axes = vv.gca()
File "/usr/lib/python2.7/site-packages/visvis/functions/gca.py", line 19, in gca
f = vv.gcf()
File "/usr/lib/python2.7/site-packages/visvis/functions/gcf.py", line 23, in gcf
return vv.figure()
File "/usr/lib/python2.7/site-packages/visvis/functions/figure.py", line 23, in figure
backends.use()
File "/usr/lib/python2.7/site-packages/visvis/backends/init.py", line 229, in use
raise RuntimeError("None of the backends could be loaded. "+tmp)
RuntimeError: None of the backends could be loaded. Install PySide, PyQt4, wxPython, GTK, or fltk.
Traceback (most recent call last):
File "../vis.py", line 20, in
t = vv.imshow(im)
File "/usr/lib/python2.7/site-packages/visvis/functions/imshow.py", line 57, in imshow
axes = vv.gca()
File "/usr/lib/python2.7/site-packages/visvis/functions/gca.py", line 19, in gca
f = vv.gcf()
File "/usr/lib/python2.7/site-packages/visvis/functions/gcf.py", line 23, in gcf
return vv.figure()
File "/usr/lib/python2.7/site-packages/visvis/functions/figure.py", line 23, in figure
backends.use()
File "/usr/lib/python2.7/site-packages/visvis/backends/init.py", line 229, in use
raise RuntimeError("None of the backends could be loaded. "+tmp)
RuntimeError: None of the backends could be loaded. Install PySide, PyQt4, wxPython, GTK, or fltk.
Traceback (most recent call last):
File "../vis.py", line 20, in
t = vv.imshow(im)
File "/usr/lib/python2.7/site-packages/visvis/functions/imshow.py", line 57, in imshow
axes = vv.gca()
File "/usr/lib/python2.7/site-packages/visvis/functions/gca.py", line 19, in gca
f = vv.gcf()
File "/usr/lib/python2.7/site-packages/visvis/functions/gcf.py", line 23, in gcf
return vv.figure()
File "/usr/lib/python2.7/site-packages/visvis/functions/figure.py", line 23, in figure
backends.use()
File "/usr/lib/python2.7/site-packages/visvis/backends/init.py", line 229, in use
raise RuntimeError("None of the backends could be loaded. "+tmp)
RuntimeError: None of the backends could be loaded. Install PySide, PyQt4, wxPython, GTK, or fltk.
Traceback (most recent call last):
File "../vis.py", line 20, in
t = vv.imshow(im)
File "/usr/lib/python2.7/site-packages/visvis/functions/imshow.py", line 57, in imshow
axes = vv.gca()
File "/usr/lib/python2.7/site-packages/visvis/functions/gca.py", line 19, in gca
f = vv.gcf()
File "/usr/lib/python2.7/site-packages/visvis/functions/gcf.py", line 23, in gcf
return vv.figure()
File "/usr/lib/python2.7/site-packages/visvis/functions/figure.py", line 23, in figure
backends.use()
File "/usr/lib/python2.7/site-packages/visvis/backends/init.py", line 229, in use
raise RuntimeError("None of the backends could be loaded. "+tmp)
RuntimeError: None of the backends could be loaded. Install PySide, PyQt4, wxPython, GTK, or fltk.
Traceback (most recent call last):
File "../vis.py", line 20, in
t = vv.imshow(im)
File "/usr/lib/python2.7/site-packages/visvis/functions/imshow.py", line 57, in imshow
axes = vv.gca()
File "/usr/lib/python2.7/site-packages/visvis/functions/gca.py", line 19, in gca
f = vv.gcf()
File "/usr/lib/python2.7/site-packages/visvis/functions/gcf.py", line 23, in gcf
return vv.figure()
File "/usr/lib/python2.7/site-packages/visvis/functions/figure.py", line 23, in figure
backends.use()
File "/usr/lib/python2.7/site-packages/visvis/backends/init.py", line 229, in use
raise RuntimeError("None of the backends could be loaded. "+tmp)
RuntimeError: None of the backends could be loaded. Install PySide, PyQt4, wxPython, GTK, or fltk.

Are PySide, PyQt4, wxPython, GTK, or fltk. necessary?

Answer 13 · 2018-12-22T13:20:18.000Z

After this, I have .txt files in examples, jpg files in scaled and .raw files in output.

Answer 14 · 2018-12-22T13:23:44.000Z

Doesn't look like an out of memory error to me. Are you connecting to a linux box without X forwarding? Instead you could use the raw2obj script and then open with something like Meshlab.

Answer 15 · 2018-12-22T19:02:48.000Z

Hi Aaron:
When I installed the nvidia driver I mused uninstall the xorg-x11 library.
I proved to use the raw2obj script with following script:

ls -1 output/*.raw | sed 's/.raw//' | while read fname ; do
python raw2obj.py
--volume output/"$i".raw
--image examples/scaled/"$i".jpg
--obj output/"$i".obj
done

but this give me following error:
[f_prado@localhost vrn]$ ./r2o.sh
Traceback (most recent call last):
File "raw2obj.py", line 2, in
import scipy.misc
ImportError: No module named scipy.misc

I can not find scipy.misc in my HD

Answer 16 · 2018-12-22T19:13:41.000Z

Sorry, I instaled scipy and now the error is:

[f_prado@localhost vrn]$ ./r2o.sh
Traceback (most recent call last):
File "raw2obj.py", line 2, in
import scipy.misc
ImportError: No module named scipy.misc
Haw can I install mcubes module?

Answer 17 · 2018-12-22T19:28:09.000Z

I make a mistake in the comment .
The output is:
[f_prado@localhost vrn]$ ./r2o.sh
Traceback (most recent call last):
File "raw2obj.py", line 3, in
import mcubes
ImportError: No module named mcubes

Answer 18 · 2018-12-22T19:40:14.000Z

Hi Aaron
Found the PyMcubes and installed.
Now all seems to run well.

but this give me following error: [f_prado@localhost vrn]$ ./r2o.sh Traceback (most recent call last): File "raw2obj.py", line 2, in import scipy.misc ImportError: No module named scipy.misc

but this give me following error:
[f_prado@localhost vrn]$ ./r2o.sh
Traceback (most recent call last):
File "raw2obj.py", line 2, in
import scipy.misc
ImportError: No module named scipy.misc