Two issues - one with mtcnn one with face_rectangles

Question

Two issues - one with mtcnn one with face_rectangles

DaddyWesker opened this issue 3 years ago · 15 comments

Hello.

Thanks for your lib. I'm currently trying to use your fer to detect emotions on webcam video. I've installed your fer using pip install fer. But i've met two issues:

When i'm initializing your detector as
detector = FER(mtcnn=True)
i'm getting
AttributeError: module 'keras.utils.generic_utils' has no attribute 'to_snake_case'
tensorflow 2.5.0, keras 2.4.3
SInce i wanted only to use your model as emotion detector not face detector, i wanted to test your detector.detect_emotions with parameter face_rectangles, so code looks like this

video_capture = cv2.VideoCapture(0)
detector = FER()
while True:
    _, frame = video_capture.read() 
    box = detector.find_faces(frame)
    emotions  = detector.detect_emotions(frame, face_rectangles=box)

So, i'm trying to use fer's find_faces to test how it performs with only emotion detect. Unfortunately, on the line
emotions = detector.detect_emotions(frame, face_rectangles=box)
i'm getting
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
on the line
if not face_rectangles:
in the file fer.py.

My understanding that it should be this if instead
if (face_rectangles is None):
With this if it works fine. Could you please change this line in your project so other people, installed fer with pip, could use it with their own face detect?

Answer 1 · 2021-07-09T07:10:38.000Z

Oh, yet another problem.
3. When using detector.top_emotion(frame) there is a chance that there won't be any face detected and you'll receive an error
IndexError: list index out of range on the line top_emotion = top_emotions[0] in the file fer.py. Probably, you should add some sort of 'if' here?

Answer 2 · 2021-07-09T09:15:45.000Z

Thank you, first off, for the suggestions!

The first issue is related to ipazc/mtcnn#114, which FER depends on. Try locking tensorflow to 2.4.1 and keras to 2.4.3 or this proposed solution: https://stackoverflow.com/a/67797729/6256888.

Regarding the second issue and third issues - good find! These have been fixed in v21.0.1. Please upgrade with pip install -U fer.

Answer 3 · 2021-07-09T09:36:43.000Z

Upgraded fer. THanks

ABout first issue. I've downgraded tf to 2.4.1 (keras already is 2.4.3). Now i've got error


Traceback (most recent call last):
  File "/home/daddywesker/EmotionRecognition/FER/testing_fer.py", line 9, in <module>
    detector = FER(mtcnn=True)
  File "/home/daddywesker/anaconda3/envs/fer/lib/python3.7/site-packages/fer/fer.py", line 98, in __init__
    from mtcnn.mtcnn import MTCNN
  File "/home/daddywesker/anaconda3/envs/fer/lib/python3.7/site-packages/mtcnn/__init__.py", line 26, in <module>
    from mtcnn.mtcnn import MTCNN
  File "/home/daddywesker/anaconda3/envs/fer/lib/python3.7/site-packages/mtcnn/mtcnn.py", line 37, in <module>
    from mtcnn.network.factory import NetworkFactory
  File "/home/daddywesker/anaconda3/envs/fer/lib/python3.7/site-packages/mtcnn/network/factory.py", line 26, in <module>
    from keras.layers import Input, Dense, Conv2D, MaxPooling2D, PReLU, Flatten, Softmax
  File "/home/daddywesker/anaconda3/envs/fer/lib/python3.7/site-packages/keras/__init__.py", line 20, in <module>
    from . import initializers
  File "/home/daddywesker/anaconda3/envs/fer/lib/python3.7/site-packages/keras/initializers/__init__.py", line 124, in <module>
    populate_deserializable_objects()
  File "/home/daddywesker/anaconda3/envs/fer/lib/python3.7/site-packages/keras/initializers/__init__.py", line 49, in populate_deserializable_objects
    LOCAL.GENERATED_WITH_V2 = tf.__internal__.tf2.enabled()
AttributeError: module 'tensorflow.compat.v2.__internal__' has no attribute 'tf2'

Process finished with exit code 1

Answer 4 · 2021-07-09T10:08:16.000Z

Can you report back if pip install tensorflow-estimator==2.1.0 fixes it? It appears related to tensorflow/tensorflow#37525.

Answer 5 · 2021-07-09T10:09:43.000Z

Well, i guess it wants me to further downgrade tensorflow?

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
tensorflow 2.4.1 requires tensorflow-estimator<2.5.0,>=2.4.0, but you have tensorflow-estimator 2.1.0 which is incompatible.
tensorflow-gpu 2.4.1 requires tensorflow-estimator<2.5.0,>=2.4.0, but you have tensorflow-estimator 2.1.0 which is incompatible.

Answer 6 · 2021-07-09T10:11:51.000Z

Otherwise trying this? https://exerror.com/module-tensorflow-compat-v2-internal-has-no-attribute-tf2/

Answer 7 · 2021-07-09T10:23:07.000Z

Well, here is the problem.
What the suggest

!pip install -U -q segmentation-models
!pip install -q tensorflow==2.1
!pip install -q keras==2.3.1
!pip install -q tensorflow-estimator==2.1.

PRoblem -

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
fer 21.0.1 requires keras==2.4.3, but you have keras 2.3.1 which is incompatible.

DOwngrading fer will return me to the the issues 2-3 =)

Answer 8 · 2021-07-09T10:41:48.000Z

Hi all,
@JustinShenk Could you perhaps run pip freeze whichever machine you're running it on and post that along with which CUDA version, OS specs you have. I suspect it has something to do with the CUDA version and not the tensorflow version so much because I've noticed a lot of people are struggling with this now and I think it might be due to CUDA updates.
I want to get the information for a system that you're running and try recreate it because it was working fine end of last year with the GPU, but now I can't get it to work either. I can detect my GPU fine and fer runs fine, it just cannot detect the GPU, so it's always using the CPU.

Answer 9 · 2021-07-09T11:21:59.000Z

Hey guys, some of these errors should have been fixed in the newly released mtcnn v0.1.1, now required in fer v21.0.2. This mtcnn version replaced all keras calls with tensorflow. Please pip install -U fer and let me know it goes.

@linguist89, I don't have a GPU in my computer unfortunately, so if you can't get the issue to show in a colab notebook, then hopefully others can help debug it.

Answer 10 · 2021-07-09T11:29:11.000Z

https://colab.research.google.com/github/justinshenk/fer/blob/master/fer-video-demo.ipynb

absl-py==0.12.0
alabaster==0.7.12
albumentations==0.1.12
altair==4.1.0
appdirs==1.4.4
argon2-cffi==20.1.0
arviz==0.11.2
astor==0.8.1
astropy==4.2.1
astunparse==1.6.3
async-generator==1.10
atari-py==0.2.9
atomicwrites==1.4.0
attrs==21.2.0
audioread==2.1.9
autograd==1.3
Babel==2.9.1
backcall==0.2.0
beautifulsoup4==4.6.3
bleach==3.3.0
blis==0.4.1
bokeh==2.3.2
Bottleneck==1.3.2
branca==0.4.2
bs4==0.0.1
CacheControl==0.12.6
cached-property==1.5.2
cachetools==4.2.2
catalogue==1.0.0
certifi==2021.5.30
cffi==1.14.5
cftime==1.5.0
chardet==3.0.4
click==7.1.2
cloudpickle==1.3.0
cmake==3.12.0
cmdstanpy==0.9.5
colorcet==2.0.6
colorlover==0.3.0
community==1.0.0b1
contextlib2==0.5.5
convertdate==2.3.2
coverage==3.7.1
coveralls==0.5
crcmod==1.7
cufflinks==0.17.3
cupy-cuda101==9.1.0
cvxopt==1.2.6
cvxpy==1.0.31
cycler==0.10.0
cymem==2.0.5
Cython==0.29.23
daft==0.0.4
dask==2.12.0
datascience==0.10.6
debugpy==1.0.0
decorator==4.4.2
defusedxml==0.7.1
descartes==1.1.0
dill==0.3.4
distributed==1.25.3
dlib==19.18.0
dm-tree==0.1.6
docopt==0.6.2
docutils==0.17.1
dopamine-rl==1.0.5
earthengine-api==0.1.272
easydict==1.9
ecos==2.0.7.post1
editdistance==0.5.3
en-core-web-sm==2.2.5
entrypoints==0.3
ephem==4.0.0.2
et-xmlfile==1.1.0
fa2==0.3.5
fastai==1.0.61
fastdtw==0.3.4
fastprogress==1.0.0
fastrlock==0.6
fbprophet==0.7.1
feather-format==0.4.1
fer==21.0.2
filelock==3.0.12
firebase-admin==4.4.0
fix-yahoo-finance==0.0.22
Flask==1.1.4
flatbuffers==1.12
folium==0.8.3
future==0.16.0
gast==0.4.0
GDAL==2.2.2
gdown==3.6.4
gensim==3.6.0
geographiclib==1.52
geopy==1.17.0
gin-config==0.4.0
glob2==0.7
google==2.0.3
google-api-core==1.26.3
google-api-python-client==1.12.8
google-auth==1.32.1
google-auth-httplib2==0.0.4
google-auth-oauthlib==0.4.4
google-cloud-bigquery==1.21.0
google-cloud-bigquery-storage==1.1.0
google-cloud-core==1.0.3
google-cloud-datastore==1.8.0
google-cloud-firestore==1.7.0
google-cloud-language==1.2.0
google-cloud-storage==1.18.1
google-cloud-translate==1.5.0
google-colab==1.0.0
google-pasta==0.2.0
google-resumable-media==0.4.1
googleapis-common-protos==1.53.0
googledrivedownloader==0.4
graphviz==0.10.1
greenlet==1.1.0
grpcio==1.34.1
gspread==3.0.1
gspread-dataframe==3.0.8
gym==0.17.3
h5py==3.1.0
HeapDict==1.0.1
hijri-converter==2.1.3
holidays==0.10.5.2
holoviews==1.14.4
html5lib==1.0.1
httpimport==0.5.18
httplib2==0.17.4
httplib2shim==0.0.3
humanize==0.5.1
hyperopt==0.1.2
ideep4py==2.0.0.post3
idna==2.10
imageio==2.4.1
imagesize==1.2.0
imbalanced-learn==0.4.3
imblearn==0.0
imgaug==0.2.9
importlib-metadata==4.6.0
importlib-resources==5.2.0
imutils==0.5.4
inflect==2.1.0
iniconfig==1.1.1
install==1.3.4
intel-openmp==2021.3.0
intervaltree==2.1.0
ipykernel==4.10.1
ipython==5.5.0
ipython-genutils==0.2.0
ipython-sql==0.3.9
ipywidgets==7.6.3
itsdangerous==1.1.0
jax==0.2.13
jaxlib==0.1.66+cuda110
jdcal==1.4.1
jedi==0.18.0
jieba==0.42.1
Jinja2==2.11.3
joblib==1.0.1
jpeg4py==0.1.4
jsonschema==2.6.0
jupyter==1.0.0
jupyter-client==5.3.5
jupyter-console==5.2.0
jupyter-core==4.7.1
jupyterlab-pygments==0.1.2
jupyterlab-widgets==1.0.0
kaggle==1.5.12
kapre==0.3.5
Keras==2.4.3
keras-nightly==2.5.0.dev2021032900
Keras-Preprocessing==1.1.2
keras-vis==0.4.1
kiwisolver==1.3.1
korean-lunar-calendar==0.2.1
librosa==0.8.1
lightgbm==2.2.3
llvmlite==0.34.0
lmdb==0.99
LunarCalendar==0.0.9
lxml==4.2.6
Markdown==3.3.4
MarkupSafe==2.0.1
matplotlib==3.2.2
matplotlib-inline==0.1.2
matplotlib-venn==0.11.6
missingno==0.4.2
mistune==0.8.4
mizani==0.6.0
mkl==2019.0
mlxtend==0.14.0
more-itertools==8.8.0
moviepy==0.2.3.5
mpmath==1.2.1
msgpack==1.0.2
mtcnn==0.1.1
multiprocess==0.70.12.2
multitasking==0.0.9
murmurhash==1.0.5
music21==5.5.0
natsort==5.5.0
nbclient==0.5.3
nbconvert==5.6.1
nbformat==5.1.3
nest-asyncio==1.5.1
netCDF4==1.5.7
networkx==2.5.1
nibabel==3.0.2
nltk==3.2.5
notebook==5.3.1
numba==0.51.2
numexpr==2.7.3
numpy==1.19.5
nvidia-ml-py3==7.352.0
oauth2client==4.1.3
oauthlib==3.1.1
okgrade==0.4.3
opencv-contrib-python==4.1.2.30
opencv-python==4.1.2.30
openpyxl==2.5.9
opt-einsum==3.3.0
osqp==0.6.2.post0
packaging==20.9
palettable==3.3.0
pandas==1.1.5
pandas-datareader==0.9.0
pandas-gbq==0.13.3
pandas-profiling==1.4.1
pandocfilters==1.4.3
panel==0.11.3
param==1.10.1
parso==0.8.2
pathlib==1.0.1
patsy==0.5.1
pexpect==4.8.0
pickleshare==0.7.5
Pillow==7.1.2
pip-tools==4.5.1
plac==1.1.3
plotly==4.4.1
plotnine==0.6.0
pluggy==0.7.1
pooch==1.4.0
portpicker==1.3.9
prefetch-generator==1.0.1
preshed==3.0.5
prettytable==2.1.0
progressbar2==3.38.0
prometheus-client==0.11.0
promise==2.3
prompt-toolkit==1.0.18
protobuf==3.17.3
psutil==5.4.8
psycopg2==2.7.6.1
ptyprocess==0.7.0
py==1.10.0
pyarrow==3.0.0
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycocotools==2.0.2
pycparser==2.20
pyct==0.4.8
pydata-google-auth==1.2.0
pydot==1.3.0
pydot-ng==2.0.0
pydotplus==2.0.2
PyDrive==1.3.1
pyemd==0.5.1
pyerfa==2.0.0
pyglet==1.5.0
Pygments==2.6.1
pygobject==3.26.1
pymc3==3.11.2
PyMeeus==0.5.11
pymongo==3.11.4
pymystem3==0.2.0
PyOpenGL==3.1.5
pyparsing==2.4.7
pyrsistent==0.18.0
pysndfile==1.3.8
PySocks==1.7.1
pystan==2.19.1.1
pytest==3.6.4
python-apt==0.0.0
python-chess==0.23.11
python-dateutil==2.8.1
python-louvain==0.15
python-slugify==5.0.2
python-utils==2.5.6
pytz==2018.9
pyviz-comms==2.1.0
PyWavelets==1.1.1
PyYAML==3.13
pyzmq==22.1.0
qdldl==0.1.5.post0
qtconsole==5.1.1
QtPy==1.9.0
regex==2019.12.20
requests==2.23.0
requests-oauthlib==1.3.0
resampy==0.2.2
retrying==1.3.3
rpy2==3.4.5
rsa==4.7.2
scikit-image==0.16.2
scikit-learn==0.22.2.post1
scipy==1.4.1
screen-resolution-extra==0.0.0
scs==2.1.4
seaborn==0.11.1
semver==2.13.0
Send2Trash==1.7.1
setuptools-git==1.2
Shapely==1.7.1
simplegeneric==0.8.1
six==1.15.0
sklearn==0.0
sklearn-pandas==1.8.0
smart-open==5.1.0
snowballstemmer==2.1.0
sortedcontainers==2.4.0
SoundFile==0.10.3.post1
spacy==2.2.4
Sphinx==1.8.5
sphinxcontrib-serializinghtml==1.1.5
sphinxcontrib-websupport==1.2.4
SQLAlchemy==1.4.20
sqlparse==0.4.1
srsly==1.0.5
statsmodels==0.10.2
sympy==1.7.1
tables==3.4.4
tabulate==0.8.9
tblib==1.7.0
tensorboard==2.5.0
tensorboard-data-server==0.6.1
tensorboard-plugin-wit==1.8.0
tensorflow==2.5.0
tensorflow-datasets==4.0.1
tensorflow-estimator==2.5.0
tensorflow-gcs-config==2.5.0
tensorflow-hub==0.12.0
tensorflow-metadata==1.1.0
tensorflow-probability==0.13.0
termcolor==1.1.0
terminado==0.10.1
testpath==0.5.0
text-unidecode==1.3
textblob==0.15.3
Theano-PyMC==1.1.2
thinc==7.4.0
tifffile==2021.7.2
toml==0.10.2
toolz==0.11.1
torch==1.9.0+cu102
torchsummary==1.5.1
torchtext==0.10.0
torchvision==0.10.0+cu102
tornado==5.1.1
tqdm==4.41.1
traitlets==5.0.5
tweepy==3.10.0
typeguard==2.7.1
typing-extensions==3.7.4.3
tzlocal==1.5.1
uritemplate==3.0.1
urllib3==1.24.3
vega-datasets==0.9.0
wasabi==0.8.2
wcwidth==0.2.5
webencodings==0.5.1
Werkzeug==1.0.1
widgetsnbextension==3.5.1
wordcloud==1.5.0
wrapt==1.12.1
xarray==0.18.2
xgboost==0.90
xkit==0.0.0
xlrd==1.1.0
xlwt==1.3.0
yellowbrick==0.9.1
zict==2.0.0
zipp==3.4.1

nvcc --version:

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2020 NVIDIA Corporation
Built on Wed_Jul_22_19:09:09_PDT_2020
Cuda compilation tools, release 11.0, V11.0.221
Build cuda_11.0_bu.TC445_37.28845127_0

!nvidia-smi:
NVIDIA-SMI 470.42.01 Driver Version: 460.32.03 CUDA Version: 11.2

Answer 11 · 2021-07-12T05:56:53.000Z

Hey guys, some of these errors should have been fixed in the newly released mtcnn v0.1.1, now required in fer v21.0.2. This mtcnn version replaced all keras calls with tensorflow. Please pip install -U fer and let me know it goes.

Hello. I've updated mtcnn and fer and after that i'm getting this


Traceback (most recent call last):
  File "/home/daddywesker/EmotionRecognition/FER/testing_fer.py", line 7, in <module>
    detector = FER(mtcnn=True)
  File "/home/daddywesker/anaconda3/envs/fer/lib/python3.7/site-packages/fer/fer.py", line 120, in __init__
    self.__emotion_classifier = load_model(emotion_model, compile=compile)
  File "/home/daddywesker/anaconda3/envs/fer/lib/python3.7/site-packages/tensorflow_core/python/keras/saving/save.py", line 146, in load_model
    return hdf5_format.load_model_from_hdf5(filepath, custom_objects, compile)
  File "/home/daddywesker/anaconda3/envs/fer/lib/python3.7/site-packages/tensorflow_core/python/keras/saving/hdf5_format.py", line 166, in load_model_from_hdf5
    model_config = json.loads(model_config.decode('utf-8'))
AttributeError: 'str' object has no attribute 'decode'

Process finished with exit code 1

Deleting .decode('utf-8') part helps but i'm editing keras code this way. And there are some more places with such a problem. DOn't know if my solution is right.

Answer 12 · 2021-07-13T17:52:02.000Z

The issue is caused by having h5py>=3.0.0 with older version TensorFlow (2.3 or below). TensorFlow 2.3 and below require h5py==2.10.0. So either try pip install -U tensorflow or

pip uninstall h5py
pip install h5py==2.10.0

I think this has been fixed in fer==21.0.3. Please try uninstalling tensorflow and pip install -U fer.

Answer 13 · 2021-07-19T07:35:44.000Z

It is weird, but i can't install fer 21.0.3, pip doesn't see such a version.

Answer 14 · 2021-07-19T11:57:40.000Z

Sorry, there was an issue with the automatic deployment. Please try again with 21.0.3.

…

On Mon, Jul 19, 2021 at 9:35 AM DaddyWesker ***@***.***> wrote: It is weird, but i can't install fer 21.0.3, pip doesn't see such a version. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#29 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACOLMZAHS2JFUJKM2XE4ATDTYPIVVANCNFSM5ACDLWPA> .

Answer 15 · 2021-07-20T06:14:49.000Z

Now it works. THanks.