nischi/MMM-Face-Reco-DNN

Not able to start up

Closed this issue · 5 comments

chienhao10 commented

Device Info
Raspberry Pi 4B+ (running Buster)
Camera: CSI Pi Cam

My steps:

  1. Followed https://github.com/cyysky/OpenCV-4.1.2-for-Raspbian installed without issue
  2. pip3 install dlib
  3. pip3 install face_recognition
  4. pip3 install imutils
  5. pip3 install numpy
  6. cd ~/MagicMirror/modules/
    git clone https://github.com/nischi/MMM-Face-Reco-DNN.git
    cd MMM-Face-Reco-DNN
    npm install
  7. Took 24 photos with iPhone XS Max and put them under MagicMirror\modules\MMM-Face-Reco-DNN\dataset\ch\
  8. Used python3 encode.py -i ../dataset/ -e encodings.pickle -d hog (seems cnn will get out of memory issue due to large photo)
  9. Edited config.js, and did a full copy from your guide
  10. git clone https://github.com/nischi/MMM-MotionControl.git in the module folder
  11. Copied config for MotionControl guide, and made useMMMFaceRecoDNN: true,
  12. Checked pi cam (CSI) is working
  13. npm start got me "ERROR:gl_surface_presentation_helper.cc(260)] GetVSyncParametersIfAvailable() failed for 1 times!" to 3 times

What I did after:

I had this

Dependency Versions
OpenCV 4.5.0
dlib 19.23.11
face_recognition 1.3.0
imutils 0.5.4

Then I installed 'pip3 install opencv-python==4.4.0.46' still got me same error.

### Config.js as below
/* MagicMirror² Config Sample
*

  • By Michael Teeuw https://michaelteeuw.nl

  • MIT Licensed.

  • For more information on how you can configure this file

  • see https://docs.magicmirror.builders/configuration/introduction.html

  • and https://docs.magicmirror.builders/modules/configuration.html
    */
    let config = {
    address: "localhost", // Address to listen on, can be:
    // - "localhost", "127.0.0.1", "::1" to listen on loopback interface
    // - another specific IPv4/6 to listen on a specific interface
    // - "0.0.0.0", "::" to listen on any interface
    // Default, when address config is left out or empty, is "localhost"
    port: 8080,
    basePath: "/", // The URL path where MagicMirror² is hosted. If you are using a Reverse proxy
    // you must set the sub path here. basePath must end with a /
    ipWhitelist: ["127.0.0.1", "::ffff:127.0.0.1", "::1"], // Set [] to allow all IP addresses
    // or add a specific IPv4 of 192.168.1.5 :
    // ["127.0.0.1", "::ffff:127.0.0.1", "::1", "::ffff:192.168.1.5"],
    // or IPv4 range of 192.168.3.0 --> 192.168.3.15 use CIDR format :
    // ["127.0.0.1", "::ffff:127.0.0.1", "::1", "::ffff:192.168.3.0/28"],

    useHttps: false, // Support HTTPS or not, default "false" will use HTTP
    httpsPrivateKey: "", // HTTPS private key path, only require when useHttps is true
    httpsCertificate: "", // HTTPS Certificate path, only require when useHttps is true

    language: "en",
    locale: "en-US",
    logLevel: ["INFO", "LOG", "WARN", "ERROR"], // Add "DEBUG" for even more logging
    timeFormat: 24,
    units: "metric",
    // serverOnly: true/false/"local" ,
    // local for armv6l processors, default
    // starts serveronly and then starts chrome browser
    // false, default for all NON-armv6l devices
    // true, force serveronly mode, because you want to.. no UI on this device

    modules: [
    {
    module: "alert",
    },
    {
    module: "updatenotification",
    position: "top_bar"
    },
    {
    module: "clock",
    position: "top_left"
    },
    {
    module: "calendar",
    header: "台灣月曆",
    position: "top_left",
    config: {
    calendars: [
    {
    symbol: "taiwan-calendar",
    url: "https://calendar.google.com/calendar/ical/zh-tw.taiwan%23holiday@group.v.calendar.google.com/public/basic.ics"
    }
    ]
    }
    },
    {
    module: "compliments",
    position: "lower_third",
    config: {
    updateInterval: 10000,
    compliments: {
    morning: [
    "Good Morning!",
    "Let's Go Back To Sleep!"
    ],
    afternoon: [
    "Good Afternoon",
    "Tea Time?"
    ],
    "....-07-14": ["HIII"],
    "....-09-30": ["HII"],
    evening: [
    "Good Evening!",
    "It's Getting Late"
    ]
    }
    }
    },
    {
    module: "weather",
    position: "top_right",
    config: {
    weatherProvider: "openweathermap",
    type: "current",
    location: "林口",
    locationID: "1665148", //ID from http://bulk.openweathermap.org/sample/city.list.json.gz; unzip the gz file and find your city
    apiKey: "c48217263d85fd0798f090dd80118c23"
    }
    },
    {
    module: "weather",
    position: "top_right",
    header: "Weather Forecast",
    config: {
    weatherProvider: "openweathermap",
    type: "forecast",
    location: "林口",
    locationID: "1665148", //ID from http://bulk.openweathermap.org/sample/city.list.json.gz; unzip the gz file and find your city
    apiKey: "c48217263d85fd0798f090dd80118c23"
    }
    },
    {
    module: "newsfeed",
    position: "bottom_bar",
    config: {
    feeds: [
    {
    title: "ETtoday新聞雲",
    url: "http://feeds.feedburner.com/ettoday/realtime?format=xml"
    //url: "https://rss.nytimes.com/services/xml/rss/nyt/HomePage.xml"
    }
    ],
    showSourceTitle: true,
    showPublishDate: true,
    broadcastNewsFeeds: true,
    broadcastNewsUpdates: true
    }
    },
    {
    module: 'MMM-DynamicWeather',
    position: 'fullscreen_above',
    config: { // See https://github.com/scottcl88/MMM-DynamicWeather for more information.
    api_key: "c48217263d85fd0798f090dd80118c23",
    locationID: "1665148",
    effects: [
    {
    month: "7",
    day: "14",
    year: "0",
    images: ["heart1.png", "heart2.png"],
    direction: "up"
    }
    ],
    }
    },
    {
    module: 'MMM-Face-Reco-DNN',
    config: {
    // Logout 15 seconds after user was not detected any more
    // If they are detected within this period, the delay will start again
    logoutDelay: 15000,
    // How often the recognition starts in milliseconds
    // With a Raspberry Pi 3+ it works well every 2 seconds
    checkInterval: 2000,
    // Module set used for when there is no face detected ie no one is in front of the camera
    noFaceClass: 'noface',
    // Module set used for when there is an unknown/unrecognised face detected
    unknownClass: 'unknown',
    // Module set used for when there is a known/recognised face detected
    knownClass: 'known',
    // Module set used for strangers and if no user is detected
    defaultClass: 'default',
    // Set of modules which should be shown for any user ie when there is any face detected
    everyoneClass: 'everyone',
    // Set of modules that are always shown - show if there is a face or no face detected
    alwaysClass: 'always',
    // XML to recognize with haarcascade
    cascade: 'modules/MMM-Face-Reco-DNN/tools/haarcascade_frontalface_default.xml',
    // Pre-encoded pickle with the faces
    encodings: 'modules/MMM-Face-Reco-DNN/tools/encodings.pickle',
    // Use Raspberry Pi camera or another type
    // 1 = RasPi camera, 0 = other camera
    usePiCamera: 1,
    // If using another type of camera, you can choose
    // i.e. 0 = /dev/video0 or 'http://link.to/live'
    source: 0,
    // Rotate camera
    rotateCamera: 0,
    // Method of facial recognition
    // dnn = deep neural network, haar = haarcascade
    method: 'dnn',
    // Which face detection model to use
    // "hog" is less accurate but faster on CPUs
    // "cnn" is a more accurate deep-learning model which is GPU/CUDA accelerated
    detectionMethod: 'hog',
    // How long in milliseconds modules take to hide and show
    animationSpeed: 0,
    // Path to Python to run the face recognition
    // null or '' means default path
    pythonPath: null,
    // Should a welcome message be shown using the MagicMirror alerts module?
    welcomeMessage: true,
    // Dictionary for person name mapping in welcome message
    // Allows for displaying name with complex character sets in welcome message e.g. jerome => Jérôme, hideyuki => 英之
    usernameDisplayMapping: null,
    // Capture new pictures of recognized people, if unknown we save it in folder "unknown"
    // So you can extend your dataset and retrain it afterwards for better recognitions
    extendDataset: false,
    // If extendDataset is true, you need to set the full path of the dataset
    dataset: 'modules/MMM-Face-Reco-DNN/dataset/',
    // How much distance between faces to consider it a match. Lower is more strict.
    tolerance: 0.6,
    // allow multiple concurrent user logins, 0=no, any other number is the maximum number of concurrent logins
    multiUser: 0,
    }
    },
    {
    module: 'MMM-MotionControl',
    config: {
    // Delay to turn the TV off
    delay: 15000,
    // Interval to check modules
    interval: 5000,
    // Use the module MMM-Facial-Recognition-OCV3
    useFacialRecognitionOCV3: false,
    // Use the module MMM-Face-Reco-DNN
    useMMMFaceRecoDNN: true,
    // Array where tv should be on
    ontime: []
    }
    },
    ]
    };

/*************** DO NOT EDIT THE LINE BELOW ***************/
if (typeof module !== "undefined") {module.exports = config;}

### Log as below:

magicmirror@2.19.0 start
DISPLAY="${DISPLAY:=:0}" ./node_modules/.bin/electron js/electron.js

[13.05.2022 18:44.14.650] [LOG] Starting MagicMirror: v2.19.0
[13.05.2022 18:44.14.658] [LOG] Loading config ...
[13.05.2022 18:44.14.665] [LOG] Loading module helpers ...
[13.05.2022 18:44.14.670] [LOG] No helper found for module: alert.
[13.05.2022 18:44.14.687] [LOG] Initializing new module helper ...
[13.05.2022 18:44.14.689] [LOG] Module helper loaded: updatenotification
[13.05.2022 18:44.14.690] [LOG] No helper found for module: clock.
[13.05.2022 18:44.14.876] [LOG] Initializing new module helper ...
[13.05.2022 18:44.14.877] [LOG] Module helper loaded: calendar
[13.05.2022 18:44.14.879] [LOG] No helper found for module: compliments.
[13.05.2022 18:44.14.880] [LOG] No helper found for module: weather.
[13.05.2022 18:44.14.905] [LOG] Initializing new module helper ...
[13.05.2022 18:44.14.906] [LOG] Module helper loaded: newsfeed
[13.05.2022 18:44.15.228] [LOG] Initializing new module helper ...
[13.05.2022 18:44.15.229] [LOG] Module helper loaded: MMM-DynamicWeather
[13.05.2022 18:44.15.236] [LOG] Initializing new module helper ...
[13.05.2022 18:44.15.237] [LOG] Module helper loaded: MMM-Face-Reco-DNN
[13.05.2022 18:44.15.238] [LOG] No helper found for module: MMM-MotionControl.
[13.05.2022 18:44.15.238] [LOG] All module helpers loaded.
[13.05.2022 18:44.15.341] [LOG] Starting server on port 8080 ...
[13.05.2022 18:44.15.359] [LOG] Server started ...
[13.05.2022 18:44.15.360] [LOG] Connecting socket for: updatenotification
[13.05.2022 18:44.15.361] [LOG] Starting module helper: updatenotification
[13.05.2022 18:44.15.361] [LOG] Connecting socket for: calendar
[13.05.2022 18:44.15.362] [LOG] Starting node helper for: calendar
[13.05.2022 18:44.15.363] [LOG] Connecting socket for: newsfeed
[13.05.2022 18:44.15.363] [LOG] Starting node helper for: newsfeed
[13.05.2022 18:44.15.364] [LOG] Connecting socket for: MMM-DynamicWeather
[13.05.2022 18:44.15.365] [LOG] Connecting socket for: MMM-Face-Reco-DNN
[13.05.2022 18:44.15.365] [LOG] Starting module helper: MMM-Face-Reco-DNN
[13.05.2022 18:44.15.366] [LOG] Sockets connected & modules started ...
[13.05.2022 18:44.15.705] [LOG] Launching application.
[13.05.2022 18:44.18.186] [LOG] Create new calendarfetcher for url: https://calendar.google.com/calendar/ical/zh-tw.taiwan%23holiday@group.v.calendar.google.com/public/basic.ics - Interval: 300000
[13.05.2022 18:44.18.264] [LOG] Create new newsfetcher for url: http://feeds.feedburner.com/ettoday/realtime?format=xml - Interval: 300000
[13.05.2022 18:44.18.278] [INFO] [MMM-DynamicWeather] Getting Holiday data
[13.05.2022 18:44.18.291] [INFO] [MMM-DynamicWeather] Getting Weather API data
[13.05.2022 18:44.18.340] [INFO] Checking git for module: MMM-DynamicWeather
[13.05.2022 18:44.18.409] [INFO] Checking git for module: MMM-Face-Reco-DNN
[13.05.2022 18:44.18.486] [INFO] Checking git for module: MMM-MotionControl
[13.05.2022 18:44.18.674] [INFO] Checking git for module: default
[13.05.2022 18:44.18.925] [INFO] Newsfeed-Fetcher: Broadcasting 50 items.
[13.05.2022 18:44.19.306] [INFO] Calendar-Fetcher: Broadcasting 26 events.
[13.05.2022 18:44.19.326] [INFO] [MMM-DynamicWeather] Received successful Weather API data
[13.05.2022 18:44.19.586] [INFO] [MMM-DynamicWeather] Received successful Holiday data
[4265:0513/184420.758194:ERROR:gl_surface_presentation_helper.cc(260)] GetVSyncParametersIfAvailable() failed for 1 times!
[13.05.2022 18:44.24.921] [LOG] [MMM-Face-Reco-DNN] loading encodings + face detector...
[13.05.2022 18:44.24.993] [LOG] [MMM-Face-Reco-DNN] starting video stream...
[13.05.2022 18:49.19.167] [INFO] Newsfeed-Fetcher: Broadcasting 50 items.
[13.05.2022 18:49.19.935] [INFO] Calendar-Fetcher: Broadcasting 26 events.
[13.05.2022 18:54.17.716] [INFO] [MMM-DynamicWeather] Getting Weather API data
[13.05.2022 18:54.18.127] [INFO] [MMM-DynamicWeather] Received successful Weather API data
[13.05.2022 18:54.19.393] [INFO] Newsfeed-Fetcher: Broadcasting 50 items.
[13.05.2022 18:54.20.663] [INFO] Calendar-Fetcher: Broadcasting 26 events.
[13.05.2022 18:56.08.567] [LOG] Shutting down server...

Side question is: where should i add USERS_LOGIN into? is it inside motioncontrol or the face-reco.

Thank you for the help

nischi commented

Hi @chienhao10

Can you test if open CV is correct installed?

python
>>> import cv2
>>> cv2.__version__
'4.0.0'
>>> exit()

But the error-message sounds like the chromium/electron is not installed properly. I remember that i needed to configure something for the browser in raspberry 4, but not sure anymore what.

The USERS_LOGIN should be catched in another Module than face-reco. If you write another module you can listen to this notification. MotionControl module uses GET_LOGGED_IN_USERS and LOGGED_IN_USERS to work.

chienhao10 commented

@nischi
Hello, thx for your time.
Yes, i've checked if CV was correctly installed. Firslty i got 4.5 version, from the tutorial provided. But then it wasn't working so I stight installed 4.4 from #79

Indeed, I've asked the same from MM community discord and got the same answer. It seems like a chromium issue, which i have no idea how to fix that. Since i have not mess areound with any of them after installed PI (only installed MM).
I've looked around and also tried re-install chromium but no luck. I have found the error message only appears when I alt-tab to another window. But if I don't alt-tab, MM just displaying black, the person from the MM disocrd said this is something about chromium overlay.
I am currently trying to reinstall with PI OS Legacy with desktop on another SD card to see if this can be solved. Please do let me know if memory came back to you about the configure for the browser. (Could it be GPU acceleration thing?)

Thx again.

nischi commented

@chienhao10
Yeah i think i remember now.

I changed the startup script in the package.json of MagicMirror

Instead of "start": "DISPLAY=\"${DISPLAY:=:0}\" ./node_modules/.bin/electron js/electron.js", i did "start": "DISPLAY=\"${DISPLAY:=:0}\" ./node_modules/.bin/electron --no-sandbox js/electron.js",

I added the --no-sandbox

Did that help?

chienhao10 commented

@nischi Unfortunately, it didnt work, the error still appear.
I also did a reinstall with the guide
Able to import cv2 and result version:

cv2. version
'4.5.0'

started MM with updatred package.json of MagicMirror root folder. Error still exist when i alt tab to another window, and the Face-Reco-DNN last LOG is "starting video steam..."

Thank you for the time, kindly close the issue if its not related to the module. It could be pi OS issue or pi 4B. I might need try purchase a pi4 or 3 then install with pi os legacy with desktop to try.

nischi commented

@chienhao10
Sorry to hear that it does not help. hope you will find an issue. i think it has nothing to do with the module. Would be awesome if you have a solution that you post it here. thank you.

Good luck to purchase a new one. I ordered it already in Januar and it does not look that i get one this year.