Click me to visit the Youtube channel
Click me to visit the Facebook Account
Click me to visit the Instagram Account
Click me to visit the Twitter Account
Click me to visit the TikTok Account
Change README Language: English ｜ Chinese
| Source | Status |
|---|---|
| helloworld.cpp |  |
| rpi4b-qemu-env |  |
| . | . |
| . | . |
SleepPanda is a sleep monitoring system based on Raspberry Pi 4b (bcm2711). SleepPanda uses a variety of sensors to collect sleep data of users, and analyzes and processes the data through dsp, convolutional neural network and other algorithms. Finally, the data processing results are presented to users in various ways.
- Heart rate, blood oxygen monitoring
- Sleep position monitoring
- Early warning of apnea syndrome
- Alarm threshold setting
| Name | ID | Role |
|---|---|---|
| Chengsen Dong | 2789825d | Embedded Engineer |
| Rui Liu | 2756138L | Project Management & Publicity & Software Engineer |
| Hui Wang | 2810919W | Hardware Engineer & Test Engineer |
| Yihan Wang | 2696441W | Software Engineer |
If you have any suggestions for this project, please email to xddcore@gmail.com
SleepPanda Demo Video
Click here to view the minutes of the development meeting
Click me to visit the Gantt chart
- Build the Github warehouse & README document, complete the basic development environment configuration, fix the Linux kernel source code, and test the compilation of the kernel module.
- build dev branch
- Buy all hardware (except touchscreen)
- Pitch Session PPT slide draft
- Final draft of Pitch Session PPT slides
- 14:20:00, 25/01/2023 Pitch Session
- Update the README document (synchronized with the development process)
- Buzzer driver development
- MAX30101 driver development
- 4K 30FPS camera (opencv c++ framework)
- Touch screen (QT-based GUI) development & top-level C++ logic
- Check memory management and memory leaks (using STL to manage memory)
- Synchronously update the contents of README_ZH.md & README.md (once a week)
- Simultaneous updates of the minutesmeeting (once a week)
- Creation of a Youtube channel to promote the SleepPanda project
- Pitch Session PPT slide draft
- Make cost accounting & original row selection Excel table
- Sound Sensor Driver Development
- WM8960 driver development (low priority)
- To do later...
- Synchronously update the contents of README_ZH.md & README.md (once a week)
- Simultaneous updates of the minutesmeeting (once a week)
- Create Gantt charts to organise tasks according to the classification of people
- Design SleepPanda Logo
- Pitch Session PPT slide final draft & pitch session speech preparation
- Make cost accounting & original row selection Excel table
- ink screen driver development (virtual zoo)
- To do later...
- Query the data sheets of all sensors/chips and upload them to the github repository
- MLX90640 driver development
- To do later...
- C++11 Standard Template Library
- Raspberry Pi 4B (4GB/8GB)｜bcm2711
- Linux distribution: Ubuntu Desktop 22.04.01 LTS (64Bit)
- Kernel version: 5.15.0-1023-raspi | Retrieve all Linux kernel
apt-cache search linux-raspi-headers - g++:
g++ (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0 - gcc:
gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0 - Unit Test Framework: Google Test + ctest
- Software that needs to be installed on a Windows computer:
1.Termius (for SSH)2.Microsoft Remote Desktop (for remote desktop) 3.Visual Studio code (universal editor) - Use Raspberry Pi Image to burn the image to SD card (ubuntu image:
https://ubuntu.com/raspberry-pi/desktop) - Use an external monitor, keyboard and mouse to complete the initialization of ubuntu. The system language is English, the created user name is
ubuntu, and the password isubuntu. - Packages to be installed: ssh vim gcc g++ screen htop git make
sudo apt-get update
sudo apt-get upgrade
sudo apt-get install ssh vim gcc g++ screen htop git make
Note: Note that the
sudo apt-get upgradecommand will upgrade the kernel to the latest version by default, please manually keep the kernel version to 5.15.0-1023-raspi
-
Open ubuntu remote desktop: Setting-share-open remote desktop-change username and password to
ubuntu4.1 Allow hdmi hot plugging to avoid inability to access remote desktop when hdmi is not plugged in. |
Unresolved! ! !|Try to use HDMI dummy load to resolve Enter the following command:sudo vim /boot/firmware/config.txt, and add the following content to the config.txt file:[hdmi] #allow remote desktop when no hdmi wire hdmi_force_hotplug=1 hdmi_drive=24.2 Fix the remote desktop login password to prevent the password from being randomly generated after each restart and unable to access the remote desktop Application-password&keys-default key&login-change password-do not set a password (leave blank)-finally cancel encryption 4.3 Use the Microsoft Remote Desktop software to access the Raspberry Pi desktop | Raspberry Pi's LAN IP + user name and password are
ubuntuHow to get Raspberry Pi IP: 1. If it is under the same router, it can be obtained by accessing the background of the router. 2. If you cannot access the router backend, obtain the Raspberry Pi’s LAN IP through
arp -a(windows)and other methods. -
Prevent Raspberry Pi from sleeping: Setting-Power-Nerver sleep
-
At this point, the Raspberry Pi is completely separated from the keyboard, mouse and monitor, and can be operated remotely by other computers
-
Use Termius software to SSH to connect to the LAN IP of the Raspberry Pi. Username:
ubuntuPassword:ubuntu
- Software to be installed on your Windows PC:
1.Termius (for SSH)2.VNC Viewer (for remote desktop, download link https://www.realvnc.com/en/connect/download/viewer/)3.Visual Studio code (Universal Editor)Raspberry Pi Image (image burning)5.Github Desktop(Optional) - Burn the
Raspberry Pi OS(32bit)image to the SD card using Raspberry Pi Image (Raspberry Pi OS image:https://www.raspberrypi.com/software/operating-systems/)
Before burning, click on the gear button in the bottom right hand corner to configure the following:
- check Set hostname -> raspberrypi.local
- check Enable SSH service -> Login with password
- Check Set username and password -> Username: pi | Password: pi
- Click Save
- Connect the Raspberry Pi to the power supply and start the Raspberry Pi. Wait for a while and then connect to the Raspberry Pi via SSH with
termiussoftware. - Enter the
sudo raspi-configcommand and use the direction keys to configure the following:
- Set the boot option to automatically log into the desktop.
System Options -> Boot/Auto Login -> Desktop Autologin- Open VNC Remote Desktop.
Interface Options -> VNC -> YES- Open the SPI port.
Interface Options -> SPI -> YES- Open the IIC port.
Interface Options -> I2C -> YES- Select Finish to save the settings.
- Enter the
rebootcommand to reboot the Raspberry Pi. (Wait a while after performing this step) - At this point, the Raspberry Pi is ready to be accessed via
VNC Viewerremote desktop.
The corresponding VNC package on the Raspberry Pi is
realvnc-vnc-server(Raspberry Pi OS comes with it, no need to install it)
- packages to be installed: ssh vim gcc g++ screen htop git make, execute the following command:
sudo apt-get update
sudo apt-get upgrade
sudo apt-get install ssh vim gcc g++ screen htop git make
- Enter the
rebootcommand to reboot the Raspberry Pi. At this point, the basic environment configuration is complete.
Considering that the computing power of the Raspberry Pi is insufficient, using a server will significantly improve compilation efficiency, version control, code backup, and collaborative work. We use a centralized server (I9-12900k+RTX3090TI+32GB DDR4+512G SSD) during the development of this project, and build a cross-compilation environment in the server.
- Install the aarch64-linux-gnu-cross-compilation toolchain, and other compilation tools required to compile the linux kernel
- Get the Raspberry Pi kernel source code
apt-get source linux-image-$(uname -r) #5.15.0-1023-raspi
- Compile the kernel source code
Ref: https://www.raspberrypi.com/documentation/computers/linux_kernel.html
- Compile
helloworldmodule code - Test the
helloworldmodule
- Compile
helloworldmodule code
make -C /usr/src/linux-headers-5.15.0-1023-raspi/ M=`pwd` modules
- Test the
helloworldmodule
Test result same as above
Q1: The module is not signed problem: It looks like your system vendor has enabled kernel module signature verification on your kernel, which means it won't load any modules that the vendor hasn't signed. In other words, your patched module is not (correctly) signed and the kernel will refuse to load it.
Solution: cmd run
CONFIG_MODULE_SIG=n
Q2: When sudo apt-get upgrade is executed, some packs (such as the linux kernel) are reserved and cannot be upgraded.
sudo apt-get --with-new-pkgs upgrade
sensor
- Speaker & Microphone: WM8960(Control:IIC;Audio:IIS)+Sound Sensor(Voltage Comparator Dout)-✅
- Body temperature (thermal imaging) & gesture control: MLX90640(IIC)-✅
- Heart rate & blood oxygen: MAX30101(IIC)-✅
- Sleeping position monitoring: 4K 30FPS camera (USB+OpenCV)-✅
Actuator
- Ink Screen:SSD1608(Virtual Zoo) (SPI)-✅
- Touch screen (QT-based GUI) (Video: HDMI, Touch: USB)
- Buzzer: Emergency abnormal alarm (High/Low Pin Level)-✅
other
- Several DuPont lines-✅
server Facing the world, it is responsible for storing user data and undertaking visits/animal exchanges between virtual zoos.
git clone https://github.com/xddcore/SleepPanda.git
Note: If you are a developer, remember to switch to the dev branch with the following command:
git checkout dev
git branch -l
#install cppunit-lib
sudo apt install -y libcppunit-dev
#include <cppunit/TestRunner.h>
because:
- Google test is relatively simple to use. Writing a test case does not need to be divided into two parts: declaration and definition, and there is no need to register the test suite yourself.
- The assert of google test is richer So this project uses google test (gtest) for unit testing
gtest dynamic link library path SleepPanda/tools/gtest/lib/, gtest header file path SleepPanda/tools/gtest/include/
# g++ build demo
# cd to workscape
cd /SleepPanda/src/app/gtest_demo
# build code
g++ -std=c++14 ./gtest_demo.cpp -I ../../../tools/gtest/include/ ../../../tools/gtest/lib/libgtest.so -lpthread -o gtest_demo
# Export the gtest dynamic link library to the system environment variable (temporary)
export LD_LIBRARY_PATH=../../../tools/gtest/lib/:$LD_LIBRARY_PATH
# run gtest demo
./gtest_demo
gtest dynamic link library pathSleepPanda/tools/gtest/rpios_32bit/lib/
gtest header file pathSleepPanda/tools/gtest/rpios_32bit/include/
- Compile using g++ (choose one of the two compilation methods, or try both)
# g++ build demo
# cd to workscape
cd ./SleepPanda/src/app/gtest_demo
# build code
g++ -std=c++14 ./gtest_demo.cpp -I ../../../tools/gtest/rpios_32bit/include/ ../../../tools/gtest/rpios_32bit/lib/libgtest.so -lpthread -o gtest_demo
# Export the gtest dynamic link library to the system environment variable (temporary)
export LD_LIBRARY_PATH=../../../tools/gtest/rpios_32bit/lib/:$LD_LIBRARY_PATH
# run gtest demo
./gtest_demo
- Compile with cmake (choose one of the two compilation methods, or try both)
# g++ build demo
# cd to workscape
cd ./SleepPanda/src/app/gtest_demo/build
# build code
cmake .. && make
# Export the gtest dynamic link library to the system environment variable (temporary)
export LD_LIBRARY_PATH=../../../tools/gtest/rpios_32bit/lib/:$LD_LIBRARY_PATH
# run gtest demo
./gtest_demo
Error: When compiling with cmake, the following error may occur:
CMake Error: The source "/home/pi/xddcore/SleepPanda/src/app/gtest_demo/CMakeLists.txt" does not match the source "/home/ubuntu/xddcore/SleepPanda/src/app/gtest_demo/CMakeLists.txt" used to generate cache. Re-run cmake with a different source directory.
Solution:
# remove /build dir
rm -rf build/
# create new /build dir
mkdir build
# go to build dir
cd build
# build code
cmake .. && make
# run gtest demo code
cd .. && ./gtest_demo
Theoretically, you will get the following running result:
./gtest_demo
[==========] Running 2 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 2 tests from FooTest
[ RUN ] FooTest.test_add
[ OK ] FooTest.test_add (0 ms)
[ RUN ] FooTest.test_minus
./gtest_demo.cpp:54: Failure
Expected equality of these values:
foo->GetNum()
Which is: 1
0
[ FAILED ] FooTest.test_minus (0 ms)
[----------] 2 tests from FooTest (1 ms total)
[----------] Global test environment tear-down
[==========] 2 tests from 1 test suite ran. (2 ms total)
[ PASSED ] 1 test.
[ FAILED ] 1 test, listed below:
[ FAILED ] FooTest.test_minus
# install dependencies
sudo apt-get install cmake
# install google test lib
# get source
git clone https://github.com/google/googletest.git
#build
cd googletest
mkdir build
cd build
cmake -DCMAKE_INSTALL_PREFIX=`pwd`/install -DBUILD_SHARED_LIBS=ON ..
make -j8
make install
# Then the corresponding dynamic library of gtest will be generated in the build/install directory
ls install /
# include lib
If you want to query GPIO more quickly, you can visit the following URL: https://pinout.xyz/pinout/
| Index | Layer | - | Comments |
|---|---|---|---|
| 7(Top) | GUI Interface Layer(QT) | - | Interaction with users |
| 6 | C++ Logic Layer | - | Opencv, TensorFlow Lite, MQTT Client, etc. |
| 5 | Hardware Driver Layer (Sensor Class) | - | Configuration/Driver of Hardware (Sensor, etc.) |
| - | |||
| 3 | Pigpiod Nested Word Communication Process | - | Drive rpi4b(bcm2711) register |
| 2 | System Call | - | Provided by the linux kernel |
| 1(Bottom) | Linux Kernel Modules | - | Allocate interrupt |
Ref: https://berndporr.github.io/realtime_cpp_coding/ https://github.com/berndporr/realtime_cpp_coding
Before you start, you need to execute the following commands to install the necessary dependencies. Here, I would like to express my sincere thanks to the developers of the pigpio library (https://github.com/joan2937/pigpio).
How to install Raspberry Pi OS
Run the following command to install piopig with one click
sudo apt-get install libpigpio-dev
Other operating system installation methods
apt-get install cmake
# Download and install latest version(pigpio)
wget https://github.com/joan2937/pigpio/archive/master.zip
unzip master.zip
cd pigpio-master
make
sudo make install
#If the Python part of the install fails it may be because you need the setup tools.
sudo apt install python-setuptools python3-setuptools
Finally, in order to verify whether pigpio lib is installed correctly, we also recommend you to run the following test command:
cd pigpio-master #(if you are not in this dir)
sudo ./x_pigpio # check C I/F
# To start the pigpio daemon
sudo pigpiod
./x_pigpiod_if2 # check C I/F to daemon
# Then you should get PASS
# To stop the pigpio daemon
sudo killall pigpiod
./x_pigpiod_if2 # check C I/F to daemon
# Then you should get 'pigpio initialisation failed (-2003).', because pigpio daemon not running.
Note: Every time you use pigpio, you should first run
sudo pigpiodto start the daemon.
Theoretically, you will get the following running result:
Testing pigpio C I/F
pigpio version 79.
Hardware revision 13644052.
Mode/PUD/read/write tests.
TEST 1.1 PASS (set mode, get mode: 0)
TEST 1.2 PASS (set pull up down, read: 1)
TEST 1.3 PASS (set pull up down, read: 0)
TEST 1.4 PASS (write, get mode: 1)
TEST 1.5 PASS (read: 0)
TEST 1.6 PASS (write, read: 1)
PWM dutycycle/range/frequency tests.
TEST 2.1 PASS (set PWM range, set/get PWM frequency: 10)
TEST 2.2 PASS (get PWM dutycycle: 0)
TEST 2.3 PASS (set PWM dutycycle, callback: 0)
TEST 2.4 PASS (get PWM dutycycle: 12
...etc
Q1: ERROR: initAllocDMAMem: mbox open failed(No such device or address)
:~$ sudo pigpiod
OR
:~$ ./x_pigpio
then get
:~$ initAllocDMAMem: mbox open failed(No such device or address)
Fixup: Run sudo modprobe vcio
OR
./x_pigpio is a program that uses mailboxes to allocate memory. If under certain conditions (e.g. qemu env), you won't have GPU memory to allocate. pigpio will use the mailbox to allocate DMA memory (unless you request a large buffer). When pigpiod is started with the -a1 option, page mapping is used to allocate DMA memory.
sudo pigpiod -a1 #force use PMAP allocate DMA memory
./x_pigpiod_if2 # check C I/F to daemon
Q2: ERROR: modprobe: FATAL: Module vcio not found in directory
:~$ sudo modprobe vcio
:~$ modprobe: FATAL: Module vcio not found in directory /lib/modules/5.15.0-1031-azure
Fixup: Run sudo apt-get install -y linux-modules-extra-$(uname -r)
OR
You need install Ubuntu Desktop Image, and then you will have vcio(videocore io)
Q3: ERROR: initInitialise: Can't lock /var/run/pigpio.pid
:~$ sudo ./x_pigpio
:~$ initInitialise: Can't lock /var/run/pigpio.pid
pigpio initialisation failed.
Fixup: Run sudo killall pigpiod
Q4: In the QEMU environment, pigpio cannot be emulated
2023-01-27 09:02:23 initAllocDMAMem: mbox open failed(No such device or address)
Can't initialize pigpio library
pigpio initialisation failed (-2003).
Fixup: Because pigpio relies on the BCM2711 chip hardware function to achieve ultra-low-latency DMA operations, and qemu's DMA cannot support this operation, so pigpio simulation cannot be completed in the QEMU environment.
Author: Chengsen Dong
Features:
The module is driven by S8550 transistor.
Working voltage 3.3V-5V.
Low level trigger.
Designed with holes for fixing with bolt, so it is easy to assemble.
VCC: External 3.3V-5V voltage (can be directly connected with 5V single-chip microcomputer and 3.3V single-chip microcomputer)
GND: External GND
I/O: External micro-controller IO port
Unit Test DEMO
When the unit test program runs: If you hear the buzzer, please use the keyboard to type "Y" and press ENTER.
Execute the following command to run unit tests:
## change to work dir
cd SleepPanda/src/app/Buzzer/build
# build, and run unit test(gtest)
cmake .. && make && sudo ctest --verbose
Author: Yihan Wang
Specification:
Working Voltage: DC 3.3V-5.5V
Nominal Frequency: 40KHz
PCB Size: 35 x 15mm / 1.38 x 0.59" (LxW)
Fixed Hole Diameter: 3mm
Main chip:LM393, Electret condenser microphone
Output form: Digital and Analog Output
Note:
AO outputs real-time voltage signal of the microphone.
DO outputs low and high lever signal when the sound intensity reaches a threshold
Sensitivity is adjustable by adjusting the potentiometer
Unit Test DEMO
What you will see: Make a sound, the DO pin level of the sound sensor will jump from low level to high level (rising edge interrupt), trigger a sound event, and the screen will print Rpi_SoundSensor Class DEBUG: SoundSensor_SoundEvent_Handle() was triggered ..
When the unit test program runs: Make a sound and press ENTER on your keyboard.
Execute the following command to run unit tests:
## change to work dir
cd SleepPanda/src/app/SoundSensor/build
# build, and run unit test(gtest)
cmake .. && make && sudo ctest --verbose
Author:Chengsen Dong
Ref:
Note: Through the
raspi-gpio getquery, it is found that the Interrupt Pin -4 originally planned to be used is occupied by other programs, resulting in the input level being always 0, and the external interrupt function cannot be realized. So will use GPIO6 to connect to INT pin of MAX30101.
Unit Test DEMO
Execute the following command to run unit tests:
## change to work dir
cd SleepPanda/src/app/MAX30101/build
# build, and run unit test(gtest)
cmake .. && make && sudo ctest --verbose
What you'll see: Put your fingertip on the MAX30101 sensor, wait 15 seconds, and press the Enter key on your keyboard. Unit testing is complete. The program will output the current heart rate and blood oxygen values. You will get output similar to:
1: Rpi_MAX30101 Class DEBUG: MAX30101_DataReadyEvent_Handle() was triggered.|heart_rate: 71, spo2: 99.525826
1: ir_mean: 136982.530000, red_mean: 121904.060000
1: beta_ir: -4.232085, beta_red: -2.549403
1: n_last_peak_interval test = GOOD
1: boundary test = GOOD
1: Rpi_MAX30101 Class DEBUG: MAX30101_DataReadyEvent_Handle() was triggered.|heart_rate: 78, spo2: 99.414008
1: ir_mean: 136578.920000, red_mean: 121704.060000
1: beta_ir: 4.154383, beta_red: 0.232127
1: n_last_peak_interval test = GOOD
1: boundary test = GOOD
1: Rpi_MAX30101 Class DEBUG: MAX30101_DataReadyEvent_Handle() was triggered.|heart_rate: 83, spo2: 99.427597
1: ir_mean: 137294.050000, red_mean: 121818.950000
1: beta_ir: 8.709769, beta_red: 1.685311
1: n_last_peak_interval test = GOOD
1: boundary test = GOOD
1: Rpi_MAX30101 Class DEBUG: MAX30101_DataReadyEvent_Handle() was triggered.|heart_rate: 75, spo2: 99.445994
1: Rpi_MAX30101.ISR_CheckPoint: 1
1: RPI DEBUG: MAX30101 Delete.
1: [ OK ] MAX30101_Test.Check_MAX30101_ISR_HeartRate_SPO2 (45287 ms)
1: [----------] 1 test from MAX30101_Test (45287 ms total)
1:
1: [----------] Global test environment tear-down
1: [==========] 1 test from 1 test suite ran. (45287 ms total)
1: [ PASSED ] 1 test.
1/1 Test #1: MAX30101_Test ................... Passed 45.30 sec
100% tests passed, 0 tests failed out of 1
Total Test time (real) = 45.30 sec
Author:Yihan Wang
Ref:
After using the arecord command that comes with the system, the system Log is as follows: Apparently, the sound card driver is not supported.
sudo arecord -D hw:3,0 -d 2 -f cd -c 2 -v -t wav test.wav
Recording WAVE 'test.wav' : Signed 16 bit Little Endian, Rate 44100 Hz, Stereo
arecord: set_params:1407: Unable to install hw params:
ACCESS: RW_INTERLEAVED
FORMAT: S16_LE
SUBFORMAT: STD
SAMPLE_BITS: 16
FRAME_BITS: 32
CHANNELS: 2
RATE: 44100
PERIOD_TIME: (125011 125012)
PERIOD_SIZE: 5513
PERIOD_BYTES: 22052
PERIODS: 4
BUFFER_TIME: (500045 500046)
BUFFER_SIZE: 22052
BUFFER_BYTES: 88208
TICK_TIME: 0
solution:
Execute the following command (using an unofficial fork to fix bugs):
git clone https://github.com/HinTak/seeed-voicecard.git
cd seen-voicecard
sudo ./install.sh
sudo reboot now
WM8960 device driver test
Playback device:
aplay -l
**** List of PLAYBACK Hardware Devices ****
card 0: Headphones [bcm2835 Headphones], device 0: bcm2835 Headphones [bcm2835 Headphones]
Subdevices: 8/8
Subdevice #0: subdevice #0
Subdevice #1: subdevice #1
Subdevice #2: subdevice #2
Subdevice #3: subdevice #3
Subdevice #4: subdevice #4
Subdevice #5: subdevice #5
Subdevice #6: subdevice #6
Subdevice #7: subdevice #7
card 1: vc4hdmi0 [vc4-hdmi-0], device 0: MAI PCM i2s-hifi-0 [MAI PCM i2s-hifi-0]
Subdevices: 1/1
Subdevice #0: subdevice #0
card 2: vc4hdmi1 [vc4-hdmi-1], device 0: MAI PCM i2s-hifi-0 [MAI PCM i2s-hifi-0]
Subdevices: 1/1
Subdevice #0: subdevice #0
card 3: seeed2micvoicec [seeed-2mic-voicecard], device 0: bcm2835-i2s-wm8960-hifi wm8960-hifi-0 [bcm2835-i2s-wm8960-hifi wm8960-hifi-0]
Subdevices: 1/1
Subdevice #0: subdevice #0
Recording equipment:
arecord -l
**** List of CAPTURE Hardware Devices ****
card 3: seeed2micvoicec [seeed-2mic-voicecard], device 0: bcm2835-i2s-wm8960-hifi wm8960-hifi-0 [bcm2835-i2s-wm8960-hifi wm8960-hifi-0]
Subdevices: 1/1
Subdevice #0: subdevice #0
Recording command:
arecord -D "plughw:3,0" -d 2 -f cd -c 2 -v -t wav test.wav
-D specifies the recording device -d set recording duration -f recording sample format -c specifies the channel -t file type of recording output path and name of test.wav output file
Play command:
aplay -D "plughw:3,0" test.wav
Unit Test DEMO
What you will see: WM8960 will record for 3 seconds and create a Record_Wav_File_Test.wav file.
When the unit test program runs: Please make a sound/play a song.
Execute the following command to run unit tests:
## change to work dir
cd SleepPanda/src/app/WM8960/build
# build, and run unit test(gtest)
cmake .. && make && sudo ctest --verbose
Note: If the Record_Wav_File_Test.wav file is successfully created, but there is no sound when it is opened and played, please use the
alsamixersound card management program to enable the microphone of the WM8960 sound card.
Author:Rui Liu Ref:
Features： 2.13" EPD display Inky pHAT pinout Compatible with all 40-pin header Raspberry Pi models Python library Comes fully assembled
Note： Overall dimensions: 65x30x8.5mm (WxHxD, depth includes header and display) Display usable area dimensions: 48.5x23.8mm (WxH), 2.13" diagonal
Author:Hui Wang
Ref:
Features： Small size, low power cells, 32*24 pixel infrared array； I2C Compatible Digital Interface； Programmable refresh rate 0.5Hz-60Hz； Supply voltage 3.3V； Current consumption ≤23mA； Working temperature: -45-85°C； Temperature range: -40-300°C；
Pin definition SDA: I²C serial interface data line (input/output)； VDD: positive power supply； GND: power negative； SCL: I²C serial interface clock line (input)；
notes
- Hardware design considerations: MLX90640 uses 3.3V power supply, and uses the power supply voltage as the reference voltage for temperature measurement, so it has high requirements for power supply stability. The power supply current needs to be able to output 100mA in balance.
- Notes on software design: calculate the corresponding functions in the API library for design.
- Pay attention to the calculated value of Vdd during operation. If the difference between this value and the actual value exceeds 0.1V, you should check the problem.
Author:Chengsen Dong
Step 0:Update GPU Memory Both CPUs and GPUs use physical RAM chips. On a Raspberry Pi 2 or 3, the GPU is allocated 64 MB by default. The Raspberry Pi 4 has a GPU memory size of 76 MB. It might be a bit small for a visual project, it's best to change it to 128 MB for now. To increase the amount of memory for the GPU, use the following menus:
Step 1: Download openCV source code
git clone https://github.com/opencv/opencv.git
Step 2: Install dependency packages
sudo apt-get install cmake
sudo apt-get install build-essential libgtk2.0-dev libavcodec-dev libavformat-dev libjpeg-dev libswscale-dev libtiff5-dev
sudo apt-get install libgtk2.0-dev
sudo apt-get install pkg-config
Step 3: Compile and install openCV
# change to your dir
cd /home/opencv
mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=Release -DOPENCV_GENERATE_PKGCONFIG=ON -DCMAKE_INSTALL_PREFIX=/usr/local ..
sudo make
sudo make install
Step 4: Configure the OpenCV environment
cd /etc/ld.so.conf.d/
sudo touch opencv4.conf
sudo sh -c 'echo "/usr/local/lib" > opencv4.conf'
Step 5: Update linker configuration (pkg-config)
sudo ldconfig
Step 6: Copy the opencv.pc file to /usr/lib/pkgconfig/
sudo cp -f /usr/local/lib/arm-linux-gnueabihf/pkgconfig/opencv4.pc /usr/lib/pkgconfig/
Step 7: Add environment variables
sudo vim /etc/bash.bashrc
#Add at the end of the file:
PKG_CONFIG_PATH=$PKG_CONFIG_PATH:/usr/lib/pkgconfig
export PKG_CONFIG_PATH
Step 8: Test whether the installation is successful
pkg-config --modversion opencv4
#maybe output: 4.7.0
Step 9: Test procedure
#include <opencv2/imgcodecs.hpp>
#include <opencv2/highgui.hpp>
#include <opencv2/imgproc.hpp>
#include <iostream>
using namespace cv;
using namespace std;
int main()
{
Mat img(512, 512, CV_8UC3, Scalar(255, 255, 255));
circle(img, Point(256, 256), 256, Scalar(0, 0, 255), FILLED);
Rect roi(128, 128, 256, 256);
rectangle(img, roi, Scalar(255, 255, 255), FILLED);
line(img, Point(256, 128), Point(256, 256), Scalar(255, 255, 0), 3);
putText(img, "I'am CV", Point(256, 128), FONT_HERSHEY_DUPLEX, 1, Scalar(0, 0, 0), 2);
imshow("img", img);
waitKey(0);
return 0;
}
Step 10: Compile the test program (SleepPanda/src/app/Camera/OpenCV_Test):
g++ OpenCV_Test.cpp -o OpenCV_Test `pkg-config --cflags --libs opencv4`
sudo ./OpenCV_Test
If an error occurs: ./OpenCV_Test: error while loading shared libraries: libopencv_highgui.so.407: cannot open shared object file: No such file or directory
Please update the linker configuration with the following command:
sudo ldconfig
Cascade Detection: Face & Profile -> Eyes
When a positive face (red circle) is detected at the same time, it means that the user is sleeping. When only the side face (blue circle) is detected, it means the user is sleeping on the side. By measuring the user's state change per unit time (the change frequency of sleeping on the side and sleeping on the side), it is judged whether the user is in deep sleep or light sleep.
wget https://github.com/opencv/opencv/raw/master/data/haarcascades/haarcascades_upperbody.xml
wget https://github.com/opencv/opencv/raw/master/data/haarcascades/haarcascade_profileface.xml
wget https://github.com/opencv/opencv/raw/master/data/haarcascades/haarcascade_frontalface_alt.xml
wget https://github.com/opencv/opencv/raw/master/data/haarcascades/haarcascade_eye_tree_eyeglasses.xml
It is realized by creating new threads, mutexes, and queues. The worker thread reads the new Frame continuously, and when the new Frame is valid, the main thread is notified to fetch the Frame through the queue.
Use Qt's QTime class, timeout() event to trigger camera frame reading.
Camera FPS=30, the processing speed of the test program is lower than the camera FPS, so it is appropriate to use the QTime class timeout() event.
Finally use Thread and Callback to achieve frame acquisition
Unit Test DEMO
What you will see: The camera image is displayed in real time, the frontal face and eyes are marked with red circles, and the side face and eyes are marked with blue circles.
Execute the following command to run unit tests:
Note: Make sure you are in a graphical desktop environment. The command line environment cannot display a GUI.
## change to work dir
cd SleepPanda/src/app/Camera/build
# build, and run unit test(gtest)
cmake .. && make && sudo ctest --verbose
You will see the following effect:
Sleeping (marked with a red circle)
Sleeping on your side (marked with a blue circle)
Package required to install QT5
sudo apt-get install qtdeclarative5-dev-tools
sudo apt-get install libqwt-qt5-dev
sudo apt-get install qtbase5-dev
sudo apt-get install qtdeclarative5-dev
Development Record
Proudly use pure C++ to write GUI, not using Qt Creator, which will avoid introducing QML.
Unit Test DEMO
What you will see: The GUI developed with QT5 will be displayed.
Execute the following command to run unit tests:
## change to work dir
cd SleepPanda/src/app/Window/build
# build, and run unit test(gtest)
cmake .. && make && sudo ctest --verbose
Note: Make sure you are in a graphical desktop environment. The command line environment cannot display a GUI.
You will see the following effect:
Sleep Panda GUI
Run the following code to execute the SleepPanda software:
Note: Make sure you are in a graphical desktop environment. The command line environment cannot display a GUI.
cd SleepPanda/src/app/build
# Rebuild code
cmake .. && make
# Run SleepPanda
sudo ./SleepPanda
You will get the following result:
SleepPanda GUI