This repo is for ros2 foxy only!!!!!!!!!!!!!!!!!!!!
Guide dogs have been used for decades to assist visually impaired individuals in navigating their surroundings. However, these dogs come at a high cost, ranging from $20,000 to $40,000. Additionally, they are red-green color blind, which limits their ability to interpret street signs and other visual cues.
To address these limitations, a I Programmed the Unitree Go1 quadruped robot dog to become an autonomous guide dog. By incorporating advanced technologies such as speech recognition, object recognition, and autonomous navigation, the robot dog will be able to help visually impaired individuals navigate their surroundings and avoid obstacles. And because a robot dog can be updated through machine learning and software updates, it is possible to continually improve its abilities over time.
Overall, this project represents an exciting advancement in assistive technology, providing a cost-effective and long-lasting alternative to traditional guide dogs. By combining the latest in robotics and machine learning, the Unitree Go1 robot dog has the potential to revolutionize the way visually impaired individuals navigate the world around them.
Demo video:
Guide_dog_Low_comp.mp4
guide_dog.launch.py:
- This launches all the required nodes for Go1 to operate in guide dog mode.
ros2 launch guide_dog_unitree_go1 guide_dog.launch.py use_nav2:=true use_object_detection:=true
- This launches all the required nodes on Go1 to operate in guide dog mode.
ros2 launch guide_dog_unitree_go1 guide_dog.launch.py use_voice_control:=true use_speech_recognition:=true use_Go1_vision:=true
The use of the Picovoice deep learning voice recognition library has enabled the creation of a custom wake word "Hey Willie" and commands like walk, stop, stand up, lay down, bark, and increase or decrease speed. By implementing this library, the user is able to control the movements of Go1 with their voice while navigating around. A ROS2 C++ and Python package was developed to hadle the voice recognition and translate the voice commands to desired controls. I utilized gTTS (Google Text-to-Speech), a Python library that generates text to speech audio files. I generate audio files that Go1 uses to effectively communicate with the user. The package was deployed on a Nvidia Jetson Nano on Go1.
Use VCS tool to clone all the required packages:
- The vcs import command clones all repositories of a previously exported file with the following command:
vcs import < guide_dog.repos
- Create a workspace:
mkdir -p ws/src
- Go into source directory:
cd ws/src
- Git clone the Guide_dog_Unitree_Go1 repository:
git clone <ssh:Guide_dog_Unitree_Go1 repository>
- Move .repos folder into source directory:
mv Guide_dog_Unitree_Go1/guide_dog.repos guide_dog.repos
- Use vcs tool to clone dependency repos:
vcs import < guide_dog.repos
- Build package in workspace directory:
colcon build
- 先下载vscode的集成环境（针对ros2-foxy）
git clone git@github.com:kanghua309/vscode_foxy_ws.git
- 进入vscode_foxy_ws目录后，下载guide_dog.repos
cd vscode_foxy_ws
wget https://raw.githubusercontent.com/kanghua309/Guide_dog_Unitree_Go1/main/guide_dog.repos
or
wget https://ghproxy.com/https://raw.githubusercontent.com/kanghua309/Guide_dog_Unitree_Go1/main/guide_dog.repos
3）打开vscode，通过devcontainer创建容器（记得启动docker服务，构建过程需要一些时间）
4）容器创建后，进行环境安装 - 执行setup.sh （从终端执行）或者从vs的task中选择setup 执行 —— 这里需要先将容器的sshkey加入到github库中,后者更高级的是参考https://code.visualstudio.com/remote/advancedcontainers/sharing-git-credentials（该步骤将安装guide_dog项目需要的ros的各种依赖库，所以需要一些时间）
5）编译ros2对应的包（即可src下的包 ， 需要少许时间）
6）我们可讲src目录或其下的工程目录添加到工作区，从而其源码被vscode可管理
7）其他debug等功能慢慢实验吧
The guide dog project was my own individual project, but some subsets of this project had collaborations with Nick Morales, Katie Hughes, Ava Zahedi and Rintaroh Shima. Thank you all for you contributions.