This repository contains all information and code necessary to be able to reproduce our FE vehicle's design and function. The files are contained in folders in the main branch, while all info regarding programming and the principles behind the code can be found in this document.
The Future Engineers challenge requires us to build a robot that can complete two tasks. The first task (called the qualifying task) requires the vehicle to make three laps of a square track autonomously, avoiding hitting any walls, as fast as possible and within a period of three minutes. The second task (called the final task) requires the vehicle to make another three laps in the same manner, while also avoiding green and red obstacles in its route and passing them from the correct side, depending on their color.
To accomplish these tasks we built a custom rear-axle driven robot, based on the Raspberry Pi SBC, which allows us to manipulate its motor functions freely, depending on input from numerous sensors located around the front of the vehicle. A wide range of sensors were used, so that the tasks could be completed as efficiently and as fast as possible. These sensors varied from ultrasonic distance sensors to cameras. All systems of the robot were powered by a custom power distribution system, stepping down voltages and supplying power to all components of the robot. Two seperate power systems were created to decrease load and improve performance and reliability, each with its own working voltage, to match the requirements of the devices it powers. In this guide we will go over all the systems in detail and explain the principles and functions behind the code used for the autonomous driving of the robot. Definitions for terms which may be unknown to some readers will be in the end of this guide.
The hardware used in this project can be divided into four distinct categories. Power, motor, control and sensing hardware. All these systems work together to complete the required tasks. In this section we will look at how each separate category works to support the completed robot. A schematic is available in the schemes directory of this repository, to better understand the information in this section and to help you replicate the design on your own. There will also be references to this section in code, so it is possible to refer to this section to understand the working principles behind the code mentioned in section 4.
The robot is powered from two 18650 lithium ion cells connected in series, with a combined nominal voltage of 7.4V (3.7V each). The output of these cells is distributed to two buck converters, each powered on with a separate switch, which step down the voltage to a stable 5V for the Raspberry Pi and 5.5V for the other components. Each buck converter is rated for up to 3A of current, more than enough for the systems to run without any voltage drops. The 5V buck converter connects directly to the Raspberry Pi 3B+ with a USB cable, while the power from the 5.5V one is distributed using WAGO terminal blocks. The Raspberry Pi is equipped with a polyfuse and the 5V step-down converter has an input fuse to protect it from any electrical failures of components connected to GPIO. Both electrical subsystems share a common ground, to enable communication between the digital equipment which use both power sources.
There is a total of 2 motors on the robot. A NEMA 17 stepper motor is used to drive the back axle. This is connected to a TMC2209 stepper motor driver and allows us to precisely manipulate the speed of the robot and the distance which it travels, by setting the exact number of steps the motor will move at any given time. This drive motor is connected by pulley to a custom made gear on the back axle and features great quiet performance and reliability, making it the best option for its selected purpose. A 9g Metal Gear Servo is used to control the front axle steering of the robot. This motor provides a holding torque of around 2.8kg.cm and a speed of approximately 0.09s/60°, making it the perfect motor for use in a steering system. Its high torque and fast response time allow us to have precise and fast steering, a necessity in such a robot. It is controlled using a PWM signal from the GPIO of the Raspberry Pi and is powered from the 5.5V power source.
The Raspberry Pi 3B+ was our SBC of choice for this project. This stems from its great computing capabilities, which are necessary for use with OpenCV, the library used for accessing and processing the camera image data. The presence of great documentation for every part of its operation with our code was also a deciding factor in the decision over its use. It has 40 GPIO ports and a dedicated CSI camera connector, making it a great choice for use in such a project. All other components are connected to it via the GPIO, providing useful data and allowing manipulation of motors using code. A button is connected to a GPIO input to give the start signal for the program.
In total 7 sensors of 4 different types are used to give the robot information about the track and its location. A camera is used to provide OpenCV with an image of the view from the front of the robot. The camera has an FOV of 160 degrees, giving us a view containing all areas necessary to compute the best path. It is connected to the Raspberry Pi by use of the dedicated CSI connection. Three HC-SR04 Ultrasonic Sensors are present on the robot, one for each side except the back of the robot. They are connected to the Raspberry Pi via the GPIO and a code function is used to calculate the distance from the walls on each side. More information on their working principle can be found here. Two infrared obstacle sensors, for the left and right sides of the robot, are also present on the robot and could be used for wall detection, even though they are not in the current version of the code due to interference problems. A TCS3200 Color Sensor mounted facing the track under the robot is also present, although this as well is not used for any function as of the latest code version. It could however be used as an alternative way to count the number of laps travelled, by counting the colored lines present on the track surface.
- 1x Raspberry Pi 3B+
- 2x 3400mAh Li-Ion 18650 Battery Cells
- 1x DFRobot DC-DC Fast Charge Module (5V)
- 1x DC-DC Adjustable Buck Converter
- 5x 3-terminal WAGO Terminal Blocks
- 1x 2-terminal WAGO Terminal Block
- 1x TMC2209 Stepper Motor Driver
- 1x NEMA 17 Stepper Motor
- 1x Active Buzzer
- 2x Infrared Obstacle Sensors
- 3x HC-SR04 Ultrasonic Distance Sensors
- 1x 9g Metal Gear Sensor
- 1x TCS3200 Color Sensor
- 1x Raspberry Pi Camera Module 5MP 160°
- Start with the Raspberry Pi turned off.
- Open a hotspot on your mobile phone or other device with the following characteristics:
- SSID: xxxx
- Password: xxxx
- Security: WPA2-Personal(AES)
- Client Isolation: Off (usually off by default).
- Initially (optional) power on the motors by turning on the right switch. Then close the left switch, turning on the Raspberry Pi. Automatically the Raspberry Pi will try to connect to the network we created and will start the necessary services for ssh and smb.
- Using an IP Scanner application (iPhone, Android) find and note the IP address of the Raspberry Pi. (In this guide we will denote it as x.x.x.x)
- Initial smb configuration for file transfer (Only once on each network):
- Open a file browser of your choice. (In this case Dolphin will be used, but the instructions are the same for all graphical programs.)
- Right click on the left bar on the Remote or Network header of your program. Then select Add Entry or an equivalent option. (Attention! Click on the header and not on the Network item).
- In the dialog that will appear add a name of your choice, the IP address and the connection protocol in the format
smb://x.x.x.xand if you want and have the option to do so, select an image for easier identification. In case there is a field for username and passwork enter the following details:Username: xxx, Password: xxx.
- Now there will be an icon in the left bar of our program which will have the name and the image you have chosen.
- Clicking on it will bring up a folder. Open it. (The name will be different)
- In the identification dialog that will appear enter the following details:
Username: xxx, Password: xxxand press OK or Connect. (If you have entered them before they may not be requested)
- From there go to Desktop and from there to the xxx folder, where our libraries are downloaded. There you can upload your code to run it in the next part of the guide. *(If you can't see Desktop in this folder, it will be in the home folder.)
- Connect via ssh to run code:
- Open your Terminal program, it will be different depending on the distribution, but the steps remain the same.
- There execute the command
ssh xxx@x.x.x.x, where x.x.x.x is the IP address you found in step 4 of the initial setup. - Immediately afterwards you will be prompted for the password, where you will enter
xxx.
- Once you press Return/Enter you will be logged in to the Raspberry Pi. Here you can run anything you want on the Raspberry Pi.
- To find the code you uploaded you need to go to the xxx folder in Desktop. Do this by using the cd command and more specifically the
cd Desktop/xxxcommand. - Here you will notice that the working directory has been changed to
~/Desktop/xxx.
- From there you can now run the code with the command
sudo python3 xxxx.py, wherexxxx.pyis the name of the code file. Here you can also run any other command you need to fix your code and the Raspberry Pi.
- To close the connection run the
exitcommand, - If you want to shut down the Raspberry Pi use the command
sudo shutdown 0. - To reboot immediately another useful command is
sudo reboot now. - To stop the execution of the code it is sufficient to use the keybind
Ctrl+C
- Details denoted with
xxxneed to be filled in by the user.
The code file that runs on the robot can be found at src/xfinal.py. To run it 6 libraries are required. These libraries are contained in the /src/libraries/ folder and consist of time, numpy, OpenCV, dlib, TMC_2209 and RPi.GPIO. The xfinal.py file contains all the code needed to complete all the tasks required by the Future Engineers competition regulations. In this section we are going to explain in detail every section of the code and how it all works to accomplish the goal required.
Lines 1-69 of the code contain all the code required to initialize the robot's systems and prepare the GPIO pins for the next code sections. In more detail:
- Lines 1-5
importthe necessary libraries for the code to work. - Line 6
GPIO.setmode(GPIO.BCM)declares the GPIO pin numbering system used in the program. - Lines 9-18 declare integrers that contain the GPIO pin numbers for all devices that use GPIO. This is done for easier and more efficient coding in the following sections.
- Lines 21-30
setupthe GPIO pins as inputs and outputs, while also enable the internal pull down resistor of pin 27, the pin used for the button. - Lines 34-47
setthe parameters used for stepper motor operations, like the microstepping resolution, max current, acceleration and speed. - Line 49 enables the motor with the command
tmc.setMotorEnabled(True) - Lines 52-53 enable the PWM output of pin 17, where the servo is connected, thus enabling the servo.
- Lines 55-69 contain the parameters for the OpenCV
masksused for isolating the red and green colors later in the code. Isolating the colors allows us to measure their prevalence in the image and adjust the robot's movement accordingly.
The code is written in such a way where most code is contained in pre-defined functions, which are called in the main loop. These functions enable control of the robot and use of sensor data. They are defined in lines 72-169.
- Lines 72-109 contain the functions for distance measurement using the ultrasonic sensors placed in the front, left and right sides of the robot. These sensors work on the principle of measuring the time between the transmission of an ultrasonic pulse and the receipt of its echo in the sensor. The functions measure the amount of time taken and perform all the necessary calculations to convert it to a centimeter
distanceoutput. - Lines 111-123 contain the functions for the operation of the steering servo motor. Steering the robot is accomplished by changing the duty cycle of the
PWMoutput of the servo pin, thus instantly changing the direction of the steering servo motor. - Lines 126-139 define the functions used for the robot's linear movement. The back-axle stepper motor is controlled with the
tmc.runToPositionStepscommand, which allows us to precisely define the steps and consequently the angle the motor will travel in a single movement. This allows us to control the exact distance the robot will travel during any movement. - Lines 141-169 contain the functions used for detecting the presence of the red and green obstacles in front of the robot. This is accomplished by use of a mask for each color and calculation of the presence of each color in the image, by calculating the number of pixels covered by the mask. These functions return a
TrueorFalseboolean value, which is used in the main loop.
The main code section, in lines 172-212, contains a loop, which works to check the values of the robot's sensor and camera inputs and manipulate the robot's movement in such a way to accomplish the WRO Future Engineers tasks. This loop repeats until the robot has completed 3 revolutions. Detection of the revolutions is done by use of a counter that raises by one in value when the robot makes a turn at a corner.
While this counter indicated that less than 3 revolutions have been completed the robot takes constant measurements of the distance measured by each of the 3 ultrasonic sensors and when there is free space in front of it it keeps going forward(). When a wall is detected the robot completes a turn to the direction where it detects is should go. This detection is done by measuring the distance on each side and turning to the one with the most space. This ensures that the robot can complete both the qualifying and final rounds, no matter the direction in which it was initially placed.
When completing the movement it also checks for the presence of obstacles in front of it and moves accordingly. The detection of obstacles works by checking the output of the OpenCV functions described in the previous sections. There is also wall detection by measuring the distance to the walls. If the distance is smaller than the threshold the robot turns away from the wall automatically.
- The motor doesn't operate normally, it runs in a random direction every time: Make sure that the login shell over serial is disabled, then reboot.
- The ultrasonic sensors aren't calculating the distance to the walls: Remove the function containing the sensor code and use its code directly in the main loop.
- FE: The Future Engineers category of the World Robotics Olympiad
- Main Branch: The main branch of this github repository
- SBC: Single Board Computer
- Buck Converter: A device which takes in a higher voltage and converts it to a lower one by use of efficient rapid switching.
- WAGO Terminal Blocks: A type of cage clamp connector which allows for easy reversible connection splicing.
- Polyfuse: A type of self-resettable fuse made out of polymer.
- GPIO: The General Purpose Input and Output pins of the Raspberry Pi. They allow for control of other hardware via a variety of modes, such as PWM and I2C, but also typical analog and digital operation.
- Stepper Motor: A type of brushless motor allowing the user to precisely define its angle and speed of rotation.
- Servo Motor: A type of motor which allows the user to define the angle at which it is positioned by use of a PWM signal.
- FOV: Field of View, a measure of the area covered by the camera.