PixEagle is an all-in-one image processing, following, and tracking solution designed for the PX4 ecosystem (with potential expansion to ArduPilot). It leverages MAVSDK Python, OpenCV, and optional YOLO for precise object tracking and drone navigation. The project emphasizes modularity and extensibility, allowing users to implement their own tracking, detection, and segmentation algorithms. The system is modular, well-commented, and designed for easy integration of new algorithms. Additionally, PixEagle includes a beta web app GUI using React for real-time monitoring and control.
Watch the latest video showcasing PixEagle v1.0, demonstrating advanced features including precision landing and intelligent target tracking in a Software in the Loop Simulation with X-Plane 12 and PX4XPlane:
The AppController is essentially the manager and coordinator of the entire application flow. It oversees and orchestrates the main loops, streaming, following, tracking, and detection processes, ensuring smooth and efficient operation of the PixEagle system.
The Tracker class has been improved to allow for building custom tracker instances. It initializes and manages the tracking algorithm, processes video frames, and updates tracking data. It supports various tracking algorithms such as CSRT, KCF, and others. The tracker can also use a position estimator for enhanced accuracy.
The BaseTracker class provides a foundation for custom tracker implementations. Templates are provided for creating new tracking algorithms, allowing for easy extension and customization.
The Follower class is designed for scenarios where a downward-looking camera follows a moving or stationary target. It uses PID control with gain scheduling to maintain accurate tracking even on a moving platform. The Follower class parameters are managed in the Parameters class.
The PX4Controller handles communication with MAVSDK and PX4 using MAVLink. It manages offboard control, allowing for precise drone navigation and control.
The FlaskHandler sends telemetry and tracker data via HTTP to any Ground Control Station (GCS) and the PixEagle dashboard. Details on the URIs, ports, and syntax can be found in the Parameters class.
The TelemetryData class is responsible for packing data for the FlaskHandler, ensuring efficient and organized data transmission.
Defines the base structure and interface for object detection modules, allowing for extensibility and integration of different detection algorithms.
Specializes in detecting objects based on feature matching, enhancing the application's ability to recognize objects in various conditions.
Stores and manages all configuration settings, including detection parameters, video source configurations, and PID gains for the Follower.
Implements advanced segmentation algorithms to improve object detection and tracking in complex scenes.
Utilizes template matching techniques for object detection, offering a straightforward method for recognizing objects based on stored templates.
Handles video inputs from various sources, including files and cameras, ensuring flexible input options for processing and analysis.
A full web-based React application serves as the Ground Control Station (GCS). It provides a basic yet functional interface where you can view live camera feeds, tracker and follower data, and plots. Future updates will include additional features such as the ability to send commands and more interactive capabilities.
To set up the PixEagle project, follow these steps:
- Clone the repository:
git clone https://github.com/alireza787b/PixEagle.git
- Navigate to the project directory:
cd PixEagle - Set up a virtual environment (optional but recommended):
python -m venv venv source venv/bin/activate # On Windows use `venv\Scripts\activate`
- Install the required dependencies:
pip install -r requirements.txt
- (Optional) If you need PyTorch for YOLO detection, run the setup script:
./setup_pytorch.sh
- Execute the main application:
python src/main.py
If you prefer, you can still use the global Python environment instead of a virtual environment.
- Install Node.js and npm for the dashboard application (if not already installed):
# Follow instructions at https://nodejs.org/en/download/ - Navigate to the dashboard directory and install dependencies:
cd dashboard npm install - Start the dashboard application:
npm start
Access the dashboard in your browser at http://127.0.0.1:3000
In the parameters.py file, you can customize various settings to suit your specific use case. This includes selecting your video source, setting descent parameters, and adjusting other important configurations such as PID gains and camera angles. The file is well-documented and easy to navigate.
Select your video source (either from a video file, USB Camera, etc.). Note that not all methods are fully tested yet, but file and USB camera methods have been tested.
For PX4 real SITL implementation, you might need X-Plane or another simulation tool that can use the output video. There is a guide to using the PX4XPlane plugin and PixEagle SITL available here.
You can define the altitude descent parameters in the parameters.py file. By default, these parameters might be set to prevent descending or to stop descending at 20 meters. Make sure to set these accordingly:
# Minimum descent height in meters.
# The drone will not descend below this altitude during operations.
MIN_DESCENT_HEIGHT = 20 # meters
# Maximum rate of descent in meters per second.
# Limits the vertical speed to prevent rapid altitude loss.
MAX_RATE_OF_DESCENT = 0.5 # meters per secondEnsure to check and adjust other parameters in the parameters.py file, such as PID gains, camera angles, and more, to optimize the system for your specific needs. The file is well-documented to assist you in making these customizations.
While in the video pop-up screen, you can use the following keys:
t: Select targetc: Cancel selectiony: YOLO detectionf: Start following offboardd: Try to re-detect targetq: Quite
Currently, the tracker with CSRT, detection with template matching, and connection to PX4 offboard using a simulated USB camera are working and tested.
- In the React UI, video overlay on the scope plot makes the telemetry update stop.
- Unable to draw the bounding box around the target from the React app yet.
- The follower with a forward-looking camera and chase mode will be implemented soon.
If you encounter an ImportError related to libGL.so.1, install the OpenGL libraries with:
sudo apt-get update
sudo apt-get install -y libgl1-mesa-glxThis is often required for OpenCV's image and video display functionalities.
We welcome contributions from developers, researchers, and enthusiasts in drone technology, AI, and robotics. You can contribute by checking out current issues, discussing your ideas, or submitting pull requests with new features or improvements.
PixEagle is under active development, focusing on leveraging PX4 for flight control, incorporating AI for smart decision-making, and utilizing advanced tracking for precise object interaction.
PixEagle has not yet been tested in real-world conditions. It has only been tested in SITL (Software In The Loop) simulations. Real-world testing can significantly differ and might lead to crashes or damage. Use it at your own risk, and the developers are not responsible for any damages or losses incurred during real-world testing.