The goal of this project is to build a self-driving car with deep learning and computer vision, which can navigate in different environments. The project is inspired by the work done by Sentdex. After experimenting with different convolutional neural networks, Google's InceptionV3 seems to offer the best prediction results. SSD MobileNet is also integrated along the inference pipeline to add collision avoidance capacity to the whole system by detection other vehicles in real-time. Detailed Explaination: https://marsolmos.com/files/projects/gtautodrive_project.html
- Grand Theft Auto-5 (no mods are required)
- Python (3.6.12)
- Tensorflow GPU (2.3.1)
- Keras
- OpenCV
- Numpy
150,000 images with respective keyboard keys pressed at that moment are collected by driving the car around the map. However, only 11,235 images are left after balancing and expanding the data.
Use 1_collect_data.py to generate your custom dataset. Ensure the GTA-5 window size same as in the collect_data.py and that it's located in the top left corner of your screen. Save the collected data into a custom path in your machine and use it to train your model.
Training code can be found here
- Set the path to your training data
- Correctly set up envirnonment (ensure GPU is enable)
- Run the code
- Download or clone this repo
- Install all dependencies in requirements.txt
- Set up paths to your local folders before collecting data
- Run 1_collect_data.py to get in-game data
- Run 2_balance_data.py to balance collected data in step 1
- Run 3_train_model.py to train your model
- Open GTA-V and run 4_test_model.py to make your model drive
- Collect more and better data (hood camera, better line detection, etc.)
- Increase object detection speed and include it on training data
- Split pipelines into urban and highway driving models
- Include analogic key input (joystick)
- Include custom sensors (GPS, LIDARs, etc.)
Sentdex: https://github.com/Sentdex/pygta5