A python library built to empower developers to build applications and systems with self-contained Computer Vision capabilities
Built with simplicity in mind, ImageAI
supports a list of state-of-the-art Machine Learning algorithms for image recognition.
ImageAI currently supports image recognition using 4 different Machine Learning algorithms
trained on the ImageNet-1000 dataset and Object Detection using RetinaNet trained on COCO dataset.
Eventually, ImageAI will provide support for a wider
and more specialized aspects of Computer Vision including and not limited to image
recognition in special environments and special fields and custom image
prediction.
New Release : ImageAI 1.0.2
What's new:
- Addition of Object Detection class
- Addition of multiple images prediction function
- Removal of the need for ImageNet-1000 JSON file for image prediction
- New Dependencies to support Object Detection (find them in the dependencies section below)
- Bug Fixes
▣ Dependencies
▣ Installation
▣ Using ImageAI for Image Prediction
▣ Multiple Images Prediction
▣ Using ImageAI in MultiThreading
▣ Using ImageAI for Object Detection
▣ Object Detection, Extraction and Fine-Tuning
▣ Real-Time and High Perfomance Implementation
▣ Sample Applications
▣ Documentation
▣ AI Practice Recommendations
▣ Contact Developers
▣ References
- Python 3.5.1 (and later versions) Download (Support for Python 2.7 coming soon)
- pip3 Install
- Tensorflow 1.4.0 (and later versions) Install or install via pip
pip3 install --upgrade tensorflow- Numpy 1.13.1 (and later versions) Install or install via pip
pip3 install numpy- SciPy 0.19.1 (and later versions) Install or install via pip
pip3 install scipy- OpenCV (Required as from ImageAI 1.0.2) Install or install via pip
pip3 install opencv-python- Matplotlib (Required as from ImageAI 1.0.2) Install or install via pip
pip3 install matplotlib- h5py (Required as from ImageAI 1.0.2) Install or install via pip
pip3 install h5py- Keras 2.x (Required as from ImageAI 1.0.2) Install or install via pip
pip3 install kerasTo install ImageAI, run the python installation instruction below in the command line:
pip3 install https://github.com/OlafenwaMoses/ImageAI/raw/master/dist/imageai-1.0.2-py3-none-any.whl
or download the Python Wheel
imageai-1.0.2-py3-none-any.whl and run the python installation instruction in the command line
to the path of the file like the one below:
pip3 install C:\User\MyUser\Downloads\imageai-1.0.2-py3-none-any.whl
- SqueezeNet (Size = 4.82 mb, fastest prediction time and moderate accuracy)
- ResNet by Microsoft Research (Size = 98 mb, fast prediction time and high accuracy)
- InceptionV3 by Google Brain team (Size = 91.6 mb, slow prediction time and higher accuracy)
- DenseNet by Facebook AI Research (Size = 31.6 mb, slower prediction time and highest accuracy)
Great! Once you have downloaded this model file, start a new python project, and then copy the model file to your project folder where your python files (.py files) will be . Then create a python file and give it a name; an example is FirstPrediction.py. Then write the code below into the python file:
from imageai.Prediction import ImagePrediction import osexecution_path = os.getcwd() prediction = ImagePrediction() prediction.setModelTypeAsResNet() prediction.setModelPath( execution_path + "\resnet50_weights_tf_dim_ordering_tf_kernels.h5") prediction.loadModel()
predictions, percentage_probabilities = prediction.predictImage("C:\Users\MyUser\Downloads\sample.jpg", result_count=5) for index in range(len(predictions)): print(predictions[index] + " : " + percentage_probabilities[index])
Sample Result:
sports_car : 90.61029553413391 car_wheel : 5.9294357895851135 racer : 0.9972884319722652 convertible : 0.8457873947918415 grille : 0.581052340567112
The code above works as follows:
from imageai.Prediction import ImagePrediction import os
The code above imports the ImageAI library and the python os class.
execution_path = os.getcwd()
The above line obtains the path to the folder that contains your python file (in this example, your FirstPrediction.py) .
prediction = ImagePrediction() prediction.setModelTypeAsResNet() prediction.setModelPath(execution_path + "\resnet50_weights_tf_dim_ordering_tf_kernels.h5")In the lines above, we created and instance of the ImagePrediction() class in the first line, then we set the model type of the prediction object to ResNet by caling the .setModelTypeAsResNet() in the second line and then we set the model path of the prediction object to the path of the model file (resnet50_weights_tf_dim_ordering_tf_kernels.h5) we copied to the python file folder in the third line.
predictions, percentage_probabilities = prediction.predictImage("C:\Users\MyUser\Downloads\sample.jpg", result_count=5) In the above line, we defined 2 variables to be equal to the function
called to predict an image, which is the .predictImage() function, into which we parsed the path to
our image and also state the number of prediction results we want to have (values from 1 to 1000) parsing
result_count=5 . The .predictImage() function will return 2 array objects with the first (predictions) being
an array of predictions and the second (percentage_probabilities) being an array of the corresponding percentage probability for each
prediction.
for index in range(len(predictions)): print(predictions[index] + " : " + percentage_probabilities[index])The above line obtains each object in the predictions array, and also obtains the corresponding percentage probability from the percentage_probabilities, and finally prints the result of both to console.
- Define your normal ImagePrediction instance
- Set the model type and model path
- Call the .loadModel() function
- Create an array and add all the string path to each of the images you want to predict to the array.
- You then perform prediction by calling the .predictMultipleImages() function and parse in the array of images, and also set the number predictions you want per image by parsing result_count_per_image=5 (default value is 2)
Find the sample code below:
import os from imageai.Prediction import ImagePredictionexecution_path = os.getcwd()
multiple_prediction = ImagePrediction() multiple_prediction.setModelTypeAsResNet() multiple_prediction.setModelPath(execution_path + "\resnet50_weights_tf_dim_ordering_tf_kernels.h5") multiple_prediction.loadModel()
all_images_array = []
all_files = os.listdir(execution_path) for each_file in all_files: if(each_file.endswith(".jpg") or each_file.endswith(".png")): all_images_array.append(each_file) results_array = multiple_prediction.predictMultipleImages(all_images_array, result_count_per_image=5)
for each_result in results_array: predictions, percentage_probabilities = each_result["predictions"], each_result["percentage_probabilities"] for index in range(len(predictions)): print(predictions[index] + " : " + percentage_probabilities[index]) print("-----------------------")
In the above code, the .predictMultipleImages() function will return an array which contains a dictionary per image. Each dictionary contains the arrays for predictions and percentage probability for each prediction.
Sample Result:
convertible : 52.459555864334106 sports_car : 37.61284649372101 pickup : 3.1751200556755066 car_wheel : 1.817505806684494 minivan : 1.7487050965428352 ----------------------- toilet_tissue : 13.99008333683014 jeep : 6.842949986457825 car_wheel : 6.71963095664978 seat_belt : 6.704962253570557 minivan : 5.861184373497963 ----------------------- bustard : 52.03368067741394 vulture : 20.936034619808197 crane : 10.620515048503876 kite : 10.20539253950119 white_stork : 1.6472270712256432 -----------------------
- You can create your prediction object, set its model type, set model path and json path outside the new thread.
- The .loadModel() must be in the new thread and image prediction (predictImage()) must take place in th new thread.
Take a look of a sample code below on image prediction using multithreading:
from imageai.Prediction import ImagePrediction import os import threadingexecution_path = os.getcwd()
prediction = ImagePrediction() prediction.setModelTypeAsResNet() prediction.setModelPath( execution_path + "\resnet50_weights_tf_dim_ordering_tf_kernels.h5") prediction.setJsonPath(execution_path + "\imagenet_class_index.json")
picturesfolder = os.environ["USERPROFILE"] + "\Pictures\" allfiles = os.listdir(picturesfolder)
class PredictionThread(threading.Thread): def init(self): threading.Thread.init(self) def run(self): prediction.loadModel() for eachPicture in allfiles: if eachPicture.endswith(".png") or eachPicture.endswith(".jpg"): predictions, percentage_probabilities = prediction.predictImage(picturesfolder + eachPicture, result_count=1) for index in range(len(predictions)): print(predictions[index] + " : " + percentage_probabilities[index])
predictionThread = PredictionThread () predictionThread.start()
As from ImageAI 1.0.2 , you can now perform Object Detection on images. ImageAI provides very convenient and powerful methods to perform object detection on images and extract each object from the image. The object detection class provided only supports the current state-of-the-art RetinaNet, while other object detection networks will be supported in the nearest future. To start performing object detection, you must download the RetinaNet object detection via the link below:
- RetinaNet (Size = 145 mb)
Once you download the RetinaNet model file, you should copy the model file to the your project folder where your .py files will be. Then create a python file and give it a name; an example is FirstObjectDetection.py. Then write the code below into the python file:
from imageai.Detection import ObjectDetection import osexecution_path = os.getcwd()
detector = ObjectDetection() detector.setModelTypeAsRetinaNet() detector.setModelPath( execution_path + "\resnet50_coco_best_v2.0.1.h5") detector.loadModel()
detections = detector.detectObjectsFromImage(execution_path + "\image2.jpg", execution_path + "\image2new.jpg") for eachObject in detections: print(eachObject["name"] + " : " + eachObject["percentage_probability"] ) print("--------------------------------")
Sample Result:
Input Image
Output Image
Let us make a breakdown of the object detection code that we used above.
from imageai.Detection import ObjectDetection import osIn the 3 lines above , we import the ImageAI object detection class in the first line, import the os in the second line and obtained the path to folder where our python file runs.execution_path = os.getcwd()
detector = ObjectDetection() detector.setModelTypeAsRetinaNet() detector.setModelPath( execution_path + "\resnet50_coco_best_v2.0.1.h5") detector.loadModel()In the 4 lines above, we created a new instance of the ObjectDetection class in the first line, set the model type to RetinaNet in the second line, set the model path to the RetinaNet model file we downloaded and copied to the python file folder in the third line and load the model in the fourth line.
detections = detector.detectObjectsFromImage(execution_path + "\image2.jpg", execution_path + "\image2new.jpg")
for eachObject in detections:
print(eachObject["name"] + " : " + eachObject["percentage_probability"] )
print("--------------------------------")
In the 2 lines above, we ran the detectObjectsFromImage() function and parse in the path to our image, and the path to the new image which the function will save. Then the function returns an array of dictionaries with each dictionary corresponding to the number of objects detected in the image. Each dictionary has the properties name (name of the object) and percentage_probability (percentage probability of the detection)
In the examples we used above, we ran the object detection on an image and it
returned the detected objects in an array as well as save a new image with rectangular markers drawn
on each object. In our next examples, we will be able to extract each object from the input image
and save it independently, and also fine-tune the object detection to detect more objects.
In the example code below which is very identical to the previous object detction code, we will save each object
detected as a seperate image.
from imageai.Detection import ObjectDetection import osexecution_path = os.getcwd()
detector = ObjectDetection() detector.setModelTypeAsRetinaNet() detector.setModelPath( execution_path + "\resnet50_coco_best_v2.0.1.h5") detector.loadModel()
detections, objects_path = detector.detectObjectsFromImage(execution_path + "\image3.jpg", execution_path + "\image3new.jpg", save_detected_objects=True) for eachObject, eachObjectPath in zip(detections, objects_path): print(eachObject["name"] + " : " + eachObject["percentage_probability"] ) print("Object's image saved in " + eachObjectPath) print("--------------------------------")
Sample Result:
Input Image
Output Images
person
person
person
person
motorcycle
dog
car
person
Let us review the part of the code that perform the object detection and extract the images:
detections, objects_path = detector.detectObjectsFromImage(execution_path + "\image3.jpg", execution_path + "\image3new.jpg", save_detected_objects=True)
for eachObject, eachObjectPath in zip(detections, objects_path):
print(eachObject["name"] + " : " + eachObject["percentage_probability"] )
print("Object's image saved in " + eachObjectPath)
print("--------------------------------")
In the above above lines, we called the detectObjectsFromImage() , parse in the input image path, output image part, and an
extra parameter save_detected_objects=True. This parameter states that the function extract each object detected from the image
and save it has a seperate image. The parameter is false by default. Once set to true, the function will create a directory
which is the output image path + "-objects" . Then it saves all the extracted images into this new directory with
each image's name being the detected object name + "-" + a number which corresponds to the order at which the objects
were detected.
This new parameter we set to extract and save detected objects as an image will make the function to return 2 values. The
first is the array of dictionaries with each dictionary corresponding to a detected object. The second is an array of the paths
to the saved images of each object detected and extracted, and they are arranged in order at which the objects are in the
first array.
from imageai.Detection import ObjectDetection import osexecution_path = os.getcwd()
detector = ObjectDetection() detector.setModelTypeAsRetinaNet() detector.setModelPath( execution_path + "\resnet50_coco_best_v2.0.1.h5") detector.loadModel()
detections = detector.detectObjectsFromImage(execution_path + "\image2.jpg", execution_path + "\image2new2.jpg", minimum_percentage_probability=30) for eachObject in detections: print(eachObject["name"] + " : " + eachObject["percentage_probability"] ) print("--------------------------------")
Sample Result:
Input Image
Output Image
<br>
apple : 35.701966285705566 -------------------------------- apple : 38.37691843509674 -------------------------------- bowl : 49.61664080619812 -------------------------------- sandwich : 34.04079973697662 -------------------------------- bowl : 36.91719174385071 -------------------------------- person : 91.946941614151 -------------------------------- person : 73.61021637916565 -------------------------------- person : 38.810619711875916 -------------------------------- laptop : 40.59039056301117 -------------------------------- laptop : 90.24320840835571 -------------------------------- laptop : 73.6881673336029 -------------------------------- person : 36.050015687942505 -------------------------------- laptop : 95.16398310661316 -------------------------------- person : 30.75830042362213 -------------------------------- person : 87.10319399833679 -------------------------------- person : 35.907524824142456 -------------------------------- bed : 48.27503561973572 -------------------------------- dining table : 43.09736788272858 --------------------------------
Interesting! You will notice we used the image above for the first object detection. Prevoiously, It detected 3 persons (which was suppose to be 4) and 3 laptops (which was suppose to be 4) when the minimum_percentage_probability was at the default value of 50 . In our new code above with minimum_percentage_probability=30 , the function detected 7 persons (not so okay), 3 laptops (not bad), 2 apples (We didn't see that before), 2 bowls (that's great), 1 sandwich, 1 bed (Oh, I forgot we had that in there) and a dining table (trust me, the meal didn't belong there). There was significant improvement in the number of items detected and a slight mis-detections. The fine-tuning can be increased to have higher detection integrity or decreased to capture as many possible items detectable.
ImageAI uses the Tensorflow backbone for it's Computer Vision operations. Tensorflow supports both CPUs and GPUs ( Specifically NVIDIA GPUs. You can get one for your PC or get a PC that has one) for machine learning and artificial intelligence algorithms' implementations. To use Tensorflow that supports the use of GPUs, follow the link below :
FOR WINDOWS
https://www.tensorflow.org/install/install_windows
FOR macOS
https://www.tensorflow.org/install/install_mac
FOR UBUNTU
https://www.tensorflow.org/install/install_linux
We also welcome submissions of applications and systems built by you and powered by ImageAI for listings here. Should you want your ImageAI powered
developements listed here, you can reach to us via our Contacts below.
The ImagePrediction class
The ImagePrediction class can be used to perform image prediction
in any python application by instanciating it and calling the available functions below:
- setModelTypeAsSqueezeNet() This function should be called should you
chose to use the SqueezeNet model file for the image prediction. You only need to call it once.
- setModelTypeAsResNet() This function should be called should you
chose to use the ResNet model file for the image prediction. You only need to call it once.
- setModelTypeAsInceptionV3() This function should be called should you
chose to use the InceptionV3Net model file for the image prediction. You only need to call it once.
- setModelTypeAsDenseNet This function should be called should you
chose to use the DenseNet model file for the image prediction. You only need to call it once.
- setModelPath() You need to call this function only once and parse the path to
the model file path into it. The model file type must correspond to the model type you set.
- loadModel() You need to call this function
once only before you attempt to call the predictImage() function .
- predictImage() To perform image
prediction on an image, you will call this function and parse in the path to the image file
you want to predict, and also state the number of predictions result you will want to
have returned by the function (1 to 1000 posible results). This functions returns two arrays.
The first one is an array of predictions while the second is an array of corresponding percentage probabilities
for each prediction in the prediction array. You can call this function as many times as you need
for as many images you want to predict and at any point in your python program as far as you
have called the required functions to set model type, set model file path, set json file path
and load the model.
- predictMultipleImages() To perform image
prediction on a collection of image, you will call this function and parse in an array that contains the strings
of the file path to each image. Then you can set the maximum number of predictions you want to retrieve
per image. This function returns an array of dictionaries, with each dictionary containing 2 arrays namely 'prediction_results' and 'prediction_probabilities'. The 'prediction_results'
contains possible objects classes arranged in descending of their percentage probabilities. The 'prediction_probabilities'
contains the percentage probability of each object class. The position of each object class in the 'prediction_results'
array corresponds with the positions of the percentage possibilities in the 'prediction_probabilities' array.
The ObjectDetection class
The ObjectDetection class can be used to perform object detection on images, object
extraction and more by instanciating it and calling the available functions below:
- setModelTypeAsRetinaNet() This function should be called
to use the RetinaNet model file for object detection. You only need to call it once.
- setModelPath() You need to call this function only once and parse the path to
the model file path into it. The model file type must correspond to the model type you set.
- loadModel() You need to call this function
once only before you attempt to call the detectObjectsFromImage function .
- detectObjectsFromImage() To perform object detection, you need to call this function and parse in
the path to input image, the path to save your output image, set the save_detected_objects (optional) and
set the minimum_percentage_probability (optional).
This function returns an array of dictionaries, with each dictionary corresponding to the objects
detected in the image. Each dictionary contains the following property:
- name
- percentage_probability
If 'save_detected_objects' is set to 'True', this function will return another array (making 2 arrays
that will be returned) that contains list of all the paths to the saved image of each object detected
For anyone interested in building AI systems and using them for business, economic, social and research purposes, it is critical that the person knows the likely positive, negative and unprecedented impacts the use of such technologies will have. They must also be aware of approaches and practices recommended by experienced industry experts to ensure every use of AI brings overall benefit to mankind. We therefore recommend to everyone that wishes to use ImageAI and other AI tools and resources to read Microsoft's January 2018 publication on AI titled "The Future Computed : Artificial Intelligence and its role in society ".
Kindly follow the link below to download the publication.
https://blogs.microsoft.com/blog/2018/01/17/future-computed-artificial-intelligence-role-society/
Moses Olafenwa
Website: https://moses.specpal.science
Twitter: @OlafenwaMoses
Medium : @guymodscientist
Facebook : moses.olafenwa
John Olafenwa
Website: https://john.specpal.science
Twitter: @johnolafenwa
Medium : @johnolafenwa
Facebook : olafenwajohn
-
Refik Can Malli, SqueezeNet v1.1 Implementation using Keras Functional framework 2.0,
https://github.com/rcmalli/keras-squeezenet/ -
Francois Chollet, Trained image classification models for Keras,
https://github.com/fchollet/deep-learning-models/ -
Somshubra Majumdar, DenseNet Implementation of the paper, Densely Connected Convolutional Networks in Keras,
https://github.com/titu1994/DenseNet/ -
Broad Institute of MIT and Harvard, Keras package for deep residual networks,
https://github.com/broadinstitute/keras-resnet -
Fizyr, Keras implementation of RetinaNet object detection,
https://github.com/fizyr/keras-retinanet