/CameraBlip

Using BLIP to identify camera and uploaded images

Primary LanguagePython

📸 Visual Question Answering App 🎉

Welcome to the Visual Question Answering (VQA) App! This fun and interactive app allows you to upload an image or capture one using your webcam, then ask questions about it. Powered by AI, the app will provide answers based on the image’s content! 🤖✨

🚀 Features

  • 📂 Image Upload: Easily upload an image from your device.
  • 📷 Camera Capture: Snap a photo with your webcam directly in the app.
  • 🧠 Interactive Q&A: Ask any question about the image, and get an AI-generated response!

🔧 Setup

Prerequisites 🛠️

  • Python: Version 3.7 or later is required.

  • Libraries: Install the necessary packages by running the following command:

    pip install streamlit transformers torch pillow opencv-python-headless numpy

Installation 🖥️

  1. Clone the Repository 📂:
git clone https://github.com/yourusername/vqa-app.git
cd vqa-app
  1. Run the Streamlit App 🎈:
streamlit run app.py
  1. Open the App 🌐: Once the app is running, open the provided URL (usually http://localhost:8501) in your browser.

🎉 How to Use the App

1. Choose an Image Source:

  • 🖼️ Select either "Upload Image" or "Use Camera" from the dropdown to choose your image source.

2. Upload or Capture an Image:

  • Upload: If uploading, choose an image file in .jpg, .jpeg, or .png format.
  • Camera: If using the camera, take a picture with your webcam.

3. Ask a Question ✍️:

  • Type a question about the image in the text input box.

4. Get the Answer 🧩:

  • Click on the "Get Answer" button to receive a response generated by the model.

📜 Code Overview

Initializing BLIP Processor and Model 🧠

The app uses the BlipProcessor and BlipForQuestionAnswering models from the Hugging Face transformers library. These models are pre-trained for visual question answering, allowing the app to generate responses based on the content of an image.

Function Definitions 📄

  • ask_question_about_image(image, question): Processes the image and question, and returns the answer.
  • load_image(image_file): Reads and converts uploaded images to RGB format.
  • convert_camera_image(img_array): Converts camera input images from Streamlit to a format compatible with the BLIP model.

Streamlit Layout 🖼️

The app layout provides users with:

  • Options to upload images or capture them with a camera.
  • A text input to enter questions about the image.
  • AI-generated answers displayed in response to the questions.

🧪 Example

Try This:

  • Upload an image of a cute cat. 🐱
  • Ask, "What is this animal?" 🤔

Expected Result:

  • The app will respond with "cat" (or a similar accurate answer based on the image content).

🧩 Dependencies

This app relies on the following libraries:

  • Streamlit: For creating the interactive web app interface.
  • transformers: To load the BLIP model for visual question answering.
  • torch: To run the model on either CPU or GPU.
  • Pillow: For image processing.
  • OpenCV: For handling camera input images.
  • NumPy: For working with image arrays.

⚠️ Troubleshooting

If you encounter any issues, try the following:

  • Compatibility: Ensure all dependencies are installed and compatible with Python 3.7 or later.
  • Streamlit Issues: Restart the Streamlit server using streamlit run app.py if any errors appear on the frontend.

📜 License

This project is licensed under the MIT License. See the LICENSE file for details.

🎉 Acknowledgments

  • 🤗 Hugging Face Transformers: For providing the BLIP model and tools for visual question answering.
  • 🌈 Streamlit: For creating an easy-to-use framework for building web apps.