This multifaceted project comprises 5 main sections including 2D Image Registration, Electrical Board Detection, IC detection, IC Recognition, and preparing a GUI to automate the entire process.
- Image Capturing: Capture images using a provided platform, ensuring significant overlap for accuracy.
- Image Analysis: Analyze image metadata for camera details and location information.
- Keypoint Detection: Detect distinct points in each image.
- Keypoint Matching: Match these points across overlapping images to identify common areas.
- Point-Cloud Generation: Generate a sparse point cloud from matched points.
- Point-Cloud Identification: Identify tie points (points appearing in multiple images) for 3D modeling.
- Image Calibration: Perform camera calibration and refine tie point positions.
- Image Alignment: Use positional data for model alignment.
- Task: Create a dense point cloud for detailed representation.
- Task: Generate a detailed Surface Model and Terrain Model from the dense point cloud.
- Image Stitching: Stitch processed images to create an orthomosaic.
- Perspective Correction: Correct perspective distortions for uniform scale and adjust for topography.
- Task: Refine the orthomosaic through editing or enhancement.
- Result: High-resolution, accurately scaled 2D map.
-
Overview:
- Description: A Visual C# GUI designed to automate and streamline the image capturing and processing stages.
-
Features:
- Camera Positioning:
- Function: Manages initial, movement, and final camera positions.
- Image Processing:
- Task: Executes PCB image processing post-capture.
- Camera Positioning:
-
Role:
- Importance: Integral for user-friendly operation and system efficiency.
- Impact: Ensures seamless workflow from image acquisition to analysis.
- Task: Process the image to isolate the electrical board, reducing background presence.
- Result: Refined image predominantly featuring the electrical board, ready for further analysis.
- Task: Apply data augmentation techniques to enhance the pre-trained YOLOv8n model's performance.
- Purpose: Generate additional training samples to prevent overfitting and improve model generalization.
- Details: The dataset includes 742 images of various boards in different colors, sizes, and perspectives.
- Status: Actively expanding the dataset with more images. Currently not publicly available.
- Note: The trained model's performance will be showcased below.
- Model Availability: The fine-tuned model can be provided upon request.
- Task: Employ the same dataset with distinct annotations for IC detection.
- Details: The dataset includes 3 types of ICs: without, four-sided, and two-sided ICs.
- Task: Experiment with different YOLO versions for fine-tuning.
- Process: Various training configurations and data augmentation techniques were tested.
- Result: The best model, based on its performance on the test set, was selected for further analysis.
- Model Availability: The fine-tuned model can be provided upon request.
- Task: Detect boxes containing text on each IC using a fine-tuned YOLO model.
- Model Details: The model is specifically trained for the precise detection of text boxes on integrated circuits.
- Model Availability: The fine-tuned YOLO model can be provided upon request.
- Task: Recognize the text within each detected box.
- Method: Utilize a fine-tuned PaddleOCR for accurate text recognition.
- Process: PaddleOCR works in conjunction with the YOLO model to effectively recognize text in the identified areas.
- Model Availability: The fine-tuned PaddleOCR model is available upon request.
- Implementation: The
ocr.pyscript is used for the detection and recognition tasks in this part.
- Task: Identify company logos on the ICs.
- Method: Use a fine-tuned YOLO model trained on a dataset of 20 different company logos.
- Objective: Facilitate the process of recognizing each IC by identifying the associated company logo.
- Model Availability: The logo detection model is accessible upon request.
- Task: Search for each IC on the internet using the recognized text from the IC surface.
- Method: Employ web-scraping techniques to find relevant information and datasheets for each IC.
- Objective: Enhance the identification process by gathering detailed information about each IC from online sources.
- Task: Create a comprehensive report for electrical boards containing multiple ICs.
- Details: The report includes datasheets and information for each identified IC.
- Feature: If an IC is successfully found online, its datasheet will be downloaded and included in the report.
- Benefit: Provides users with accessible and detailed information about each IC on the electrical board.
- Description: A Windows-based application crafted to automate the entire process of IC recognition and reporting.
- Technology: Written in Python and utilizes PyQt for the graphical user interface.
- Functionality: Handles input and output files for each electrical board.
- Includes:
- The generated report for each board.
- Images of detected electronic boards.
- Patches of detected ICs.
- Design Approach: The application is developed with multithreading and multiprocessing capabilities.
- Purpose: To efficiently handle and process multiple electrical boards simultaneously.
- Benefit: Enhances performance and responsiveness, especially when dealing with multiple tasks or large datasets.
- Aspect: Features a user-friendly interface for easy navigation and operation.
- Functionality: Allows users to seamlessly interact with the application for processing and retrieving results.
- Objective: Streamline the entire process from image input to report generation, integrating all the previously mentioned components (YOLO model, PaddleOCR, web scraping).
- Outcome: A unified system that efficiently processes electrical boards, identifies ICs, retrieves information, and compiles comprehensive reports.
- Ali Amini: Machine Vision Engineer and Software Developer at RCDAT.
- GitHub Repo: Repo Adress