This project, is a comprehensive exploration and implementation of human activity understanding for enhanced furniture assembly tools. It combines state-of-the-art techniques in computer vision and machine learning to address complex challenges in the domain of assembly processes.
The primary motivation behind this project is to revolutionize the way furniture assembly tasks are performed by leveraging advanced technologies. Our purpose is to enhance the efficiency and precision of furniture assembly processes with the help of a digital assistant (e.g., LLM), ultimately contributing to the advancement of manufacturing and related industries.
At its core, this project aims to understand and analyze human activity within an assembly environment. We break down the project into the following key components:
- Object Detection: This component focuses on the accurate detection and localization of objects within the assembly environment, enabling precise tracking of components.
- Hand Landmarks: We delve into the intricate details of hand gestures and movements to facilitate interactions between human operators and assembly tools.
- MLP and LSTM: The use of Multi-Layer Perceptrons (MLP) and Long Short-Term Memory (LSTM) networks enhances our ability to recognize and predict human activity patterns.
- Transformer Network: Leveraging Transformer networks allows us to capture complex spatial and temporal relationships, enhancing the overall understanding of assembly processes.
Large Language Model (LLM) instructions are seamlessly integrated into our pipeline to enable real-time decision-making and control of assembly tools based on human activity.