I embarked on the GenAI module journey to stay current with industry trends and expand my expertise in artificial intelligence and data analysis. This module offered me a comprehensive learning experience, enhancing my skills in prompt engineering, data cleaning, and data preprocessing through five distinct projects.
Diving into the Google Play Store dataset was an exciting start. Using Python, I tackled the challenge of cleaning and preprocessing the data. The process involved identifying and fixing errors, removing duplicates, and handling missing values. This groundwork set the stage for future exploratory data analysis using SQL. Although the primary focus was on data cleaning, I also learned to visualize data trends through charts and graphs, which significantly improved my understanding of making data-driven decisions.
Libraries used:
- langchain: For building applications based on large language models (LLMs).
- langchain_experimental: Advanced features for natural language processing.
- openai: State-of-the-art NLP capabilities for various tasks.
The second project was an insightful dive into the healthcare sector. Analyzing medical appointment data to understand patient behavior was crucial. I prepared the dataset by importing libraries, setting up API keys, and reading data. Cleaning the data involved identifying duplicates, handling missing data, and preprocessing. Converting categorical data to binary and creating age groups were particularly interesting tasks. Using LangChain for data cleaning ensured that the dataset was ready for detailed analysis, ultimately aiming to improve healthcare services.
Food delivery has become incredibly popular, and understanding this sector was fascinating. The Zomato dataset, with information on restaurants, cuisines, and user reviews, provided a comprehensive view. Using LangChain, I addressed errors, inconsistencies, and missing information, preparing the dataset for analysis. Cleaning and preprocessing nearly 50,000 rows and 13 columns of data was a significant task, but it was rewarding to see the dataset ready for in-depth analysis, which can help businesses thrive and consumers make informed choices.
Libraries used:
- langchain: Simplifying the data cleaning process.
- langchain_experimental: Advanced features for natural language processing.
- openai: State-of-the-art NLP capabilities.
Building a resume storage and search system using a dataset of over 100 resumes was a practical and engaging project. I read and processed resumes, created vector storage in SingleStore, and implemented a system to query and identify top matches based on similarity scores. This project demonstrated the practical application of vector storage in real-world scenarios, enhancing my understanding of efficient data storage and retrieval.
The final project was a highlight, where I developed an AI-powered recipe recommendation system. Using state-of-the-art NLP models to analyze user queries and generate relevant recipe recommendations was fascinating. I created an intuitive user interface for seamless navigation and interaction. Utilizing the text-embedding-3-small model for effective recipe representation and retrieval was particularly rewarding. Integrating retrieval techniques with generation models provided personalized recommendations, making this project both sophisticated and user-friendly.
Project highlights:
- Data Source: Indian dessert recipes dataset from Hugging Face.
- Database: MongoDB Vector Database for efficient storage and retrieval.
- System Integration: Combining retrieval techniques with generation models for personalized recommendations.
Through these projects, I have significantly advanced my skills in data analysis and AI application development. I am eager to continue learning and contributing to this exciting field, leveraging the knowledge and experience gained from the GenAI module to tackle new challenges and innovate in the industry.