The Find Your Mate AI project is designed to scrape and process cofounder matching profiles from startupschool.org. It involves scraping the web pages, extracting relevant information, and storing it in a structured format for Querying them.
This is a project born out of my own itch and helps me learn how to use LLM's to build a search based apps.
- Python 3.8 or higher
- macOS operating system
- MongoDB Account Register here
- YC Account Profile Signup here
- EditThisCookie Chrome Extension Install here
- Clone the repository:
git clone https://github.com/your-repository/find_your_mate_ai.git
cd find_your_mate_ai
- Install PDM (Python Dependency Manager):
brew install pdm
- Install the project dependencies:
pdm install
pdm run playwright install
- Export Cookies by following the instructions at EditThisCookie Chrome Extension
- Create settings.toml and .secrets.toml in the root directory of the project and add the following environment variables:
To scrape the cofounder matching profiles, run the following command:
pdm run python src/find_your_mate_ai/scraper.py --cookie-path ".startupschool-cookie.json"
To ingest the data from the specified directory path and index it using LlamaIndex, run the following command:
pdm run python src/find_your_mate_ai/data_ingestion.py