InsightRed is an LLM-powered tool adept at extracting the latest Reddit comments from subreddits, sorted by "Hot", and pinpointing users who exhibit potential interest in your project or product. It's a Reddit marketing tool to help you get your initial users for your product/project.
This project was build for the ANARCHY October 2023 Hackathon. Checkout ANARCHY on their website and github.
click the image to view the YouTube video or click here: https://youtu.be/xhKwnKxmg5k
-
(OPTIONAL) This is tool is designed for Unix based operating systems (MacOS or Linux). So if you are using Windows, I recommend using Docker. To set that up, do the following:
- Install Docker: https://docs.docker.com/engine/install/
- Grab an Ubuntu install:
docker pull ubuntu
- Run, and go into, the Ubuntu image:
docker run -it ubuntu
- In the image, install these things:
apt update apt upgrade apt install vim apt install git apt install python3-pip
-
Clone this project and cd into it:
git clone git@github.com:MehmetMHY/insightred.git cd insightred
-
For this tool, you will need to get these API keys:
- Reddit API Keys:
- Why? This is used to scrape data from Reddit
- How? Watch this video from timestamps 00:00 to 03:05. You will also need to grab your Reddit account's username and password.
- OpenAI API Key
- Why? To use the GPT-4 LLM model. This can be changed in the config file but by default we use the GPT-4 model.
- How? Go to OpenAI's API Page to setup your account. After this, setup billing for the API by going to this link: https://platform.openai.com/account/billing/overview. With billing setup, create & get the API key from this link: https://platform.openai.com/account/api-keys.
- Pinecone API Key
- Why? We use Retrieval Augmented Generation, so we need a vector database. For this, we decided to go with Pinecone as our vector database.
- How? Go to Pinecone's Website & sign up or log in if you already have an account. After you login, you should be in the Pinecone dashboard. From there, make sure you don't create an Index. If you do, delete it because InsightRed will generate the Index for you. Then, go to the "API Keys" tab and click on "Create API Key". Create your API key and save it.
- Reddit API Keys:
-
Set your environment variables in a file called ".env". Check out the "env_template" file to get an idea of the env file layout. This file should contain the following content (you set your key values):
# reddit bot keys export CLIENT_ID="" export SECRET_KEY="" export ACCOUNT_USERNAME="" export ACCOUNT_PASSWORD="" # OpenAI API key export OPENAI_API_KEY="" # Pinecone API key export PINECONE_API_KEY=""
- Make sure the create and save these values. You will need to source them later on.
-
Install dependencies:
# create local python environment python3 -m venv env # activate local python environment source env/bin/activate # install python dependencies pip3 install -r requirements.txt # install LLM-VM git clone https://github.com/anarchy-ai/LLM-VM.git cd LLM-VM pip3 install . cd ..
-
Make sure to source/activate your environment variables and your python environment:
# activate local python environment source env/bin/activate # source/activate environment variables source .env
-
Run the CLI tool to use this application. Run the following command and fill out ALL of the questions:
python3 main.py
-
When the CLI is running, fill out all the questions. You can skip some of those questions if allowed. After the questions are answered, the tool should output Reddit comments you can reply to for your project/product.
- ChatGPT (GPT-4)
- James Briggs (YouTuber)
- https://openai.com/blog/openai-api
- https://docs.pinecone.io/docs/overview
- https://www.reddit.com/r/redditdev/comments/14nbw6g/updated_rate_limits_going_into_effect_over_the/
- https://www.reddit.com/prefs/apps
- https://www.youtube.com/watch?v=FdjVoOf9HN4
- https://medium.com/analytics-vidhya/scraping-reddit-data-using-reddit-api-3eba4bd7a56a
- https://gist.github.com/davestevens/4257bbfc82b1e59eeec7085e66314215
- https://github.com/praw-dev/praw
- https://chat.openai.com/