Pipeline to scrape prompt + image url pairs from LAION share-gpt4v-images
discord channel
This is currently syncing to huggingface here: https://huggingface.co/datasets/TwoAbove/LAION-discord-gpt4v
DISCORD_TOKEN
- Discord bot token with read access and "MESSAGE CONTENT INTENT" toggled onHF_DATASET_NAME
- Name of the dataset to sync to on huggingfaceHF_TOKEN
- Huggingface token with write access to the dataset
channel_id
- ID of the discord channel to scrapelimit
- Number of messages to scrape per requesthf_dataset_name
- Falback dataset name incase ENV is not set