This is an illustrated story generator app allowing users to create stories based on a prompt and the pictures of themself.
You can try it out here.
This app use Next.js, Firebase, Runpod, ChatGPT, and Stable Diffusion XL with IP Adapter.
- User uploads a picture of themself.
- User enters a prompt for the story.
- ChatGPT generates a story based on the prompt.
- ChatGPT split chapters and generate Stable Diffusion prompts for each splits
- Stable Diffusion XL with IP Adapter with low weight generates an avatar based on the picture of the user
- Stable Diffusion XL with IP Adapter with high weight generates illustrations for each prompts based on the generated avatar --> this allow the generated pictures to have consistent style and head across the story
- Web App:
./web-app - Story and Prompts generation:
./back/functions/ - Image Generation:
./back/runpod/
- refactor image generation for standalone use
- use diffuser's IP Adapter for image generation to use prompt weighting (when IP Face is supported)
- don't use IP Adapter for splits where the hero is not present