The idea is to build a crew of video production with ability to direct videos, write scripts and character descriptions. We generate a story using a text prompt. CrewAI Agents generated the story, character description and scene descriptions (illustrations). This is used to prompt stable diffusion to generate images while the story is used to generate audio. Finally the images and videos are put together manually to generate a video.
figure above shows the current architecture. We plan to experiment and refine this further.
-
poetry install --no-root -
poetry shell