🥞🦜 askFSDL 🦜🥞
askFSDL is a demonstration of a document-backed question-answering pipeline.
We use the materials from the Full Stack Deep Learning course as our document corpus, so the resulting application is great at answering questions like
- What are the differences between PyTorch, TensorFlow, and JAX?
- How do I build an ML team?
- Which is cheaper: running experiments on cheap, slower GPUs or fast, more expensive GPUs?
- What's a data flywheel?
You can try it out via the Discord bot frontend in the FSDL Discord.
Stack
We use langchain
to organize our LLM invocations and prompt magic.
We stood up a MongoDB instance on Atlas to store our cleaned and organized document corpus, as shown in the "Building the FSDL Corpus" notebook.
For fast search of relevant documents to insert into our prompt, we use a FAISS index.
We host the application backend, which communicates with OpenAI's language modeling API and other services, on Modal, which provides serverless execution and scaling. That's also where we execute batch jobs, like syncing the document and vector stores.
We host the Discord bot,
written in py-cord,
on a free-tier
AWS EC2
instance.
We use Gantry to monitor model behvaior in production and collect feedback from users.