jpandeinge/notesGPT

Record voice notes & transcribe, summarize, and get tasks

notesGPT

Generate action items from your notes in seconds. Powered by Convex, Together.ai, and Whisper.

Tech Stack · Deploy Your Own · Future Tasks

Tech Stack

• Convex for the database and cloud functions
• Next.js App Router for the framework
• Together Inference for the LLM (Mixtral)
• Together Embeddings for the embeddings for search
• Convex File Storage for storing voice notes
• Convex Vector search for vector search
• Replicate for Whisper transcriptions
• Clerk for user authentication
• Tailwind CSS for styling

Deploy Your Own

You can deploy this template by setting up the following services and adding their environment variables:

1. Run npm install to install dependencies.
2. Run npm run dev. It will prompt you to log into Convex and create a project.
3. It will then ask you to supply the CLERK_ISSUER_URL. To do this:
   1. Make a Clerk account.
   2. Copy both the CLERK_SECRET_KEY and NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY API keys into .env.local.
   3. Do steps 1-3 here and copy the Issuer URL. It should look something like https://some-animal-123.clerk.accounts.dev.
   4. Add CLERK_ISSUER_URL to your Convex Environment Variables (deep link also available in your terminal). Paste the Issuer URL as the value and click "Save".
4. Now your frontend and backend should be running and you should be able to log in but not record.
5. Make a Together account to get your API key.
6. Make a Replicate account to get your API key.
7. Save your environment variables in Convex as REPLICATE_API_KEY and TOGETHER_API_KEY.

Future tasks

• Keep recording for future playback and display it on the page somewhere
• Animate the purple microphone to be in sync with your voice
• Store completed action items for the future instead of fully deleting them
• Make text/images in the landing page smaller to account for multiple screen sizes.
• Make the search experience a little smoother overall by searching automatically on entering text
• Be able to have this as a PWA if there's an easy step to do that
• Prompt engineer the summary a little more to be more useful than what's currently displaying
• Add a Notion integration to be able to get the transcript + summary + action items on there
• UI updates to make it look a little nicer based on Youssef's redesign
• Be able to edit action items after the fact and set a due date for them
• Account for layout shift on the dashboard page when refreshing – show a loading state on content?
• Make action items animate out + make checkbox rounded + add a little check icon on hover
• Migrate to incredibly fast whisper