/quoordinates

Use OpenAI Embeddings to visualize Kindle Highlights from Readwise!

Primary LanguageJavaScript

Quoordinates (Quote Coordinates)

check out the general purpose PostgresDB prototype version of Quoordinates I'm working on -- Commonbase!!

  • Readwise
  • OpenAI Embeddings
  • Nomic
  • PSQL (PostgreSQL)
  1. fetch highlights from readwise
    1. (optional) start at date
  2. upsert data in database
  3. embed highlights with openai
  4. place embeddings into nomic visualization

embed-into-supabase.js is a node script that embeds highlights into supabase nomic-viz.py is a python script that visualizes highlights in nomic readwise-highlights.py is a python script that fetches highlights from readwise

Downstream Dependents

The apps below depend on Quoordinates and display how versatile this technology is:

  • Commonplace Bot - A Discord Bot where quotes can be surfaced, discussed, and transformed in an MMO like format.
  • Quo Host - A realtime speech to semantic search to "higlight within highlight" surfacer. Sound smarter on podcasts.
  • Kindle Streamer - Use OCR on a cron job to take a picture of Kindle, and generate a visual bookmark using DALL-E.

Setup

  1. Create a .env file with the following variables:
READWISE_ACCESS_TOKEN=
SUPABASE_URL=
SUPABASE_KEY=
OPENAI_API_KEY=
OPENAI_ORG=
DAY_LAST_FETCHED=
LOG_PATH=
  1. Install dependencies
poetry init
poetry shell
poetry install

and

npm install
  1. Run the script to get readwise highlights
python readwise-highlights.py
  1. Run the script to embed highlights
node embed-into-supabase.js
  1. Run the script to visualize highlights
python nomic-viz.py

Other

There are other tools in this repo too!

  • compile quotes - use k-means to make a cluster of all quotes from a book and give them topic titles
  • share-pic - use a quote, dalle, and cf images to save a quote and image in one eye catching piece

Images

Screenshot 2023-07-26 01-46-45 Screenshot 2023-07-26 01-35-19 Screenshot 2023-07-26 01-35-27

Screenshot 2023-07-28 15-11-18 Screenshot 2023-07-28 15-10-35