/shoppy-gpt

A Nextjs + Pinecone + OpenAI GPT site to surface answers to Shopify help center content

Primary LanguageJavaScript

**This is a research demo. Support is not provided.

Shopify Help Center Search via GPT

Quickly surface answers from Shopify's help center using GPT.

Technologies used

  • ScrapingBee to scrape list of help center urls
  • Mongodb to store scraped data
  • OpenAI to created embeddings vector points and completion prompt
  • Pinecone to store vectors in db

How this works

  1. Run tasks/1-sitemap-to-csv.js to convert Shopify's Help Center Sitemap.xml into CSV and drop all columns except urls.
  2. Convert CSV into array of links.
  3. Run tasks/2-scrape.js to scrape the article text from every link using ScrapingBee and isnert text into Mongodb using url as unique index.
  4. Run tasks/3-generate-embeddings.js to generate OpenAI embeddings and upsert into Pinecone.

Why?

Created this as a research experiment in order to learn OpenAI embeddings + Pinecone. Added bonus was to have a way to quickly surface answers for my Shopify platform questions.

How to optimize this further

  • Split article text into smaller documents to decrease cost of token usage. Split by H2/section.
  • Test different models to see one cost. Curie is 10x cheaper than Davinci.
  • Search documents with a normal search engine (Algolia) and pass that document into open AI rather than using embedding’s and Pinecone.
  • Cache results for common queries.
  • Test a shorter prompt to further save tokens.

Preview

preview.png