bitcoinsearch/scraper

add raw text to markdown scrapers

Opened this issue · 1 comments

Presently, markdown documents are formatted in JSON to capture titles and other semantics but complicates things when analyzing the tokenized text or rending it on an app like bitcoinsearch.xyz.

e.g.
Screen Shot 2023-03-09 at 10 49 35 AM

We need another way to access the rendered text, maybe it's another field or maybe the body field has two elements to it, raw and JSON.

will transition to uniform html scrape and add raw html. Need update for bitcoin transcripts and bitcoin ops.