Disclaimer: this is a test repo, maybe it will be a real project in the future.
If you want to use a GPTs that use this output file as Knowledge Base, you can use this Ruby and Rails Agent I'm working on.
This project contains scripts that automate the process of downloading web content from a list of URLs and compiling it into a single, usable HTML file.
The main script reads a markdown file containing a list of URLs, downloads the HTML content from each URL, extracts the main content using the Readability library, and writes the extracted content into a single HTML file.
- Ruby
- Readability gem
- Open-uri (built-in Ruby library)
-
Create a markdown file with a list of URLs you want to extract content from. Each URL should be on a new line.
-
Run the script with the input markdown file and the desired output HTML file as arguments:
ruby test.rb