➹ Clean-mark

Convert a blog article into a clean Markdown text file.

Example

For example, this article:

Is converted into this text file:

Usage

$ clean-mark "http://some-website.com/fancy-article"

The article will be automatically named using the URL path name. In the case, above, the name will be some-fancy-article.md.

The file type can be specified:

$ clean-mark "http://some-website.com/fancy-article" -t html

The available types are: HTML, TEXT and Markdown.

The output file and path can be also specified:

$ clean-mark "http://some-website.com/fancy-article" -o /tmp/article

In that case the output will be /tmp/article.md. The extension is added automatically.

Installation

Simply install with npm:

$ npm install clean-mark --global

Why ?

to save interesting articles offline, in a highly readable text format
it's easy to read on a tablet, or a Kindle (as it is, or exported to PDF)
Markdown is easy to export into different formats
for offline text analysis of multiple articles, using machine learning / AI

How ?

Implementation steps:

Downloads the content of a web page
Meta-scrape page details (title, author, date, etc)
Sanitizes the ugly HTML
Minifies the disinfected HTML
Converts the result into clean Markdown text

This project depends on the A-Extractor project, a database of expressions used for extracting content from blogs and articles.

Vision

The goals of the project are are:

Good text extraction
More useless text is preferred, instead of cutting from the actual article by mistake
Extracting media (images, videos, audio) is not that important
Extraction speed is not that important

Contributing

Clean-mark was tested on all major news sites. On some websites, the text, or links are cut from the article. In this case, you have to manually edit the resulted text,

AND

please raise an issue on A-Extractor with the link that doesn't work and we'll add it in the database, so that next time, the text will be extracted correctly.

Also, see how to contribute.

HpWens/clean-mark