vxern/robots_txt

Remove non-developer dependencies.

Closed this issue · 1 comments

vxern commented

For it to be a legitimate claim that the package is 'lightweight', the package should have as few dependencies as possible.

  • Remove sprint.
    This will be a breaking change, as the library will have to throw an exception instead.
  • Remove web_scraper.
    Instead of scraping webpages itself to obtain the contents of the robots.txt file, the package could instead accept a content parameter passed into it by the developer using the library, thus making the package both more flexible and legitimately 'lightweight'. This will also be a breaking change.
vxern commented

Removed in #5.